WEBVTT - This Is What Happens When Governments Build Software

0:00:10.119 --> 0:00:14.120
<v Speaker 1>Hello, and welcome to another episode of the Odd Lots Podcast.

0:00:14.200 --> 0:00:16.560
<v Speaker 2>I'm Jill Wisenthal and I'm Tracy Alloway.

0:00:16.760 --> 0:00:19.120
<v Speaker 1>Tracy, you know, what I felt was a really interesting

0:00:19.200 --> 0:00:21.720
<v Speaker 1>thing in a recent debt ceiling episode that we did

0:00:21.920 --> 0:00:24.439
<v Speaker 1>was about all the coding it would take if the

0:00:24.480 --> 0:00:27.680
<v Speaker 1>Treasury were to introduce a new kind of a bill.

0:00:27.960 --> 0:00:31.280
<v Speaker 2>Yes, that was interesting to me too, because I think

0:00:31.320 --> 0:00:34.639
<v Speaker 2>there's a perception out there that if Treasury wakes up

0:00:34.720 --> 0:00:37.680
<v Speaker 2>tomorrow and decides to sell a new type of bond,

0:00:38.280 --> 0:00:40.320
<v Speaker 2>they could just do it. But actually there are all

0:00:40.360 --> 0:00:43.160
<v Speaker 2>these back end changes that would need to be made.

0:00:43.720 --> 0:00:46.879
<v Speaker 2>And as you and I know from multiple conversations at

0:00:46.880 --> 0:00:50.760
<v Speaker 2>this point, it feels like government technology can be a

0:00:50.760 --> 0:00:53.760
<v Speaker 2>little bit clunky sometimes. Is that a fair way of

0:00:53.760 --> 0:00:54.320
<v Speaker 2>putting it?

0:00:54.320 --> 0:00:54.640
<v Speaker 3>It is?

0:00:54.760 --> 0:00:57.000
<v Speaker 1>I would say, you know, in defense, I mean get

0:00:57.000 --> 0:01:00.600
<v Speaker 1>into yes, you know, corporate technology can be clunky. But

0:01:00.760 --> 0:01:04.640
<v Speaker 1>that point reminded me that, like we've done, software is

0:01:04.640 --> 0:01:07.440
<v Speaker 1>like a fascinating thing. And we talk a lot on

0:01:07.480 --> 0:01:10.200
<v Speaker 1>the show. I think about like the sort of built economy,

0:01:10.319 --> 0:01:13.400
<v Speaker 1>like the Chips Act and battery investment and semiconductor and

0:01:13.440 --> 0:01:16.440
<v Speaker 1>all these things are like can the US build things again?

0:01:16.520 --> 0:01:20.280
<v Speaker 1>Like as a government state capacity these ideas, but in

0:01:20.319 --> 0:01:23.759
<v Speaker 1>the twenty first century also like building software in tech

0:01:23.959 --> 0:01:24.920
<v Speaker 1>has to be part of that.

0:01:24.959 --> 0:01:27.800
<v Speaker 2>Question, right, So it's one thing to announce that you're

0:01:27.840 --> 0:01:31.240
<v Speaker 2>going to do this new thing, but almost everything that

0:01:31.280 --> 0:01:34.800
<v Speaker 2>we do nowadays comes with some sort of software requirement.

0:01:35.120 --> 0:01:38.520
<v Speaker 2>Like I'm sure there is a system for processing applications

0:01:38.560 --> 0:01:41.479
<v Speaker 2>for chips, funding and things like that, and someone has

0:01:41.520 --> 0:01:44.600
<v Speaker 2>to actually build that. So there's two questions contained in

0:01:44.640 --> 0:01:48.040
<v Speaker 2>the can America build things question? It's like, can it

0:01:48.120 --> 0:01:50.800
<v Speaker 2>build the actual things and then can it build the

0:01:50.840 --> 0:01:53.200
<v Speaker 2>software systems that would allow it to build the things?

0:01:53.320 --> 0:01:56.280
<v Speaker 1>And you know, in the pandemic period, we got sort

0:01:56.320 --> 0:01:58.200
<v Speaker 1>of smacked in the face, I would say, by this

0:01:58.320 --> 0:02:02.120
<v Speaker 1>realization that the saw fair component is not trivial at all.

0:02:02.200 --> 0:02:05.680
<v Speaker 1>And most notably it was like every state had some

0:02:05.880 --> 0:02:09.440
<v Speaker 1>sort of problem with their unemployment insurance system and there

0:02:09.440 --> 0:02:12.200
<v Speaker 1>were just such a surge in claims obviously in those

0:02:12.200 --> 0:02:14.160
<v Speaker 1>initial you know, really for that first year, just on

0:02:14.160 --> 0:02:16.920
<v Speaker 1>the scale that none had ever seen. And it reminded

0:02:16.960 --> 0:02:20.480
<v Speaker 1>people's like, oh, yeah, fifty different stage systems much with

0:02:20.600 --> 0:02:23.480
<v Speaker 1>like tech that was probably built by some contractor, and

0:02:23.720 --> 0:02:25.600
<v Speaker 1>I don't know the eighties or nineties and the people

0:02:25.639 --> 0:02:28.280
<v Speaker 1>who knew how to run it have left, and you know,

0:02:28.280 --> 0:02:30.520
<v Speaker 1>it was a I gues Eventually, I guess it got

0:02:30.760 --> 0:02:34.200
<v Speaker 1>smoothed out, but it sort of exposed like, Okay, why

0:02:34.200 --> 0:02:36.399
<v Speaker 1>did that happen? And what else is lurking out there

0:02:36.400 --> 0:02:38.400
<v Speaker 1>that we don't really know how it works until it's broken.

0:02:38.560 --> 0:02:39.919
<v Speaker 2>I'm sure this is going to end up being a

0:02:39.960 --> 0:02:42.920
<v Speaker 2>classic episode where we mentioned Kobyl quite a few times,

0:02:43.000 --> 0:02:45.880
<v Speaker 2>like it's inevitable, right, it's coming, you can feel it.

0:02:45.919 --> 0:02:47.640
<v Speaker 1>One day, We're going to do it like actually, just

0:02:47.680 --> 0:02:50.840
<v Speaker 1>like what is Kobol episode where we just like focus

0:02:50.880 --> 0:02:52.720
<v Speaker 1>on that directly. But yes, we're going to talk about

0:02:52.760 --> 0:02:54.280
<v Speaker 1>software and we're going to talk about it from the

0:02:54.320 --> 0:02:57.480
<v Speaker 1>public sector perspective. What does it mean when the government

0:02:57.560 --> 0:03:00.480
<v Speaker 1>tries to fix something, what happens when the govern tries

0:03:00.480 --> 0:03:02.880
<v Speaker 1>to build something, what happens when the government tries to

0:03:02.880 --> 0:03:05.799
<v Speaker 1>buy something. Some of these themes we talked about them

0:03:05.800 --> 0:03:10.240
<v Speaker 1>from a private sector perspective with Patrick McKenzie earlier in

0:03:10.280 --> 0:03:12.160
<v Speaker 1>the year, but more to do on this topic.

0:03:12.480 --> 0:03:14.800
<v Speaker 2>Yeah, I'm excited. I have a lot This is going

0:03:14.880 --> 0:03:17.880
<v Speaker 2>to be a really good chance to actually ask like

0:03:18.320 --> 0:03:22.280
<v Speaker 2>the decisions that are going into some of these things

0:03:22.320 --> 0:03:25.600
<v Speaker 2>being built. I'm still floored every time I log into

0:03:25.680 --> 0:03:28.000
<v Speaker 2>the Treasury Direct website that you know you have to

0:03:28.000 --> 0:03:31.959
<v Speaker 2>click the little the buttons like the actual letters and

0:03:32.040 --> 0:03:34.640
<v Speaker 2>numbers to type your password. It won't let you just

0:03:34.720 --> 0:03:35.640
<v Speaker 2>type your password.

0:03:36.240 --> 0:03:37.680
<v Speaker 4>How did that come to be?

0:03:38.200 --> 0:03:39.279
<v Speaker 2>I have questions?

0:03:39.880 --> 0:03:42.720
<v Speaker 1>That is a great question. So we have I think

0:03:42.840 --> 0:03:46.720
<v Speaker 1>literally the two perfect guests to discuss this, two guests

0:03:46.720 --> 0:03:51.280
<v Speaker 1>with extensive experience answering and working on exactly these problems.

0:03:51.280 --> 0:03:53.880
<v Speaker 1>We're going to be speaking with Jennifer Palka. She is

0:03:53.920 --> 0:03:57.200
<v Speaker 1>the founder of Code for America. She helped found the

0:03:57.320 --> 0:04:00.880
<v Speaker 1>US Digital Service. She co chaired the cal Fornia task

0:04:00.960 --> 0:04:05.480
<v Speaker 1>force on fixing its unemployment insurance system during the pandemic,

0:04:05.600 --> 0:04:08.680
<v Speaker 1>and she's the author of a new book called Recoding America,

0:04:09.080 --> 0:04:11.240
<v Speaker 1>and we're going to be speaking with it. Dave Guarino,

0:04:11.360 --> 0:04:15.200
<v Speaker 1>who's a number of things currently an independent consultant researcher

0:04:15.240 --> 0:04:18.520
<v Speaker 1>focused on this stuff. Founding engineer at Code for America,

0:04:18.839 --> 0:04:22.920
<v Speaker 1>helped create get cal Fresh, which was like allowed Californians

0:04:22.920 --> 0:04:28.440
<v Speaker 1>to easily access snap has recently worked on the unemployment

0:04:28.440 --> 0:04:32.279
<v Speaker 1>insurance modernization at the Department of Labor. Also worked at California,

0:04:32.360 --> 0:04:35.880
<v Speaker 1>also worked in California. So two people's extensive experience on

0:04:35.960 --> 0:04:38.839
<v Speaker 1>exactly these things. So, Jennifer and David, thank you so

0:04:38.920 --> 0:04:40.000
<v Speaker 1>much for coming on odd.

0:04:39.800 --> 0:04:41.720
<v Speaker 4>Lots great, thanks for having us.

0:04:41.880 --> 0:04:43.279
<v Speaker 3>Absolutely great to be here.

0:04:44.440 --> 0:04:46.640
<v Speaker 1>Now I want to ask Tracy, could I steal your

0:04:46.640 --> 0:04:47.160
<v Speaker 1>first question?

0:04:47.200 --> 0:04:47.360
<v Speaker 3>Yeah?

0:04:47.360 --> 0:04:47.720
<v Speaker 5>I do it.

0:04:47.839 --> 0:04:48.920
<v Speaker 4>How has just happened that.

0:04:48.880 --> 0:04:52.279
<v Speaker 1>A government website doesn't let you type in something and

0:04:52.320 --> 0:04:55.479
<v Speaker 1>you have to like click buttons like that like resemble

0:04:55.600 --> 0:04:57.960
<v Speaker 1>mini keyboard on a website. What happened when you see that?

0:04:58.320 --> 0:04:59.880
<v Speaker 1>Do you have like in your mind you're like, oh,

0:04:59.920 --> 0:05:02.520
<v Speaker 1>I know exactly the meeting that caused this to happen.

0:05:02.720 --> 0:05:05.400
<v Speaker 4>I think Dave has been in those meetings and has

0:05:05.440 --> 0:05:07.919
<v Speaker 4>come out like with his hair on fire, fuming, So

0:05:08.040 --> 0:05:09.760
<v Speaker 4>I will let him answer that.

0:05:09.360 --> 0:05:12.120
<v Speaker 3>I mean, well, I don't have Yeah, I wasn't in

0:05:12.160 --> 0:05:15.880
<v Speaker 3>that specific meeting. Maybe the best answer I can give is,

0:05:16.400 --> 0:05:20.719
<v Speaker 3>and there's deeper issues behind it, is that the way

0:05:20.839 --> 0:05:23.359
<v Speaker 3>Garment tends to build technology. The reason for that is

0:05:23.400 --> 0:05:27.480
<v Speaker 3>no one wrote down specifically as a requirement upfront that

0:05:27.600 --> 0:05:30.800
<v Speaker 3>people should be able to use the keypad and the

0:05:30.880 --> 0:05:35.680
<v Speaker 3>numbers on their keyboard to enter the digits, and so

0:05:35.720 --> 0:05:37.920
<v Speaker 3>someone's like, well, I've met the requirements. Like the requirements

0:05:37.960 --> 0:05:41.120
<v Speaker 3>are that someone can do this, they can enter it,

0:05:41.160 --> 0:05:43.680
<v Speaker 3>and the way they do that is they click in

0:05:43.760 --> 0:05:46.359
<v Speaker 3>this somewhat I suppose insane way.

0:05:46.640 --> 0:05:49.160
<v Speaker 1>This is gonna be a conversation or like bang your

0:05:49.160 --> 0:05:51.120
<v Speaker 1>head on a disk a bunch. I feel like as

0:05:51.160 --> 0:05:52.880
<v Speaker 1>we listen these answers, anyway.

0:05:52.680 --> 0:05:53.400
<v Speaker 4>Yeah, I get ready.

0:05:53.480 --> 0:05:56.480
<v Speaker 2>Okay, Well, let me ask maybe a big picture question

0:05:56.760 --> 0:06:00.560
<v Speaker 2>to start, but what is the process by which government

0:06:00.640 --> 0:06:04.800
<v Speaker 2>software is developed? Because you know that answer just then

0:06:04.960 --> 0:06:07.560
<v Speaker 2>it makes it sound like someone gets a mandate we

0:06:07.640 --> 0:06:10.640
<v Speaker 2>need a website, for instance, through which people can buy

0:06:10.920 --> 0:06:13.960
<v Speaker 2>government bonds, and then someone else goes off and I

0:06:13.960 --> 0:06:16.600
<v Speaker 2>guess they execute it along the lines of the mandate

0:06:16.640 --> 0:06:17.280
<v Speaker 2>that's been set.

0:06:17.360 --> 0:06:19.000
<v Speaker 4>I mean, I'll jump in on that one. I mean,

0:06:19.160 --> 0:06:21.320
<v Speaker 4>first of all, it is changing. So just before we

0:06:21.400 --> 0:06:24.200
<v Speaker 4>get everyone sort of bounding their heads on the table,

0:06:24.240 --> 0:06:26.800
<v Speaker 4>there's a lot of movement in good directions. But what

0:06:27.000 --> 0:06:29.880
<v Speaker 4>has been the process is exactly that there's sort of

0:06:29.960 --> 0:06:33.600
<v Speaker 4>this mandated often comes from legislation or regulation that very

0:06:33.600 --> 0:06:36.120
<v Speaker 4>frequently now like says there will be a website that'll

0:06:36.120 --> 0:06:39.440
<v Speaker 4>even put the URL in the legislation, and then it

0:06:39.560 --> 0:06:43.440
<v Speaker 4>kicks off what's a really long process that does exactly

0:06:43.600 --> 0:06:48.320
<v Speaker 4>what Dave was saying. It is first centered around requirements gathering,

0:06:48.480 --> 0:06:51.599
<v Speaker 4>and that process of just requirements gathering can easily take

0:06:51.640 --> 0:06:53.920
<v Speaker 4>ten years. Now, when it's something like healthcare doc of

0:06:54.320 --> 0:06:56.400
<v Speaker 4>you know, and they have a launch date that's three

0:06:56.440 --> 0:07:00.320
<v Speaker 4>years out, they don't take ten years, but they still

0:07:00.320 --> 0:07:04.520
<v Speaker 4>gather sort of every imaginable requirement and they kind of

0:07:04.560 --> 0:07:08.720
<v Speaker 4>throw it in one giant bucket. And that's how the

0:07:08.920 --> 0:07:12.240
<v Speaker 4>RFP is created. The request for proposal, that's what vendors

0:07:12.400 --> 0:07:16.320
<v Speaker 4>bid on, and it's sort of like an undifferentiated, unprioritized

0:07:16.400 --> 0:07:20.000
<v Speaker 4>mess of everything everyone could think of, plus things that

0:07:20.040 --> 0:07:22.960
<v Speaker 4>have sort of pulled over from previous RFPs, plus a

0:07:23.000 --> 0:07:28.000
<v Speaker 4>bunch of compliance stuff around security, compliance stuff around well

0:07:28.080 --> 0:07:30.720
<v Speaker 4>various other things. Dave can maybe chime in, but what

0:07:30.760 --> 0:07:34.080
<v Speaker 4>that isn't is product management. Right, there's no one saying

0:07:34.200 --> 0:07:37.600
<v Speaker 4>here's the important things this should do. And as Dave said,

0:07:37.640 --> 0:07:40.560
<v Speaker 4>there's never a requirement for it to just work. So

0:07:41.040 --> 0:07:43.600
<v Speaker 4>then the vendor, I mean, I'll skip several steps in

0:07:43.640 --> 0:07:45.720
<v Speaker 4>the middle where you know, it takes a long time

0:07:45.760 --> 0:07:47.960
<v Speaker 4>to bid it out, and then there's like a protest

0:07:48.080 --> 0:07:51.000
<v Speaker 4>because one vendor thinks the other vendor shouldn't have got it,

0:07:51.040 --> 0:07:52.600
<v Speaker 4>and that delays it a couple more years. But then

0:07:52.640 --> 0:07:55.960
<v Speaker 4>they go into development and their job, as Dave said,

0:07:56.040 --> 0:07:58.760
<v Speaker 4>is just like check all these boxes, and it's thousands.

0:07:58.800 --> 0:08:02.480
<v Speaker 4>I mean, when we were at California, State of California

0:08:02.560 --> 0:08:05.679
<v Speaker 4>working on unemployment insurance, they were actually about to put

0:08:05.720 --> 0:08:10.240
<v Speaker 4>out a Business System Modernization RFP. Actually I think about

0:08:10.240 --> 0:08:12.320
<v Speaker 4>it to award it to a vendor, and I think

0:08:12.320 --> 0:08:15.800
<v Speaker 4>it had seven six seven hundred requirements. We can check that,

0:08:15.840 --> 0:08:19.320
<v Speaker 4>but it's in that's really a normal number of requirements.

0:08:19.360 --> 0:08:21.880
<v Speaker 4>And so they do all these things that where you

0:08:21.880 --> 0:08:24.240
<v Speaker 4>can check a box, but what they don't do is

0:08:24.280 --> 0:08:27.560
<v Speaker 4>actually check that it works. So I mean, maybe i'll

0:08:27.600 --> 0:08:29.480
<v Speaker 4>just tell a quick story from the book. And you know,

0:08:29.520 --> 0:08:34.480
<v Speaker 4>there was famously this application for veterans' healthcare benefits at

0:08:34.520 --> 0:08:39.200
<v Speaker 4>the VA that didn't work outside the building and they

0:08:39.840 --> 0:08:43.240
<v Speaker 4>didn't couldn't see it. So basically, somewhere in the specs

0:08:43.360 --> 0:08:46.640
<v Speaker 4>it had that this form needed to work on a

0:08:46.800 --> 0:08:51.840
<v Speaker 4>very specific and outdated combination of Internet Explorer and Adobe

0:08:51.880 --> 0:08:54.600
<v Speaker 4>Reader and The reason no one knew that it didn't

0:08:54.600 --> 0:08:57.000
<v Speaker 4>work inside the building is that's how all the computers

0:08:57.040 --> 0:08:59.439
<v Speaker 4>in there were set up. But if you were outside

0:08:59.440 --> 0:09:02.800
<v Speaker 4>the building, had any other possible combination of those two

0:09:02.800 --> 0:09:05.360
<v Speaker 4>pieces of software, it literally wouldn't load. And so they

0:09:05.360 --> 0:09:09.520
<v Speaker 4>had very very few people applying for these benefits online,

0:09:09.920 --> 0:09:12.920
<v Speaker 4>and you know, veterans were really, really, really frustrated. And

0:09:12.920 --> 0:09:16.720
<v Speaker 4>it took a team going out there recording a veteran

0:09:16.880 --> 0:09:19.679
<v Speaker 4>who had tried to do this dozens and dozens of times,

0:09:20.120 --> 0:09:24.360
<v Speaker 4>bringing that video back and showing it to the Deputy

0:09:24.400 --> 0:09:26.560
<v Speaker 4>secretary for them to be able to say, oh, okay,

0:09:26.600 --> 0:09:30.000
<v Speaker 4>actually there is something to be fixed here. Okay, you know,

0:09:30.040 --> 0:09:31.720
<v Speaker 4>you can go ahead and make a new form. But

0:09:31.800 --> 0:09:34.760
<v Speaker 4>up to them, they said, sorry, it's fine, we're looking

0:09:34.760 --> 0:09:37.719
<v Speaker 4>at the requirements. The requirements have been met. There's technically

0:09:37.720 --> 0:09:39.760
<v Speaker 4>nothing wrong. That's amazing.

0:09:39.840 --> 0:09:42.120
<v Speaker 2>It's also kind of crazy to think that people would

0:09:42.160 --> 0:09:43.839
<v Speaker 2>have been looking at that and just been like, oh, well,

0:09:43.840 --> 0:09:47.120
<v Speaker 2>I guess demand for veterans benefits is lower than we

0:09:47.200 --> 0:09:49.360
<v Speaker 2>thought it would be. But actually it was a tech issue.

0:09:49.400 --> 0:09:51.880
<v Speaker 4>I think what they said was demand for them doing

0:09:51.880 --> 0:09:54.360
<v Speaker 4>it online is low. These veterans must.

0:09:54.120 --> 0:09:56.240
<v Speaker 2>Not have access, which is absolutely not true, or they

0:09:56.240 --> 0:09:58.520
<v Speaker 2>can't figure out the computer when in fact, yeah, I

0:09:58.520 --> 0:10:00.640
<v Speaker 2>mean them couldn't figure out the software.

0:10:00.679 --> 0:10:02.520
<v Speaker 4>They would tell them. And they told this guy Dominic,

0:10:02.520 --> 0:10:04.480
<v Speaker 4>the veteran that they interviewed, They kept telling him it's

0:10:04.559 --> 0:10:06.040
<v Speaker 4>user error. There's something wrong with you.

0:10:06.840 --> 0:10:09.160
<v Speaker 1>Can I ask you a question about that bidding process?

0:10:09.240 --> 0:10:14.760
<v Speaker 1>When I hear government goes out to bidders for a website,

0:10:15.160 --> 0:10:21.280
<v Speaker 1>I imagine these like big sort of faceless offices in Roslin, Virginia,

0:10:21.800 --> 0:10:24.199
<v Speaker 1>where you know, people get like chop salid for lunch

0:10:24.240 --> 0:10:25.959
<v Speaker 1>and stuff like that, and there's like three or four

0:10:26.000 --> 0:10:27.800
<v Speaker 1>of them and they sort of like rotate who wins

0:10:27.840 --> 0:10:30.439
<v Speaker 1>the bid and stuff like that. How competitive, like is

0:10:30.480 --> 0:10:33.800
<v Speaker 1>it would a typical rfp auction or bidding process be

0:10:33.840 --> 0:10:37.640
<v Speaker 1>and how hard would it be forcea a more agile,

0:10:37.840 --> 0:10:40.840
<v Speaker 1>innovative firm or someone wanting to do it differently to

0:10:41.440 --> 0:10:43.600
<v Speaker 1>break into the pursuit of that contract.

0:10:43.760 --> 0:10:46.920
<v Speaker 4>That's changing a lot. So now you've got a set

0:10:46.960 --> 0:10:49.760
<v Speaker 4>of vendors. I think there's about thirty five of them

0:10:50.040 --> 0:10:52.400
<v Speaker 4>in this thing called the Digital Services Alliance, which are

0:10:52.559 --> 0:10:55.520
<v Speaker 4>smaller companies that work in a more sort of agile,

0:10:55.600 --> 0:10:58.880
<v Speaker 4>user centered way, and you know, they're just a lot

0:10:58.960 --> 0:11:02.280
<v Speaker 4>smaller than the Beltway, so there's still a tiny portion

0:11:02.880 --> 0:11:06.319
<v Speaker 4>of the market, but you know, they're really viable businesses,

0:11:06.400 --> 0:11:09.000
<v Speaker 4>and people in government can contract out to them. In fact,

0:11:09.000 --> 0:11:10.959
<v Speaker 4>I think there's a mechanism by which you can put

0:11:11.000 --> 0:11:13.800
<v Speaker 4>an RFP out just to those companies. But what has

0:11:13.840 --> 0:11:17.560
<v Speaker 4>happened in the past is like Dave can pile onto this.

0:11:17.679 --> 0:11:21.080
<v Speaker 4>Like if you were bidding out, say you know benefits

0:11:21.120 --> 0:11:23.880
<v Speaker 4>system in a state, and pick your benefit. Usually they

0:11:23.920 --> 0:11:28.200
<v Speaker 4>would require in the RFP that the bidder had to

0:11:28.240 --> 0:11:32.240
<v Speaker 4>have done a similar system in a different state in

0:11:32.320 --> 0:11:35.280
<v Speaker 4>the same category, which means that right there it limited

0:11:35.280 --> 0:11:39.000
<v Speaker 4>to like three vendors, and so you were never going

0:11:39.080 --> 0:11:42.880
<v Speaker 4>to get any disruptors because they were boxed out in

0:11:42.920 --> 0:11:44.080
<v Speaker 4>the very first step.

0:11:44.200 --> 0:11:47.040
<v Speaker 3>Just to maybe jump in. That's exactly right, and if

0:11:47.080 --> 0:11:49.440
<v Speaker 3>you I mean, to some extent, it's rational. If you

0:11:49.480 --> 0:11:52.239
<v Speaker 3>are thinking about this as someone who's not a technologist,

0:11:52.360 --> 0:11:55.640
<v Speaker 3>you run an agency, and you're thinking, we need to

0:11:55.679 --> 0:11:58.000
<v Speaker 3>do this very large project. It's going to take a

0:11:58.120 --> 0:12:01.640
<v Speaker 3>very long time. It we really need it to go right.

0:12:02.320 --> 0:12:06.760
<v Speaker 3>It is hard to think about, well, why wouldn't I

0:12:06.840 --> 0:12:10.640
<v Speaker 3>look for a vendor who's done this five, ten, fifteen times.

0:12:10.640 --> 0:12:13.840
<v Speaker 3>The problem is, as Jen said, well, I think there's

0:12:13.840 --> 0:12:17.120
<v Speaker 3>two problems in that. One is you've now very very

0:12:17.200 --> 0:12:20.400
<v Speaker 3>drastically narrowed your vendor pool, so there's a lot less competition,

0:12:20.800 --> 0:12:25.200
<v Speaker 3>and two you also may have too big of a project,

0:12:25.280 --> 0:12:27.680
<v Speaker 3>Like really, what level of attraction are you trying to

0:12:27.720 --> 0:12:30.120
<v Speaker 3>work towards? And is it the case that really you

0:12:30.200 --> 0:12:33.439
<v Speaker 3>need someone or a vendor who's worked on this specific

0:12:33.640 --> 0:12:36.280
<v Speaker 3>type of system in another entity that it works exactly

0:12:36.360 --> 0:12:38.959
<v Speaker 3>the same way and all of those details of the

0:12:39.000 --> 0:12:42.880
<v Speaker 3>exact same program, the exact same structure of agencies. Probably not,

0:12:43.040 --> 0:12:47.600
<v Speaker 3>because probably it's about more generically software. But if you

0:12:47.679 --> 0:12:50.520
<v Speaker 3>really focus only on the track record there, you kind

0:12:50.520 --> 0:12:54.680
<v Speaker 3>of get stuck, as Jema was saying, in this trap

0:12:54.760 --> 0:12:57.839
<v Speaker 3>where there's only a small number of vendors who can

0:12:57.880 --> 0:13:01.240
<v Speaker 3>meet those criteria, even if you think they're And it's

0:13:01.280 --> 0:13:03.120
<v Speaker 3>hard because the other thing I would say is as

0:13:03.200 --> 0:13:05.640
<v Speaker 3>much as there are new vendors, and maybe Jenny can

0:13:05.640 --> 0:13:08.280
<v Speaker 3>speak more of this, but it's also the case that

0:13:08.320 --> 0:13:10.280
<v Speaker 3>if you have a new vendor that wants to work

0:13:10.320 --> 0:13:12.160
<v Speaker 3>in a more agile way, wants to work in a

0:13:12.160 --> 0:13:16.360
<v Speaker 3>more use this CRD way, but they are bidding on

0:13:16.440 --> 0:13:19.880
<v Speaker 3>an RFP that is structured in a very waterfall top

0:13:20.000 --> 0:13:22.920
<v Speaker 3>down meet other requirements, and that's what success looks like.

0:13:23.160 --> 0:13:26.280
<v Speaker 3>It's also not going to be successful because you're not

0:13:26.320 --> 0:13:30.040
<v Speaker 3>giving them any space to say, oh, well, I know

0:13:30.120 --> 0:13:33.840
<v Speaker 3>your requirement says a way to ender phone numbers using

0:13:34.040 --> 0:13:37.000
<v Speaker 3>the key the mouse, but when we tested it with

0:13:37.080 --> 0:13:40.840
<v Speaker 3>people like people, turns out really don't like using the

0:13:40.920 --> 0:13:43.520
<v Speaker 3>mouse to click numbers to enter digits. They prefer to

0:13:43.600 --> 0:13:46.880
<v Speaker 3>use the keyboard. Could we change that requirement? And honestly,

0:13:47.120 --> 0:13:50.040
<v Speaker 3>I mean, that's an extreme example, but a lot of

0:13:51.160 --> 0:13:52.760
<v Speaker 3>and there's a lot of what Jen's book is about

0:13:52.880 --> 0:13:56.200
<v Speaker 3>and what resonated with me in it is that in

0:13:56.679 --> 0:13:59.640
<v Speaker 3>doing these things and building software, you learn those details

0:13:59.640 --> 0:14:02.040
<v Speaker 3>that you never could have gotten right up front. And

0:14:02.120 --> 0:14:07.640
<v Speaker 3>so if you make success defined by implementing everything as

0:14:07.679 --> 0:14:11.600
<v Speaker 3>we understand it before we ever get started, you almost

0:14:11.720 --> 0:14:14.200
<v Speaker 3>ensure that you're not going to get what you really

0:14:14.240 --> 0:14:17.559
<v Speaker 3>wanted because you're not allowing any information after that cut

0:14:17.600 --> 0:14:19.800
<v Speaker 3>off point, right when you might be getting to the

0:14:19.800 --> 0:14:22.160
<v Speaker 3>point where in doing it you're gonna learn a lot.

0:14:39.280 --> 0:14:41.320
<v Speaker 2>I definitely want to ask you more about that top

0:14:41.400 --> 0:14:44.800
<v Speaker 2>down waterfall nature of how a lot of these software

0:14:44.840 --> 0:14:47.760
<v Speaker 2>projects get commissioned. But before I do, I'm just curious

0:14:47.840 --> 0:14:51.600
<v Speaker 2>about the actual vendors because I'm sort of imagining these

0:14:51.640 --> 0:14:57.200
<v Speaker 2>like big faceless offices. But what types of vendors are

0:14:57.240 --> 0:14:59.720
<v Speaker 2>we talking about? And I'm trying to think how to

0:15:00.120 --> 0:15:03.680
<v Speaker 2>use those kind of diplomatically. But I can imagine a

0:15:03.800 --> 0:15:08.000
<v Speaker 2>company that is putting itself out bidding for government contracts

0:15:08.120 --> 0:15:13.360
<v Speaker 2>on software development, Like maybe they're not as experimental or

0:15:13.440 --> 0:15:17.320
<v Speaker 2>cutting edge as a software company that's, you know, somewhere

0:15:17.360 --> 0:15:21.920
<v Speaker 2>in San Francisco and it's building like new exciting products

0:15:21.640 --> 0:15:25.240
<v Speaker 2>for the corporate world. Although we've also done an episode

0:15:25.240 --> 0:15:29.400
<v Speaker 2>on how bad internal corporate tech can be, but it

0:15:29.520 --> 0:15:32.680
<v Speaker 2>seems like it might be a little bit more I

0:15:32.720 --> 0:15:37.080
<v Speaker 2>hesitate to say boring, but sort of basic playing it safe.

0:15:37.160 --> 0:15:39.520
<v Speaker 4>Well, you know one thing that I will say, like

0:15:39.600 --> 0:15:43.960
<v Speaker 4>I hear both incredible frustration and even anger from folks

0:15:44.000 --> 0:15:46.720
<v Speaker 4>that have to hire these companies. They're just like, there's

0:15:46.720 --> 0:15:50.400
<v Speaker 4>no alternatives. This has gone badly, it will go badly.

0:15:50.440 --> 0:15:52.880
<v Speaker 4>They kind of know that, Like I had this woman

0:15:53.040 --> 0:15:57.120
<v Speaker 4>in a major state. I won't say which who. When

0:15:57.120 --> 0:15:59.080
<v Speaker 4>I was said, you know your project's going to fail,

0:15:59.160 --> 0:16:01.440
<v Speaker 4>she said, do you think we don't know that? The

0:16:01.560 --> 0:16:04.720
<v Speaker 4>last seven projects have failed. So there's this a huge frustration.

0:16:05.240 --> 0:16:11.560
<v Speaker 4>But there's also this real feeling that the company in

0:16:11.600 --> 0:16:15.280
<v Speaker 4>San Francisco with its forty people who make consumer software

0:16:15.760 --> 0:16:18.560
<v Speaker 4>like great, people love using that software, but they don't

0:16:18.600 --> 0:16:21.880
<v Speaker 4>understand us, right, They're not going to actually understand the

0:16:21.880 --> 0:16:25.040
<v Speaker 4>constraints of government. So we have to go with these

0:16:25.040 --> 0:16:29.120
<v Speaker 4>big companies. And they're not wrong in the sense that

0:16:29.240 --> 0:16:32.880
<v Speaker 4>the constraints of working with government are real. There's a

0:16:32.960 --> 0:16:35.960
<v Speaker 4>huge compliance burden that forty person company in Sum Francisco

0:16:36.640 --> 0:16:40.520
<v Speaker 4>mostly does not want these jobs, Like there's so much

0:16:40.560 --> 0:16:44.520
<v Speaker 4>that goes into getting the you know, getting the bid,

0:16:45.360 --> 0:16:48.160
<v Speaker 4>you know these it's sort of famously said that these

0:16:48.200 --> 0:16:51.760
<v Speaker 4>companies know how to get the get the work, not

0:16:52.320 --> 0:16:54.760
<v Speaker 4>how to deliver on the work. That's not probably what

0:16:54.800 --> 0:16:58.400
<v Speaker 4>a fordy person company wants to do, but you know,

0:16:59.000 --> 0:17:02.320
<v Speaker 4>it's true that you have to deliver on stuff that

0:17:02.440 --> 0:17:06.840
<v Speaker 4>is mind bogglingly complex. Like when we're working in unemployment

0:17:06.840 --> 0:17:12.200
<v Speaker 4>insurance again during the pandemic, my colleague was talking with

0:17:12.400 --> 0:17:15.760
<v Speaker 4>the claims processors like week over week, and we're trying

0:17:15.800 --> 0:17:18.520
<v Speaker 4>to like dissect it and figure out what's going wrong

0:17:18.600 --> 0:17:21.600
<v Speaker 4>and like clear this backlog. And one of these guys

0:17:21.680 --> 0:17:24.600
<v Speaker 4>keep saying, well, I'm not quite sure about that answer.

0:17:24.680 --> 0:17:26.560
<v Speaker 4>I'm the new guy. I'm the new guy. And she

0:17:26.640 --> 0:17:29.600
<v Speaker 4>finally says, how long have you been here? And he says,

0:17:29.680 --> 0:17:33.119
<v Speaker 4>I've been here seventeen years. The guys who really know

0:17:33.200 --> 0:17:35.879
<v Speaker 4>how this works have been here twenty five years or more.

0:17:36.440 --> 0:17:40.880
<v Speaker 4>So think about, like, you know, going from doing some simple, cool,

0:17:41.080 --> 0:17:44.720
<v Speaker 4>you know tech app, you know, easy consumer app, to

0:17:45.200 --> 0:17:48.640
<v Speaker 4>trying to build or fix or improve upon a system

0:17:49.240 --> 0:17:52.440
<v Speaker 4>that is so complex that it takes twenty five years

0:17:52.520 --> 0:17:56.320
<v Speaker 4>to learn how to process a claim. That's that's sort

0:17:56.320 --> 0:17:57.919
<v Speaker 4>of I think what needs to be on the table

0:17:57.960 --> 0:18:00.680
<v Speaker 4>as part of this agenda is not just like can

0:18:00.720 --> 0:18:03.000
<v Speaker 4>the tech be better? But can we go back and

0:18:03.000 --> 0:18:07.440
<v Speaker 4>simplify the accumulated like ninety years of policy and process

0:18:07.840 --> 0:18:09.560
<v Speaker 4>that's making that so hard to make?

0:18:09.920 --> 0:18:10.120
<v Speaker 3>Well?

0:18:10.240 --> 0:18:11.880
<v Speaker 1>Why do we sort of back up?

0:18:11.920 --> 0:18:12.080
<v Speaker 3>Then?

0:18:12.119 --> 0:18:14.800
<v Speaker 1>And I again kind of steal Tracy's question because I

0:18:14.840 --> 0:18:17.399
<v Speaker 1>had the same thought, what is it about government buying

0:18:18.040 --> 0:18:23.800
<v Speaker 1>that creates these massive waterfold RFPs in which you can't

0:18:23.880 --> 0:18:27.719
<v Speaker 1>you know, in which a like create these constraints that

0:18:27.760 --> 0:18:31.200
<v Speaker 1>the software developer would not necessarily have with a private buyer,

0:18:31.640 --> 0:18:35.879
<v Speaker 1>and b puts government agencies in a position where they

0:18:35.920 --> 0:18:37.359
<v Speaker 1>will say to you, of course, we know it's going

0:18:37.440 --> 0:18:40.840
<v Speaker 1>to fail, which again presumably that actually probably does happen

0:18:40.880 --> 0:18:43.320
<v Speaker 1>in the private sector too. Or but what is what

0:18:43.359 --> 0:18:45.800
<v Speaker 1>are the to some extent, but what are the dynamics

0:18:45.920 --> 0:18:48.560
<v Speaker 1>that make these things like so hard to change and

0:18:48.640 --> 0:18:52.040
<v Speaker 1>so hard to you know, rip, you know, come up

0:18:52.080 --> 0:18:52.760
<v Speaker 1>with a new system.

0:18:54.280 --> 0:18:56.320
<v Speaker 4>Well, this is a good Dave question, but I'm gonna start.

0:18:57.600 --> 0:18:59.960
<v Speaker 4>You know, I really spend a lot of time reflect

0:19:00.200 --> 0:19:02.200
<v Speaker 4>on this, you know, I had. I've had sort of

0:19:02.240 --> 0:19:04.080
<v Speaker 4>three years since I stepped down from Code for America.

0:19:04.119 --> 0:19:05.720
<v Speaker 4>I just like sat in a room and well, I

0:19:05.760 --> 0:19:08.760
<v Speaker 4>interviewed people and thought about it, and I thought about

0:19:08.760 --> 0:19:12.080
<v Speaker 4>my own experiences, and I think that there's a deep

0:19:12.119 --> 0:19:16.040
<v Speaker 4>seated culture in government where the policy people are the

0:19:16.080 --> 0:19:20.840
<v Speaker 4>important people. They do the important stuff, and technology digital

0:19:21.000 --> 0:19:23.760
<v Speaker 4>is just part of implementation, which is at the not

0:19:23.800 --> 0:19:27.520
<v Speaker 4>just the bottom of like a software development waterfall, it's

0:19:27.520 --> 0:19:31.760
<v Speaker 4>the bottom of a big, rigid hierarchy and which information

0:19:31.920 --> 0:19:34.280
<v Speaker 4>and power and insights only flows from the top to

0:19:34.320 --> 0:19:38.119
<v Speaker 4>the bottom. And so it's problematic in part because the

0:19:38.160 --> 0:19:41.800
<v Speaker 4>people who are doing the tech are really just sort

0:19:41.840 --> 0:19:46.560
<v Speaker 4>of downstream of everything else and the power and ability

0:19:46.640 --> 0:19:49.480
<v Speaker 4>and willingness to step up and say, hey, like we

0:19:49.680 --> 0:19:53.560
<v Speaker 4>probably shouldn't do those sixy seven hundred requirements, we should

0:19:53.560 --> 0:19:57.440
<v Speaker 4>probably focus on these two hundred. Get that out the door,

0:19:57.480 --> 0:19:59.960
<v Speaker 4>and then you know, adage cases as later like that's

0:20:00.000 --> 0:20:03.360
<v Speaker 4>it's just not there's no permission really to say that,

0:20:03.480 --> 0:20:07.439
<v Speaker 4>and until we you compare that with say, we'll call

0:20:07.440 --> 0:20:09.879
<v Speaker 4>it metaphysical silicon valleys. That I don't mean actually like the

0:20:09.920 --> 0:20:13.680
<v Speaker 4>silicon valley, Like the people I write code started the companies,

0:20:13.680 --> 0:20:16.879
<v Speaker 4>they're in power, they're at the top, Like compliance like

0:20:17.400 --> 0:20:23.160
<v Speaker 4>is below them. In sort of the DC government hierarchy,

0:20:23.240 --> 0:20:26.640
<v Speaker 4>compliance is like way above, right, Like policy is way

0:20:26.680 --> 0:20:29.440
<v Speaker 4>above and there's not this sort of build measure learn

0:20:29.560 --> 0:20:33.960
<v Speaker 4>cycle the Dave was referring to, where you know, people

0:20:34.000 --> 0:20:36.720
<v Speaker 4>are learning from each other as they're doing the work

0:20:36.760 --> 0:20:40.040
<v Speaker 4>and that technology. You know, certainly the policy has an

0:20:40.040 --> 0:20:42.440
<v Speaker 4>impact on the technology, but the building of the technology

0:20:42.480 --> 0:20:45.400
<v Speaker 4>needs to have an impact on the policy, like that

0:20:45.680 --> 0:20:50.680
<v Speaker 4>fundamental culture is something that needs to be like looked

0:20:50.680 --> 0:20:54.120
<v Speaker 4>at and called out, and where it's not happening, where

0:20:54.160 --> 0:21:01.639
<v Speaker 4>there's actual conversation, you know, dialogue beats right. Where that's happening,

0:21:01.680 --> 0:21:04.280
<v Speaker 4>we should be lifting that up and saying this is

0:21:04.320 --> 0:21:07.520
<v Speaker 4>possible within government because that's like the foundation I think

0:21:07.520 --> 0:21:12.119
<v Speaker 4>of getting away from these mega projects that fail. But

0:21:12.200 --> 0:21:14.040
<v Speaker 4>Dave you you are you're going to disagree with me

0:21:14.040 --> 0:21:14.919
<v Speaker 4>on this, and I'm gonna love it.

0:21:15.000 --> 0:21:17.119
<v Speaker 3>I think that's I actually do think that's right. I

0:21:17.160 --> 0:21:21.480
<v Speaker 3>think a big part of it flows downstream from budgeting,

0:21:22.359 --> 0:21:26.119
<v Speaker 3>meaning a lot of this the way that and again

0:21:26.320 --> 0:21:28.920
<v Speaker 3>everything that I think Eyegen sate like, there's some degree

0:21:28.960 --> 0:21:31.520
<v Speaker 3>of generalization here, and there's always outliers and there's always exceptions,

0:21:31.520 --> 0:21:33.080
<v Speaker 3>and of course there's also a lot of people try

0:21:33.080 --> 0:21:37.280
<v Speaker 3>and change these things. That said, maybe the dominant mode

0:21:37.520 --> 0:21:41.879
<v Speaker 3>is we're going to budget for a large project. We

0:21:42.000 --> 0:21:44.640
<v Speaker 3>get one time funding to spend to do that project,

0:21:45.280 --> 0:21:47.399
<v Speaker 3>so we got to get everything right. We're only get

0:21:47.440 --> 0:21:50.800
<v Speaker 3>this chance once every so often. We're going to replace everything.

0:21:51.080 --> 0:21:53.080
<v Speaker 3>We're gonna have this new system. It's going to be great.

0:21:53.840 --> 0:21:56.679
<v Speaker 3>And the way that this paradigm breaks down is to

0:21:56.840 --> 0:22:00.679
<v Speaker 3>in what's called a design development phase, where you build

0:22:00.680 --> 0:22:03.200
<v Speaker 3>the system, you design what it should do, you think

0:22:03.240 --> 0:22:05.560
<v Speaker 3>about that. Again, you're kind of making all these decisions

0:22:05.640 --> 0:22:08.359
<v Speaker 3>up front, which I think is very counter to a

0:22:08.400 --> 0:22:10.000
<v Speaker 3>lot of the if you want to call the Silicon

0:22:10.080 --> 0:22:12.520
<v Speaker 3>Valley approach. But then you're building the system, and then

0:22:12.560 --> 0:22:15.879
<v Speaker 3>you enter what's called maintenance and operation, and it's supposed

0:22:15.880 --> 0:22:18.320
<v Speaker 3>to be very very little. You're not going to change much.

0:22:18.320 --> 0:22:22.399
<v Speaker 3>It's just kind of keeping the system going. And the

0:22:22.480 --> 0:22:27.280
<v Speaker 3>problem is modern software you kind of learn the most

0:22:27.480 --> 0:22:30.600
<v Speaker 3>the second you've actually deployed to real users. That is

0:22:30.640 --> 0:22:33.199
<v Speaker 3>the point when you are getting people saying, oh, this

0:22:33.240 --> 0:22:35.560
<v Speaker 3>doesn't work, or you're getting an error, or you're getting hey,

0:22:35.640 --> 0:22:37.119
<v Speaker 3>I tried to put in this address, but I have

0:22:37.160 --> 0:22:38.919
<v Speaker 3>a funky address and it won't let me do that. So,

0:22:38.920 --> 0:22:41.840
<v Speaker 3>all of a sudden, you have all this information, and unfortunately,

0:22:42.280 --> 0:22:44.400
<v Speaker 3>in sort of what gets called a big bang launch

0:22:44.400 --> 0:22:46.200
<v Speaker 3>project where you just put it up and you're done

0:22:46.280 --> 0:22:49.000
<v Speaker 3>and you're not going to touch it again, you now,

0:22:49.200 --> 0:22:52.000
<v Speaker 3>almost immediately on day one, can see potentially all of

0:22:52.040 --> 0:22:54.000
<v Speaker 3>these issues. And a lot of people prepare for this

0:22:54.040 --> 0:22:55.800
<v Speaker 3>in these kinds of projects. They're like, the first month

0:22:55.840 --> 0:22:57.240
<v Speaker 3>is going to be really hard. You all of a

0:22:57.280 --> 0:22:59.400
<v Speaker 3>sudden know all the things that were assumptions you made

0:22:59.400 --> 0:23:02.400
<v Speaker 3>that you got wrong. But the model that you had

0:23:02.400 --> 0:23:04.560
<v Speaker 3>that you were given is, well, we're going to do

0:23:04.600 --> 0:23:09.000
<v Speaker 3>this one time. Now. Contrast that with actually going live

0:23:09.280 --> 0:23:11.680
<v Speaker 3>is the first step, and you're going to have funding

0:23:11.800 --> 0:23:15.119
<v Speaker 3>that's continuous and kind of flat over ten, over twenty,

0:23:15.119 --> 0:23:17.320
<v Speaker 3>over thirty years, and you're going to have those people

0:23:17.359 --> 0:23:20.880
<v Speaker 3>on STEPH and you're going to continuously change the system

0:23:21.160 --> 0:23:23.560
<v Speaker 3>as you learn more about how users interact with with it,

0:23:23.760 --> 0:23:25.680
<v Speaker 3>as you learn more about like what things you didn't

0:23:25.720 --> 0:23:29.359
<v Speaker 3>get right upfront? Did you You made it possible to

0:23:30.480 --> 0:23:34.159
<v Speaker 3>upload a certain type of document, You accept PDFs and

0:23:34.280 --> 0:23:38.359
<v Speaker 3>tiff images Tiff this is a real example, but turns

0:23:38.359 --> 0:23:42.200
<v Speaker 3>out you don't support JPEGs or the I think iPhone

0:23:42.200 --> 0:23:45.879
<v Speaker 3>proprietary default image format. So now all your mobile users

0:23:45.880 --> 0:23:47.840
<v Speaker 3>are saying, hey, we can't upload, or a bunch of

0:23:47.880 --> 0:23:49.520
<v Speaker 3>them were saying, I can't upload documents. It says I

0:23:49.520 --> 0:23:51.959
<v Speaker 3>can't submit this file type right now. A lot of

0:23:52.320 --> 0:23:55.040
<v Speaker 3>you know, well intentioned folks in government get to that

0:23:55.119 --> 0:23:58.359
<v Speaker 3>situation and then they're stuck because they don't have the

0:23:58.880 --> 0:24:00.480
<v Speaker 3>mandate or the funding the same now we're going to

0:24:00.560 --> 0:24:03.000
<v Speaker 3>change that. They're waiting for the next big project to

0:24:03.040 --> 0:24:05.919
<v Speaker 3>replace the system and now support those new things, Whereas instead,

0:24:05.920 --> 0:24:08.400
<v Speaker 3>if you have a model where you're saying we expect

0:24:08.400 --> 0:24:11.640
<v Speaker 3>to continuously change this, it's a very very different thing.

0:24:12.240 --> 0:24:14.200
<v Speaker 3>And that's a little bit that is more how Silicon

0:24:14.280 --> 0:24:17.280
<v Speaker 3>Valley operates. I mean, this maybe is sort of hyperbole,

0:24:17.400 --> 0:24:21.600
<v Speaker 3>but Google didn't build search and then lay off ninety

0:24:21.600 --> 0:24:23.480
<v Speaker 3>percent of their staff and say we're done. This is great,

0:24:23.960 --> 0:24:26.360
<v Speaker 3>you know, like, look we've got this search thing. And

0:24:26.880 --> 0:24:30.439
<v Speaker 3>I think that difference flowing downstream from how tech is

0:24:30.480 --> 0:24:33.160
<v Speaker 3>budgeted for where you're budgeting for a one time project,

0:24:33.240 --> 0:24:36.840
<v Speaker 3>and this comes from legislatures, you know, from Congress. A

0:24:36.880 --> 0:24:40.040
<v Speaker 3>one time project versus we want to have ongoing fundings

0:24:40.119 --> 0:24:42.760
<v Speaker 3>because this is a living, breathing system. Is a huge

0:24:42.760 --> 0:24:46.040
<v Speaker 3>paradigm shift and has an uncomfortable aspect, which is it

0:24:46.119 --> 0:24:49.679
<v Speaker 3>may seem like it's costing more money because it's potentially

0:24:50.080 --> 0:24:53.439
<v Speaker 3>dollars ongoing because it's staff, not one time by but

0:24:53.640 --> 0:24:55.679
<v Speaker 3>I think a lot of the dysfunction that we see

0:24:56.119 --> 0:24:59.320
<v Speaker 3>probably flows down from that root. Cause.

0:24:59.440 --> 0:25:02.440
<v Speaker 2>Okay, so here's here's a big question. If we can

0:25:02.480 --> 0:25:06.080
<v Speaker 2>identify the cause of a lot of this dysfunction and

0:25:06.160 --> 0:25:09.360
<v Speaker 2>if we can trace it back to this waterfall structure,

0:25:09.960 --> 0:25:12.960
<v Speaker 2>the sort of top down mandates or maybe like theorists

0:25:13.040 --> 0:25:15.720
<v Speaker 2>versus practitioners. So you have a politician who has a

0:25:15.720 --> 0:25:18.720
<v Speaker 2>big idea and they want that big bang reveal of

0:25:18.760 --> 0:25:20.919
<v Speaker 2>the software they're going to expand benefits, and it's going

0:25:20.960 --> 0:25:23.439
<v Speaker 2>to come with a really cool website and everyone's going

0:25:23.480 --> 0:25:26.560
<v Speaker 2>to be able to access them easily. And yet it

0:25:26.800 --> 0:25:28.960
<v Speaker 2>leads to these problems that we've been discussing.

0:25:29.720 --> 0:25:30.800
<v Speaker 4>Why do we keep doing it?

0:25:31.880 --> 0:25:36.359
<v Speaker 2>That's a hard question, Well, what is the system that

0:25:36.520 --> 0:25:40.600
<v Speaker 2>is like keeping this way of doing things in place,

0:25:40.920 --> 0:25:45.240
<v Speaker 2>versus maybe you know, after a few decades of developing

0:25:45.320 --> 0:25:48.440
<v Speaker 2>these types of systems, why isn't someone going well, actually,

0:25:48.960 --> 0:25:51.520
<v Speaker 2>the way we've been doing things hasn't really been working well.

0:25:51.520 --> 0:25:53.280
<v Speaker 4>I think a lot of people have been saying that,

0:25:53.359 --> 0:25:55.320
<v Speaker 4>and that's why I say things are changing, like and

0:25:55.359 --> 0:25:58.440
<v Speaker 4>I will offer as evidence COVID tests dot gov. Do

0:25:58.480 --> 0:26:02.400
<v Speaker 4>you remember that one beginning of twenty twenty one, the

0:26:02.440 --> 0:26:05.080
<v Speaker 4>President announced that there would be this site where you

0:26:05.119 --> 0:26:10.200
<v Speaker 4>could request tests, and I think he gave a date

0:26:10.320 --> 0:26:12.720
<v Speaker 4>that was I want to say six weeks out, it

0:26:12.800 --> 0:26:16.280
<v Speaker 4>launched a day early, it launched in multiple languages. It

0:26:16.320 --> 0:26:19.120
<v Speaker 4>took eleven seconds for me to use other people say

0:26:19.600 --> 0:26:23.399
<v Speaker 4>eight seconds, fifteen seconds, right, and you're test showed up

0:26:23.440 --> 0:26:26.640
<v Speaker 4>a couple days later. Like that sounds like it could

0:26:26.640 --> 0:26:29.600
<v Speaker 4>have been really like okay, so it's much simpler than

0:26:29.640 --> 0:26:32.159
<v Speaker 4>say signing up for healthcare ducov or whatever. But like

0:26:32.240 --> 0:26:35.120
<v Speaker 4>they could have made it way more complex. They could

0:26:35.160 --> 0:26:40.520
<v Speaker 4>have said, like, let's gather every possible requirement and try

0:26:40.520 --> 0:26:45.240
<v Speaker 4>to fulfill every possible requirement. But you had internal capacity,

0:26:45.359 --> 0:26:47.840
<v Speaker 4>Like to Dave's point, there are people in house who

0:26:47.920 --> 0:26:51.199
<v Speaker 4>were saying, hey, we have been doing it away that

0:26:51.280 --> 0:26:55.359
<v Speaker 4>hasn't been working. Let's try this way and it's great.

0:26:55.440 --> 0:26:58.160
<v Speaker 4>Like it serves as a fantastic example I think within

0:26:58.240 --> 0:27:00.800
<v Speaker 4>government and outside government. Right, So part of it is

0:27:00.800 --> 0:27:04.120
<v Speaker 4>how people in government think what they believe, and part

0:27:04.119 --> 0:27:06.080
<v Speaker 4>of it is what the public thinks. Right. The public

0:27:06.200 --> 0:27:08.760
<v Speaker 4>can't couldn't imagine something that easy, but we have to

0:27:08.760 --> 0:27:12.720
<v Speaker 4>start imagining it and holding our government accountable to doing that. So,

0:27:12.800 --> 0:27:17.240
<v Speaker 4>I mean, I will say it is changing, but the

0:27:17.280 --> 0:27:20.520
<v Speaker 4>belief that we aren't good at it, that we couldn't

0:27:20.600 --> 0:27:26.040
<v Speaker 4>ever do like a COVID test. Dot of drives politicians, pundits,

0:27:26.440 --> 0:27:29.880
<v Speaker 4>vendors to say government's bad at this, and so we

0:27:29.920 --> 0:27:33.600
<v Speaker 4>should outsource everything. Now, of course we're going to have vendors,

0:27:34.000 --> 0:27:36.760
<v Speaker 4>and we should have vendors, but there has to be

0:27:36.920 --> 0:27:41.199
<v Speaker 4>enough internal capacity and competency to outsource well and to

0:27:41.280 --> 0:27:44.760
<v Speaker 4>make those decisions like, hey, let's have this thing, only

0:27:44.880 --> 0:27:48.480
<v Speaker 4>ask like name, address and like, you know, not ask

0:27:48.560 --> 0:27:51.480
<v Speaker 4>for their health insurance, you know, numbers and not like

0:27:52.040 --> 0:27:55.800
<v Speaker 4>do different numbers of tests per household. Like that's an

0:27:55.840 --> 0:28:02.200
<v Speaker 4>internal capacity question that is really important. And we don't

0:28:02.960 --> 0:28:07.840
<v Speaker 4>have enough internal capacity for these kinds of decisions in

0:28:07.920 --> 0:28:10.240
<v Speaker 4>government because we believe we're not good at it, and

0:28:10.240 --> 0:28:11.840
<v Speaker 4>we believe the only people who are good at out

0:28:11.840 --> 0:28:15.080
<v Speaker 4>of these outside companies. Plenty of evidence that that's totally wrong.

0:28:32.119 --> 0:28:34.359
<v Speaker 1>I want to go back to internal capacity in a minute,

0:28:34.400 --> 0:28:38.240
<v Speaker 1>but since you mentioned the COVID test website, it's sort

0:28:38.240 --> 0:28:40.160
<v Speaker 1>of a good chance to pivot a little bit here.

0:28:40.600 --> 0:28:43.800
<v Speaker 1>You know, COVID specifically seem to open people's minds a

0:28:43.840 --> 0:28:45.680
<v Speaker 1>little bit about how fast we can work and how

0:28:45.720 --> 0:28:49.360
<v Speaker 1>much money we can spend, and obviously a historically fast

0:28:49.400 --> 0:28:53.080
<v Speaker 1>pace of vaccine development that we've never seen prior to that,

0:28:53.480 --> 0:28:55.680
<v Speaker 1>and so it sort of had this brief moment where

0:28:55.680 --> 0:28:57.360
<v Speaker 1>it sort of opened people's minds. And I'm not sure

0:28:57.360 --> 0:28:59.920
<v Speaker 1>if it's closing again, but you know, since both of

0:29:00.120 --> 0:29:05.440
<v Speaker 1>you worked on the California unemployment insurance system and generally,

0:29:05.480 --> 0:29:07.720
<v Speaker 1>like I think almost every state on some level was

0:29:07.720 --> 0:29:11.320
<v Speaker 1>seen by some tobaccle Like what was I guess the

0:29:11.360 --> 0:29:15.640
<v Speaker 1>biggest eye opening thing that going into that environment taught

0:29:15.680 --> 0:29:18.160
<v Speaker 1>you or about the system. And then what was like,

0:29:18.280 --> 0:29:20.720
<v Speaker 1>you know, what's the key sort of takeaway of like, Okay,

0:29:20.720 --> 0:29:23.080
<v Speaker 1>here's a thing that we can learn from this that

0:29:23.120 --> 0:29:26.960
<v Speaker 1>then whether it's UI and other states are more broadly, okay,

0:29:26.960 --> 0:29:29.160
<v Speaker 1>this is something that you learned that then can like

0:29:29.360 --> 0:29:30.560
<v Speaker 1>have extensible lessons.

0:29:31.160 --> 0:29:35.040
<v Speaker 4>So I think you're right. COVID had sort of twin

0:29:35.640 --> 0:29:38.640
<v Speaker 4>impacts on our thinking. One was, holy cow, we can

0:29:38.680 --> 0:29:41.440
<v Speaker 4>actually do stuff, and the other one was, holy cow,

0:29:41.560 --> 0:29:44.880
<v Speaker 4>we're really screwed. Right, Yeah, you're way to put it,

0:29:45.840 --> 0:29:48.680
<v Speaker 4>And there's evidence of both, which is which is really fair.

0:29:49.400 --> 0:29:51.680
<v Speaker 4>I mean I think for me, you know, i'd been

0:29:51.720 --> 0:29:54.840
<v Speaker 4>doing this for you know, ten years when I got

0:29:54.840 --> 0:30:00.360
<v Speaker 4>pulled into the the unemployment insurance I'll call it a rescue.

0:30:00.600 --> 0:30:05.200
<v Speaker 4>We were clearing the backlot, and I felt like, nothing

0:30:05.280 --> 0:30:08.360
<v Speaker 4>here can scare me. I have seen the VA, I've

0:30:08.400 --> 0:30:13.200
<v Speaker 4>seen the operating of you know, some pretty janky, you know,

0:30:14.120 --> 0:30:18.800
<v Speaker 4>government agencies. But I actually really did learn a lot

0:30:18.840 --> 0:30:22.440
<v Speaker 4>and was kind of shocked at how the unemployment insurance

0:30:22.440 --> 0:30:25.880
<v Speaker 4>system worked. And the metaphor that came to mind for

0:30:25.960 --> 0:30:27.959
<v Speaker 4>me was like everyone kept saying, well, it's a system,

0:30:28.040 --> 0:30:31.080
<v Speaker 4>obviously you can just fix it, and it isn't a system.

0:30:31.400 --> 0:30:33.400
<v Speaker 4>And I think that's true of other things we think

0:30:33.440 --> 0:30:37.440
<v Speaker 4>of the systems too, Like it is archaeological layers of

0:30:37.520 --> 0:30:43.240
<v Speaker 4>technology that sort of map to archaeological layers of policy

0:30:44.000 --> 0:30:50.320
<v Speaker 4>that didn't ever get designed. It just accrued, Like so

0:30:50.400 --> 0:30:53.200
<v Speaker 4>nobody ever goes back and says, okay, how is this

0:30:53.280 --> 0:30:56.040
<v Speaker 4>going to work with this? There's such little appetite for

0:30:56.080 --> 0:30:59.200
<v Speaker 4>anything that's like backward looking that would actually update and

0:30:59.240 --> 0:31:03.760
<v Speaker 4>make these layers work together well, that it just isn't

0:31:03.840 --> 0:31:06.360
<v Speaker 4>what people think it is. When you go look at it,

0:31:06.480 --> 0:31:09.480
<v Speaker 4>you're just like, WHOA, of course you can't do these things,

0:31:10.200 --> 0:31:12.760
<v Speaker 4>like this thing isn't connected in any meaningful way to

0:31:12.840 --> 0:31:15.160
<v Speaker 4>this thing, and like this was built in a certain

0:31:15.200 --> 0:31:18.719
<v Speaker 4>era for a certain purpose, and we just glombed stuff

0:31:18.760 --> 0:31:22.800
<v Speaker 4>onto it when we needed to say, add internet access,

0:31:22.880 --> 0:31:26.719
<v Speaker 4>like let people apply online. I mean, the biggest thing was,

0:31:26.760 --> 0:31:29.440
<v Speaker 4>like I think we haven't grappled with the fact that

0:31:29.880 --> 0:31:32.400
<v Speaker 4>when these systems were quote unquote designed, I actually was

0:31:32.520 --> 0:31:34.320
<v Speaker 4>to say when they were started, because they really have

0:31:34.520 --> 0:31:37.520
<v Speaker 4>never been designed, and a lot of them and there's

0:31:37.600 --> 0:31:41.680
<v Speaker 4>huge advantage to actually doing some design. But you went

0:31:41.720 --> 0:31:44.040
<v Speaker 4>into an office and showed who you were, like you

0:31:44.560 --> 0:31:48.760
<v Speaker 4>identified you, you validated your identity by going in, and

0:31:48.800 --> 0:31:51.280
<v Speaker 4>then we moved online and we never figured out how

0:31:51.280 --> 0:31:55.320
<v Speaker 4>to validate people's identity. And there were a number of

0:31:55.360 --> 0:31:59.280
<v Speaker 4>problems in California with unemployment assurance. The fact that you

0:31:59.280 --> 0:32:01.240
<v Speaker 4>know it took twenty five years to learn how to

0:32:01.280 --> 0:32:05.360
<v Speaker 4>do it meant that the any claim that couldn't be

0:32:05.440 --> 0:32:11.400
<v Speaker 4>processed automatically was a huge, huge bottleneck. And you couldn't

0:32:11.400 --> 0:32:13.640
<v Speaker 4>add claims process to do that, Like you would have

0:32:13.680 --> 0:32:16.440
<v Speaker 4>to go back in time twenty five years, start new

0:32:16.480 --> 0:32:19.200
<v Speaker 4>claims browsers and have them ready to come online during

0:32:19.400 --> 0:32:23.360
<v Speaker 4>a downturn, right, Like that's never going to scale. But

0:32:23.440 --> 0:32:27.280
<v Speaker 4>the other big problem was that that you only went

0:32:27.400 --> 0:32:32.640
<v Speaker 4>through the automatic sort of pipeline if they felt like

0:32:32.760 --> 0:32:35.440
<v Speaker 4>they could verify your identity, but they weren't really doing

0:32:35.520 --> 0:32:42.400
<v Speaker 4>any meaningful identity verification. And we, unlike most other industrialized countries,

0:32:43.040 --> 0:32:46.000
<v Speaker 4>don't have a national system for that, Like we don't

0:32:46.000 --> 0:32:50.280
<v Speaker 4>have a national technology platform for that. But also like

0:32:50.360 --> 0:32:53.120
<v Speaker 4>again back to this all derives from like the legal

0:32:53.120 --> 0:32:57.640
<v Speaker 4>and policy framework, but actually have a reasonable policy framework

0:32:58.080 --> 0:33:00.720
<v Speaker 4>for identifying people and knowing who they are when we

0:33:00.720 --> 0:33:04.680
<v Speaker 4>start to do a transaction. And that is causing problems everywhere,

0:33:04.840 --> 0:33:07.640
<v Speaker 4>and there is interesting conversations about how to fix it.

0:33:07.640 --> 0:33:11.640
<v Speaker 4>There's some stuff out there, but it's it's really contested,

0:33:12.240 --> 0:33:15.360
<v Speaker 4>and it's not just a problem for the pandemic. It's

0:33:15.360 --> 0:33:16.680
<v Speaker 4>going to continue to be a problem.

0:33:16.800 --> 0:33:19.920
<v Speaker 3>Yeah, I mean, I'll just add and my vantage point

0:33:20.120 --> 0:33:22.360
<v Speaker 3>was was somewhat different, right, I was coming at it

0:33:22.360 --> 0:33:28.520
<v Speaker 3>from a different angle. I think the major thing that

0:33:28.600 --> 0:33:32.600
<v Speaker 3>I took away, especially with so much focus on technology

0:33:32.640 --> 0:33:35.000
<v Speaker 3>and so much focus on COBAL mainframes, and that was,

0:33:35.040 --> 0:33:35.280
<v Speaker 3>you know.

0:33:35.240 --> 0:33:39.360
<v Speaker 2>That was set we need to do a Cobyl like

0:33:39.480 --> 0:33:42.240
<v Speaker 2>drinking take a shot time.

0:33:42.400 --> 0:33:44.360
<v Speaker 4>How long it took us to get to that.

0:33:44.240 --> 0:33:47.360
<v Speaker 3>Word oh, I can't believe, you know. And the best

0:33:47.360 --> 0:33:49.000
<v Speaker 3>part is that. I mean my next point was that's

0:33:49.040 --> 0:33:52.120
<v Speaker 3>misplaced causation. And yet I am the one who brought

0:33:52.160 --> 0:33:57.479
<v Speaker 3>up cobal. Yeah, I mean I think I think we

0:33:58.320 --> 0:34:00.800
<v Speaker 3>that is a you know, for so much focus on Oh,

0:34:00.840 --> 0:34:04.160
<v Speaker 3>these tech systems are falling over they're not good. I

0:34:04.200 --> 0:34:08.080
<v Speaker 3>think there's the number one lesson I learned was that

0:34:08.920 --> 0:34:10.359
<v Speaker 3>and I think this is true, but we don't tend

0:34:10.400 --> 0:34:12.719
<v Speaker 3>to think about it, which is the technology systems are

0:34:12.719 --> 0:34:15.960
<v Speaker 3>part of a larger sociotechnical system. What I mean by

0:34:15.960 --> 0:34:19.080
<v Speaker 3>that is, as Jen mentioned, you have staff who can

0:34:19.080 --> 0:34:21.480
<v Speaker 3>do stuff manually, and you have a system that might

0:34:21.480 --> 0:34:24.080
<v Speaker 3>be able to do stuff in an automated way. One

0:34:24.120 --> 0:34:27.480
<v Speaker 3>of the problems with programs that are so complex is

0:34:27.480 --> 0:34:29.759
<v Speaker 3>if you have a massive spike of volume in your

0:34:29.760 --> 0:34:32.960
<v Speaker 3>technology system isn't set up to just cover all those

0:34:33.000 --> 0:34:36.000
<v Speaker 3>cases in an automated way. You try to throw bodies

0:34:36.000 --> 0:34:37.640
<v Speaker 3>at it, and you try to hire people. But if

0:34:37.680 --> 0:34:39.720
<v Speaker 3>it takes a year and a half to train someone

0:34:40.080 --> 0:34:42.960
<v Speaker 3>and like train someone to a relatively basic level, with

0:34:43.040 --> 0:34:45.560
<v Speaker 3>some of the complexity of these programs, you're not able

0:34:45.600 --> 0:34:51.640
<v Speaker 3>to hire fast enough and Unfortunately, you know, this is why.

0:34:54.360 --> 0:34:57.840
<v Speaker 4>We were hiring really fast. We heard five thousand people

0:34:58.320 --> 0:35:02.080
<v Speaker 4>in California to help process these claims, but because they

0:35:02.080 --> 0:35:05.520
<v Speaker 4>couldn't do anything, they were taking up the time of

0:35:05.600 --> 0:35:10.120
<v Speaker 4>the experienced claims processors. And we calculated that every single

0:35:10.239 --> 0:35:15.759
<v Speaker 4>new hire the state made slowed processing down. And one

0:35:15.760 --> 0:35:17.839
<v Speaker 4>of the big things we did was actually just get

0:35:18.000 --> 0:35:22.239
<v Speaker 4>them to reassign staff away. I mean, some of it

0:35:22.280 --> 0:35:24.279
<v Speaker 4>was just reassigned staff to other things, Like nobody was

0:35:24.320 --> 0:35:28.160
<v Speaker 4>opening the mail, which was pretty critical. And yet opening

0:35:28.200 --> 0:35:30.480
<v Speaker 4>the mail is something you can do if you just

0:35:30.840 --> 0:35:33.960
<v Speaker 4>joined and have no background, but like, yeah, it was

0:35:34.280 --> 0:35:36.799
<v Speaker 4>pretty franny. So sorry to interruptive. It's just like we

0:35:36.840 --> 0:35:39.680
<v Speaker 4>were hiring. It's just that the hiring was having a

0:35:39.719 --> 0:35:40.520
<v Speaker 4>perverse effect.

0:35:40.960 --> 0:35:44.040
<v Speaker 3>Well, and that's I guess that's my point is that's

0:35:44.080 --> 0:35:48.600
<v Speaker 3>exactly right. And I guess the number one lesson I

0:35:48.760 --> 0:35:52.440
<v Speaker 3>learned was if you look at the history of a program,

0:35:52.840 --> 0:35:56.040
<v Speaker 3>like un employment trends, but specifically UI, you have this

0:35:56.440 --> 0:35:59.080
<v Speaker 3>thing where once every ten years or so, there's a

0:35:59.080 --> 0:36:01.480
<v Speaker 3>major recession, or in this case, there's a pandemic, which

0:36:01.520 --> 0:36:03.840
<v Speaker 3>was like a ten x recession, right because it was

0:36:03.880 --> 0:36:07.040
<v Speaker 3>overnight as opposed to graduate was overnight. So much of

0:36:07.040 --> 0:36:09.560
<v Speaker 3>the economy, so many of the working force, labor force,

0:36:09.760 --> 0:36:12.600
<v Speaker 3>were just out of work immediately, so all those claims

0:36:12.600 --> 0:36:15.120
<v Speaker 3>came in simultaneously. But we have this cycle of every

0:36:15.200 --> 0:36:17.000
<v Speaker 3>ten years or so way of recession or I mean,

0:36:17.040 --> 0:36:18.960
<v Speaker 3>I don't know, I'm non economists, I can tell you

0:36:19.000 --> 0:36:21.960
<v Speaker 3>exactly what that is, but it happens in a recurring way,

0:36:22.120 --> 0:36:25.560
<v Speaker 3>and then everyone gets frustrated with how difficult it is

0:36:25.600 --> 0:36:28.839
<v Speaker 3>to get unemployment insurance because they're overloaded with volume. And

0:36:28.880 --> 0:36:32.600
<v Speaker 3>then the claims volume goes down, recession ends, and then

0:36:33.480 --> 0:36:35.720
<v Speaker 3>there's a lot there. Honestly, isn't a lot of focus

0:36:35.760 --> 0:36:38.960
<v Speaker 3>on Okay, we're gonna sort of put more money into

0:36:39.000 --> 0:36:41.040
<v Speaker 3>the UI system so that we're ready for next time.

0:36:41.560 --> 0:36:45.520
<v Speaker 3>But unfortunately, you can't have resilient systems if you optimize

0:36:45.520 --> 0:36:48.680
<v Speaker 3>for pure, pure, pure efficiency, like in the down years,

0:36:48.719 --> 0:36:50.440
<v Speaker 3>in the years when you have very very low volume,

0:36:51.200 --> 0:36:53.680
<v Speaker 3>if you are just going to cut staff down to

0:36:53.880 --> 0:36:56.759
<v Speaker 3>very very bare bone skeleton crew. It comes back to

0:36:56.800 --> 0:36:58.880
<v Speaker 3>this point of not only are you not able to

0:36:58.880 --> 0:37:01.920
<v Speaker 3>do preparation if you don't have the funding to have staff,

0:37:02.280 --> 0:37:05.640
<v Speaker 3>you can't even make the changes that would be good,

0:37:05.719 --> 0:37:07.960
<v Speaker 3>and you can't when you get the money in a

0:37:08.040 --> 0:37:10.439
<v Speaker 3>new recession hire the people because you can't train them,

0:37:10.640 --> 0:37:12.160
<v Speaker 3>and so we kind of get stuck in the cycle.

0:37:12.239 --> 0:37:14.920
<v Speaker 3>So that was the number one from a systems modeling

0:37:14.960 --> 0:37:16.799
<v Speaker 3>perspective lesson I took away with it.

0:37:16.840 --> 0:37:19.440
<v Speaker 4>From it, I got to add, though, you know, we

0:37:19.560 --> 0:37:23.480
<v Speaker 4>did invest several billion across states during the Great recession

0:37:23.719 --> 0:37:26.839
<v Speaker 4>or after a Great recession, so I think about half

0:37:26.880 --> 0:37:30.520
<v Speaker 4>of the states technically modernized then they did take advantage

0:37:30.520 --> 0:37:34.359
<v Speaker 4>of pretty significant funds. None of those states did any

0:37:34.400 --> 0:37:39.279
<v Speaker 4>better than average in this downturn. So I agree with Dave,

0:37:39.400 --> 0:37:43.080
<v Speaker 4>and I would argue that taking a different approach to

0:37:43.280 --> 0:37:48.719
<v Speaker 4>quote unquote modernization, to policy and process simplification, to you know,

0:37:48.760 --> 0:37:54.440
<v Speaker 4>having goal driven modernization like all of those things, is

0:37:54.560 --> 0:37:59.480
<v Speaker 4>equally important as investing during the downturn. Investing the way

0:37:59.520 --> 0:38:03.000
<v Speaker 4>we have been doing it is not working. There are

0:38:03.040 --> 0:38:05.719
<v Speaker 4>so many different threads to pick out of there. But

0:38:06.160 --> 0:38:07.960
<v Speaker 4>just on that last point, you know, when you were

0:38:08.000 --> 0:38:10.319
<v Speaker 4>talking about how a lot of these government systems were

0:38:10.360 --> 0:38:13.520
<v Speaker 4>not designed to be that way, I kept thinking of

0:38:13.560 --> 0:38:16.640
<v Speaker 4>a lot of the software used by the banks, where

0:38:16.800 --> 0:38:20.080
<v Speaker 4>you know, you think, you think of like JP Morgan's

0:38:20.120 --> 0:38:24.520
<v Speaker 4>software system. It wasn't designed to be JP morgan software system.

0:38:24.520 --> 0:38:28.040
<v Speaker 4>It's the result of many, many acquisitions of other banks,

0:38:28.040 --> 0:38:30.320
<v Speaker 4>and you know, every time they take over this business,

0:38:30.400 --> 0:38:33.759
<v Speaker 4>it gets sort of duct taped onto the rest of

0:38:33.800 --> 0:38:37.880
<v Speaker 4>the system. But I'm curious, is there a moment that

0:38:38.000 --> 0:38:42.160
<v Speaker 4>you have seen in your careers at which someone says,

0:38:42.719 --> 0:38:46.000
<v Speaker 4>this system is just like irredeemable. We're going to have

0:38:46.080 --> 0:38:48.320
<v Speaker 4>to start from scratch, And like, what is the trigger

0:38:48.360 --> 0:38:52.840
<v Speaker 4>point for making that decision versus reaching for the band

0:38:52.840 --> 0:38:55.759
<v Speaker 4>aids and the duct tape and just trying to make

0:38:55.800 --> 0:38:56.239
<v Speaker 4>it work.

0:38:56.960 --> 0:38:59.200
<v Speaker 3>I mean, and I forget if someone else has mentioned this.

0:38:59.600 --> 0:39:02.160
<v Speaker 3>I lit the Patrick McKenzie episode, maybe he mentioned this,

0:39:02.160 --> 0:39:06.479
<v Speaker 3>But in general, a good rule of thumb is never

0:39:06.560 --> 0:39:09.839
<v Speaker 3>rewrite a system from scratch that's working. I mean, other

0:39:09.880 --> 0:39:13.680
<v Speaker 3>people might disagree with that. But the reason I don't

0:39:13.719 --> 0:39:16.200
<v Speaker 3>have a good answer for sort of a moment. Lots

0:39:16.200 --> 0:39:17.839
<v Speaker 3>of people have tried to do that thing where they

0:39:17.880 --> 0:39:20.440
<v Speaker 3>just rebuild it from scratch, and again it tends to

0:39:20.560 --> 0:39:24.520
<v Speaker 3>go poorly because and this is the specific thing, the

0:39:24.600 --> 0:39:28.799
<v Speaker 3>old system encodes so much tacit knowledge and so many

0:39:28.880 --> 0:39:31.560
<v Speaker 3>of the many, many many little details, and some of

0:39:31.600 --> 0:39:34.600
<v Speaker 3>those details might no longer be relevant, but many of

0:39:34.640 --> 0:39:40.760
<v Speaker 3>them might encode some really hard learned truth about the world,

0:39:41.000 --> 0:39:43.799
<v Speaker 3>because at the end of the day, the software is

0:39:44.000 --> 0:39:46.520
<v Speaker 3>just modeling the real world. It's just saying, okay, well

0:39:46.560 --> 0:39:49.840
<v Speaker 3>we have customers. They have addresses. Okay, so maybe you

0:39:49.840 --> 0:39:52.560
<v Speaker 3>have two address fields in your old system and you think, oh, well,

0:39:52.640 --> 0:39:55.080
<v Speaker 3>let's just streamline that down to one. But then you

0:39:55.160 --> 0:39:58.120
<v Speaker 3>launch your new system has one address, and everybody calls

0:39:58.120 --> 0:39:59.759
<v Speaker 3>you up and like, what are you doing? Like, we

0:40:00.080 --> 0:40:02.880
<v Speaker 3>the reason we have two addresses is because our billing

0:40:02.880 --> 0:40:05.520
<v Speaker 3>department is over here and the customer facing address is different.

0:40:05.800 --> 0:40:08.680
<v Speaker 3>So I think the throw it away and start from

0:40:08.719 --> 0:40:13.000
<v Speaker 3>scratch is an impulse everybody has, but it has a

0:40:13.000 --> 0:40:17.000
<v Speaker 3>lot of risk as well, and I think the better

0:40:17.080 --> 0:40:19.680
<v Speaker 3>approach tends to be can you get to a place

0:40:20.080 --> 0:40:25.600
<v Speaker 3>where you can start to incrementally change your systems as

0:40:25.600 --> 0:40:28.359
<v Speaker 3>you go in small ways that are safe, where it

0:40:28.400 --> 0:40:31.000
<v Speaker 3>doesn't take it. You know, it's not like we have

0:40:31.040 --> 0:40:34.759
<v Speaker 3>to test for six weeks any change before making it live.

0:40:35.520 --> 0:40:38.680
<v Speaker 3>We can ship weekly, we can ship daily, we can

0:40:38.680 --> 0:40:42.840
<v Speaker 3>ship multiple times a day. That tends to yield better things.

0:40:43.560 --> 0:40:47.239
<v Speaker 3>There are plenty of projects out there that we're going

0:40:47.280 --> 0:40:50.879
<v Speaker 3>to build this from scratch, from the bottom up. Most

0:40:50.880 --> 0:40:54.640
<v Speaker 3>of the ones I'm aware of don't have great launch stories.

0:40:54.920 --> 0:40:56.400
<v Speaker 3>Maybe JO have disagrees, that's my no.

0:40:56.520 --> 0:40:58.960
<v Speaker 4>They're still in that mode of like get it all

0:40:58.960 --> 0:41:01.600
<v Speaker 4>in once and then we're done, and we have sort

0:41:01.600 --> 0:41:04.600
<v Speaker 4>of maintenance, and so they're pretty bad. But I will

0:41:04.600 --> 0:41:07.480
<v Speaker 4>say two things. One on a practical level, just because

0:41:07.800 --> 0:41:10.239
<v Speaker 4>I was talking with a colleague on the way over

0:41:10.280 --> 0:41:13.239
<v Speaker 4>here about this, there are great examples now of what

0:41:13.320 --> 0:41:16.319
<v Speaker 4>Dave's talking about, like not waiting for this big bang

0:41:16.360 --> 0:41:19.640
<v Speaker 4>project but saying we are just going to be making

0:41:19.640 --> 0:41:22.520
<v Speaker 4>this a little bit better every day. And one of

0:41:22.560 --> 0:41:26.640
<v Speaker 4>them is New Jersey's Department of Labor. Their unemployment insurance

0:41:27.280 --> 0:41:30.960
<v Speaker 4>response during the pandemic was quite good, but they've also

0:41:31.080 --> 0:41:34.200
<v Speaker 4>learned from that and said we're not going to wait

0:41:34.239 --> 0:41:36.719
<v Speaker 4>for some like big system to come that's going to

0:41:36.719 --> 0:41:39.720
<v Speaker 4>fix all of our problems. We have a team now

0:41:40.000 --> 0:41:44.759
<v Speaker 4>that can just incrementally fix things as we go and

0:41:45.560 --> 0:41:49.000
<v Speaker 4>eventually that will involve some bigger investments as they learn X, Y,

0:41:49.040 --> 0:41:52.359
<v Speaker 4>and Z. But they're just not waiting and it's just

0:41:52.400 --> 0:41:56.319
<v Speaker 4>a great, great example, But I will say, Tracy, like

0:41:56.440 --> 0:41:58.920
<v Speaker 4>I think about what you're asking about all the time.

0:42:00.120 --> 0:42:02.480
<v Speaker 4>I don't think this will ever happen, but if it,

0:42:03.480 --> 0:42:05.239
<v Speaker 4>if you do get a chance to sort of zero

0:42:05.320 --> 0:42:08.600
<v Speaker 4>base it and start over, you have to zero base

0:42:08.920 --> 0:42:13.080
<v Speaker 4>the policy, Like you can't. Could you do a new

0:42:13.160 --> 0:42:17.799
<v Speaker 4>unemployment insurance system in California that encodes the twenty five

0:42:17.880 --> 0:42:21.080
<v Speaker 4>years of knowledge that those claims processors have. Well, I

0:42:21.080 --> 0:42:22.960
<v Speaker 4>mean maybe I will help you now. I don't want

0:42:22.960 --> 0:42:27.120
<v Speaker 4>to talk about any but like, but like that's a problem.

0:42:27.360 --> 0:42:29.560
<v Speaker 4>Like you know, I say in the book, there's this

0:42:29.640 --> 0:42:32.560
<v Speaker 4>great team that's working on this Medicare problem, and it's

0:42:32.600 --> 0:42:36.600
<v Speaker 4>like there's nine different definitions of even like what a

0:42:36.680 --> 0:42:40.080
<v Speaker 4>medical group is. Like even the first question doctors are

0:42:40.080 --> 0:42:43.840
<v Speaker 4>supposed to answer is like wildly complex. And the policy

0:42:44.080 --> 0:42:47.160
<v Speaker 4>the delivery team is sort of fighting with the policy team,

0:42:47.200 --> 0:42:50.439
<v Speaker 4>and she's like, I get that it's complex. It has

0:42:50.520 --> 0:42:54.080
<v Speaker 4>to make sense to a person, And I like, fight, fight, fight,

0:42:54.200 --> 0:42:58.000
<v Speaker 4>not over the tech, but out of like over collapsing

0:42:58.040 --> 0:43:00.560
<v Speaker 4>these nine definitions into two. They want I get to one.

0:43:00.600 --> 0:43:02.960
<v Speaker 4>They can't. At least they get to two. And like

0:43:03.160 --> 0:43:05.400
<v Speaker 4>this idea that it has to make sense to a person.

0:43:06.080 --> 0:43:09.080
<v Speaker 4>We've like lost track of that. I mean, of course

0:43:09.080 --> 0:43:12.080
<v Speaker 4>you're frustrated when you apply for unemployment insurance, Like it's

0:43:12.160 --> 0:43:16.440
<v Speaker 4>wildly complex in its policy and the process that accrued

0:43:16.440 --> 0:43:19.560
<v Speaker 4>from those policies. Like if you were going to start over,

0:43:20.080 --> 0:43:22.200
<v Speaker 4>you can't start over with the tech. You would have

0:43:22.280 --> 0:43:25.640
<v Speaker 4>to say, okay, let's figure out let's like look at

0:43:25.680 --> 0:43:30.560
<v Speaker 4>this ninety years of accumulated policy and say what was

0:43:30.600 --> 0:43:34.160
<v Speaker 4>this actually trying to do. Can we kind of you know,

0:43:34.239 --> 0:43:37.239
<v Speaker 4>easier go back and just like rationalize all the changes, right,

0:43:37.239 --> 0:43:40.240
<v Speaker 4>It's not like like I say, like there's not one

0:43:40.480 --> 0:43:45.200
<v Speaker 4>binder of regulations that covers youI there's ninety years of

0:43:45.320 --> 0:43:48.839
<v Speaker 4>memos that change the previous memo that change the previous thing,

0:43:49.120 --> 0:43:52.000
<v Speaker 4>that changed the previous thing. And you could at least

0:43:52.080 --> 0:43:55.520
<v Speaker 4>go back and like condense that and like figure out

0:43:55.560 --> 0:43:58.120
<v Speaker 4>all of the conflicts in it. But it would make actually,

0:43:58.120 --> 0:44:00.239
<v Speaker 4>I think a lot more sense. And I will be clear,

0:44:00.440 --> 0:44:02.759
<v Speaker 4>this is never going to happen if you just said,

0:44:02.880 --> 0:44:08.040
<v Speaker 4>let's design the policy for an unemployment insurance system that

0:44:08.200 --> 0:44:10.960
<v Speaker 4>makes sense in twenty twenty three, and then you could

0:44:10.960 --> 0:44:12.360
<v Speaker 4>build tech for that.

0:44:12.400 --> 0:44:14.560
<v Speaker 1>There's so many different threads and places we could go.

0:44:14.600 --> 0:44:17.720
<v Speaker 1>There's such a fascinating conversation. I just have one more question,

0:44:18.000 --> 0:44:21.000
<v Speaker 1>and it sort of relates to something, Jennifer that you

0:44:21.000 --> 0:44:25.120
<v Speaker 1>said earlier about like in government, the tech people are

0:44:25.160 --> 0:44:29.240
<v Speaker 1>like secondary likely like the policy wonks and people designing

0:44:29.280 --> 0:44:31.640
<v Speaker 1>these things, and that that's a really big difference between

0:44:32.160 --> 0:44:35.279
<v Speaker 1>government of public and private sector. And you know, this

0:44:35.440 --> 0:44:38.600
<v Speaker 1>internal capacity, how much does that hinder And I think

0:44:38.680 --> 0:44:41.239
<v Speaker 1>this is another thing that Patrick McKenzie brought up, but

0:44:41.320 --> 0:44:44.880
<v Speaker 1>like hiring and so and the challenge of like, you know,

0:44:44.960 --> 0:44:47.720
<v Speaker 1>public sector pay skills are not private sector pay skills.

0:44:47.760 --> 0:44:49.600
<v Speaker 1>I have to imagine for a lot of these things

0:44:49.840 --> 0:44:52.360
<v Speaker 1>he talked about a little bit, you know, and you

0:44:52.360 --> 0:44:54.319
<v Speaker 1>know what you've worked on some of these like sort

0:44:54.320 --> 0:44:57.080
<v Speaker 1>of volunteer things that sort of bridge the private and

0:44:57.320 --> 0:44:59.480
<v Speaker 1>public sector. But can you talk a little bit about

0:44:59.560 --> 0:45:03.040
<v Speaker 1>that can strength of like even if like just the

0:45:03.160 --> 0:45:06.680
<v Speaker 1>challenge of in those in those roles getting like experienced

0:45:06.680 --> 0:45:07.360
<v Speaker 1>talented people.

0:45:08.080 --> 0:45:10.600
<v Speaker 4>I'm glad you brought that up because it's a little

0:45:10.600 --> 0:45:13.200
<v Speaker 4>bit of a soapbox I like to get on because

0:45:13.360 --> 0:45:16.400
<v Speaker 4>we have you know, like politicians, like the leaders who

0:45:16.440 --> 0:45:20.120
<v Speaker 4>are so frustrated with delivery and technology, and like they're

0:45:20.480 --> 0:45:24.160
<v Speaker 4>generally they're well intentioned, but the sort of things they

0:45:24.200 --> 0:45:27.600
<v Speaker 4>push on are kind of unhelpful. You know, they kind

0:45:27.600 --> 0:45:31.000
<v Speaker 4>of increase the risk aversion of the bureaucracy by yelling

0:45:31.080 --> 0:45:35.080
<v Speaker 4>them about things and not recognizing the constraints on the bureaucracy.

0:45:35.120 --> 0:45:37.799
<v Speaker 4>I call this the accountability trap. But there are things

0:45:37.880 --> 0:45:41.800
<v Speaker 4>they actually could do, and fixing the civil service and

0:45:41.840 --> 0:45:46.279
<v Speaker 4>civil service hiring rules is really really important. I do

0:45:46.600 --> 0:45:49.640
<v Speaker 4>know what Patrick McKenzie said on your show about pay

0:45:49.680 --> 0:45:53.560
<v Speaker 4>scales is not totally I don't totally agree with that.

0:45:53.960 --> 0:45:57.640
<v Speaker 4>The biggest problem is not the pay, it's the time

0:45:57.680 --> 0:46:01.200
<v Speaker 4>to hire. So if it takes, as is really common

0:46:01.280 --> 0:46:05.040
<v Speaker 4>right now in federal government, nine months to make an offer,

0:46:05.400 --> 0:46:08.040
<v Speaker 4>of course you lose everybody you've got. Actually, you know,

0:46:08.120 --> 0:46:10.880
<v Speaker 4>when when I started this work, there was not like

0:46:11.080 --> 0:46:14.560
<v Speaker 4>a line of people out the door, one with great

0:46:14.640 --> 0:46:17.480
<v Speaker 4>tech and design skills wanting to work for government. Now

0:46:17.880 --> 0:46:20.560
<v Speaker 4>there actually is, and we can't get them in the door.

0:46:20.880 --> 0:46:23.479
<v Speaker 4>I mean, I know people who do wait nine months

0:46:23.520 --> 0:46:27.319
<v Speaker 4>for that offer because they really want to help and

0:46:27.360 --> 0:46:30.919
<v Speaker 4>the pay is fine, Like it's not fantastic. I think

0:46:30.920 --> 0:46:33.680
<v Speaker 4>he was comparing to like Google whatever, but like you know,

0:46:34.400 --> 0:46:37.400
<v Speaker 4>compared to a startup where yes, you have the hope

0:46:37.400 --> 0:46:40.000
<v Speaker 4>of getting an exit, but you're probably not gonna make

0:46:40.160 --> 0:46:42.879
<v Speaker 4>like five hundred thousand dollars right as a programmer, you're

0:46:42.880 --> 0:46:44.919
<v Speaker 4>gonna make what half of that. I mean, we don't

0:46:44.960 --> 0:46:46.960
<v Speaker 4>have to get into like exact numbers, but like the

0:46:47.000 --> 0:46:51.240
<v Speaker 4>pay is in the right ballpark for like mid level folks.

0:46:51.360 --> 0:46:54.080
<v Speaker 4>It's not going to pay at the executive level. But

0:46:54.160 --> 0:46:56.439
<v Speaker 4>as he mentioned, like at the executive level, if you've

0:46:56.440 --> 0:46:58.600
<v Speaker 4>been at Google for twenty years, pay is not your

0:46:58.640 --> 0:47:00.879
<v Speaker 4>biggest concern. I do want to push back a little

0:47:00.880 --> 0:47:02.920
<v Speaker 4>bit on the notion that that is he is in government.

0:47:02.960 --> 0:47:05.439
<v Speaker 4>There are some of those people, but there are far

0:47:05.520 --> 0:47:08.560
<v Speaker 4>more people who just have good tech and design skills

0:47:08.920 --> 0:47:12.280
<v Speaker 4>who just want to make a difference more than anything else.

0:47:12.360 --> 0:47:16.120
<v Speaker 4>And we should be working on all of the constraints

0:47:16.360 --> 0:47:20.839
<v Speaker 4>that are keeping that hiring process at nine months more

0:47:20.920 --> 0:47:24.200
<v Speaker 4>than we're than we're like calling up the people who

0:47:25.440 --> 0:47:28.200
<v Speaker 4>manage the Department of Labor in a state and yelling

0:47:28.239 --> 0:47:30.840
<v Speaker 4>at them because they had these failures, like fix the

0:47:31.040 --> 0:47:34.759
<v Speaker 4>environment in which they're working. Give them that capacity, then

0:47:34.840 --> 0:47:37.320
<v Speaker 4>you can hold them accountable, but they can't hire people

0:47:37.360 --> 0:47:37.799
<v Speaker 4>right now.

0:47:38.760 --> 0:47:40.880
<v Speaker 3>You also you need to be able to hire people,

0:47:40.920 --> 0:47:44.080
<v Speaker 3>and you need them to be able to use their

0:47:44.280 --> 0:47:50.080
<v Speaker 3>judgment to make higher order decisions than just do we

0:47:50.280 --> 0:47:53.200
<v Speaker 3>enter the phone number with the keyboard or by clicking

0:47:53.280 --> 0:47:56.080
<v Speaker 3>on a virtual pad of numbers. Like it is also

0:47:56.120 --> 0:47:58.759
<v Speaker 3>the case that some of these systems are the way

0:47:58.760 --> 0:48:04.680
<v Speaker 3>they are because it's just jen this might be your phrase,

0:48:04.920 --> 0:48:08.320
<v Speaker 3>it's like policy vomit. It's just the rules put onto

0:48:08.760 --> 0:48:11.439
<v Speaker 3>a screen, just like all the rules in statute put

0:48:11.480 --> 0:48:14.439
<v Speaker 3>onto a screen. And so I think the thing I'd

0:48:14.480 --> 0:48:18.520
<v Speaker 3>like to add is the whole point of product management

0:48:19.560 --> 0:48:26.360
<v Speaker 3>is to arbitrate the trade offs between user experience, compliance goals,

0:48:26.760 --> 0:48:30.279
<v Speaker 3>technical cost of change, and technical trade offs. And if

0:48:30.280 --> 0:48:33.520
<v Speaker 3>you don't give those people if you hire them, but

0:48:33.920 --> 0:48:36.880
<v Speaker 3>they don't have the ability to say, well, maybe we

0:48:36.960 --> 0:48:40.160
<v Speaker 3>need to simplify this to make it make sense to

0:48:40.400 --> 0:48:44.400
<v Speaker 3>a person. If instead the only response they get is

0:48:44.640 --> 0:48:47.399
<v Speaker 3>what we need to put on the page exactly what's

0:48:47.440 --> 0:48:52.279
<v Speaker 3>in statute then, or whatever other compliance goals there are, like, oh,

0:48:52.320 --> 0:48:54.920
<v Speaker 3>we need to strictly follow this process. It's procedurally what

0:48:54.960 --> 0:48:57.360
<v Speaker 3>it is. If those are counter to the outcome. You

0:48:57.440 --> 0:49:00.200
<v Speaker 3>need a change in government where outcomes start to be

0:49:00.280 --> 0:49:04.880
<v Speaker 3>it outcomes in this area around technology start to sort

0:49:04.880 --> 0:49:08.880
<v Speaker 3>of have a higher importance than the strict process and

0:49:08.920 --> 0:49:11.080
<v Speaker 3>the strict procedures. And I do think that that's a

0:49:11.120 --> 0:49:16.520
<v Speaker 3>necessary corollary to bringing smart capable people in because if

0:49:16.560 --> 0:49:23.000
<v Speaker 3>you bring in a superb product manager and they're just

0:49:23.280 --> 0:49:25.680
<v Speaker 3>they have no ability to sort of bring these trade

0:49:25.719 --> 0:49:28.719
<v Speaker 3>offs up and make those calls on the ground, you're

0:49:28.760 --> 0:49:30.359
<v Speaker 3>going to get the same status quo.

0:49:30.440 --> 0:49:30.800
<v Speaker 4>I think.

0:49:31.360 --> 0:49:34.799
<v Speaker 2>So I can't resist asking this question. I take all

0:49:34.800 --> 0:49:36.560
<v Speaker 2>the points about, like, here's what we need to do

0:49:36.680 --> 0:49:38.839
<v Speaker 2>in order for this to change, But can I just ask,

0:49:39.200 --> 0:49:43.839
<v Speaker 2>because both of you have that practical experience of doing this,

0:49:44.160 --> 0:49:47.359
<v Speaker 2>what was the craziest thing that you saw during your

0:49:47.520 --> 0:49:51.440
<v Speaker 2>respective tenures, Like, what was something that just really surprised

0:49:51.520 --> 0:49:54.160
<v Speaker 2>you or shocked you. I'm back to the banging your

0:49:54.200 --> 0:49:56.640
<v Speaker 2>head on the table a portion of this conversation.

0:49:57.440 --> 0:49:59.640
<v Speaker 4>I mean, I guess the thing that comes to mind

0:50:00.719 --> 0:50:05.200
<v Speaker 4>just because it really made an impression on me, and

0:50:05.239 --> 0:50:08.040
<v Speaker 4>it's it's a negative story that I want to sort

0:50:08.080 --> 0:50:11.960
<v Speaker 4>of balance out with something positive. But I was working

0:50:12.000 --> 0:50:16.520
<v Speaker 4>in the White House and had decided to do what

0:50:16.560 --> 0:50:20.200
<v Speaker 4>we convinced the Veterans Administration to do what we call

0:50:20.280 --> 0:50:25.080
<v Speaker 4>discovery sprint on the Veterans Benefit management system. And it

0:50:25.200 --> 0:50:29.319
<v Speaker 4>was going really slowly. There was like huge latency, and

0:50:29.360 --> 0:50:31.520
<v Speaker 4>I got these two folks in the help out for

0:50:31.680 --> 0:50:33.600
<v Speaker 4>just a couple of weeks and just you know, do

0:50:33.640 --> 0:50:35.759
<v Speaker 4>an analysis of what was wrong before they go, like,

0:50:35.880 --> 0:50:37.759
<v Speaker 4>you know, trying to figure you can't solve the problem

0:50:37.840 --> 0:50:39.919
<v Speaker 4>till you know what it is. So like the first

0:50:40.000 --> 0:50:42.719
<v Speaker 4>thing was we show up and the guy's like, oh,

0:50:42.760 --> 0:50:44.920
<v Speaker 4>I'm glad they sent the White House to you know,

0:50:45.400 --> 0:50:49.240
<v Speaker 4>verify that everything's fine now. And it turns out everything

0:50:49.280 --> 0:50:53.279
<v Speaker 4>was fine because the leader had defined the latency as

0:50:53.400 --> 0:50:58.000
<v Speaker 4>under two minutes. Okay, so you've just defined the problem away.

0:50:58.040 --> 0:51:01.320
<v Speaker 4>So people who are trying to process benefit fit applications,

0:51:01.360 --> 0:51:04.000
<v Speaker 4>like if they clicked and it took one minute and

0:51:04.080 --> 0:51:07.840
<v Speaker 4>fifty nine seconds for the application to respond, it was fine,

0:51:08.440 --> 0:51:12.120
<v Speaker 4>no problem here. Wow. But then I kept talking to

0:51:12.160 --> 0:51:14.799
<v Speaker 4>the guy and I kept asking him questions about the system, like, well,

0:51:14.840 --> 0:51:16.960
<v Speaker 4>why is it designed this way? Why was it designed

0:51:16.960 --> 0:51:19.760
<v Speaker 4>that way? And he said, I've spent my career teaching

0:51:19.840 --> 0:51:22.799
<v Speaker 4>people not to have an opinion on business requirements. The

0:51:22.880 --> 0:51:26.919
<v Speaker 4>IT people should have no opinions there. And I said why,

0:51:27.400 --> 0:51:29.520
<v Speaker 4>He said, well, if they ask us to build a

0:51:29.560 --> 0:51:34.319
<v Speaker 4>concrete boat, will build a concrete boat, because then it's

0:51:34.400 --> 0:51:37.919
<v Speaker 4>not our fault. And I was just like, I felt

0:51:37.960 --> 0:51:41.600
<v Speaker 4>like I'd been gut punched, Like it was so hard

0:51:41.640 --> 0:51:44.080
<v Speaker 4>to hear that, Like there were I think at the time,

0:51:44.200 --> 0:51:49.040
<v Speaker 4>eighteen veterans a day committing suicide, most of whom couldn't

0:51:49.120 --> 0:51:53.160
<v Speaker 4>get their healthcare benefits. And I just could understand how

0:51:53.160 --> 0:51:56.840
<v Speaker 4>someone could take that approach. But honestly, you know, in

0:51:56.880 --> 0:51:58.960
<v Speaker 4>the ten years since then, I've seen a lot of

0:51:59.000 --> 0:52:02.359
<v Speaker 4>reasons why he felt like he could say that, And

0:52:02.560 --> 0:52:03.920
<v Speaker 4>you know, I don't want to blame him, I want

0:52:03.920 --> 0:52:08.120
<v Speaker 4>to blame the system. But yeah, if you tell us

0:52:08.120 --> 0:52:10.640
<v Speaker 4>to build a concrete boat, we'll build a concrete boat. Like,

0:52:10.719 --> 0:52:13.480
<v Speaker 4>how what way is that to run a country?

0:52:13.840 --> 0:52:17.600
<v Speaker 3>I have a much more technical problem, and please yell

0:52:17.640 --> 0:52:19.760
<v Speaker 3>at me if this is too technically in the weeds.

0:52:19.920 --> 0:52:23.120
<v Speaker 3>So at one point, we in my old work on snap,

0:52:23.200 --> 0:52:26.719
<v Speaker 3>we observed a system, one of the one of the

0:52:26.760 --> 0:52:29.600
<v Speaker 3>official systems that was user facing, so external users like

0:52:29.719 --> 0:52:31.839
<v Speaker 3>folks trying to apply for for essentially use it. And

0:52:31.880 --> 0:52:35.839
<v Speaker 3>we noticed one day that it wasn't loading right, and

0:52:35.880 --> 0:52:38.440
<v Speaker 3>it wasn't loading in the following way. You would go

0:52:38.480 --> 0:52:41.880
<v Speaker 3>to the website, it would have buttons and everything loaded,

0:52:42.600 --> 0:52:44.600
<v Speaker 3>but then when you try to interclick on anything, it

0:52:44.600 --> 0:52:47.960
<v Speaker 3>wouldn't work, and it wouldn't work weirdly for about a

0:52:48.120 --> 0:52:51.600
<v Speaker 3>minute and a half, like then all of a sudden,

0:52:51.600 --> 0:52:54.200
<v Speaker 3>you can click on things and we're like, what is

0:52:54.239 --> 0:52:59.440
<v Speaker 3>going on here? This is bizarre, And we looked around,

0:52:59.239 --> 0:53:02.080
<v Speaker 3>and so we we did some debugging on the client side,

0:53:02.080 --> 0:53:04.279
<v Speaker 3>meaning like we opened up the web browser inspector and

0:53:04.320 --> 0:53:06.360
<v Speaker 3>we said, okay, what's going on? And what we found

0:53:06.560 --> 0:53:11.120
<v Speaker 3>was this website was loading I forget exactly the name.

0:53:11.120 --> 0:53:14.560
<v Speaker 3>It was like translations dot js or translations dot xml,

0:53:15.600 --> 0:53:17.320
<v Speaker 3>and it was taking a min and a half. Everything

0:53:17.320 --> 0:53:19.560
<v Speaker 3>else was loading fine, but this was taking a minut

0:53:19.520 --> 0:53:23.400
<v Speaker 3>and a half, and counter to best practice, it was

0:53:24.320 --> 0:53:27.120
<v Speaker 3>you know, blocking any interaction with the page until it

0:53:27.160 --> 0:53:29.480
<v Speaker 3>was fully fully loaded. And it was loading very very slowly.

0:53:29.840 --> 0:53:31.399
<v Speaker 3>And we looked at it and it says giant file.

0:53:31.440 --> 0:53:32.560
<v Speaker 3>I was like, I don't know. It was like fifty

0:53:32.600 --> 0:53:34.839
<v Speaker 3>megab items a giant file, Like what is going on here.

0:53:35.120 --> 0:53:38.520
<v Speaker 3>Looked into it and it was a translation of every

0:53:39.120 --> 0:53:45.600
<v Speaker 3>page on the entire system into like eight languages, and

0:53:46.680 --> 0:53:49.399
<v Speaker 3>we call the thing and that was like, okay, this

0:53:49.480 --> 0:53:51.600
<v Speaker 3>is not great, and there's ways to fix that. Like

0:53:51.640 --> 0:53:53.760
<v Speaker 3>I said, you can let people interact with the page

0:53:53.840 --> 0:53:56.600
<v Speaker 3>until before that's fully load and it's fine. Also, you

0:53:57.080 --> 0:53:59.600
<v Speaker 3>definitely shouldn't put all the translations for every page in

0:53:59.600 --> 0:54:02.160
<v Speaker 3>your website on eight pages into a single file that

0:54:02.200 --> 0:54:05.400
<v Speaker 3>you sends to the client, Like that's just bad development practice.

0:54:05.480 --> 0:54:09.680
<v Speaker 3>But the thing that maddened me is we being good

0:54:10.200 --> 0:54:13.440
<v Speaker 3>hearted people who were not out to, you know, just

0:54:13.440 --> 0:54:17.400
<v Speaker 3>sort of lob bombs and talk trash. We contacted the

0:54:17.560 --> 0:54:20.240
<v Speaker 3>entity who ran it and we said, hey, this is happening.

0:54:20.480 --> 0:54:22.960
<v Speaker 3>We're seeing consistently, this is the issue that's going on.

0:54:23.680 --> 0:54:25.839
<v Speaker 3>And they said, no, no, no, no, that's not having us load.

0:54:25.920 --> 0:54:31.160
<v Speaker 3>Totally fine for us. And the reason is because we

0:54:31.880 --> 0:54:33.839
<v Speaker 3>diagnosed it down to one or two reasons. One was

0:54:33.960 --> 0:54:36.000
<v Speaker 3>they'd already loaded the file once so it was cash

0:54:36.040 --> 0:54:37.400
<v Speaker 3>and so it was on their computers every time they

0:54:37.440 --> 0:54:39.239
<v Speaker 3>went to it. It was fine and like, oh yeah,

0:54:39.239 --> 0:54:42.120
<v Speaker 3>it works for us. The other was and this was

0:54:42.160 --> 0:54:45.640
<v Speaker 3>our better guess is their server that was serving this

0:54:45.960 --> 0:54:48.960
<v Speaker 3>was on their network and actually sending it much more quickly,

0:54:49.480 --> 0:54:51.520
<v Speaker 3>and they were just like, well, it works on our network.

0:54:51.520 --> 0:54:54.160
<v Speaker 3>It works totally fine. And I don't know exactly what

0:54:54.200 --> 0:54:56.640
<v Speaker 3>it was. But I think that is the reason that's

0:54:56.680 --> 0:54:59.440
<v Speaker 3>so maddening. And it gets back to a little bit

0:54:59.440 --> 0:55:02.680
<v Speaker 3>like Gem's point about accountability is the truth is we

0:55:02.760 --> 0:55:06.040
<v Speaker 3>can't fix problems that we don't see or don't recognize.

0:55:06.280 --> 0:55:09.279
<v Speaker 3>And I do think that they're a big part of

0:55:09.320 --> 0:55:13.200
<v Speaker 3>all of this is we have some expectations that websites

0:55:13.200 --> 0:55:15.880
<v Speaker 3>should work, that technology should work. We also need to

0:55:15.960 --> 0:55:18.919
<v Speaker 3>give more. We have to set an expectation I think,

0:55:19.040 --> 0:55:22.960
<v Speaker 3>government wide about what stuff working looks like. Like Jen

0:55:23.000 --> 0:55:26.360
<v Speaker 3>gave an implicit one of a normal human should be

0:55:26.400 --> 0:55:29.439
<v Speaker 3>able to understand this. That should be tested on every

0:55:29.440 --> 0:55:32.680
<v Speaker 3>system in the e commerce world. Like what's the number

0:55:32.719 --> 0:55:35.200
<v Speaker 3>one metric you care about? It's from start to finish?

0:55:35.280 --> 0:55:37.840
<v Speaker 3>How many people start transaction and actually end up purchasing

0:55:37.960 --> 0:55:40.360
<v Speaker 3>that conversion rate that I mean, it would be a

0:55:40.360 --> 0:55:43.239
<v Speaker 3>really useful feedback loop to measure the success rate of

0:55:43.360 --> 0:55:46.680
<v Speaker 3>users trying to do a transaction across every transaction in government,

0:55:46.760 --> 0:55:49.360
<v Speaker 3>right like people applying for food, SAMs, people applying for whatever.

0:55:49.760 --> 0:55:53.640
<v Speaker 3>If you saw massive drop off or massive disparities across systems,

0:55:53.800 --> 0:55:56.600
<v Speaker 3>at least it would tell you, oh, there's something we

0:55:56.719 --> 0:55:59.480
<v Speaker 3>need to fix here. So I think there's another layer

0:55:59.480 --> 0:56:03.040
<v Speaker 3>of all this, and that's my depressing story because eventually

0:56:03.040 --> 0:56:05.560
<v Speaker 3>it got fixed and I think probably they figured out

0:56:05.600 --> 0:56:07.799
<v Speaker 3>we were right like a week later, but it took

0:56:07.800 --> 0:56:10.799
<v Speaker 3>a week and it was a very I think that's

0:56:10.840 --> 0:56:13.640
<v Speaker 3>why in my mind, there's also if we can get

0:56:13.680 --> 0:56:17.479
<v Speaker 3>to better accountability and give people better on the government side,

0:56:17.480 --> 0:56:21.640
<v Speaker 3>better visibility into these issues with possible routes to solving them.

0:56:21.719 --> 0:56:24.719
<v Speaker 3>They want to solve them, but right now we don't

0:56:24.719 --> 0:56:27.960
<v Speaker 3>have a system that necessarily looks for those issues. And

0:56:28.000 --> 0:56:29.520
<v Speaker 3>I think that would be a better form of the

0:56:29.520 --> 0:56:33.840
<v Speaker 3>accountability we have relative to say, did you follow every

0:56:34.040 --> 0:56:37.759
<v Speaker 3>IT checklist bullet point in policy? Right? Like, if we're

0:56:37.760 --> 0:56:41.120
<v Speaker 3>getting bad outcomes while following that, then maybe something else

0:56:41.160 --> 0:56:43.320
<v Speaker 3>needs to change and maybe there's a different form of

0:56:43.320 --> 0:56:44.759
<v Speaker 3>accountability necessary there.

0:56:44.960 --> 0:56:47.000
<v Speaker 4>We should give a quick shout out to the President

0:56:47.040 --> 0:56:54.680
<v Speaker 4>Biden's Customer Experience Executive Order that is trying to get agencies.

0:56:54.719 --> 0:56:58.000
<v Speaker 1>Maybe in that direction, there is so much to talk

0:56:58.040 --> 0:57:01.120
<v Speaker 1>about it, so many different subversions of this conversation we have,

0:57:01.400 --> 0:57:05.760
<v Speaker 1>but uh, that was amazing conversation. Jennifer Palka, Dave Gorino,

0:57:05.920 --> 0:57:08.400
<v Speaker 1>thank you both so much for our coming on odlocks,

0:57:08.600 --> 0:57:11.359
<v Speaker 1>sharing your time and insight and direct experience and all

0:57:11.400 --> 0:57:11.800
<v Speaker 1>this stuff.

0:57:11.920 --> 0:57:27.480
<v Speaker 5>Yes, delight, thanks so much, Thank you, Thank you, Tracy.

0:57:27.680 --> 0:57:31.120
<v Speaker 1>That was an extremely illuminating conversation I found. I mean,

0:57:31.160 --> 0:57:33.040
<v Speaker 1>it's like easy enough to say like, oh, government is

0:57:33.120 --> 0:57:35.480
<v Speaker 1>bureaucratic and they don't pay enough and that's why they

0:57:35.480 --> 0:57:37.320
<v Speaker 1>can't build software, but they were like there was like

0:57:37.400 --> 0:57:41.439
<v Speaker 1>so much richer and more complex than I like appreciated.

0:57:41.760 --> 0:57:41.920
<v Speaker 5>Right.

0:57:42.040 --> 0:57:45.120
<v Speaker 2>The emphasis on incentive was really interesting and sort of

0:57:45.120 --> 0:57:47.200
<v Speaker 2>fits into a lot of the discussions we have here.

0:57:47.240 --> 0:57:50.560
<v Speaker 2>And also Jen's point about a lot of this emanates

0:57:50.560 --> 0:57:54.160
<v Speaker 2>from the policy yes side, yes, right, and you know

0:57:54.160 --> 0:57:56.760
<v Speaker 2>it's people trying to tick various boxes that have been

0:57:56.800 --> 0:57:58.120
<v Speaker 2>set in stone by policy.

0:57:58.320 --> 0:57:59.360
<v Speaker 4>I got to say one.

0:57:59.160 --> 0:58:01.720
<v Speaker 2>Thing coming out of that, though, I feel a newfound

0:58:01.720 --> 0:58:04.760
<v Speaker 2>appreciation for being a journalist. And actually, you know, when

0:58:04.800 --> 0:58:08.440
<v Speaker 2>you and I produce an episode or write an article,

0:58:08.520 --> 0:58:10.160
<v Speaker 2>we sort of send it out in the world and

0:58:10.200 --> 0:58:12.040
<v Speaker 2>then it's done, and we can move on to the

0:58:12.080 --> 0:58:12.480
<v Speaker 2>next thing.

0:58:12.640 --> 0:58:15.200
<v Speaker 4>We don't have to go back in ten years refine it.

0:58:15.320 --> 0:58:20.200
<v Speaker 2>Yeah, in response to customer feedback and various testing and

0:58:20.280 --> 0:58:21.000
<v Speaker 2>things like that.

0:58:21.240 --> 0:58:23.800
<v Speaker 1>If we were if we were Wikipedia editors, we would

0:58:23.880 --> 0:58:26.800
<v Speaker 1>have to be continually editing them for years. I thought

0:58:26.840 --> 0:58:29.600
<v Speaker 1>that was really great. And again Jen's point that you

0:58:29.680 --> 0:58:32.080
<v Speaker 1>just brought up, it's like this sort of like reflection,

0:58:32.280 --> 0:58:35.440
<v Speaker 1>like the real solution like has to emanate from the

0:58:35.440 --> 0:58:37.800
<v Speaker 1>policy level. I was actually getting drinks recently with a

0:58:37.800 --> 0:58:40.520
<v Speaker 1>friend of mine who's a software engineer, and he said,

0:58:40.560 --> 0:58:43.720
<v Speaker 1>there's like this phenomenon. He said, it's called Conway's law.

0:58:43.800 --> 0:58:46.600
<v Speaker 1>Like every organization, he said, like ships, its organ structure

0:58:47.080 --> 0:58:49.080
<v Speaker 1>and this then it makes like so much sense, Like

0:58:49.240 --> 0:58:52.720
<v Speaker 1>the software is complicated because US politics is like complicated

0:58:52.880 --> 0:58:56.760
<v Speaker 1>ninety years of policy changes and Democrats and Republicans switching

0:58:56.800 --> 0:58:59.360
<v Speaker 1>who's in power, et cetera. It's like, no wonder, like

0:58:59.400 --> 0:59:03.120
<v Speaker 1>the ultimate like product of that thing is going to

0:59:03.280 --> 0:59:05.560
<v Speaker 1>not look like you know, Google dot Com.

0:59:05.560 --> 0:59:08.280
<v Speaker 2>Well, also the story of the guy who is like

0:59:08.320 --> 0:59:10.800
<v Speaker 2>it takes twenty five years to really know how to

0:59:10.880 --> 0:59:14.440
<v Speaker 2>file acclaim. You can imagine if you spend twenty five

0:59:14.520 --> 0:59:18.960
<v Speaker 2>years developing expertise in this one very specific skill set.

0:59:19.200 --> 0:59:23.800
<v Speaker 2>You're not really incentivized to start like changing that anytime soon.

0:59:23.920 --> 0:59:26.840
<v Speaker 2>And so the system just kind of perpetuates itself so

0:59:26.920 --> 0:59:29.840
<v Speaker 2>many different directions, So we can go with that conversation.

0:59:29.480 --> 0:59:32.640
<v Speaker 1>So many different ways. The system is a self perpetuating Yeah,

0:59:32.800 --> 0:59:33.640
<v Speaker 1>I learned a lot from that.

0:59:33.720 --> 0:59:33.919
<v Speaker 4>Yep.

0:59:33.960 --> 0:59:34.800
<v Speaker 2>Shall we leave it there.

0:59:34.880 --> 0:59:35.560
<v Speaker 1>Let's leave it there.

0:59:35.600 --> 0:59:38.360
<v Speaker 2>This has been another episode of the odd Lots podcast.

0:59:38.600 --> 0:59:41.160
<v Speaker 2>I'm Tracy Alloway. You can follow me on Twitter at

0:59:41.160 --> 0:59:41.960
<v Speaker 2>Tracy Alloway.

0:59:42.120 --> 0:59:44.280
<v Speaker 1>And I'm Joe Why Isn't Thal? You can follow me

0:59:44.480 --> 0:59:47.960
<v Speaker 1>on Twitter at the Stalwart. Follow our guests on Twitter.

0:59:48.240 --> 0:59:51.320
<v Speaker 1>Jennifer Palka, She's at Paulka dot and she is the

0:59:51.400 --> 0:59:54.720
<v Speaker 1>author of the new book Recoding America. Definitely check it out.

0:59:55.080 --> 0:59:59.880
<v Speaker 1>Follow Dave Guarino, He's at Dave Underscore Glorrino, follow our

1:00:00.520 --> 1:00:04.440
<v Speaker 1>Carmen Rodriguez at Carmen Arman, and dash Oll Bennett at dashbot.

1:00:04.720 --> 1:00:07.160
<v Speaker 1>And check out all of the Bloomberg podcasts under the

1:00:07.200 --> 1:00:10.880
<v Speaker 1>handle at Podcasts and for more odlots content, go to

1:00:10.880 --> 1:00:14.040
<v Speaker 1>Bloomberg dot com slash odd Lots, where we post transcripts.

1:00:14.240 --> 1:00:16.560
<v Speaker 1>Tracy and I have a blog and a newsletter that

1:00:16.640 --> 1:00:19.320
<v Speaker 1>comes out every Friday, and you can talk about all

1:00:19.360 --> 1:00:22.360
<v Speaker 1>of these things twenty four to seven with fellow listeners

1:00:22.440 --> 1:00:25.520
<v Speaker 1>on the Odlogs Discord. It's really fun. I hang out

1:00:25.520 --> 1:00:29.400
<v Speaker 1>there a lot, go to discord dot, gg slash, odd lots,

1:00:29.920 --> 1:00:32.200
<v Speaker 1>so it's a real blast and a fun place to

1:00:32.200 --> 1:00:34.400
<v Speaker 1>hang out on the Internet. Thanks for listening.

1:01:00.680 --> 1:01:00.720
<v Speaker 3>E