Hello, and welcome to another episode of the Odd Lots Podcast.
I'm Jill Wisenthal and I'm Tracy Alloway.
Tracy, you know, what I felt was a really interesting thing in a recent debt ceiling episode that we did was about all the coding it would take if the Treasury were to introduce a new kind of a bill.
Yes, that was interesting to me too, because I think there's a perception out there that if Treasury wakes up tomorrow and decides to sell a new type of bond, they could just do it. But actually there are all these back end changes that would need to be made. And as you and I know from multiple conversations at this point, it feels like government technology can be a little bit clunky sometimes. Is that a fair way of putting it?
It is?
I would say, you know, in defense, I mean get into yes, you know, corporate technology can be clunky. But that point reminded me that, like we've done, software is like a fascinating thing. And we talk a lot on the show. I think about like the sort of built economy, like the Chips Act and battery investment and semiconductor and all these things are like can the US build things again?
Like as a government state capacity these ideas, but in the twenty first century also like building software in tech has to be part of that.
Question, right, So it's one thing to announce that you're going to do this new thing, but almost everything that we do nowadays comes with some sort of software requirement. Like I'm sure there is a system for processing applications for chips, funding and things like that, and someone has to actually build that. So there's two questions contained in
the can America build things question? It's like, can it build the actual things and then can it build the software systems that would allow it to build the things?
And you know, in the pandemic period, we got sort of smacked in the face, I would say, by this realization that the saw fair component is not trivial at all. And most notably it was like every state had some sort of problem with their unemployment insurance system and there were just such a surge in claims obviously in those initial you know, really for that first year, just on
the scale that none had ever seen. And it reminded people's like, oh, yeah, fifty different stage systems much with like tech that was probably built by some contractor, and I don't know the eighties or nineties and the people who knew how to run it have left, and you know, it was a I gues Eventually, I guess it got smoothed out, but it sort of exposed like, Okay, why did that happen? And what else is lurking out there that we don't really know how it works until it's broken.
I'm sure this is going to end up being a classic episode where we mentioned Kobyl quite a few times, like it's inevitable, right, it's coming, you can feel it.
One day, We're going to do it like actually, just like what is Kobol episode where we just like focus on that directly. But yes, we're going to talk about software and we're going to talk about it from the public sector perspective. What does it mean when the government tries to fix something, what happens when the govern tries to build something, what happens when the government tries to
buy something. Some of these themes we talked about them from a private sector perspective with Patrick McKenzie earlier in the year, but more to do on this topic.
Yeah, I'm excited. I have a lot This is going to be a really good chance to actually ask like the decisions that are going into some of these things being built. I'm still floored every time I log into the Treasury Direct website that you know you have to click the little the buttons like the actual letters and numbers to type your password. It won't let you just type your password.
How did that come to be?
I have questions?
That is a great question. So we have I think literally the two perfect guests to discuss this, two guests with extensive experience answering and working on exactly these problems. We're going to be speaking with Jennifer Palka. She is the founder of Code for America. She helped found the US Digital Service. She co chaired the cal Fornia task force on fixing its unemployment insurance system during the pandemic, and she's the author of a new book called Recoding America,
and we're going to be speaking with it. Dave Guarino, who's a number of things currently an independent consultant researcher focused on this stuff. Founding engineer at Code for America, helped create get cal Fresh, which was like allowed Californians to easily access snap has recently worked on the unemployment insurance modernization at the Department of Labor. Also worked at California, also worked in California. So two people's extensive experience on
exactly these things. So, Jennifer and David, thank you so much for coming on odd.
Lots great, thanks for having us.
Absolutely great to be here.
Now I want to ask Tracy, could I steal your first question?
Yeah?
I do it.
How has just happened that.
A government website doesn't let you type in something and you have to like click buttons like that like resemble mini keyboard on a website. What happened when you see that? Do you have like in your mind you're like, oh, I know exactly the meeting that caused this to happen.
I think Dave has been in those meetings and has come out like with his hair on fire, fuming, So I will let him answer that.
I mean, well, I don't have Yeah, I wasn't in that specific meeting. Maybe the best answer I can give is, and there's deeper issues behind it, is that the way Garment tends to build technology. The reason for that is no one wrote down specifically as a requirement upfront that people should be able to use the keypad and the numbers on their keyboard to enter the digits, and so
someone's like, well, I've met the requirements. Like the requirements are that someone can do this, they can enter it, and the way they do that is they click in this somewhat I suppose insane way.
This is gonna be a conversation or like bang your head on a disk a bunch. I feel like as we listen these answers, anyway.
Yeah, I get ready.
Okay, Well, let me ask maybe a big picture question to start, but what is the process by which government software is developed? Because you know that answer just then it makes it sound like someone gets a mandate we need a website, for instance, through which people can buy government bonds, and then someone else goes off and I guess they execute it along the lines of the mandate that's been set.
I mean, I'll jump in on that one. I mean, first of all, it is changing. So just before we get everyone sort of bounding their heads on the table, there's a lot of movement in good directions. But what has been the process is exactly that there's sort of this mandated often comes from legislation or regulation that very frequently now like says there will be a website that'll even put the URL in the legislation, and then it kicks off what's a really long process that does exactly
what Dave was saying. It is first centered around requirements gathering, and that process of just requirements gathering can easily take ten years. Now, when it's something like healthcare doc of you know, and they have a launch date that's three years out, they don't take ten years, but they still gather sort of every imaginable requirement and they kind of throw it in one giant bucket. And that's how the
RFP is created. The request for proposal, that's what vendors bid on, and it's sort of like an undifferentiated, unprioritized mess of everything everyone could think of, plus things that have sort of pulled over from previous RFPs, plus a bunch of compliance stuff around security, compliance stuff around well various other things. Dave can maybe chime in, but what that isn't is product management. Right, there's no one saying here's the important things this should do. And as Dave said,
there's never a requirement for it to just work. So then the vendor, I mean, I'll skip several steps in the middle where you know, it takes a long time to bid it out, and then there's like a protest because one vendor thinks the other vendor shouldn't have got it, and that delays it a couple more years. But then they go into development and their job, as Dave said, is just like check all these boxes, and it's thousands.
I mean, when we were at California, State of California working on unemployment insurance, they were actually about to put out a Business System Modernization RFP. Actually I think about it to award it to a vendor, and I think it had seven six seven hundred requirements. We can check that, but it's in that's really a normal number of requirements. And so they do all these things that where you can check a box, but what they don't do is actually check that it works. So I mean, maybe i'll
just tell a quick story from the book. And you know, there was famously this application for veterans' healthcare benefits at the VA that didn't work outside the building and they didn't couldn't see it. So basically, somewhere in the specs it had that this form needed to work on a very specific and outdated combination of Internet Explorer and Adobe Reader and The reason no one knew that it didn't work inside the building is that's how all the computers
in there were set up. But if you were outside the building, had any other possible combination of those two pieces of software, it literally wouldn't load. And so they had very very few people applying for these benefits online,
and you know, veterans were really, really, really frustrated. And it took a team going out there recording a veteran who had tried to do this dozens and dozens of times, bringing that video back and showing it to the Deputy secretary for them to be able to say, oh, okay, actually there is something to be fixed here. Okay, you know, you can go ahead and make a new form. But up to them, they said, sorry, it's fine, we're looking
at the requirements. The requirements have been met. There's technically nothing wrong. That's amazing.
It's also kind of crazy to think that people would have been looking at that and just been like, oh, well, I guess demand for veterans benefits is lower than we thought it would be. But actually it was a tech issue.
I think what they said was demand for them doing it online is low. These veterans must.
Not have access, which is absolutely not true, or they can't figure out the computer when in fact, yeah, I mean them couldn't figure out the software.
They would tell them. And they told this guy Dominic, the veteran that they interviewed, They kept telling him it's user error. There's something wrong with you.
Can I ask you a question about that bidding process? When I hear government goes out to bidders for a website, I imagine these like big sort of faceless offices in Roslin, Virginia, where you know, people get like chop salid for lunch and stuff like that, and there's like three or four of them and they sort of like rotate who wins
the bid and stuff like that. How competitive, like is it would a typical rfp auction or bidding process be and how hard would it be forcea a more agile, innovative firm or someone wanting to do it differently to break into the pursuit of that contract.
That's changing a lot. So now you've got a set of vendors. I think there's about thirty five of them in this thing called the Digital Services Alliance, which are smaller companies that work in a more sort of agile, user centered way, and you know, they're just a lot smaller than the Beltway, so there's still a tiny portion of the market, but you know, they're really viable businesses, and people in government can contract out to them. In fact, I think there's a mechanism by which you can put
an RFP out just to those companies. But what has happened in the past is like Dave can pile onto this. Like if you were bidding out, say you know benefits system in a state, and pick your benefit. Usually they would require in the RFP that the bidder had to have done a similar system in a different state in the same category, which means that right there it limited to like three vendors, and so you were never going to get any disruptors because they were boxed out in the very first step.
Just to maybe jump in. That's exactly right, and if you I mean, to some extent, it's rational. If you are thinking about this as someone who's not a technologist, you run an agency, and you're thinking, we need to do this very large project. It's going to take a very long time. It we really need it to go right. It is hard to think about, well, why wouldn't I look for a vendor who's done this five, ten, fifteen times. The problem is, as Jen said, well, I think there's
two problems in that. One is you've now very very drastically narrowed your vendor pool, so there's a lot less competition, and two you also may have too big of a project, Like really, what level of attraction are you trying to work towards? And is it the case that really you need someone or a vendor who's worked on this specific type of system in another entity that it works exactly the same way and all of those details of the
exact same program, the exact same structure of agencies. Probably not,
because probably it's about more generically software. But if you really focus only on the track record there, you kind of get stuck, as Jema was saying, in this trap where there's only a small number of vendors who can meet those criteria, even if you think they're And it's hard because the other thing I would say is as much as there are new vendors, and maybe Jenny can speak more of this, but it's also the case that if you have a new vendor that wants to work
in a more agile way, wants to work in a more use this CRD way, but they are bidding on an RFP that is structured in a very waterfall top down meet other requirements, and that's what success looks like. It's also not going to be successful because you're not giving them any space to say, oh, well, I know your requirement says a way to ender phone numbers using the key the mouse, but when we tested it with people like people, turns out really don't like using the
mouse to click numbers to enter digits. They prefer to use the keyboard. Could we change that requirement? And honestly, I mean, that's an extreme example, but a lot of and there's a lot of what Jen's book is about and what resonated with me in it is that in doing these things and building software, you learn those details
that you never could have gotten right up front. And so if you make success defined by implementing everything as we understand it before we ever get started, you almost ensure that you're not going to get what you really wanted because you're not allowing any information after that cut off point, right when you might be getting to the point where in doing it you're gonna learn a lot.
I definitely want to ask you more about that top down waterfall nature of how a lot of these software projects get commissioned. But before I do, I'm just curious about the actual vendors because I'm sort of imagining these like big faceless offices. But what types of vendors are we talking about? And I'm trying to think how to
use those kind of diplomatically. But I can imagine a company that is putting itself out bidding for government contracts on software development, Like maybe they're not as experimental or cutting edge as a software company that's, you know, somewhere in San Francisco and it's building like new exciting products
for the corporate world. Although we've also done an episode on how bad internal corporate tech can be, but it seems like it might be a little bit more I hesitate to say boring, but sort of basic playing it safe.
Well, you know one thing that I will say, like I hear both incredible frustration and even anger from folks that have to hire these companies. They're just like, there's no alternatives. This has gone badly, it will go badly. They kind of know that, Like I had this woman in a major state. I won't say which who. When I was said, you know your project's going to fail, she said, do you think we don't know that? The last seven projects have failed. So there's this a huge frustration.
But there's also this real feeling that the company in San Francisco with its forty people who make consumer software like great, people love using that software, but they don't understand us, right, They're not going to actually understand the constraints of government. So we have to go with these big companies. And they're not wrong in the sense that
the constraints of working with government are real. There's a huge compliance burden that forty person company in Sum Francisco mostly does not want these jobs, Like there's so much that goes into getting the you know, getting the bid, you know these it's sort of famously said that these companies know how to get the get the work, not how to deliver on the work. That's not probably what a fordy person company wants to do, but you know, it's true that you have to deliver on stuff that
is mind bogglingly complex. Like when we're working in unemployment insurance again during the pandemic, my colleague was talking with the claims processors like week over week, and we're trying to like dissect it and figure out what's going wrong and like clear this backlog. And one of these guys keep saying, well, I'm not quite sure about that answer. I'm the new guy. I'm the new guy. And she finally says, how long have you been here? And he says,
I've been here seventeen years. The guys who really know how this works have been here twenty five years or more. So think about, like, you know, going from doing some simple, cool, you know tech app, you know, easy consumer app, to trying to build or fix or improve upon a system that is so complex that it takes twenty five years to learn how to process a claim. That's that's sort of I think what needs to be on the table as part of this agenda is not just like can
the tech be better? But can we go back and simplify the accumulated like ninety years of policy and process that's making that so hard to make?
Well?
Why do we sort of back up?
Then?
And I again kind of steal Tracy's question because I had the same thought, what is it about government buying that creates these massive waterfold RFPs in which you can't you know, in which a like create these constraints that the software developer would not necessarily have with a private buyer, and b puts government agencies in a position where they will say to you, of course, we know it's going to fail, which again presumably that actually probably does happen
in the private sector too. Or but what is what are the to some extent, but what are the dynamics that make these things like so hard to change and so hard to you know, rip, you know, come up with a new system.
Well, this is a good Dave question, but I'm gonna start. You know, I really spend a lot of time reflect on this, you know, I had. I've had sort of three years since I stepped down from Code for America. I just like sat in a room and well, I interviewed people and thought about it, and I thought about my own experiences, and I think that there's a deep seated culture in government where the policy people are the
important people. They do the important stuff, and technology digital is just part of implementation, which is at the not just the bottom of like a software development waterfall, it's the bottom of a big, rigid hierarchy and which information and power and insights only flows from the top to
the bottom. And so it's problematic in part because the people who are doing the tech are really just sort of downstream of everything else and the power and ability and willingness to step up and say, hey, like we probably shouldn't do those sixy seven hundred requirements, we should
probably focus on these two hundred. Get that out the door, and then you know, adage cases as later like that's it's just not there's no permission really to say that, and until we you compare that with say, we'll call it metaphysical silicon valleys. That I don't mean actually like the silicon valley, Like the people I write code started the companies, they're in power, they're at the top, Like compliance like
is below them. In sort of the DC government hierarchy, compliance is like way above, right, Like policy is way above and there's not this sort of build measure learn cycle the Dave was referring to, where you know, people are learning from each other as they're doing the work
and that technology. You know, certainly the policy has an impact on the technology, but the building of the technology needs to have an impact on the policy, like that fundamental culture is something that needs to be like looked at and called out, and where it's not happening, where there's actual conversation, you know, dialogue beats right. Where that's happening, we should be lifting that up and saying this is possible within government because that's like the foundation I think
of getting away from these mega projects that fail. But Dave you you are you're going to disagree with me on this, and I'm gonna love it.
I think that's I actually do think that's right. I think a big part of it flows downstream from budgeting, meaning a lot of this the way that and again everything that I think Eyegen sate like, there's some degree of generalization here, and there's always outliers and there's always exceptions, and of course there's also a lot of people try and change these things. That said, maybe the dominant mode
is we're going to budget for a large project. We get one time funding to spend to do that project, so we got to get everything right. We're only get this chance once every so often. We're going to replace everything. We're gonna have this new system. It's going to be great. And the way that this paradigm breaks down is to in what's called a design development phase, where you build the system, you design what it should do, you think
about that. Again, you're kind of making all these decisions up front, which I think is very counter to a lot of the if you want to call the Silicon Valley approach. But then you're building the system, and then you enter what's called maintenance and operation, and it's supposed to be very very little. You're not going to change much. It's just kind of keeping the system going. And the problem is modern software you kind of learn the most
the second you've actually deployed to real users. That is the point when you are getting people saying, oh, this doesn't work, or you're getting an error, or you're getting hey, I tried to put in this address, but I have
a funky address and it won't let me do that. So, all of a sudden, you have all this information, and unfortunately, in sort of what gets called a big bang launch project where you just put it up and you're done and you're not going to touch it again, you now, almost immediately on day one, can see potentially all of these issues. And a lot of people prepare for this in these kinds of projects. They're like, the first month
is going to be really hard. You all of a sudden know all the things that were assumptions you made that you got wrong. But the model that you had that you were given is, well, we're going to do
this one time. Now. Contrast that with actually going live is the first step, and you're going to have funding that's continuous and kind of flat over ten, over twenty, over thirty years, and you're going to have those people on STEPH and you're going to continuously change the system as you learn more about how users interact with with it, as you learn more about like what things you didn't
get right upfront? Did you You made it possible to upload a certain type of document, You accept PDFs and tiff images Tiff this is a real example, but turns out you don't support JPEGs or the I think iPhone proprietary default image format. So now all your mobile users are saying, hey, we can't upload, or a bunch of them were saying, I can't upload documents. It says I
can't submit this file type right now. A lot of you know, well intentioned folks in government get to that situation and then they're stuck because they don't have the mandate or the funding the same now we're going to change that. They're waiting for the next big project to replace the system and now support those new things, Whereas instead, if you have a model where you're saying we expect to continuously change this, it's a very very different thing.
And that's a little bit that is more how Silicon Valley operates. I mean, this maybe is sort of hyperbole, but Google didn't build search and then lay off ninety percent of their staff and say we're done. This is great, you know, like, look we've got this search thing. And I think that difference flowing downstream from how tech is budgeted for where you're budgeting for a one time project,
and this comes from legislatures, you know, from Congress. A one time project versus we want to have ongoing fundings because this is a living, breathing system. Is a huge paradigm shift and has an uncomfortable aspect, which is it may seem like it's costing more money because it's potentially dollars ongoing because it's staff, not one time by but I think a lot of the dysfunction that we see probably flows down from that root. Cause.
Okay, so here's here's a big question. If we can identify the cause of a lot of this dysfunction and if we can trace it back to this waterfall structure, the sort of top down mandates or maybe like theorists versus practitioners. So you have a politician who has a big idea and they want that big bang reveal of the software they're going to expand benefits, and it's going to come with a really cool website and everyone's going
to be able to access them easily. And yet it leads to these problems that we've been discussing.
Why do we keep doing it?
That's a hard question, Well, what is the system that is like keeping this way of doing things in place, versus maybe you know, after a few decades of developing these types of systems, why isn't someone going well, actually, the way we've been doing things hasn't really been working well.
I think a lot of people have been saying that, and that's why I say things are changing, like and I will offer as evidence COVID tests dot gov. Do you remember that one beginning of twenty twenty one, the President announced that there would be this site where you could request tests, and I think he gave a date that was I want to say six weeks out, it
launched a day early, it launched in multiple languages. It took eleven seconds for me to use other people say eight seconds, fifteen seconds, right, and you're test showed up a couple days later. Like that sounds like it could have been really like okay, so it's much simpler than say signing up for healthcare ducov or whatever. But like they could have made it way more complex. They could have said, like, let's gather every possible requirement and try
to fulfill every possible requirement. But you had internal capacity, Like to Dave's point, there are people in house who were saying, hey, we have been doing it away that hasn't been working. Let's try this way and it's great. Like it serves as a fantastic example I think within government and outside government. Right, So part of it is how people in government think what they believe, and part
of it is what the public thinks. Right. The public can't couldn't imagine something that easy, but we have to start imagining it and holding our government accountable to doing that. So, I mean, I will say it is changing, but the belief that we aren't good at it, that we couldn't ever do like a COVID test. Dot of drives politicians, pundits, vendors to say government's bad at this, and so we
should outsource everything. Now, of course we're going to have vendors, and we should have vendors, but there has to be enough internal capacity and competency to outsource well and to make those decisions like, hey, let's have this thing, only ask like name, address and like, you know, not ask for their health insurance, you know, numbers and not like do different numbers of tests per household. Like that's an
internal capacity question that is really important. And we don't have enough internal capacity for these kinds of decisions in government because we believe we're not good at it, and we believe the only people who are good at out of these outside companies. Plenty of evidence that that's totally wrong.
I want to go back to internal capacity in a minute, but since you mentioned the COVID test website, it's sort of a good chance to pivot a little bit here. You know, COVID specifically seem to open people's minds a little bit about how fast we can work and how much money we can spend, and obviously a historically fast pace of vaccine development that we've never seen prior to that, and so it sort of had this brief moment where
it sort of opened people's minds. And I'm not sure if it's closing again, but you know, since both of you worked on the California unemployment insurance system and generally, like I think almost every state on some level was seen by some tobaccle Like what was I guess the biggest eye opening thing that going into that environment taught
you or about the system. And then what was like, you know, what's the key sort of takeaway of like, Okay, here's a thing that we can learn from this that then whether it's UI and other states are more broadly, okay, this is something that you learned that then can like have extensible lessons.
So I think you're right. COVID had sort of twin impacts on our thinking. One was, holy cow, we can actually do stuff, and the other one was, holy cow, we're really screwed. Right, Yeah, you're way to put it, And there's evidence of both, which is which is really fair. I mean I think for me, you know, i'd been doing this for you know, ten years when I got pulled into the the unemployment insurance I'll call it a rescue. We were clearing the backlot, and I felt like, nothing
here can scare me. I have seen the VA, I've seen the operating of you know, some pretty janky, you know, government agencies. But I actually really did learn a lot and was kind of shocked at how the unemployment insurance system worked. And the metaphor that came to mind for me was like everyone kept saying, well, it's a system, obviously you can just fix it, and it isn't a system.
And I think that's true of other things we think of the systems too, Like it is archaeological layers of technology that sort of map to archaeological layers of policy that didn't ever get designed. It just accrued, Like so nobody ever goes back and says, okay, how is this going to work with this? There's such little appetite for anything that's like backward looking that would actually update and make these layers work together well, that it just isn't
what people think it is. When you go look at it, you're just like, WHOA, of course you can't do these things, like this thing isn't connected in any meaningful way to this thing, and like this was built in a certain era for a certain purpose, and we just glombed stuff onto it when we needed to say, add internet access,
like let people apply online. I mean, the biggest thing was, like I think we haven't grappled with the fact that when these systems were quote unquote designed, I actually was to say when they were started, because they really have never been designed, and a lot of them and there's
huge advantage to actually doing some design. But you went into an office and showed who you were, like you identified you, you validated your identity by going in, and then we moved online and we never figured out how to validate people's identity. And there were a number of problems in California with unemployment assurance. The fact that you know it took twenty five years to learn how to do it meant that the any claim that couldn't be
processed automatically was a huge, huge bottleneck. And you couldn't add claims process to do that, Like you would have to go back in time twenty five years, start new claims browsers and have them ready to come online during a downturn, right, Like that's never going to scale. But the other big problem was that that you only went through the automatic sort of pipeline if they felt like they could verify your identity, but they weren't really doing
any meaningful identity verification. And we, unlike most other industrialized countries, don't have a national system for that, Like we don't have a national technology platform for that. But also like again back to this all derives from like the legal and policy framework, but actually have a reasonable policy framework for identifying people and knowing who they are when we
start to do a transaction. And that is causing problems everywhere, and there is interesting conversations about how to fix it. There's some stuff out there, but it's it's really contested, and it's not just a problem for the pandemic. It's going to continue to be a problem.
Yeah, I mean, I'll just add and my vantage point was was somewhat different, right, I was coming at it from a different angle. I think the major thing that I took away, especially with so much focus on technology and so much focus on COBAL mainframes, and that was, you know.
That was set we need to do a Cobyl like drinking take a shot time.
How long it took us to get to that.
Word oh, I can't believe, you know. And the best part is that. I mean my next point was that's misplaced causation. And yet I am the one who brought up cobal. Yeah, I mean I think I think we that is a you know, for so much focus on Oh, these tech systems are falling over they're not good. I think there's the number one lesson I learned was that and I think this is true, but we don't tend to think about it, which is the technology systems are
part of a larger sociotechnical system. What I mean by that is, as Jen mentioned, you have staff who can do stuff manually, and you have a system that might be able to do stuff in an automated way. One of the problems with programs that are so complex is if you have a massive spike of volume in your technology system isn't set up to just cover all those cases in an automated way. You try to throw bodies
at it, and you try to hire people. But if it takes a year and a half to train someone and like train someone to a relatively basic level, with some of the complexity of these programs, you're not able to hire fast enough and Unfortunately, you know, this is why.
We were hiring really fast. We heard five thousand people in California to help process these claims, but because they couldn't do anything, they were taking up the time of the experienced claims processors. And we calculated that every single new hire the state made slowed processing down. And one of the big things we did was actually just get them to reassign staff away. I mean, some of it was just reassigned staff to other things, Like nobody was
opening the mail, which was pretty critical. And yet opening the mail is something you can do if you just joined and have no background, but like, yeah, it was pretty franny. So sorry to interruptive. It's just like we were hiring. It's just that the hiring was having a perverse effect.
Well, and that's I guess that's my point is that's exactly right. And I guess the number one lesson I learned was if you look at the history of a program, like un employment trends, but specifically UI, you have this thing where once every ten years or so, there's a major recession, or in this case, there's a pandemic, which was like a ten x recession, right because it was
overnight as opposed to graduate was overnight. So much of the economy, so many of the working force, labor force, were just out of work immediately, so all those claims came in simultaneously. But we have this cycle of every ten years or so way of recession or I mean, I don't know, I'm non economists, I can tell you exactly what that is, but it happens in a recurring way, and then everyone gets frustrated with how difficult it is
to get unemployment insurance because they're overloaded with volume. And then the claims volume goes down, recession ends, and then there's a lot there. Honestly, isn't a lot of focus on Okay, we're gonna sort of put more money into the UI system so that we're ready for next time. But unfortunately, you can't have resilient systems if you optimize for pure, pure, pure efficiency, like in the down years, in the years when you have very very low volume, if you are just going to cut staff down to
very very bare bone skeleton crew. It comes back to this point of not only are you not able to do preparation if you don't have the funding to have staff, you can't even make the changes that would be good, and you can't when you get the money in a new recession hire the people because you can't train them, and so we kind of get stuck in the cycle. So that was the number one from a systems modeling perspective lesson I took away with it.
From it, I got to add, though, you know, we did invest several billion across states during the Great recession or after a Great recession, so I think about half of the states technically modernized then they did take advantage of pretty significant funds. None of those states did any
better than average in this downturn. So I agree with Dave, and I would argue that taking a different approach to quote unquote modernization, to policy and process simplification, to you know, having goal driven modernization like all of those things, is equally important as investing during the downturn. Investing the way we have been doing it is not working. There are
so many different threads to pick out of there. But just on that last point, you know, when you were talking about how a lot of these government systems were not designed to be that way, I kept thinking of a lot of the software used by the banks, where you know, you think, you think of like JP Morgan's software system. It wasn't designed to be JP morgan software system.
It's the result of many, many acquisitions of other banks, and you know, every time they take over this business, it gets sort of duct taped onto the rest of the system. But I'm curious, is there a moment that you have seen in your careers at which someone says, this system is just like irredeemable. We're going to have to start from scratch, And like, what is the trigger point for making that decision versus reaching for the band aids and the duct tape and just trying to make it work.
I mean, and I forget if someone else has mentioned this. I lit the Patrick McKenzie episode, maybe he mentioned this, But in general, a good rule of thumb is never rewrite a system from scratch that's working. I mean, other people might disagree with that. But the reason I don't
have a good answer for sort of a moment. Lots of people have tried to do that thing where they just rebuild it from scratch, and again it tends to go poorly because and this is the specific thing, the old system encodes so much tacit knowledge and so many of the many, many many little details, and some of those details might no longer be relevant, but many of them might encode some really hard learned truth about the world, because at the end of the day, the software is
just modeling the real world. It's just saying, okay, well we have customers. They have addresses. Okay, so maybe you have two address fields in your old system and you think, oh, well, let's just streamline that down to one. But then you launch your new system has one address, and everybody calls you up and like, what are you doing? Like, we the reason we have two addresses is because our billing department is over here and the customer facing address is different.
So I think the throw it away and start from scratch is an impulse everybody has, but it has a lot of risk as well, and I think the better approach tends to be can you get to a place where you can start to incrementally change your systems as you go in small ways that are safe, where it doesn't take it. You know, it's not like we have to test for six weeks any change before making it live. We can ship weekly, we can ship daily, we can ship multiple times a day. That tends to yield better things.
There are plenty of projects out there that we're going to build this from scratch, from the bottom up. Most of the ones I'm aware of don't have great launch stories. Maybe JO have disagrees, that's my no.
They're still in that mode of like get it all in once and then we're done, and we have sort of maintenance, and so they're pretty bad. But I will say two things. One on a practical level, just because I was talking with a colleague on the way over here about this, there are great examples now of what Dave's talking about, like not waiting for this big bang project but saying we are just going to be making this a little bit better every day. And one of
them is New Jersey's Department of Labor. Their unemployment insurance response during the pandemic was quite good, but they've also learned from that and said we're not going to wait for some like big system to come that's going to
fix all of our problems. We have a team now that can just incrementally fix things as we go and eventually that will involve some bigger investments as they learn X, Y, and Z. But they're just not waiting and it's just a great, great example, But I will say, Tracy, like I think about what you're asking about all the time. I don't think this will ever happen, but if it, if you do get a chance to sort of zero base it and start over, you have to zero base
the policy, Like you can't. Could you do a new unemployment insurance system in California that encodes the twenty five years of knowledge that those claims processors have. Well, I mean maybe I will help you now. I don't want to talk about any but like, but like that's a problem. Like you know, I say in the book, there's this great team that's working on this Medicare problem, and it's like there's nine different definitions of even like what a
medical group is. Like even the first question doctors are supposed to answer is like wildly complex. And the policy the delivery team is sort of fighting with the policy team, and she's like, I get that it's complex. It has to make sense to a person, And I like, fight, fight, fight, not over the tech, but out of like over collapsing these nine definitions into two. They want I get to one. They can't. At least they get to two. And like this idea that it has to make sense to a person.
We've like lost track of that. I mean, of course you're frustrated when you apply for unemployment insurance, Like it's wildly complex in its policy and the process that accrued from those policies. Like if you were going to start over, you can't start over with the tech. You would have to say, okay, let's figure out let's like look at this ninety years of accumulated policy and say what was
this actually trying to do. Can we kind of you know, easier go back and just like rationalize all the changes, right, It's not like like I say, like there's not one binder of regulations that covers youI there's ninety years of memos that change the previous memo that change the previous thing, that changed the previous thing. And you could at least go back and like condense that and like figure out all of the conflicts in it. But it would make actually,
I think a lot more sense. And I will be clear, this is never going to happen if you just said, let's design the policy for an unemployment insurance system that makes sense in twenty twenty three, and then you could build tech for that.
There's so many different threads and places we could go. There's such a fascinating conversation. I just have one more question, and it sort of relates to something, Jennifer that you said earlier about like in government, the tech people are like secondary likely like the policy wonks and people designing these things, and that that's a really big difference between
government of public and private sector. And you know, this internal capacity, how much does that hinder And I think this is another thing that Patrick McKenzie brought up, but like hiring and so and the challenge of like, you know, public sector pay skills are not private sector pay skills. I have to imagine for a lot of these things he talked about a little bit, you know, and you know what you've worked on some of these like sort of volunteer things that sort of bridge the private and
public sector. But can you talk a little bit about that can strength of like even if like just the challenge of in those in those roles getting like experienced talented people.
I'm glad you brought that up because it's a little bit of a soapbox I like to get on because we have you know, like politicians, like the leaders who are so frustrated with delivery and technology, and like they're generally they're well intentioned, but the sort of things they push on are kind of unhelpful. You know, they kind of increase the risk aversion of the bureaucracy by yelling them about things and not recognizing the constraints on the bureaucracy.
I call this the accountability trap. But there are things they actually could do, and fixing the civil service and civil service hiring rules is really really important. I do know what Patrick McKenzie said on your show about pay scales is not totally I don't totally agree with that. The biggest problem is not the pay, it's the time to hire. So if it takes, as is really common right now in federal government, nine months to make an offer,
of course you lose everybody you've got. Actually, you know, when when I started this work, there was not like a line of people out the door, one with great tech and design skills wanting to work for government. Now there actually is, and we can't get them in the door. I mean, I know people who do wait nine months for that offer because they really want to help and
the pay is fine, Like it's not fantastic. I think he was comparing to like Google whatever, but like you know, compared to a startup where yes, you have the hope of getting an exit, but you're probably not gonna make like five hundred thousand dollars right as a programmer, you're gonna make what half of that. I mean, we don't have to get into like exact numbers, but like the pay is in the right ballpark for like mid level folks.
It's not going to pay at the executive level. But as he mentioned, like at the executive level, if you've been at Google for twenty years, pay is not your biggest concern. I do want to push back a little bit on the notion that that is he is in government. There are some of those people, but there are far more people who just have good tech and design skills who just want to make a difference more than anything else.
And we should be working on all of the constraints that are keeping that hiring process at nine months more than we're than we're like calling up the people who manage the Department of Labor in a state and yelling at them because they had these failures, like fix the environment in which they're working. Give them that capacity, then you can hold them accountable, but they can't hire people right now.
You also you need to be able to hire people, and you need them to be able to use their judgment to make higher order decisions than just do we enter the phone number with the keyboard or by clicking on a virtual pad of numbers. Like it is also the case that some of these systems are the way they are because it's just jen this might be your phrase, it's like policy vomit. It's just the rules put onto a screen, just like all the rules in statute put
onto a screen. And so I think the thing I'd like to add is the whole point of product management is to arbitrate the trade offs between user experience, compliance goals, technical cost of change, and technical trade offs. And if you don't give those people if you hire them, but they don't have the ability to say, well, maybe we need to simplify this to make it make sense to
a person. If instead the only response they get is what we need to put on the page exactly what's in statute then, or whatever other compliance goals there are, like, oh, we need to strictly follow this process. It's procedurally what it is. If those are counter to the outcome. You need a change in government where outcomes start to be it outcomes in this area around technology start to sort of have a higher importance than the strict process and
the strict procedures. And I do think that that's a necessary corollary to bringing smart capable people in because if you bring in a superb product manager and they're just they have no ability to sort of bring these trade offs up and make those calls on the ground, you're going to get the same status quo.
I think.
So I can't resist asking this question. I take all the points about, like, here's what we need to do in order for this to change, But can I just ask, because both of you have that practical experience of doing this, what was the craziest thing that you saw during your respective tenures, Like, what was something that just really surprised you or shocked you. I'm back to the banging your head on the table a portion of this conversation.
I mean, I guess the thing that comes to mind just because it really made an impression on me, and it's it's a negative story that I want to sort of balance out with something positive. But I was working in the White House and had decided to do what we convinced the Veterans Administration to do what we call discovery sprint on the Veterans Benefit management system. And it
was going really slowly. There was like huge latency, and I got these two folks in the help out for just a couple of weeks and just you know, do an analysis of what was wrong before they go, like, you know, trying to figure you can't solve the problem till you know what it is. So like the first thing was we show up and the guy's like, oh, I'm glad they sent the White House to you know,
verify that everything's fine now. And it turns out everything was fine because the leader had defined the latency as under two minutes. Okay, so you've just defined the problem away. So people who are trying to process benefit fit applications, like if they clicked and it took one minute and fifty nine seconds for the application to respond, it was fine, no problem here. Wow. But then I kept talking to the guy and I kept asking him questions about the system, like, well,
why is it designed this way? Why was it designed that way? And he said, I've spent my career teaching people not to have an opinion on business requirements. The IT people should have no opinions there. And I said why, He said, well, if they ask us to build a concrete boat, will build a concrete boat, because then it's
not our fault. And I was just like, I felt like I'd been gut punched, Like it was so hard to hear that, Like there were I think at the time, eighteen veterans a day committing suicide, most of whom couldn't get their healthcare benefits. And I just could understand how someone could take that approach. But honestly, you know, in the ten years since then, I've seen a lot of reasons why he felt like he could say that, And you know, I don't want to blame him, I want
to blame the system. But yeah, if you tell us to build a concrete boat, we'll build a concrete boat. Like, how what way is that to run a country?
I have a much more technical problem, and please yell at me if this is too technically in the weeds. So at one point, we in my old work on snap, we observed a system, one of the one of the official systems that was user facing, so external users like folks trying to apply for for essentially use it. And we noticed one day that it wasn't loading right, and
it wasn't loading in the following way. You would go to the website, it would have buttons and everything loaded, but then when you try to interclick on anything, it wouldn't work, and it wouldn't work weirdly for about a minute and a half, like then all of a sudden, you can click on things and we're like, what is going on here? This is bizarre, And we looked around, and so we we did some debugging on the client side, meaning like we opened up the web browser inspector and
we said, okay, what's going on? And what we found was this website was loading I forget exactly the name. It was like translations dot js or translations dot xml, and it was taking a min and a half. Everything else was loading fine, but this was taking a minut and a half, and counter to best practice, it was you know, blocking any interaction with the page until it was fully fully loaded. And it was loading very very slowly. And we looked at it and it says giant file.
I was like, I don't know. It was like fifty megab items a giant file, Like what is going on here. Looked into it and it was a translation of every page on the entire system into like eight languages, and we call the thing and that was like, okay, this is not great, and there's ways to fix that. Like I said, you can let people interact with the page
until before that's fully load and it's fine. Also, you definitely shouldn't put all the translations for every page in your website on eight pages into a single file that you sends to the client, Like that's just bad development practice. But the thing that maddened me is we being good hearted people who were not out to, you know, just sort of lob bombs and talk trash. We contacted the entity who ran it and we said, hey, this is happening.
We're seeing consistently, this is the issue that's going on. And they said, no, no, no, no, that's not having us load. Totally fine for us. And the reason is because we diagnosed it down to one or two reasons. One was they'd already loaded the file once so it was cash and so it was on their computers every time they went to it. It was fine and like, oh yeah,
it works for us. The other was and this was our better guess is their server that was serving this was on their network and actually sending it much more quickly, and they were just like, well, it works on our network. It works totally fine. And I don't know exactly what it was. But I think that is the reason that's so maddening. And it gets back to a little bit like Gem's point about accountability is the truth is we can't fix problems that we don't see or don't recognize.
And I do think that they're a big part of all of this is we have some expectations that websites should work, that technology should work. We also need to give more. We have to set an expectation I think, government wide about what stuff working looks like. Like Jen gave an implicit one of a normal human should be able to understand this. That should be tested on every system in the e commerce world. Like what's the number one metric you care about? It's from start to finish?
How many people start transaction and actually end up purchasing that conversion rate that I mean, it would be a really useful feedback loop to measure the success rate of users trying to do a transaction across every transaction in government, right like people applying for food, SAMs, people applying for whatever. If you saw massive drop off or massive disparities across systems, at least it would tell you, oh, there's something we
need to fix here. So I think there's another layer of all this, and that's my depressing story because eventually it got fixed and I think probably they figured out we were right like a week later, but it took a week and it was a very I think that's why in my mind, there's also if we can get to better accountability and give people better on the government side, better visibility into these issues with possible routes to solving them. They want to solve them, but right now we don't
have a system that necessarily looks for those issues. And I think that would be a better form of the accountability we have relative to say, did you follow every IT checklist bullet point in policy? Right? Like, if we're getting bad outcomes while following that, then maybe something else needs to change and maybe there's a different form of accountability necessary there.
We should give a quick shout out to the President Biden's Customer Experience Executive Order that is trying to get agencies.
Maybe in that direction, there is so much to talk about it, so many different subversions of this conversation we have, but uh, that was amazing conversation. Jennifer Palka, Dave Gorino, thank you both so much for our coming on odlocks, sharing your time and insight and direct experience and all this stuff.
Yes, delight, thanks so much, Thank you, Thank you, Tracy.
That was an extremely illuminating conversation I found. I mean, it's like easy enough to say like, oh, government is bureaucratic and they don't pay enough and that's why they can't build software, but they were like there was like so much richer and more complex than I like appreciated.
Right.
The emphasis on incentive was really interesting and sort of fits into a lot of the discussions we have here. And also Jen's point about a lot of this emanates from the policy yes side, yes, right, and you know it's people trying to tick various boxes that have been set in stone by policy.
I got to say one.
Thing coming out of that, though, I feel a newfound appreciation for being a journalist. And actually, you know, when you and I produce an episode or write an article, we sort of send it out in the world and then it's done, and we can move on to the next thing.
We don't have to go back in ten years refine it.
Yeah, in response to customer feedback and various testing and things like that.
If we were if we were Wikipedia editors, we would have to be continually editing them for years. I thought that was really great. And again Jen's point that you just brought up, it's like this sort of like reflection, like the real solution like has to emanate from the policy level. I was actually getting drinks recently with a friend of mine who's a software engineer, and he said, there's like this phenomenon. He said, it's called Conway's law.
Like every organization, he said, like ships, its organ structure and this then it makes like so much sense, Like the software is complicated because US politics is like complicated ninety years of policy changes and Democrats and Republicans switching who's in power, et cetera. It's like, no wonder, like the ultimate like product of that thing is going to not look like you know, Google dot Com.
Well, also the story of the guy who is like it takes twenty five years to really know how to file acclaim. You can imagine if you spend twenty five years developing expertise in this one very specific skill set. You're not really incentivized to start like changing that anytime soon. And so the system just kind of perpetuates itself so many different directions, So we can go with that conversation.
So many different ways. The system is a self perpetuating Yeah, I learned a lot from that.
Yep.
Shall we leave it there.
Let's leave it there.
This has been another episode of the odd Lots podcast. I'm Tracy Alloway. You can follow me on Twitter at Tracy Alloway.
And I'm Joe Why Isn't Thal? You can follow me on Twitter at the Stalwart. Follow our guests on Twitter. Jennifer Palka, She's at Paulka dot and she is the author of the new book Recoding America. Definitely check it out. Follow Dave Guarino, He's at Dave Underscore Glorrino, follow our Carmen Rodriguez at Carmen Arman, and dash Oll Bennett at dashbot.
And check out all of the Bloomberg podcasts under the handle at Podcasts and for more odlots content, go to Bloomberg dot com slash odd Lots, where we post transcripts. Tracy and I have a blog and a newsletter that comes out every Friday, and you can talk about all of these things twenty four to seven with fellow listeners on the Odlogs Discord. It's really fun. I hang out there a lot, go to discord dot, gg slash, odd lots, so it's a real blast and a fun place to
hang out on the Internet. Thanks for listening.
E