Speaker 1: Hello, and welcome to another episode of the Odd Locks Podcast.
Speaker 1: I'm Joe Wisenthal and I'm Tracy Hallo. Tracy, I forgot
Speaker 1: to ask you, and it's kind of embarrassing like this
Speaker 1: late in January, but how is your how is your
Speaker 1: New Year's? Like? How is your holidays? Do you have
Speaker 1: a good Christmas? And stuff? Thanks? Thanks, Joe. I had
Speaker 1: excellent Christmas. I stayed at home for a week with
Speaker 1: my husband and my dog and we did hardly anything
Speaker 1: and it was absolutely glorious. How about you? It was
Speaker 1: all right. So the thing was I was down in
Speaker 1: Texas visiting a family, which is but there was that
Speaker 1: huge cold blast and it's worse in It's worth when
Speaker 1: when the weather gets really cold, it's worse in a
Speaker 1: place like Texas because like none of the buildings are
Speaker 1: insulated particularly well, none of them have that heat. So
Speaker 1: it's like, when it's really cold, is that like better
Speaker 1: to be in a cold place where people are used
Speaker 1: to it. So it's a little uncomfortable. But the good
Speaker 1: news is somehow we managed to travel back and forth
Speaker 1: without getting having any major airline disruptions. Right, So this
Speaker 1: is the key thing that happened right before Christmas, which
Speaker 1: is we had that very big winter storm, the Arctic
Speaker 1: bomb blast, and it disrupted a ton of flights first off,
Speaker 1: because of the weather, But then what happened is you
Speaker 1: had this sort of cascade effect because the weather event
Speaker 1: was so large, a number of airlines, but one airline
Speaker 1: in particular experienced a lot of problems with its software
Speaker 1: and Southwest had to cancel I think in the end
Speaker 1: it was something like sixteen thousand flights. You had millions
Speaker 1: of passengers affected, and you had disruptions that you know,
Speaker 1: a weather related disruption that lasted one or two days
Speaker 1: ended up lasting I think more than a week because
Speaker 1: of the impact of the computer litches. I guess, yeah, yeah,
Speaker 1: that's right. Like you know, airline traveled like it always
Speaker 1: sort of cascades and ripples out right, because it canceled
Speaker 1: flight is going to affect other flights and so forth.
Speaker 1: But it seemed like Southwest experienced something unique, which is
Speaker 1: that there's turned into this major software problem. And it's
Speaker 1: sort of like a reminder that like, okay, when we
Speaker 1: use the Internet, when we use like sort of like
Speaker 1: modern consumer software, it's all very zippy and quick, and
Speaker 1: it has nice interfaces and then when you use like
Speaker 1: sort of like back end corporate software, particularly at large
Speaker 1: legacy institutions, it's just like it's nothing like the consumer Internet.
Speaker 1: It's like clunky. It's like we all know what it's like, right.
Speaker 1: So this is something that came up quite a lot.
Speaker 1: I used to cover the big banks, and one of
Speaker 1: the crazy things that I learned relatively early on while
Speaker 1: I was doing this was just how much of their
Speaker 1: I T system was still these old, creaky, big iron mainframes,
Speaker 1: some of them still running on Cobalt, which is the
Speaker 1: programming language that I think it dates back to the
Speaker 1: nineteen fifties or nineteen sixties. And I remember, you know,
Speaker 1: you hear this, You hear like, oh, I can't believe
Speaker 1: that these big banks, our entire financial system in some respects,
Speaker 1: is still running on these legacy computer systems. But on
Speaker 1: the other hand, if you look at what a big
Speaker 1: bank is, it's structurally a series of mergers and acquisitions.
Speaker 1: There used to be the no, no, that's there used
Speaker 1: to be this great flow chart that showed the formation
Speaker 1: of like a JP Morgan or a Bank of America,
Speaker 1: and you can just see it's like a series of
Speaker 1: roll ups of smaller banks, and you think every time
Speaker 1: they acquire a new bank, they have to integrate another
Speaker 1: system into their own system, and in the end you
Speaker 1: kind of end up with this just incredibly like complex
Speaker 1: and kind of patchy i T structure that in some
Speaker 1: ways very much resembles the amalgamation of all these smaller
Speaker 1: banks into a larger bank. There's too the right, like
Speaker 1: you think of like banks and when they merge, it
Speaker 1: just like okay, like you know, the capitals all merged
Speaker 1: together and the assets, etcetera. But like, right, like none
Speaker 1: of them are going to have like i T systems
Speaker 1: that work perfectly together, so they're like glued together with
Speaker 1: like duct tape and just over time. I think technical
Speaker 1: debt is a term that software engineers, and so you
Speaker 1: can to relate all this technical debt. And I'm surely
Speaker 1: nobody likes it inside the bank at any level. But
Speaker 1: I've always been curious, like, what are the economics such
Speaker 1: that it is just so impossible for these legacy institutions, banking, airlines,
Speaker 1: absolute public sector to like get with the times, you know, absolutely,
Speaker 1: and when it comes to Southwest, So first of all,
Speaker 1: let me declare a small self interest in Southwest which
Speaker 1: is my My dad was a pilot for Southwest for
Speaker 1: a long time and I remember when the outages were
Speaker 1: happening this Christmas. I sent him a text going like, well,
Speaker 1: what do you think about this? And he just said, well,
Speaker 1: all the crew are out of position and they need
Speaker 1: to get them back. So very straightforward ex pilot answer
Speaker 1: about what was going on. Not that helpful for this podcast.
Speaker 1: But I do think in the case of Southwest, this
Speaker 1: has been a long running issue and you have had,
Speaker 1: you know, people talking about it at various times, the
Speaker 1: need to upgrade the infrastructure, of the technological infrastructure, and
Speaker 1: yet it hasn't happened. And so the question is why.
Speaker 1: The question is whether or not is it cheaper just
Speaker 1: to keep running these old mainframes and assume that you
Speaker 1: are going to have these outages during major events, cheaper
Speaker 1: than it would cost to actually upgrade it. No. Absolutely, Well,
Speaker 1: let's let's talk to somebody who knows about software and
Speaker 1: knows about economics and can walk us through and help
Speaker 1: us understand the problem. We're gonna be speaking with Patrick mackenzie.
Speaker 1: He's an expert on software and infrastructure. He's a writer
Speaker 1: of the Bits about Money newsletter and knows a lot
Speaker 1: about finance, and he just left a stripe after six years.
Speaker 1: He's still an adviser there. Been reading his stuff for
Speaker 1: a long time. One of these people I was sort
Speaker 1: of trust on almost any topic. Patrick, Thank you so
Speaker 1: much for coming on odd lots. Thanks very much for
Speaker 1: having me. What is the deal? Let's just start like
Speaker 1: straight up, like why do you like, I mean Tracy
Speaker 1: mentioned like all these institutions, they just like they swish
Speaker 1: it all together. They sort of like tie it together
Speaker 1: with duct tape and so forth. You get these unwieldy things.
Speaker 1: Just like give us the high level view of like
Speaker 1: why is it so difficult from at an abstract level
Speaker 1: to modernize legacy software? So I'll start with a disclaimer,
Speaker 1: which is sort of mandatory and engineering culture, we have
Speaker 1: this thing that we've come up with over the last
Speaker 1: course of two decades, or so called blameless postmortems, where
Speaker 1: when there is a failure within a company where you know,
Speaker 1: planes cannot be up in the air for a week
Speaker 1: at a time. Rather than like trying to point the
Speaker 1: finger at someone and say it was your decision or
Speaker 1: your inaction that caused this event, we as engineers want
Speaker 1: to like look at the rejective reality of the system,
Speaker 1: figure out what went wrong for the benefit of both
Speaker 1: that organization and for the larger community. And so this
Speaker 1: isn't too grind their nose in it, but just you know,
Speaker 1: as an engineering matter, what probably happened in an ideal world,
Speaker 1: And if this had happened at a Google, for example,
Speaker 1: the engineering teams would push, because of the culture of
Speaker 1: these things, to do a very public postmortem of what
Speaker 1: the decisions were, what the background is, etcetera, etcetera. In
Speaker 1: more traditional industries, I don't think that culture is fully
Speaker 1: baked yet as it were, although they're very well might
Speaker 1: be a postmortem by uh, you know, the FA and
Speaker 1: federal regulators, because you know, liveness constraint is a real
Speaker 1: thing for extremely important economic systems like airlines. Anyhowp what
Speaker 1: probably happened. It wouldn't surprise anyone in the bowels of
Speaker 1: an airline that if you put off maintenance on airplanes
Speaker 1: for decades at a time, that eventually bad things would
Speaker 1: happen and no one would countenance that. However, software systems
Speaker 1: are quite similar. Where aren't a build once and then
Speaker 1: run for the rest of eternity sort of things? There
Speaker 1: were some decisions made early in their lifetimes which share
Speaker 1: no longer accurate for the world we live in. They
Speaker 1: do suffer from something engineers you from misply called bit
Speaker 1: rot where software which worked back in the past will
Speaker 1: tend to like succumbed entropy over time and not work
Speaker 1: exactly perfectly for all time afterwards, and so you need
Speaker 1: to be doing a ongoing program of maintenance for your software,
Speaker 1: just like you would for your airplanes. That bluntly was
Speaker 1: not done, and it seems to be credibly reported that's
Speaker 1: the sort of cultural factors at Southwest that caused that
Speaker 1: to not be done. Might have been caused by a
Speaker 1: sort of like overly accounting the slash penny pinching focused
Speaker 1: management culture, which thought, well, it costs money and the
Speaker 1: short term to do to do maintenance, we can like
Speaker 1: cram down or engineering costs by doing less of this
Speaker 1: and relying more on external vendors, etcetera, etcetera. And then
Speaker 1: when stuff HiT's the oscillating plate, the right people have
Speaker 1: not done the years of work that are required to
Speaker 1: get to a posture where you can like quickly recover
Speaker 1: from failures, and so they were left in a part
Speaker 1: where we use euphemistically in the in street called heroics
Speaker 1: was required from folks in operations trying to you know,
Speaker 1: contact thousands and tens of thousands of employees by fille
Speaker 1: and pass around their information and probably spreadsheets to figure
Speaker 1: out where the crew is actually where to be able
Speaker 1: to take the boxes that are required due to regulation
Speaker 1: to allow people to get back and up in the air.
Speaker 1: So that's probably the high level, like root cause of
Speaker 1: what happened. But like a reason to conduct these postmortems
Speaker 1: is it's there for one, like one single decision made
Speaker 1: by one person. It's the result of cultural factors, business
Speaker 1: decisions made over a course of probably decades in this place,
Speaker 1: and we want to like keys out the various nuances
Speaker 1: there and like make both Southwest and other organizations where
Speaker 1: of that so that they don't suffer critical systemic failures
Speaker 1: in their own places. So, first of all, bit brought
Speaker 1: is a fantastic term that I am going to have
Speaker 1: to try to work into all my conversations going forward.
Speaker 1: But secondly, just to step back a bit, can you
Speaker 1: maybe explain, you know, when we talk about mainframe computer systems,
Speaker 1: what exactly are we talking about? And like what is
Speaker 1: the counterpoint to mainframe. I'm assuming it's more like cloud
Speaker 1: based applications and things like that, but could you maybe
Speaker 1: define those that basic term very quickly. And then secondly,
Speaker 1: my understanding of the Southwest debacle was that they had
Speaker 1: this in house software. I think it was called sky Solver,
Speaker 1: but it was based on an application that g E
Speaker 1: had been selling, and then Southwest kind of customized it.
Speaker 1: And I guess my question is how endemic is that
Speaker 1: type of software where you get something off the shelf,
Speaker 1: but then you customize it in such a way that
Speaker 1: it becomes um, I guess special to you and therefore
Speaker 1: your problem when it goes awry. So let's talk about
Speaker 1: the the main frame first, and then we'll talk about
Speaker 1: the problem of like who owns the problem, whether it's
Speaker 1: your problem or some of the vendor's problem, and the
Speaker 1: way that balls tend to end uh in the middle
Speaker 1: of people and things get dropped. So mainframes back in
Speaker 1: the day, many many decades ago, computers were approximately the
Speaker 1: size of a room. This is the before the personal
Speaker 1: computer revolution and banks, which were some of the earliest
Speaker 1: adopters of computers for sort of scaled usage in industry.
Speaker 1: And it's funny like the earliest users and computers tended
Speaker 1: to be either the financial industry or the military, either
Speaker 1: attempting to like move numbers which represented money, or numbers
Speaker 1: which represented like literally artillery shells flying through the air,
Speaker 1: and both were a very important society for better or worse,
Speaker 1: Like banks because they standardized on what was the best
Speaker 1: available technology at the time, ended up with a lot
Speaker 1: of main frames, and they have kept those main frames
Speaker 1: running for a good portion of seventy years now. In
Speaker 1: some cases, as you mentioned earlier, the alternative to main frames.
Speaker 1: Cloud is a bit of a buzzword. So there's the
Speaker 1: personal computer form factor that you're familiar with, where something
Speaker 1: sits on your desk. There are servers which would typically
Speaker 1: sit in a server X somewhere, uh. And the difference
Speaker 1: between like servers and quote unquote the cloud is like
Speaker 1: the traditional way to manage servers would be you would
Speaker 1: have a data center that would be owned or at
Speaker 1: least by yourself, and you would put hardware that you
Speaker 1: owned in that data center and in the cloud. The
Speaker 1: cloud case, the data center is owned by Amazon or
Speaker 1: Google or Microsoft. You have rented access to a machine
Speaker 1: there which sits on their balance sheet and it's probably
Speaker 1: not one machine, and it's probably like an awful lot
Speaker 1: of machines potentially with some virtualization layer, and your engineers
Speaker 1: can cause systems to scale up or scale down based
Speaker 1: on how many machines you need on a you know,
Speaker 1: minute by minute or second by second basis. Is sort
Speaker 1: of like the high level version of the sales pitch
Speaker 1: that cloud vendors will give you. So that is a
Speaker 1: hey quick run through between like four generations of the
Speaker 1: main stay of how technology gets done at scale the
Speaker 1: question of where software gets written. So it is quite
Speaker 1: common for businesses in the traditional economy to not have
Speaker 1: an internal software engineering competence, and as a result, they'll
Speaker 1: go to vendors. Like in this case, you probably know
Speaker 1: this better than me. But for example g and the
Speaker 1: vendor will sell quote unquote package software which might not
Speaker 1: be fully responsive to the needs of the business, and
Speaker 1: then some customization happens. Um what could happen is the
Speaker 1: customization might happen within the business if they have some
Speaker 1: level of software engineers internally. What happens more frequently in
Speaker 1: a lot of places, particular in depend where I live,
Speaker 1: is that the business will contract with we call them
Speaker 1: system integrators here that may be called consultancies in America,
Speaker 1: you know a Deloitte or a another large consulting firm
Speaker 1: like that Accenture and say, okay, we have this package system,
Speaker 1: we have these business requirements. Clearly some software engineers have
Speaker 1: to be involved. We don't have enough on our staff.
Speaker 1: Can you like figure out the missing link for us?
Speaker 1: And so there is an ex sense of contract negotiation.
Speaker 1: Consultancy goes off and does the customizations, and they delivered
Speaker 1: the work to the organization. And then there are various
Speaker 1: different ways to do maintenance, but often that will involve
Speaker 1: the original team standing down for a while, which that
Speaker 1: always sounds like a great idea when it's pitched, because
Speaker 1: it's like, oh great, I don't have to pay expensive
Speaker 1: engineers every week to just sit around waiting for something
Speaker 1: to happen. And then when something actually happens, you sort
Speaker 1: of like realize that the other end of the option
Speaker 1: value there where oh not so great. I don't have
Speaker 1: a team of like experts who understand the system, ready
Speaker 1: to call up at a moment's notice and bring in
Speaker 1: to debug the problems that we're seeing right now. And
Speaker 1: so I have no specific knowledge about went in this
Speaker 1: individual instance. But a thing that happens during a lot
Speaker 1: of outages is uh a, you know, quick look into
Speaker 1: the history of the system to say, hey, wait, who
Speaker 1: actually built this for us? Are they still in business?
Speaker 1: Can we get on their calendar immediately? Is there an
Speaker 1: engineering team there that is ready to hop in and
Speaker 1: work at this? Like goodness, they're going to charge us
Speaker 1: I bleeding rates to to do it, but we're going
Speaker 1: to have to pay that. That's a you know, tertiary
Speaker 1: consideration at this point. The bigger consideration is like, how
Speaker 1: quickly can we get our you know, organization to organization
Speaker 1: like legal paperwork, etcetera, spun up? And then how quickly
Speaker 1: can you get they get the engineering team spun up
Speaker 1: to address the system in real time? And this sort
Speaker 1: of thing is why the sort of like major scaled
Speaker 1: software companies and the economy that Google, Microsoft, etcetera, etcetera,
Speaker 1: you all know the names, largely treat engineering as an
Speaker 1: internal competence rather than you know, the software for the
Speaker 1: iPhone isn't built by Debloid, It's built by Apple, and
Speaker 1: almost everything in the critical path will either be open
Speaker 1: source or something that there is a team on Apple
Speaker 1: that owns the sort of totality of the experience for
Speaker 1: a goal. So something that I sort of hinted at
Speaker 1: in the beginning, and we talked about this a little bit.
Speaker 1: We did an episode on petroleum engineering. But I'm really
Speaker 1: curious about like the distribution of engineering talent. Like in
Speaker 1: my mind, I would have to imagine that a talented
Speaker 1: engineer would be more excited to work for Stripe than Google,
Speaker 1: and we're excited to work for Google than Deloitte, and
Speaker 1: we're excited to work for Deloitte than Southwest. And I'm
Speaker 1: just sort of and I'm curious, like a if that's
Speaker 1: the case, and whether like this is a problem, and
Speaker 1: whether like it would be better, whether it be better
Speaker 1: if like there were just more high tech experts, software
Speaker 1: experts who would want to work for a Southwest directly
Speaker 1: or work for a bank directly, whether this sort of
Speaker 1: like distribution of engineering talent contributes to software body all act.
Speaker 1: So I have nuanced thoughts that engineering talent, on one hand,
Speaker 1: like that is certainly thing that exists in the world,
Speaker 1: and like talent is not necessarily distributed equally among different industries, individuals, etcetera.
Speaker 1: On the other hand, I think that Silicon Valley and
Speaker 1: ecosystem to which I owe a lot occasionally has an
Speaker 1: overly enamored view of itself, thinking that, oh, yes, we
Speaker 1: have the best engineers in the world, and all the
Speaker 1: engineers who made every other system that we rely on
Speaker 1: in our lives are sort of second rate, And clearly
Speaker 1: that's not true. Like the phone system works, airplanes like
Speaker 1: what's the old Lui kiss came on a love like
Speaker 1: it flies through the sky and then teleports beams up
Speaker 1: to space like none of that happened by accident, And
Speaker 1: so it's important to not focus on like the You know,
Speaker 1: engineers working traditional industry are worse engineers that engineers their
Speaker 1: work in software companies. The larger problement they have is
Speaker 1: one they don't drive the bus. They have less ability
Speaker 1: to control the situations within their organizations and control like
Speaker 1: larger decisions made such that they have influence over decisions
Speaker 1: on like what the main and schedule would look like,
Speaker 1: or who gets to make decisions with respect to whether
Speaker 1: something ships or not. And there is a little bit
Speaker 1: of a sort of like the life cycle of engineers
Speaker 1: thing that happens where the original architects of the system
Speaker 1: did it forty years ago. A careers are about as
Speaker 1: long as they are. Many of the original architects of
Speaker 1: the system will sort of be aging into the retirement
Speaker 1: years at this point, and so there is a question
Speaker 1: of did the organization put in the work over the
Speaker 1: years to recruit newer engineers to inculcate them into how
Speaker 1: the system is made, etcetera, etcetera, Or do they just
Speaker 1: allow all of the knowledge to walk out the door
Speaker 1: and did they Like I think that comes up a
Speaker 1: lot is do they put enough work on a day
Speaker 1: to day basis to maintain an engineering brand so that
Speaker 1: they can get new talent engineers to join them in
Speaker 1: three such that those engineers are the word used as
Speaker 1: often gray beard, but you know that the wise and
Speaker 1: veteran thirty year is from now, so that when something
Speaker 1: happens in there are people who have been around the
Speaker 1: black that know where the skeletons are buried in the system. Interestingly,
Speaker 1: traditional industry is getting better over the years at having
Speaker 1: an engineering brand, sort of moving away from this this
Speaker 1: world where engineering was largely seen as a cost center,
Speaker 1: where like the goal is just too cram down the
Speaker 1: amount of money you spend on it and improve your margins.
Speaker 1: I think there's a few things that played into that.
Speaker 1: One of them was, particularly in the Internet age, it
Speaker 1: became obvious that you know, in finance, you talk about
Speaker 1: like the front office and the back office. Engineering used
Speaker 1: to be in the back office, and the front office,
Speaker 1: where the sales people live, is the one that generates
Speaker 1: all the money for the bank. And increasingly, because like
Speaker 1: experiences that were in the palm of the users hands
Speaker 1: were the thing that we're actually generating in the sales,
Speaker 1: those experiences became sort of institutionally important within banks and
Speaker 1: airlines and other firms. And then when those experiences became
Speaker 1: important took a while, but gradually the people and teams
Speaker 1: that build those experiences became more institutionally important than they
Speaker 1: had previously been. And so like, if you look at
Speaker 1: the large money center banks in the US, they certainly
Speaker 1: have no small number of technical challenges. But the thing
Speaker 1: that is like largely true which was not true, is
Speaker 1: that their mobile apps are actually kind of good these days.
Speaker 1: Like if you download, you know, not to endorse anybody
Speaker 1: in particular, but like Chase or Capital One, when you
Speaker 1: play with their mobile apps, like, oh, this kind of
Speaker 1: feels like a mobile app made in Silicon Valley. And
Speaker 1: the reason is, well, yeah, they hired a lot of
Speaker 1: people who made the apps from Silicon Valley, and those
Speaker 1: folks brought their skills and sort of like level of
Speaker 1: competence with this and their taste and now exercise and
Speaker 1: on behalf of old world companies. One hopes, knock on wood,
Speaker 1: that that seeps into the back end of these systems
Speaker 1: to where Apple, Google, Microsoft, they have very talented teams
Speaker 1: on the front end of their systems, but they also
Speaker 1: have very talented teams on the back end of the systems.
Speaker 1: And that's bluntly why you don't see like core systems
Speaker 1: that Google going down for a week at a time
Speaker 1: that is almost unimaginable. You know how much of capitalism
Speaker 1: would break if Google Docs was just down for a week.
Speaker 1: Given that there are many parts of the world where
Speaker 1: there is a liveness constraint, we would like all else
Speaker 1: being equal. If it is safe to fly airplanes, we
Speaker 1: would prefer that there be airplanes flying versus all the
Speaker 1: airplanes being on the ground, because airplanes generated valley for
Speaker 1: human society. If that is true, then it must be
Speaker 1: the case that the back end systems of airlines that
Speaker 1: control whether airplanes are allowed to fly at a given time.
Speaker 1: That has to be at least as important as you know,
Speaker 1: Google docs is, which implies that the airlines need to
Speaker 1: put at least as much work as Google does, roughly
Speaker 1: into like having true mastery of their own back end
Speaker 1: systems and the various problems that could happen there. So
Speaker 1: just on this note, you know, there's another travel disruption
Speaker 1: that happened recently and we haven't even mentioned it, but
Speaker 1: that was the f a A experiencing some sort of
Speaker 1: computer event and grounding I think all the domestic departures
Speaker 1: for for one morning. And it was recently reported that
Speaker 1: the approximate cause of that was because there were some
Speaker 1: software engineers who were trying to upgrade the system and
Speaker 1: they accidentally deleted a bunch of critical files as they
Speaker 1: were trying to do this. Can you talk a little
Speaker 1: bit more about the technical challenges when it comes to
Speaker 1: trying to fix some of these legacy systems, Like why
Speaker 1: exactly is it so difficult? You sort of talked about
Speaker 1: it from a organizational perspective, but from a technical perspective,
Speaker 1: why does this seem to be such a big challenge? Sure,
Speaker 1: so that one thing is that that that symptom, the
Speaker 1: the underlying cause is that a problem happened during an
Speaker 1: upgrade is extremely well understood in the software engineering field.
Speaker 1: It's called out in, among other places, Google's book about
Speaker 1: Site Reliability Engineers, which is essentially the subcategory of engineers
Speaker 1: that Google relies on keeping the world running. And it
Speaker 1: is often the case that the people who are attempting
Speaker 1: to make an incremental change do not have full context
Speaker 1: of how the system came to be in the state
Speaker 1: that is currently in, and a change that they thought
Speaker 1: will have a limited sort of area of impact ends
Speaker 1: up having a larger area of impact. The term of
Speaker 1: art we use in the industry is blast radius. Like
Speaker 1: you you hope to quantify the amount of blast radius
Speaker 1: of something that goes wrong, such that you know, if
Speaker 1: I'm make a mistake, am I going to like bring
Speaker 1: down our blog? Or am I going to bring down
Speaker 1: like credit card processing worldwide? And you know, be much
Speaker 1: more careful if the blast radius includes like worldwide credit
Speaker 1: card processing plausibly, like you should engineer systems such that
Speaker 1: there's no way to take down worldwide credit card process
Speaker 1: but that's actually harder to do, then then it just sounds.
Speaker 1: So anyhow, how do you does one get to the
Speaker 1: point where it is difficult to understand, like what the
Speaker 1: implications of the changes who make and truly are These
Speaker 1: are like meat and potatoes questions, and then the meat
Speaker 1: and potato's answers are often things like was the system
Speaker 1: adequately documented when it was made? Frequently the answer is no,
Speaker 1: and a lot of information about like how systems are
Speaker 1: put together survives this oral lore within the engineering teamate
Speaker 1: various companies, which, yeah, that's an uncomfortable bit of information
Speaker 1: to holding your head when you start talking about like
Speaker 1: the life cycle engineers and the fact that the original
Speaker 1: architects of many of these systems are like literally no
Speaker 1: longer with us, either because they have retired or they
Speaker 1: might be like um, beyond our ability to call up
Speaker 1: out of retirement at this point, and so you have
Speaker 1: to write down what you do. And that concept was
Speaker 1: not like new to governments and bureaucracies as a result
Speaker 1: of software engineering happening in the last seventy years, sort
Speaker 1: of fundamental to the operation of large organizations. Software is
Speaker 1: just how people choose to do work with each other,
Speaker 1: but is a lesson that we keep relearning there is
Speaker 1: often that issue where because software is how people an
Speaker 1: organization choose to work with each other, often software will
Speaker 1: interface various systems together and problems will happen at the
Speaker 1: boundaries between systems, either between literal computer systems or between
Speaker 1: other breakages between organizations. So the thing that you will
Speaker 1: see frequently is my software didn't fail. Your software didn't fail.
Speaker 1: We mutually failed together at that point where where we are,
Speaker 1: you know, supposed to transfer information and then both sides
Speaker 1: end up pointing their their at their counterparty. And so
Speaker 1: part of the discipline of software engineering is one like
Speaker 1: creating a culture where you don't want to point fingers
Speaker 1: of the counterparty into creating structures and incentives such that,
Speaker 1: you know, like complex systems that involved multiple different parties
Speaker 1: with multiple different engineering teams who might not report to
Speaker 1: the same payerial department will like converge on correct outcomes.
Speaker 1: And there are a variety of ways to do that
Speaker 1: in the industry. Some of them are better than others.
Speaker 1: So without like naming the particular company, there exists to
Speaker 1: credit card system which is extremely so credit card systems
Speaker 1: as a baseline are extremely reliable. You probably don't remember
Speaker 1: the last time that you were unable to use a
Speaker 1: credit card for a week, because like literally that does
Speaker 1: not happen. So like a plus plus for achieving that outcome.
Speaker 1: What one credit card company does to achieve that is
Speaker 1: they are willing to make changes for their system precisely
Speaker 1: twice a year, after six months of testing every change
Speaker 1: that they make. That's an incredible amount of upfront work
Speaker 1: to do, like relatively small amounts of engineering, and so
Speaker 1: the pace at which that ecosystem evolves is much much
Speaker 1: slower than more software forward company is like Google, App, Apple,
Speaker 1: ms on, etcetera, etcetera, where they're shipping thousands of changes
Speaker 1: to their systems every day. So one of the interesting
Speaker 1: bits about the remixing the skills and techniques of Silicon
Speaker 1: Valley is attempting to get people who are comfortable with
Speaker 1: the engineering practices that allow you to ship software thousands
Speaker 1: of times a day into positions of authority at old
Speaker 1: line companies such that they can gradually transition from you know,
Speaker 1: the point that they're at where they might be able
Speaker 1: to ship software once or twice a year, maybe quarterly,
Speaker 1: to the point where they will be shipping software like
Speaker 1: let's start with by week move up from there. So
Speaker 1: we've been talking a lot about like failures or sort
Speaker 1: of like collapses. But the other thing that I like
Speaker 1: sort of associate with large business software is just like
Speaker 1: the user the user experience is just not as good.
Speaker 1: And I think that was, you know, the sort of
Speaker 1: like the u X I believe was part of the
Speaker 1: story with like the infamous like city error where they
Speaker 1: transmitted nine million dollars they shouldn't have to some counterparties,
Speaker 1: and I think some of the users were confused by
Speaker 1: the internal software, whether they were actually sending that or not.
Speaker 1: I heard story from someone who worked at the VCR
Speaker 1: once of a major bank about like the hoops that
Speaker 1: they have to go through just to share documents with
Speaker 1: each other like power points, because they get flagged off.
Speaker 1: And internally I assume that there's some sort of like
Speaker 1: regulatory issues or booking travel, like you know, going to
Speaker 1: like booking dot com is really easy when Tracy and
Speaker 1: I book travel for work here, it's not that bad.
Speaker 1: But like the usability of the software, internally, it's not
Speaker 1: as smooth and snappy as consumer travel sides. Why is that?
Speaker 1: Like what is why is this sort of like business
Speaker 1: software internally just like is not as uh, you know, yeah,
Speaker 1: easy to use and sort of visually appealing as the
Speaker 1: consumer internet. So this is getting better over time, and
Speaker 1: we'll talk about that in a moment, but broadly, like
Speaker 1: your your observation is entirely accurate. If you were the
Speaker 1: softwares are for the entire world, you might like rank
Speaker 1: applications by their importance to the world and say, okay,
Speaker 1: if you were an online application on someone's phone that
Speaker 1: allows someone to share cat photos, it's like important in
Speaker 1: some sense, but probably not as important as sending a
Speaker 1: billion dollars outside of a bank, And so we would
Speaker 1: have a lot more talent and time spent on the
Speaker 1: question of can you send a billion dollars out of
Speaker 1: a bank? Versus does a thirteen year old have an
Speaker 1: awesome experience when sending a cat photo? In actual fact,
Speaker 1: though much much more time and talent is spent the
Speaker 1: cat photo question, that is spent on the like wiring
Speaker 1: billions of dollars out of banks question, that was a choice.
Speaker 1: It sounds silly to say maybe we should stop choosing
Speaker 1: stupid things, but the the like true answer is business
Speaker 1: software gets better when the you know, people and organizations
Speaker 1: that cause that software to be built, choose that the
Speaker 1: quality of that software is something that is very relevant
Speaker 1: to their interests. And so one way to like come
Speaker 1: to the realization is to lose a billion dollars and
Speaker 1: then you know, hopefully the next time you're level engineering
Speaker 1: management says we should spend a little more on maintenance,
Speaker 1: you will say, yes, I agree with you, we should
Speaker 1: spend a little more on maintenance versus taking another billion
Speaker 1: dollar charge at a time, not of our choosing. Part
Speaker 1: of it is just the culture of quality coming back
Speaker 1: to these things. Part of it is also through sort
Speaker 1: of teaching the user inside of organizations that software doesn't
Speaker 1: have to be terrible because most software in the world
Speaker 1: exists inside of companies and runs business processes. There's something
Speaker 1: that is not broadly known, but like, of all the
Speaker 1: lines of software in the world, most exist inside of companies,
Speaker 1: and for a very long time, because most software people
Speaker 1: interacted with was after employer and it was generally kind
Speaker 1: of terrible. They just had an image like software is
Speaker 1: like generally kind of terrible. And then the iPhone came around.
Speaker 1: Everyone has a powerful computer in their hand. For x
Speaker 1: number of hours a day, you've used applications you've tapped
Speaker 1: three times, and you know, interesting things happened in the world.
Speaker 1: There's a result of you tapping three times and you
Speaker 1: broadly like the experience, and then you go back to
Speaker 1: work and say, wait, I've used software that doesn't suck.
Speaker 1: All the stuff I use at work sucks. Hey, I
Speaker 1: t department, Hey senior management, can you please make like
Speaker 1: our expense tracking software not be terrible? And that is
Speaker 1: starting to happen built as a result of like internal
Speaker 1: software producers advocating for change, the result of that user
Speaker 1: feedback within companies, and also as a result of various
Speaker 1: startups happening to say, like, not to throw a particular
Speaker 1: expense solution under the bus, but the thing that most
Speaker 1: like oldline economy companies probably use is not a thing
Speaker 1: that people love using to book their travel. And if
Speaker 1: you you know, use trip actions or something that is
Speaker 1: designed by a modern team with modern sort of ux offordances,
Speaker 1: it's a much nicer solution for the end user. And
Speaker 1: in some companies, end users are starting to have some
Speaker 1: level of ability to advocate for what software gets adopted,
Speaker 1: whereas previously that was made by processes that were not
Speaker 1: user centric, where which team was better at doing whining
Speaker 1: and dining the person in charge of the purchase and
Speaker 1: decision and not winning on the basis of product quality.
Speaker 1: And the couple of years, even like some enterprise software
Speaker 1: is starting to win largely end the basis of product
Speaker 1: quality versus on sort of the more traditional sales motion,
Speaker 1: although goodness knows that the traditional sales motion is still
Speaker 1: very important to enterprise software companies. So I have a
Speaker 1: slightly weird question, but I'm thinking a lot about it
Speaker 1: as as we have this conversation. But it feels to me,
Speaker 1: like software engineering and computer programming, it always seems to
Speaker 1: be in flux. Like if your job is a software engineer,
Speaker 1: it feels like there's always something to do. You're always
Speaker 1: trying to fix a problem or adapt a system. And
Speaker 1: I guess, I guess my question is why you know?
Speaker 1: I fully admit my own programming experience is confined to
Speaker 1: like HTML, which I learned from that website HTML goodies
Speaker 1: in like But back then, you know, you program your website,
Speaker 1: you design it in HTML, you release it into the wild,
Speaker 1: and you're kind of done. And yet it seems with
Speaker 1: these large scale systems that there's always change, something is
Speaker 1: always in motion. Something is always influx. Why is that? So,
Speaker 1: let me push back a tiny bit on this year,
Speaker 1: Like how many years have we had lawyers available? And
Speaker 1: does anyone ever go up to the lawyers and say, like,
Speaker 1: come on, guys, it's three haven't you like figured out
Speaker 1: all the laws? Yeah? And so why does the law
Speaker 1: change on a week to week basis? Well, it doesn't
Speaker 1: change per se. It's just the world is complicated. The
Speaker 1: number of commercial relationships between organizations is increasing all the time.
Speaker 1: We have increasing demands on what those relationships will do.
Speaker 1: And the job of lawyers is to adapt to that
Speaker 1: increasingly complex world every weekend, continue delivering like the law
Speaker 1: that society needs and the outcomes that come as a
Speaker 1: result of like competently executing on the ability of organizations
Speaker 1: to collaborate internally with their employees and with other organizations.
Speaker 1: What's my answer for software engineering, Well, software engineers, they
Speaker 1: are you know, working this week on increasingly complex world
Speaker 1: where software is more leverage than it had even last week,
Speaker 1: where they're increasing demands on the world, etcetera, etcetera, etcetera.
Speaker 1: Is there going to be a time where the last
Speaker 1: line of software is written? Probably not. There will never
Speaker 1: be a last a bit of software written. There will
Speaker 1: never be a less contract written, there will never be
Speaker 1: a last book written. Because humans want more things out
Speaker 1: of the world than we have, kind of like infinite
Speaker 1: capacity for want. At the margin, there was a compelling answer,
Speaker 1: can you talk a little bit about you know, we've
Speaker 1: been talking about banks, airlines for startups. Can you tell
Speaker 1: us like, how are the challenges for the public sector?
Speaker 1: And I remember like Obamba had a thing about like
Speaker 1: I want to like bring government websites or government tech
Speaker 1: and the modern age. It just seems like whatever problems
Speaker 1: exist for big companies seemed to be even worse or
Speaker 1: more tricky when you're dealing with the public sector. Could
Speaker 1: I think at one point, at one point, New Jersey
Speaker 1: was explicitly like begging the Internet for cobal programmers, wasn't it.
Speaker 1: In I think that was a thing that happened. Yeah,
Speaker 1: so full disclosure here at today UH nonprofit organization. Last
Speaker 1: year where a few of us in the tech industry
Speaker 1: abandoned together to work on the vaccine location information infrastructure
Speaker 1: for the United States because the public sector was having
Speaker 1: a great deal of difficulty creating websites that would track
Speaker 1: where the vaccine was and route vaccine sekers to it.
Speaker 1: So have lots of thoughts here, so many issues. Again,
Speaker 1: we did not wake up in three with these issues magically.
Speaker 1: It's a result of like decisions that we've collectively made
Speaker 1: as a society go for many years. One decision that
Speaker 1: we've made in the United States in particulars that government
Speaker 1: pay of scales are what they are. If you compare
Speaker 1: those government pay scales to what private industry pays for technologists,
Speaker 1: they are sharply out of whack. And so then you know,
Speaker 1: if you look at GS whatever, the highest paid public
Speaker 1: sector employees in the United States make less than Google
Speaker 1: uh interns. To help for the equilibrium, if you can
Speaker 1: get hired by Google, you know, it would require you
Speaker 1: to really does matter in this realm, right. And one
Speaker 1: of the things that the government has been attempting to
Speaker 1: do over the years to create things where there are
Speaker 1: groups of people who are like officially their government employees. Unofficially,
Speaker 1: I think they're sort of doing an active service to
Speaker 1: the nation in places like the Digital Services Agency, etcetera, etcetera,
Speaker 1: where they already made their money in tech there now
Speaker 1: on the GS whatever and making a fraction of what
Speaker 1: they previously made, but are contributing software expertise to these
Speaker 1: various problems, where like what the government needs is like
Speaker 1: some competent software written, and that requires having competent software
Speaker 1: people available in quantity. Another issue that governments have is
Speaker 1: like what is the true goal you are solving for?
Speaker 1: Without getting too political about it, In some parts of
Speaker 1: the government, like, you know, an organization might exist as
Speaker 1: largely a job's program, and i T modernization might sharply
Speaker 1: decrease the effectiveness of that organization at employing a large
Speaker 1: number of people to like repeatedly to a process that
Speaker 1: a machine could do in a faster fashion. And so
Speaker 1: sometimes like the the powers that be with an organization
Speaker 1: so like, well, you know, I don't necessarily consider i
Speaker 1: T modernization one of my top priorities at the moment,
Speaker 1: because that would cause me to need to break faith
Speaker 1: with a number of people that that I employees slash.
Speaker 1: You know, sometimes my own career trajectory as a bureaucrat
Speaker 1: is and this is true within private industry as well.
Speaker 1: You know, there's a bit of empire building involved where
Speaker 1: you want your number of people that you manage and
Speaker 1: your budgets to go up every year, and you don't
Speaker 1: want to say, okay, like I've solved my problems, so
Speaker 1: I can deal with five percent as much budget next year,
Speaker 1: thank you. That is incentive incompatible. That's not great from
Speaker 1: the perspective of the parts of society which aren't employed
Speaker 1: by government, but which nonetheless dependent on government for you know,
Speaker 1: providing goods and services. And so this is ultimately a
Speaker 1: thing that we have to resolve through the political system
Speaker 1: on pushing back a little bit on saying like, hey,
Speaker 1: you kind of have to be good at what you do,
Speaker 1: and these days that involves making software that is also
Speaker 1: good at what you do. I have no magic bullet
Speaker 1: for how to cause that to be a you know,
Speaker 1: a stunning rallying cry for political parties, but probably something
Speaker 1: that needs to get said in a lot of places
Speaker 1: for enough decades until the message thinks in Patrick, this
Speaker 1: has been an amazing conversation, and I feel like a
Speaker 1: I already want to have you back and be like
Speaker 1: almost each one of your answers could be like it's
Speaker 1: or like have a full conversation. I have one last question,
Speaker 1: how does bit rot happen? And I mean, like you know,
Speaker 1: even I like you know, you like, go away on
Speaker 1: vacation for two weeks, you come back to your office
Speaker 1: computer and like, face are weird and sort of jaki,
Speaker 1: they don't quite work the same, Like what is that process?
Speaker 1: Because you would think that just like words on a database,
Speaker 1: like what did rot? So what's actually going on there?
Speaker 1: So the sardonic but true answer that you have to
Speaker 1: think of its scale is like bits in a computer
Speaker 1: can literally flipped by gamma rays coming from outer space
Speaker 1: that interact with like the physical manifestation of your your
Speaker 1: memory in the computer. And that's one cause of this
Speaker 1: that's true. That does happen, and that isn't like the
Speaker 1: dominant thing that happens. The dominant thing that happens is
Speaker 1: like there exists change in the broader system that must
Speaker 1: happen on any given basis. Change is a sort of risk.
Speaker 1: It is not always managed well. Um. This gets back
Speaker 1: to that's like a commanding majority of systemic downtime at
Speaker 1: well managed software companies is caused by attempts to upgrade
Speaker 1: the system that go less than optimally. Is the thing
Speaker 1: that is amenable to study, Like a bit happens in
Speaker 1: some cases because you know you had a constellation of software, etcetera.
Speaker 1: Installed on your machine and installed on other machines that
Speaker 1: your machine connected to, which was working. You might say
Speaker 1: exactly perfectly, exactly perfect because unknown as software, but like
Speaker 1: it was working right now. Something about the constellation changed
Speaker 1: as a result of a decision made about a machine
Speaker 1: that is not directly under your control, and that decision
Speaker 1: must be made at scale in the economy because software
Speaker 1: can't be software can't allow to be static to deliver
Speaker 1: the things that we want from software. Is a society,
Speaker 1: and then that change caused some other part of the
Speaker 1: system to behave in a less great manner, and then eventually,
Speaker 1: you know, you see the ripple effects of it in
Speaker 1: your daily life. That's the dominant way bit rot happens.
Speaker 1: It is not the bits actually getting corrupted over time,
Speaker 1: but again a thing that does happen, and we have
Speaker 1: things in engineering to control against that. Well, Patrick, you
Speaker 1: are the perfect guest for this topic. Really appreciate you
Speaker 1: coming out on odd locks. I really appreciate you having
Speaker 1: me and would be glad to be back sometimes. Definitely,
Speaker 1: Thanks so much, Patrick, that was great. I learned so
Speaker 1: many excellent new terms like bit rot, blast, radius, heroics.
Speaker 1: We mutually failed together. That one will come in hand
Speaker 1: Ea Joe, every everything we do wrong mutual. No. I
Speaker 1: love that, Patrick, Thank you so much, Thanks very much,
Speaker 1: happing Tracy. I think really Patrick was like the perfect
Speaker 1: guest for that topic, Like we needed to do this
Speaker 1: for a while. I'm glad we like did it with Patrick. Yeah. Well,
Speaker 1: I also feel like this is something that's going to
Speaker 1: keep coming up, and so we might have more opportunities
Speaker 1: from Patrick to do uh interesting post mortems on various
Speaker 1: tech failures. Yeah. Absolutely, I mean there's so many interesting things.
Speaker 1: I really like some of the questions that you asked about,
Speaker 1: like ownership of software, and I think like it's sort
Speaker 1: of like that really click to me because like if
Speaker 1: you're a software company and software is the main product,
Speaker 1: and you know, I'm thinking about like in the manufacturing analogy,
Speaker 1: you know, it's like a Taiwan semiconductor, this sort of
Speaker 1: like institutional knowledge to build something exists within the firm
Speaker 1: and it just you know gets handed down, handed down
Speaker 1: when software isn't your main product, Like if you're a
Speaker 1: Southwest like if you're a city group, etcetera. Then you
Speaker 1: can sort of see why that process of like internal
Speaker 1: knowledge that works in manufacturing, you don't get that sort
Speaker 1: of ongoing feedback, you know, sort of distribution of knowledge
Speaker 1: in some of these large organizations. Yeah. Absolutely, And it
Speaker 1: feels like, I mean Patrick kind of I think he
Speaker 1: used the expression like dropping the ball in the middle
Speaker 1: of both of us. But it does seem like that
Speaker 1: system kind of produces opportunities for I guess I'm trying
Speaker 1: to think how to phrase this for people to sort
Speaker 1: of like, no one takes total responsibility for a systems
Speaker 1: failure like that, right, because on the one hand, someone
Speaker 1: designed the software, but on the other hand, maybe it
Speaker 1: was customized by someone else. Maybe you have two different
Speaker 1: systems talking to each other and both of them mess
Speaker 1: up in some way, or there's some sort of misunderstanding.
Speaker 1: It just seems like there's such a like gray area,
Speaker 1: and maybe this is one of the reasons why it's
Speaker 1: so difficult to fix, because you have all these different
Speaker 1: things that are sort of operating together. Well, even like
Speaker 1: you know, like his last answer about how bit rot happens, right,
Speaker 1: like somewhere somewhere, like all these interconnected computers, somewhere someone
Speaker 1: has to make a change. Because he pointed out in
Speaker 1: his answer to your question about why software is never
Speaker 1: like done nor why it's never solved problem. It's like
Speaker 1: we're always demanding more so there will never be a
Speaker 1: time where someone like has the luxury of not making
Speaker 1: a change and then everyone else has to. Intercomputer we
Speaker 1: figured out technology is solved. Well, maybe that's what the AI, like,
Speaker 1: what's it, you know, the singularity, GPT it could be,
Speaker 1: but like, you know, that seems like it makes a
Speaker 1: lot of sense. Someone has to make a change because
Speaker 1: that's just how the world works. And then all these
Speaker 1: other interconnected systems like maybe they're fine with the change,
Speaker 1: but something happens and then eventually they have to change too,
Speaker 1: and so it's just like this constant state of flux.
Speaker 1: Can I tell you my one internalized programming lesson? So, um,
Speaker 1: you know I mentioned HTML and then when I was
Speaker 1: in high school, part of our computer science class we
Speaker 1: had to learn JavaScript and we had to create a program.
Speaker 1: And so I wrote this program. Again, like keep in
Speaker 1: mind that this was the year two thousand or something
Speaker 1: like that. I wrote a program. It was like a
Speaker 1: digital fortune cookie, and you know, you could click on
Speaker 1: it and it would give you a fortune. And then
Speaker 1: at the end of it, at the end of this module,
Speaker 1: we had to sign a contract signing over our program
Speaker 1: to our computer teacher took ownership. No, it was an
Speaker 1: extremely valuable lesson, which is all the coding that you're
Speaker 1: doing will ultimately belong to someone else and they'll be
Speaker 1: able to monetize it. That's the downside, as they'll monetize
Speaker 1: it for you and maybe you won't get as much.
Speaker 1: But the upside, I guess, is that you don't have
Speaker 1: to take responsibility for it. Once you write it, it
Speaker 1: goes out into the world. The computer professor owns it
Speaker 1: and he can do with it. I love that that
Speaker 1: was actually the lesson. Also, I feel like in another universe,
Speaker 1: like you could have sold that startup for a hundred
Speaker 1: million dollars to face book and it went like super viral.
Speaker 1: It seems like, how did this person make their fortune?
Speaker 1: They made like a digital fortune cookie, but they also
Speaker 1: like I loved his answer about like the public sector
Speaker 1: because it's like, there, you really do have this problem
Speaker 1: with like salary disparities, and it is pretty crazy that
Speaker 1: we sort of treat like the way we're sort of
Speaker 1: solving this problem in this country is kind of getting
Speaker 1: people to volunteer, like going to work for the government
Speaker 1: in I T and tech. It's like what you do
Speaker 1: after you're like rich and you want to give something
Speaker 1: back is like, Okay, I'm gonna like go work for
Speaker 1: the federal government and like try to help them like
Speaker 1: update their systems. Which is great that people want to
Speaker 1: do that, and like I love that, but like that
Speaker 1: does not seem like a great sustainable solution to having
Speaker 1: like a modern government that can like communicate with people
Speaker 1: and provide services for people in the way that they expect. No, Absolutely,
Speaker 1: and it's something that we see again and again in
Speaker 1: various ways. Shall we leave it there, Let's leave at
Speaker 1: the Okay, this has been another episode of the All
Speaker 1: Thoughts podcast. I'm Tracy Alloway. You can follow me on
Speaker 1: Twitter at Tracy Alloway and I'm Joe Why Isn't Though.
Speaker 1: You can follow me on Twitter at the Stalwart. Follow
Speaker 1: our guest Patrick McKenzie. He's on Twitter at Patio eleven
Speaker 1: and check out his Bits about Money newsletter. Follow our
Speaker 1: producers Carmen Rodriguez at kerman Ermine and Dash Bennett at Dashbot.
Speaker 1: And check out all of the podcasts here at Bloomberg
Speaker 1: under the handle at podcasts and For more odd Lots content,
Speaker 1: go to Bloomberg dot com slash odd Lots, where we
Speaker 1: push the transcripts of the episodes. Tracy and I blog,
Speaker 1: and we have a weekly newsletter that comes out every Friday.
Speaker 1: Go there, sign up and get into your inboxes. Thanks
Speaker 1: for listening to ye to