Speaker 1: Welcome to tech Stuff, a production from iHeartRadio. Hey there,
Speaker 1: and welcome to tech Stuff. I'm your host, Jonathan Strickland.
Speaker 1: I'm an executive producer with iHeartRadio, and how the tech
Speaker 1: are you? It is time for a tech Stuff classics episode.
Speaker 1: This episode originally published on December seventh, twenty sixteen. It
Speaker 1: is called bad Computer Bugs. No, it was twenty sixteen,
Speaker 1: the year where like bedbugs became a big news item.
Speaker 1: Like there was a year where it just bed bugs
Speaker 1: were in the news, and I'm wondering if that was
Speaker 1: twenty sixteen. At this point it goes to show that
Speaker 1: I haven't actually listened back to this episode, but yeah,
Speaker 1: Bad Computer Bugs. It originally published December seventh, twenty sixteen.
Speaker 1: Let's take a listen, now, shall we. I have to
Speaker 1: address a bit of apocryphal history, and regrettably it's a
Speaker 1: story that we've repeated on tech Stuff. So I'm sad
Speaker 1: to admit that I was complicit, although unknowingly, in the
Speaker 1: spread of misinformation, and that all has to do with
Speaker 1: the origin of the term bug to describe a flaw
Speaker 1: in programming. So here's the popular story, the one that
Speaker 1: we have accidentally promoted on tech stuff without knowing that
Speaker 1: we were in the wrong. It goes that Grace Hopper,
Speaker 1: who was an early computer scientist who rose to the
Speaker 1: rank of Rear admiral in the US Navy, coined the
Speaker 1: phrase bug after discovering a moth guming up Harvard's Mark
Speaker 1: II calculator, a literal bug. Generally speaking, the story tends
Speaker 1: to be set in nineteen forty five, and there is
Speaker 1: even a note in the logbook that reads first actual
Speaker 1: case of bug being found that's attributed to Grace Hopper.
Speaker 1: But there are several points that are wrong in this story. First,
Speaker 1: the year. It didn't happen in nineteen forty five. It
Speaker 1: happened on September ninth, nineteen forty seven. We know because
Speaker 1: there's a logbook. At the logbook that marks the incident
Speaker 1: not only has the notes, it actually has the moth
Speaker 1: taped into the book itself. It's taped onto the page. Second,
Speaker 1: Grace Hopper wasn't the person to discover the moth or
Speaker 1: make that log entry. She did tell the story about
Speaker 1: the moth several times, but it wasn't in the context
Speaker 1: of finding it or logging it. She just told the
Speaker 1: story that, yeah, we really did have a bug in
Speaker 1: the system, and most importantly, the word bug had already
Speaker 1: been used to describe design flaws for decades before the
Speaker 1: Mark II was even designed. In fact, if you look
Speaker 1: at the logbook, this makes sense. It says first actual
Speaker 1: case of bug being found. That sentence doesn't make sense
Speaker 1: unless you've already used the word bug to describe a flaw,
Speaker 1: because you wouldn't say first actual case of bug being found.
Speaker 1: The wording doesn't make any sense. The context makes no sense. Sadly,
Speaker 1: there are documented quotes dating back to the nineteenth century
Speaker 1: using the word bug to mean a design fault, and
Speaker 1: it could go back even further than that. So is
Speaker 1: with much regret that I admit I have unwittingly contributed
Speaker 1: to a bit of misleading folklore making the rounds. But
Speaker 1: I'm glad I can take this opportunity to address it.
Speaker 1: All Right, So let's talk about design bugs, and I'll
Speaker 1: be covering several goofs, mistakes, flubs, flaws, and outright catastrophes
Speaker 1: in this episode. But one thing I'm not necessarily going
Speaker 1: to cover our software vulnerabilities that were later exploited either
Speaker 1: by opportunistic hackers or white hats who are just trying
Speaker 1: to improve system security. Those vulnerabilities are common in many
Speaker 1: types of software and arise not just through mistakes, but
Speaker 1: sometimes simple oversights. And I think it might be more
Speaker 1: fun to look at some real bugs, like stuff that
Speaker 1: made things go wrong, stuff that may have rendered a
Speaker 1: program defunct or otherwise caused headaches. Now I'm going to
Speaker 1: make an exception to this. I'm going to start off
Speaker 1: with the Ping of death. And I only mention it
Speaker 1: because it has an awesome name. Now, this flaw caused
Speaker 1: headaches back in nineteen ninety five and ninety six. It
Speaker 1: was a flawed IP fragmentation reassembly code, and it became
Speaker 1: possible to crash lots of different types of computers using
Speaker 1: different operating systems, although Windows machines were particularly vulnerable, and
Speaker 1: this particular flaw would make a Windows machine revert to
Speaker 1: the dreaded blue screen of death. And it all happened
Speaker 1: by sending a special ping packet over the Internet. So,
Speaker 1: for those of you who aren't familiar with what that is,
Speaker 1: a ping is essentially a simple message that checks for
Speaker 1: a connection between two computers. You send one ping from
Speaker 1: a computer to a another one and look for a
Speaker 1: response so that way you verify there is in fact
Speaker 1: a connection. You can also tell other things like how
Speaker 1: fast is that connection between those two computers. Now, in
Speaker 1: this case, you would have to actually design a malformed
Speaker 1: ping request and send that to a target and it
Speaker 1: would bring that target down. That's the only security vulnerability
Speaker 1: story I really wanted to focus on. The others are
Speaker 1: all just design flaws. And let's begin with the bug
Speaker 1: that inspired me to do this episode in the first place,
Speaker 1: that Spotify bug I mentioned earlier. Ours Tetnica wrote a
Speaker 1: piece on it in November twenty sixteen, but the problem
Speaker 1: seems to date back at least as far as June
Speaker 1: twenty sixteen, and that's when a few savvy Spotify users
Speaker 1: noticed some unusual activities on their computers, and it took
Speaker 1: a little bit of detective work, but they discovered that
Speaker 1: Spotify was apparently generating a huge amount of data on
Speaker 1: a daily basis, like gigabytes of data per day. And
Speaker 1: the culprit turned out to be a vacuum process for
Speaker 1: a database file containing the string mercury dot dB. Now,
Speaker 1: the vacuum process is the digital equivalent of vacuum ceiling.
Speaker 1: It's meant to repack data so that it takes up
Speaker 1: less space on a drive. Now, this involves building a
Speaker 1: new file to maximize efficiency, which is a good thing
Speaker 1: generally speaking. The problem was that Spotify's version was making
Speaker 1: it happen way too frequently, like on the order of
Speaker 1: once every few minutes. So that's not generally necessary. You
Speaker 1: don't need to rebuild a database file every few minutes
Speaker 1: to make sure it's the most efficient size it can be.
Speaker 1: So each rebuild represented a relatively small amount of data,
Speaker 1: but over time it added up, which meant that if
Speaker 1: you had Spotify on on your computer, even if it
Speaker 1: was just running in the background, it would be generating
Speaker 1: gigabytes worth of information rewriting this file over and over. Now,
Speaker 1: it wasn't filling up a hard drive. It was just
Speaker 1: overwriting the same file. Now, if it had been filling
Speaker 1: up a hard drive, people would have noticed much earlier,
Speaker 1: and it wouldn't have just been savvy Spotify users, because
Speaker 1: you would suddenly notice, hey, I can't save anything to
Speaker 1: my hard drive because everything's filling up. Instead, again, it
Speaker 1: was just sort of writing and deleting, and writing and
Speaker 1: deleting the same file over and over again, and that
Speaker 1: probably doesn't sound like a big deal, but it is
Speaker 1: a problem if you're using a solid state drive or SSD.
Speaker 1: So one of the drawbacks of an SSD is that
Speaker 1: over time it loses storage capacity. Like you can store
Speaker 1: less data on an SSD over time. Now, by overtime
Speaker 1: I generally mean over a great deal of time and
Speaker 1: a lot of different data being written to it and overwritten.
Speaker 1: Generally speaking, most of us end up replacing our drives
Speaker 1: before we get to a point where the loss of
Speaker 1: capacity is a real issue. But similar in a way
Speaker 1: to how a battery can lose its ability to hold
Speaker 1: a full charge after you've gone through lots of charging
Speaker 1: and discharging cycles, you know how a battery won't be
Speaker 1: able to hold as much even if it says it's
Speaker 1: up to one hundred percent, But one hundred percent doesn't
Speaker 1: last you as long as it used to. That's because
Speaker 1: its capacity to hold a full charge has decreased over time.
Speaker 1: But let's say you've got a program that's just constantly
Speaker 1: overwriting data to your drive, you might discover that your
Speaker 1: SSD's useful lifespan has been drastically reduced. So as I
Speaker 1: record this episode, Spotify has already rolled out an updated
Speaker 1: version of its desktop application, and that, by the way,
Speaker 1: is the only version of Spotify that was affected. If
Speaker 1: you use web based Spotify or mobile Spotify, you're in
Speaker 1: the clear already. If you use a desktop version, as
Speaker 1: long as you have version one point zero point four
Speaker 1: to two or later, you are fine. But if you
Speaker 1: did have that earlier version and you just had Spotify
Speaker 1: running on in the background, chances are it was writing
Speaker 1: to your hard drive like crazy. So what about some
Speaker 1: of the other big bugs in computer history. Well, some
Speaker 1: of the real doozies involve our attempts to explore the
Speaker 1: final frontier. So we'll be talking about space a few
Speaker 1: times in this episode, and we'll start with an early
Speaker 1: US satellite. So first up is a nineteen sixty two
Speaker 1: blunder involving the Mariner one. So some backstory on this one.
Speaker 1: We're going to talk a lot about the Soviet Union
Speaker 1: in this episode too. It takes a couple of roles
Speaker 1: as we go on, But in this case, the then
Speaker 1: USSR had launched Sputnik into orbit in nineteen fifty seven,
Speaker 1: which really kicked off the space race and also was
Speaker 1: a big shot in the Cold War because the Soviet
Speaker 1: Union was essentially saying, hey, we can launch this into space,
Speaker 1: we could also launch something at you. In response, the
Speaker 1: US done sort of the same thing. They had launched
Speaker 1: some satellites into space, and the Mariner one was going
Speaker 1: to be a big, big feather in the cap of
Speaker 1: the US. The whole idea was to launch a probe
Speaker 1: that would be a flyby probe and it would go
Speaker 1: by Venus. So NASA, which was newly formed in nineteen
Speaker 1: sixty two, was taking control of this, and the budget
Speaker 1: for this particular project was eighteen point five million dollars, which,
Speaker 1: if you were to adjust for inflation, would be almost
Speaker 1: one hundred and fifty million dollars today, So one hundred
Speaker 1: and fifty million dollar project to launch the Mariner one
Speaker 1: and have it fly by Venus. But, as I'm sure
Speaker 1: you guys have figured out by now based upon the
Speaker 1: topic of this podcast, not all went according to plan.
Speaker 1: Not long at all. After the rocket launched from the
Speaker 1: launch pad, it began to veer off course, and neither
Speaker 1: the computer controls on the rocket or manual controls back
Speaker 1: at HQ could correct for the problem. The rocket's course
Speaker 1: was such that it was going to take it over
Speaker 1: shipping lanes, which meant there could be a potential catastrophe,
Speaker 1: and so a range safety officer made the difficult call
Speaker 1: and issued the command to blow the whole thing up
Speaker 1: just shy of three hundred seconds after it launched. So
Speaker 1: what happened? What why did it go off course in
Speaker 1: the first place? Well, there was a flaw in the
Speaker 1: spacecraft's guidance software which diverted the rocket, and no amount
Speaker 1: of commands from ground control could correct for it. After
Speaker 1: a lengthy investigation, NASA discovered the error was the result
Speaker 1: of a mistake transcribing handwritten notes into computer code. So
Speaker 1: someone just took some handwritten notes and misinterpreted one of them,
Speaker 1: and that one mistake was enough to crash the rocket
Speaker 1: or to necessitate it being destroyed. The great science fiction
Speaker 1: author Arthur C. Clark wrote that the Mariner one was
Speaker 1: wrecked by the most expensive hyphen in history, which isn't
Speaker 1: quite right, but it's pretty funny, I mean, come on,
Speaker 1: its humorous phrase. So the actual punctuation mark that caused
Speaker 1: the problem was not technically a hyphen. It was a
Speaker 1: superscript bar. Superscript bars, by the way, not a place
Speaker 1: where playwrights hang out. To get tore up. A superscript
Speaker 1: bar just means it's a horizontal bar that is above
Speaker 1: some other symbol. In this case, it was a radius symbol,
Speaker 1: and that was a symbol along with the superscript bar
Speaker 1: to describe a smoothing function, which means the formula was
Speaker 1: meant to calculate smoothed values over the time derivative of
Speaker 1: a radius. Now, without the smoothing function, tiny deviations in
Speaker 1: course sent commands to the rocket's thrusters to kick in
Speaker 1: big time to overcorrect for that problem. As an analogy,
Speaker 1: imagine you're driving a vehicle and you see a pothole
Speaker 1: in the road and you're approaching it, and instead of
Speaker 1: gently steering out of the way, you wrench the wheel
Speaker 1: really hard to the left or to the right in
Speaker 1: order to try and get around this pothole. That's kind
Speaker 1: of what was happening with the rocket. It didn't have
Speaker 1: the smoothing function and so as a result, it was
Speaker 1: having these wild deviations and course. So it wasn't a
Speaker 1: hyphen that caused the problem, was close enough. Our next
Speaker 1: space story takes place in nineteen ninety six with the
Speaker 1: European Space Agency's Ari Anne five flight five oh one rocket. Now,
Speaker 1: this rocket was to launch into space on June fourth,
Speaker 1: nineteen ninety six, and instead the rocket disintegrated forty seconds
Speaker 1: after taking off. So what the heck happened? Well, it
Speaker 1: largely had to do with the ESA reusing old work.
Speaker 1: This actually becomes a theme in this episode. One of
Speaker 1: the morals the of this entire podcast is, if you're
Speaker 1: designing something a successor to an earlier product, and you'd
Speaker 1: want to reuse some of the features that you created
Speaker 1: in your previous product, test the heck out of it
Speaker 1: in its new form, factor, because it could be that
Speaker 1: things that worked perfectly fine in the earlier model will
Speaker 1: go awry in the new one. That's what happened here. So,
Speaker 1: as you might guess from the name, the Ariyan five
Speaker 1: marked the fifth generation of launch vehicles under that name.
Speaker 1: The Arian four's inertial reference system would convert sixty four
Speaker 1: bit floating point numbers into a sixteen bit signed integer,
Speaker 1: and it worked just fine. But the Arian five's stats
Speaker 1: were beefier than its predecessor with faster engines, and that
Speaker 1: was where the problem really started. The engine output meant
Speaker 1: those sixty four bit floating point numbers were significantly larger
Speaker 1: than the ones generated by the engines on the Arean four.
Speaker 1: They didn't anticipate this, so during the conversion process there
Speaker 1: was actually data overflow, and that overflow caused both the
Speaker 1: backup computer and the primary computer board the Area N
Speaker 1: five to crash, and they crashed in that order. The
Speaker 1: backup computer crashed first, followed by the primary computer a
Speaker 1: couple of seconds later. The whole thing took less than
Speaker 1: a minute to go from launch to disintegration. Oops, now
Speaker 1: we're going to stick with space. But jump forward to
Speaker 1: nineteen ninety eight and the Mars Climate Orbiter. This was
Speaker 1: an unfortunate problem. So this particular spacecraft was meant to
Speaker 1: study Mars's climate, atmosphere and surface changes, and it was
Speaker 1: also supposed to be a kind of relay station for
Speaker 1: landers that would explore Mars's surface, but none of that
Speaker 1: would last because of some pretty significant goofs. So on
Speaker 1: September twenty third, nineteen ninety nine, the orbiter passed into
Speaker 1: the upper atmosphere of Mars and did so at a
Speaker 1: pretty low altitude. And this is what folks in the
Speaker 1: space industry call a bad thing. The drag on the
Speaker 1: spacecraft was significant, it began to fall apart and it
Speaker 1: was destroyed upon entering Mars's atmosphere. That's what happened. So
Speaker 1: the software guiding the orbiter was to blame, and it's
Speaker 1: a dumb, dumb mistake. It was supposed to make adjustments
Speaker 1: to the orbiter's flight in SI units, specifically in Newton's seconds.
Speaker 1: That's what the contract with Lockheed and NASA said, Newton seconds,
Speaker 1: use Si units for all of your all of your calculations.
Speaker 1: But the software instead made calculations in non SI units,
Speaker 1: namely pounds seconds. So Lockheed software gave information to NASA's
Speaker 1: systems using the wrong units of measure. NASA systems then
Speaker 1: took that information, assuming it was with the right units
Speaker 1: of measure, and executed commands based upon that. So this
Speaker 1: is why if you're ever in a math course and
Speaker 1: the teacher makes you stop in the middle of writing
Speaker 1: a problem on the board and says, where are your units?
Speaker 1: This is why you have to make sure you're using
Speaker 1: the right units, because if you're saying a number and
Speaker 1: you don't associate a unit with it, someone could make
Speaker 1: an incorrect decision based on that, and it could be disastrous,
Speaker 1: as it was with the case of this orbiter, the
Speaker 1: thrusters fired at four point four or five times the
Speaker 1: power they were supposed to, and the orbiter didn't stand
Speaker 1: a chance. And this was a pre expensive mistake. That
Speaker 1: mission's cost came in at three hundred and twenty seven
Speaker 1: point six million dollars. But on the bright side, with
Speaker 1: all of these stories, at least no human lives were
Speaker 1: ever in real danger as a result of the mistake.
Speaker 1: We're gonna take a quick break, and when we come back,
Speaker 1: we'll talk more about bad computer bugs. All right, Now,
Speaker 1: let's make a switch to AT and T, which is
Speaker 1: a company that had a pretty big problem with switches
Speaker 1: once upon a time. I'm talking about an issue that
Speaker 1: popped up on January fifteenth, nineteen ninety. That's when AT
Speaker 1: and T long distance customers discovered they were unable to
Speaker 1: make any long distance calls. Why why could they no
Speaker 1: longer reach anybody? Well? AT and t's long distance switches,
Speaker 1: which control that and allow for the actual connections to
Speaker 1: be made, were on the frints. They were trying to
Speaker 1: reboot over and over again. They were just stuck in
Speaker 1: a reboot cycle. Now, initially the company thought it was
Speaker 1: being hacked, but like I said at the top of
Speaker 1: the show, I'm not covering stories about hackers here. I'm
Speaker 1: talking about big design flaws that caused problems. So they
Speaker 1: weren't getting hacked. That's not what was going on with
Speaker 1: those one hundred and fourteen long distance switches. No, there
Speaker 1: was a design problem at fault. So what had happened
Speaker 1: was AT and T had rolled out an update to
Speaker 1: the code that managed the switches, and it was meant
Speaker 1: to increase the efficiency. It was meant to speed things up.
Speaker 1: But the problem was it sped things up so much
Speaker 1: that the system got caught up in itself. It gets
Speaker 1: pretty technical, but I can give you kind of an
Speaker 1: overview of what the problem was. All right, So each
Speaker 1: switch had a function that allowed it to alert the
Speaker 1: next switch down the line if things were starting to
Speaker 1: get hairy. So imagine that switch number one is handling traffic,
Speaker 1: but it's getting really close to capacity. So it sends
Speaker 1: a message over to switch number two and says, I
Speaker 1: can't take on any more work because if I do,
Speaker 1: I'll be overloaded. Switch too then says, no problem, I'll
Speaker 1: take on any oncoming work for you and we'll handle
Speaker 1: it from there. And if switched number two were to
Speaker 1: get into the same sort of situation, it would say
Speaker 1: the same thing to switch number three, and so on
Speaker 1: and so forth. Now, eventually each switch will contact the
Speaker 1: one below it and say, hey, how you doing there,
Speaker 1: And if the answer is okay, then everything switches back
Speaker 1: and you go back to normal operation. That's how it's
Speaker 1: supposed to work up. But AT and t's updated code
Speaker 1: sped things up so much it caused some real issues,
Speaker 1: and there was some poor timing, just coincidental timing that
Speaker 1: made things worse. So switch number one starts to get
Speaker 1: overwhelmed and sends a message over to switch number two,
Speaker 1: But switch number two was just in the middle of
Speaker 1: resetting itself, So switch number two goes into reset mode,
Speaker 1: which says do not disturb. Sends a message over to
Speaker 1: switch number three. That prompted switch number three to overload
Speaker 1: and put up a do not disturb sign. Move that
Speaker 1: down to switch number four. This whole thing goes down
Speaker 1: the entire line of one hundred and fourteen switches. They
Speaker 1: all end up getting overloaded as a result of this,
Speaker 1: and all go into reset mode and they get stuck there.
Speaker 1: That problem lasted for nine hours before be for AT
Speaker 1: and T was finally able to address the message load
Speaker 1: on the entire system and get the switches back to normal.
Speaker 1: The estimated cost of lost revenue for that time was
Speaker 1: about sixty million dollars in long distance calls, and there
Speaker 1: were a lot of angry customers to boot, so to
Speaker 1: placate them, AT and T offered reduced long distance rates
Speaker 1: on Valentine's Day. Pretty ugly, but AT and T tried
Speaker 1: to handle it, at least in a way that didn't
Speaker 1: turn it into a pr nightmare. Not so with Intel.
Speaker 1: That's what brings us to the Pendium problem. I don't
Speaker 1: know if you guys remember when pentium processors first came out,
Speaker 1: but they were a big deal. It was a redesign
Speaker 1: of the architecture of the microprocessor and it was meant
Speaker 1: to really speed things up. Well, Intel had a massive
Speaker 1: nightmare in nineteen ninety four thanks to a flaw in
Speaker 1: the entire first generation of Pentium processors. Now, when you
Speaker 1: break it all down, a CPU is all about performing math,
Speaker 1: medical operations on data, so it's kind of important that
Speaker 1: it does this correctly. Unfortunately, the flaw on the Pentium
Speaker 1: processors kind of messed that up, and the issue has
Speaker 1: to do with floating point operations. So the predecessor to
Speaker 1: the Pentium, the four eighty six, used a shift and
Speaker 1: subtract algorithm for floating point operations, which was effective but
Speaker 1: relatively slow compared to what Intel thought they could do
Speaker 1: by totally redesigning that structure and using a lookup table approach. Now,
Speaker 1: the table was supposed to have one thousand and sixty
Speaker 1: six entries programmed directly onto the logic array of the
Speaker 1: Pentium processor, but for some reason only one thousand and
Speaker 1: sixty one entries made it. Five entries went missing and
Speaker 1: essentially returned an answer of zero instead of what they
Speaker 1: were supposed to say, so if a calculation accessed one
Speaker 1: of those missing cells, it got zero, even though that's
Speaker 1: not the correct answer. All the first generation pentiums went
Speaker 1: out with this error because it was so minor that
Speaker 1: it wasn't even picked up by Intel's quality control at
Speaker 1: the time. Now, processes worked just fine up to the
Speaker 1: eighth decimal point. Beyond that things got messy, but for
Speaker 1: most folks that wasn't a problem because they weren't doing
Speaker 1: mathematical calculations that needed that level of precision. It just
Speaker 1: wasn't a thing. In fact, there was only a one
Speaker 1: in three hundred and sixty billion chance that this error
Speaker 1: would cause a big enough problem to reach up to
Speaker 1: the fourth decimal place. So most calculations that were simple
Speaker 1: were bulletproof. You were fine. But if you needed that precision,
Speaker 1: if you needed that really fine degree, that's when you
Speaker 1: would encounter the flaw. And that happened because they're math
Speaker 1: professors in this world, and one of those, Thomas Nicely
Speaker 1: discovered in October nineteen ninety four that he was getting
Speaker 1: errors because of this issue. He needed the processor to
Speaker 1: work correctly, and so he contacted Intel about the problem.
Speaker 1: And this is where we take a moment to acknowledge
Speaker 1: there's a right way and a wrong way to handle
Speaker 1: an issue. That's your fault. Intel decided to go the
Speaker 1: wrong way. My opinion is if you make a mistake,
Speaker 1: it's usually a good idea to just own up to
Speaker 1: it and try to make it better. But Intel's response
Speaker 1: was more along the lines of, yeah, we didn't think
Speaker 1: it was a big deal. And then Intel made other
Speaker 1: pr blenders. But because people began to hear, hey, that
Speaker 1: pentium processor in your computer that you just bought it
Speaker 1: doesn't work properly. So people wanted to get replacements, but
Speaker 1: Intel said, oh, we're only going to replace the ones
Speaker 1: if you can prove that the mistakes that it makes
Speaker 1: affect you in some meaningful way. So they weren't They
Speaker 1: weren't denying that there was a problem. They were just saying, hey,
Speaker 1: unless you can prove the problem affects you, we don't care.
Speaker 1: That didn't go well. If you create a product and
Speaker 1: you market it as the future of computing and then
Speaker 1: it's discovered there's a flaw in the design, and then
Speaker 1: you say we'll replace it, but only if you prove
Speaker 1: you deserve it, it doesn't tend to make your customer
Speaker 1: base very happy. So ultimately Intel reverse that decision and
Speaker 1: offered to replace the processor for anyone who wanted it
Speaker 1: who had a first generation Pentium, and that mistake ended
Speaker 1: up costing the company four hundred and seventy five million dollars. Yikes.
Speaker 1: All right, now we're gonna switch gears over to Microsoft. First.
Speaker 1: I think you could claim that all of Microsoft Bob,
Speaker 1: the nineteen ninety five product that was supposed to be
Speaker 1: an easy accessible computer interface, was really just a massive
Speaker 1: software bug. I mean it introduced comic sands. For goodness sakes,
Speaker 1: the cluttered organization system, the lack of meaningful security, and
Speaker 1: other numerous issues plagued that software. But we did an
Speaker 1: entire episode of tech Stuff about Microsoft Bob a couple
Speaker 1: of years ago, So I'm not going to dwell on
Speaker 1: it anymore, but if you want to hear more about it,
Speaker 1: go find that episode. It was a fun one. Now.
Speaker 1: In two thousand and seven, Microsoft experienced a massive headache
Speaker 1: when a bug on their servers notified thousands of Windows
Speaker 1: customers that they were filthy, dirty software pirates and they
Speaker 1: should be punished. These include people who actually had legitimate
Speaker 1: legal purchase copies of Windows XP or Vista. So the
Speaker 1: problem here was Microsoft had an initiative called Windows Genuine Advantage,
Speaker 1: and it was a nice name for a strategy meant
Speaker 1: to curtail operating system piracy. Essentially, it was a component
Speaker 1: in Windows that would allow Microsoft to figure out if
Speaker 1: the copy of Windows on any given computer was legit.
Speaker 1: In other words, it was a DRM strategy. But in
Speaker 1: two thousand and seven, a buggy install of software on
Speaker 1: a server misidentified thousands of legitimate, law abiding customers as pirates.
Speaker 1: For nineteen hours, the software just laid down the law,
Speaker 1: and so people began to receive sternly written warnings about
Speaker 1: their choice to indulge in bad behavior. And if you
Speaker 1: were a Windows Vista customer, you had it the worst,
Speaker 1: not just because you were using Windows Vista, which I
Speaker 1: think we all agree was not one of the bright
Speaker 1: points in Microsoft's operating system history, but also because Microsoft
Speaker 1: had built in the ability for Windows Genuine Advantage to
Speaker 1: switch off certain operating system features in Windows Vista if
Speaker 1: it determined that the copy someone was using was a
Speaker 1: pirated version. So it was misidentifying real versions as pirated ones,
Speaker 1: turning off features. And these are for people who have
Speaker 1: bought legitimate copies. This, by the way, is one of
Speaker 1: the big arguments people have against DRM. It has the
Speaker 1: tendency to punish legitimate CUS customers. And you feel like
Speaker 1: you're stupid for buying a copy of a piece of
Speaker 1: software rather than just stealing one that has had those
Speaker 1: features or those defenses removed. Like why you're creating more
Speaker 1: incentives for people to go outside and get a pirated copy.
Speaker 1: All right, so imagine you've purchased this legitimate copy of
Speaker 1: Windows Vista. First of all, you already feel bad. Then
Speaker 1: you're told you're a thief, so you feel worse. Then
Speaker 1: someone remotely switches off several features of your operating system.
Speaker 1: That was not a great PR message, So that was
Speaker 1: a real issue. They did eventually fix it after that
Speaker 1: nineteen hours, but by then people were already very upset. Also,
Speaker 1: I don't want to just you know, pile lots of
Speaker 1: abuse onto Microsoft. I gotta talk about Apple here too.
Speaker 1: So the company prides itself on a high standard of quality,
Speaker 1: and in general it's pretty good about living up to
Speaker 1: that standard of quality, depending upon your point of view
Speaker 1: of their various products. But that hasn't stopped a few
Speaker 1: clunkers getting through and into the public hands. And that
Speaker 1: was the case in twenty twelve with Apple Maps. If
Speaker 1: you owned an iPhone back in twenty twelve when Apple
Speaker 1: Maps came out, you may remember this problem. It's pretty
Speaker 1: well publicized. Maps were inaccurate, sometimes leaving out important details, like,
Speaker 1: you know, a river or a lake between you and
Speaker 1: your destination, things that might be important if I don't
Speaker 1: know you don't drive an amphibious vehicle, might not have
Speaker 1: a road on there that's important, might misidentify the location
Speaker 1: of a historical landmark. For instance, it thought the Washington
Speaker 1: Monument was across the street from where it is, but nope,
Speaker 1: it's just where we left it. Despite all of Rolanimerick's
Speaker 1: best attempts to move it or destroy it, it's still there.
Speaker 1: The real problem here was that the Apple software just
Speaker 1: wasn't ready for public unveiling. It needed a lot more testing.
Speaker 1: It was trying to play catch up to Google Maps,
Speaker 1: but Google had the advantage of working with companies that
Speaker 1: had been doing mapping software for years. Google acquired those
Speaker 1: companies and acquired the expertise of people had been working
Speaker 1: on that software, and Apple was really just trying to
Speaker 1: create their own version and get it out as fast
Speaker 1: as it could. But it got out a little too early,
Speaker 1: and the company spent the next several months tweaking maps
Speaker 1: and trying to keep control of the situation. But by
Speaker 1: that time, many of Apple's fans, even the most devoted ones,
Speaker 1: had kind of given up and switched over to Google
Speaker 1: Maps instead. All right, we've got another break ahead of us,
Speaker 1: but don't worry. Once that's done, we're back to conclude
Speaker 1: our discussion about bad computer bugs. Now I'm going to
Speaker 1: transition into some serious bugs. These are ones that either
Speaker 1: threaten the lives of people or they contributed to people dying.
Speaker 1: The ones I've talked about now up to out rather
Speaker 1: have cost companies millions of dollars, but no one's life
Speaker 1: was truly threatened. Unfortunately, that's not the case with all
Speaker 1: software bugs. Now, a couple of bugs had the potential
Speaker 1: to kill millions of people. One of those happened in
Speaker 1: nineteen eighty a famous famous bug, or at least a
Speaker 1: faulty circuit, and that was a faulty circuit in Norrad's
Speaker 1: computer system, which caused it to mistakenly conclude the US
Speaker 1: was under nuclear attack from the Soviet Union. So displays
Speaker 1: on nor Ad systems showed seemingly random attacks, and they
Speaker 1: didn't correspond with each other. So the display might show, Hey,
Speaker 1: they're two missiles heading over from the Soviet Union. No,
Speaker 1: they're two hundred. No they're fifty. No there's three. And
Speaker 1: it wasn't consistent, and command posts around the US all
Speaker 1: had conflicting information, which led leaders to conclude the whole
Speaker 1: thing was a regrettable computer error, and they were right
Speaker 1: to do so. To be fair, they were kind of
Speaker 1: prepared for this because there was another incident that had
Speaker 1: actually happened in nineteen seventy nine that was way scarier,
Speaker 1: and in that case, someone mistakenly inserted a training scenario
Speaker 1: into the computer system that made it seem like the
Speaker 1: Soviet Union had launched an all out nuclear attack on
Speaker 1: the US. But that wasn't a bug. That was a
Speaker 1: mistake on the part of a human who had accidentally
Speaker 1: uploaded the wrong or rather executed the wrong command. It
Speaker 1: didn't have something to do with a flaw in the
Speaker 1: computer system itself. However, because that thing happened and everybody
Speaker 1: was freaked out and then was able to determine that
Speaker 1: in fact, it was a false alarm, it meant that
Speaker 1: calmer heads could prevail in the nineteen eighty incident, so
Speaker 1: the Soviets also had a close call just a few
Speaker 1: years later. It was a bug in the early warning
Speaker 1: detection software that the USSR was using in the early eighties,
Speaker 1: and on September twenty third, nineteen eighty three, so Union
Speaker 1: received an alert that the US had launched a nuclear
Speaker 1: attack in the form of five nuclear warheads, technically two
Speaker 1: different attacks. The first would have been a single nuclear
Speaker 1: warhead and the second was four nuclear warheads, and this
Speaker 1: was during a particularly stressful period in the history of
Speaker 1: both countries and their relationship with each other, at the
Speaker 1: height of the Cold War nineteen eighty three. Now, Fortunately,
Speaker 1: Soviet Air Defense Forces Lieutenant Colonel Stanislav Petrov suspected that
Speaker 1: this report was an error and that there was some
Speaker 1: sort of bug in the software or a mistake in
Speaker 1: the reporting system that caused this. He gave a command
Speaker 1: to hold off on any sort of retaliatory strike, which
Speaker 1: would have initiated a full scale nuclear war had it happened.
Speaker 1: Petrov was the officer in charge at a bunker that
Speaker 1: served as the command center for this early warning system,
Speaker 1: and he had said afterward that his reckoning was any
Speaker 1: real attack would consist of hundreds of warheads, not five.
Speaker 1: No one would start an attack with just five warheads,
Speaker 1: so it was more likely to be an error than
Speaker 1: a genuine attack, So he gave the command to wait
Speaker 1: until the reported missiles would pass into the range of radar,
Speaker 1: which only extended as far as the horizon, so if
Speaker 1: it had in fact been a real attack, it would
Speaker 1: have potentially limited the Soviet Union's ability to respond. But
Speaker 1: no missile showed up, and he was vindicated in his decision. Now,
Speaker 1: the cause of the false alarm in this case was
Speaker 1: a combination of factors that the designers didn't anticipate, which
Speaker 1: largely consisted of sunlight hitting high altitude clouds at a
Speaker 1: particular angle from a particular perspective of the satellites. So
Speaker 1: the satellites misidentified that reflection as a warhead. Now they
Speaker 1: were where the Soviets were able to address this error
Speaker 1: in the future by adding another step in which these
Speaker 1: satellites would cross reference data from other geostationary satellites to
Speaker 1: make certain that they are identifying actual rockets as opposed
Speaker 1: to high altitude clouds. Now, there are several cases of
Speaker 1: software bugs leading to actual deaths. For example, the Therak
Speaker 1: twenty five was such a case. Now, that was a
Speaker 1: radiation therapy machine that could deliver two different modes of
Speaker 1: radiation treatments. The first was a low powered direct electron
Speaker 1: beam and the second was a megavault X ray beam. Now,
Speaker 1: the X ray beam was far more intense and it
Speaker 1: required physicians to provide shielding to patients to limit exposure
Speaker 1: to the beam. But the Therak twenty five had inherited
Speaker 1: its code from its predecessor, which had different hardware constraints.
Speaker 1: Now the new machine meant that these constraints were aren't there,
Speaker 1: and it created a deadly problem. If operators change the
Speaker 1: machine's mode too quickly from one to the other, it
Speaker 1: would actually send two sets of instructions to the processor,
Speaker 1: one for each mode of operation, and whichever set of
Speaker 1: instructions reached the processor first, that's what the machine would
Speaker 1: switch to. So let's say you've been operating the the
Speaker 1: Act twenty five in the Megavault X ray mode, but
Speaker 1: now you're going to have a patient come in. You
Speaker 1: need to administer radiation therapy, so you want to switch
Speaker 1: it to low electron beam. You switch it too quickly,
Speaker 1: it sends two sets of instructions to the processor, and
Speaker 1: the one that arises the Megavault X ray instruction, So
Speaker 1: instead of switching it, you confirm to stay on the
Speaker 1: more intense, deadlier radiation. The tragic news is this did
Speaker 1: happen several times. Six patients were documented as dying from
Speaker 1: complications due to radiation poisoning from THERA Act twenty five
Speaker 1: machines between nineteen eighty five and nineteen eighty six, and
Speaker 1: while the machine would send error messages when these conditions
Speaker 1: were present. The documentation for the machine didn't explain what
Speaker 1: the errors meant. It didn't say, hey, if you get
Speaker 1: this error, it means that you've switched modes too quickly
Speaker 1: and you need to address this. So, since operators weren't
Speaker 1: told that this was necessarily a hazardous condition, they would
Speaker 1: just clear the error and proceed, and there were deadly results.
Speaker 1: In a similar vein in Panama City, Panama, there was
Speaker 1: an incident involving a Cobalt sixty system, actually several incidents
Speaker 1: involving this Cobalt sixty system that was running therapy planning
Speaker 1: software made by a company called Multi Data Systems International. Now,
Speaker 1: the software's purpose was to calculate the amount of radiation
Speaker 1: that cancer patients should receive in radiation therapy sessions. Now,
Speaker 1: during these radiation therapy sessions, the therapists were meant to
Speaker 1: place metal shields on the patient to protect healthy tissue
Speaker 1: from radiation damage, and the software would allow therapists to
Speaker 1: use a methodology to show where those shields were on
Speaker 1: the patient to indicate where the shields are present, But
Speaker 1: they could only draw up to four shields, and the
Speaker 1: doctors in Panama wanted to use five shields for particular
Speaker 1: therapy sessions. They were overloaded, they had a long waiting
Speaker 1: list of patients, and they were trying to make things
Speaker 1: more efficient, and they discovered that they could kind of
Speaker 1: work around this limitation of four shields by drawing a
Speaker 1: design on the computer screen as if they were using
Speaker 1: just one large shield that has a hole in the
Speaker 1: middle of it. And so what they would do is
Speaker 1: they would arrange the five shields to essentially be in
Speaker 1: the same sort of shape with the middle of it
Speaker 1: being opened, so that they could have the radiation therapy
Speaker 1: pass through it. But they didn't realize that the software
Speaker 1: had a bug in it, and that bug was if
Speaker 1: you drew the hole in one direction, you get the
Speaker 1: correct dose of radiation, but if you drew it in
Speaker 1: the other direction, so like clockwise versus counterclockwise, the software
Speaker 1: would recommend a dosage twice as strong as what was needed,
Speaker 1: and the result was devastating. Eight patients died as a
Speaker 1: result of this, and another twenty received doses high enough
Speaker 1: to potentially cause health problems. Later on, the physicians were
Speaker 1: actually arrested and brought up on murder charges because they
Speaker 1: were supposed to double check all calculations by hand to
Speaker 1: ensure that they were going to give the proper dose
Speaker 1: of radiation treatment. So while the software was calculating the
Speaker 1: incorrect dose, the physicians were responsible for making sure that
Speaker 1: any dose that was calculated was in fact the correct one,
Speaker 1: and they failed to do so, or at least that
Speaker 1: was the charge. There are also bugs that involve military
Speaker 1: applications that have resulted in the loss of life. During
Speaker 1: the Persian Gulf War in Iraq, he fired scud missile
Speaker 1: hit a US base in Saudi Arabia and it killed
Speaker 1: twenty eight soldiers. Now the base had detected the missile
Speaker 1: and had launched and fired a Patriot missile in return.
Speaker 1: The purpose of the Patriot missile was to intercept and
Speaker 1: destroy incoming missiles, and the way a Patriot missile did
Speaker 1: this was to use radar pulses to guide trajectory calculations
Speaker 1: so that it would end up getting close to the
Speaker 1: incoming missile. This is harder than it sounds because both
Speaker 1: missiles are moving very very quickly, so a NEA very
Speaker 1: precise information in order to adjust its trajectory properly and
Speaker 1: make sure it was on target. Now, once it gets
Speaker 1: within range, which is between five and ten meters. I
Speaker 1: think it would then fire out one thousand pellets from
Speaker 1: the Patriot missile at high velocity, with the goal of
Speaker 1: causing the incoming warhead to explode prematurely. In this case,
Speaker 1: the Patriot missile missed, and the military investigated the issue
Speaker 1: in the wake of the loss of life and found
Speaker 1: a problem with the software guiding the Patriot missile. And
Speaker 1: it was a problem that actually the military kind of
Speaker 1: knew about already. So one of the processes in the
Speaker 1: Patriots programming was to convert time into floating point operations
Speaker 1: for increased accuracy, but not all subroutines that depended on
Speaker 1: tracking time did this. Some of them remained clock units
Speaker 1: rather than floating point operations, which meant that they would
Speaker 1: get out of sync after a while. There'd be a
Speaker 1: disagreement in various subroutines as to how much time had
Speaker 1: actually passed. And like I said, the military was aware
Speaker 1: of this issue and they had a workaround which was
Speaker 1: not ideal. The workaround was you would occasionally reboot the system,
Speaker 1: which would reset the clocks and synchronize them, but over
Speaker 1: time they would fall out of sync because they're not
Speaker 1: tracking time the same way. And since there was no
Speaker 1: hard and fast rule as to how frequently you'd reset
Speaker 1: the system, problems like this one were possible, and in fact,
Speaker 1: in this case it did happen. So prior to this
Speaker 1: particular incident, that specific Patriot system had been running for
Speaker 1: one hundred hours without a reboot, and the clock disagreement
Speaker 1: amounted to about one third of a second. Now that
Speaker 1: seems like it's no time at all. One third of
Speaker 1: a second is so short, but a scud missile's top
Speaker 1: speed is about one point one miles per second or
Speaker 1: one point seven kilometers per second, which means if you
Speaker 1: take a third of a second, the missile could travel
Speaker 1: more than five hundred meters. And since a Patriot needs
Speaker 1: to be within ten meters of a target to destroy it,
Speaker 1: that resulted in a catastrophic failure. So software bugs can
Speaker 1: be a matter of life or death. It's not all
Speaker 1: just Hey, this irritating thing meant people couldn't make long
Speaker 1: distance phone calls, or this issue caused my computer to
Speaker 1: start writing massive amounts of data to its hard drive.
Speaker 1: And this is why it's so important to have really
Speaker 1: qualified QA personnel go through code and make sure it's
Speaker 1: doing what it's supposed to do, because the problems that
Speaker 1: can arise can be non trivial and in fact life
Speaker 1: or death situations depending upon the application of technology. So
Speaker 1: technology is a fascinating thing. It's a wonderful thing. It
Speaker 1: has benefited us in ways that I can't even begin
Speaker 1: to describe. It's just too broad a topic, and it's
Speaker 1: something I've been tackling for, you know, eight years, and
Speaker 1: I haven't even gotten close to getting toward a finishing point.
Speaker 1: So I don't want to suggest that technology is bad,
Speaker 1: but we definitely have the need to check, double check,
Speaker 1: and triple check all this work to make certain things
Speaker 1: are working properly before we release them ount into the wild.
Speaker 1: That particularly applies if, again, you are reusing old code
Speaker 1: or old components in a new way, because you have
Speaker 1: to make absolutely certain that there's not going to be
Speaker 1: some unintended problem that results when a new form factor
Speaker 1: is using old code. Alrighty, And that was our Bad
Speaker 1: Computer Bugs episode, which I still haven't listened back to yet.
Speaker 1: So maybe I talked about bedbugs. I probably talked about
Speaker 1: Grace Hopper in that episode. Just did an episode about
Speaker 1: Grace Hopper earlier this year for a Memorial Day remembering
Speaker 1: Grace Hopper and her contributions because of the somewhat apocryphal
Speaker 1: story that she coined the term computer bug, but I
Speaker 1: have a feeling I might have addressed that in this
Speaker 1: episode too. Hopefully I did. If not, make sure you
Speaker 1: find that episode about Grace Hopper. She was a truly
Speaker 1: phenomenal innovator and computer scientist, someone who is extremely important
Speaker 1: in the history of computer science, especially here in the
Speaker 1: United States. Check that out and I hope you enjoyed
Speaker 1: this classic episode. I hope you're all well, and I'll
Speaker 1: talk to you again really soon. Tech Stuff is an
Speaker 1: iHeartRadio production. For more podcasts from iHeartRadio, visit the iHeartRadio app,
Speaker 1: Apple Podcasts, or wherever you listen to your favorite shows.