Speaker 1: Pushkin. Nobody really wants to be deliberately bombarded with radiation,
Speaker 1: but if you have cancer, radiation therapy might just save
Speaker 1: your life. That must have been what a young man
Speaker 1: called Voyin Ray Cox hoped as he made visit after
Speaker 1: visit to the East Texas Cancer Center. Just thirty three
Speaker 1: years old, Ray, to his friends, was young to be
Speaker 1: a cancer survivor. He had a tumor cut out of
Speaker 1: his shoulder, and now March nineteen eighty six, he was
Speaker 1: there for his ninth session of radiation therapy designed to
Speaker 1: ensure that no traces of the cancer remained. Despite his
Speaker 1: bad luck, Ray was a cheerful, resilient man. He knew
Speaker 1: the drill, press his bare chest and stomach onto the
Speaker 1: cold metal treatment table, chat to the operator while she
Speaker 1: maneuvered him into position underneath the looming bulk of the
Speaker 1: THEAK twenty five radiation therapy machine. The operator knew the
Speaker 1: drill too. In his account of the case, the ergonomics
Speaker 1: expert Stephen Casey calls her Mary Beth, although that's not
Speaker 1: her real name, will do the same. Mary Beth cheerfully
Speaker 1: caught up with Ray as she used a console control
Speaker 1: to precisely position him under the THAK twenty five's radiation beamgun.
Speaker 1: Then she walked down the corridor to the control room,
Speaker 1: which was at a safe distance. Ordinarily, the control room
Speaker 1: and the treatment room would be linked up by CCTV
Speaker 1: and microphones, but neither the cameras nor the audio were
Speaker 1: connected that day, and that didn't seem to matter. Normally,
Speaker 1: they'd be useful for some reassuring chat or to give
Speaker 1: the patient a word of instruction, and Ray, being an
Speaker 1: old hand, didn't need any of that. Mary Beth typed
Speaker 1: the treatment instructions into the computer a series of letters.
Speaker 1: She pressed X to choose the mode, then straightway realized
Speaker 1: her mistake. Ray needed the other mode. She deleted the
Speaker 1: X and pressed E should check the instructions. They were
Speaker 1: all correct, beam ready, the computer told her. She pressed
Speaker 1: B to administer the treatment. Down the hall. On the
Speaker 1: treatment table, Ray Cox heard a sizzling sound and saw
Speaker 1: a blue flash and then agony. It was like someone
Speaker 1: had thrust a hot skewer through his sh shoulder. This
Speaker 1: wasn't right. He knew it couldn't be right. The last
Speaker 1: eight treatments had been nothing like this. Back in the
Speaker 1: control room, Mary Beth couldn't hear Ray's cry of pain,
Speaker 1: and she couldn't see his body contorting on the treatment table.
Speaker 1: All she saw was a bland little notification malfunction fifty four.
Speaker 1: It wasn't clear what that meant. The machine would often
Speaker 1: pause and produce an unexplained error code, sometimes thirty forty
Speaker 1: fifty times a day. As one operator later commented, I
Speaker 1: can't remember all the reasons it would stop, but there
Speaker 1: were a lot of them. The machine indicated that Ray
Speaker 1: had received only a tiny fraction of the intended dose,
Speaker 1: and Mary Beth had been assured that the Therak twenty
Speaker 1: five had so many safeguards it was almost impossible to
Speaker 1: overdose a patient. It only took a single keypress for
Speaker 1: her to reset the machine and try again. I'm Tim Harford,
Speaker 1: and you're listening to cautionary tales. Mary Beth was an
Speaker 1: experienced operator of for Ferrak twenty five. She must have
Speaker 1: seen it crash and pop up an error message ten
Speaker 1: thousand times or more. Although she didn't know what those
Speaker 1: error messages meant. How could she? The machine's manual didn't
Speaker 1: explain them, it didn't even mention them. Because she'd been
Speaker 1: in that position so many times. It took her mere
Speaker 1: seconds to reset the machine for another try at giving
Speaker 1: Ray his treatment. That wasn't enough time for Ray to
Speaker 1: get off the table. He'd rolled onto his side, but
Speaker 1: mary Beth, of course couldn't see that, and she couldn't
Speaker 1: hear his agonized yelling. The machine fired again, another flash
Speaker 1: of blue light, Another sizzled, and this time a hot
Speaker 1: skewer went through Ray's neck. He was in too much
Speaker 1: pain even to scream. Then the agony started to fade.
Speaker 1: He gulped in some air and blew it out again.
Speaker 1: Tried to calm himself, Hey, are you pushing the long bottom?
Speaker 1: But mary Beth couldn't hear him. Something had gone wrong
Speaker 1: with the Tharak twenty five. But what loyal listeners to
Speaker 1: cautionary tales will be familiar with the Swiss cheese model
Speaker 1: of accidents, made famous by the psychologist James Reason. Imagine
Speaker 1: slices of Swiss cheese with those distinctive holes in them.
Speaker 1: Each slice represents some kind of safeguard against an accident.
Speaker 1: Maybe it's a fail safe in the hardware of a system,
Speaker 1: so it simply won't work if the right pieces aren't
Speaker 1: in position. Maybe it's a subroutine in the software, monitoring
Speaker 1: what the system's doing and shutting it down if another
Speaker 1: part of the software happens to glitch. But no safeguard
Speaker 1: is perfect. Every slice of cheese has holes in it.
Speaker 1: In James Reasons model, an accident becomes possible when all
Speaker 1: the holes line up. That means that every safeguard in
Speaker 1: the system becomes simultananeously vulnerable to the same kind of problem.
Speaker 1: To prevent accidents, then get extra lines of defense and
Speaker 1: try to strengthen the defenses you already have. To put
Speaker 1: it another way, get more slices of cheese with fewer
Speaker 1: holes in them. So did the THEAC twenty five need
Speaker 1: better hardware or better software to prevent the accident that
Speaker 1: happened to ray Cox? As it turns out, yes, but
Speaker 1: that's also the wrong question. We should be looking at
Speaker 1: a different kind of cheese slice. Altogether. Nine months before
Speaker 1: ray Cox's excruciating experience, in July nineteen eighty five, forty
Speaker 1: year old Francis Hill arrived at the Ontario Cancer Foundation
Speaker 1: Clinic for her twenty fourth round of radiation treatment for
Speaker 1: cervical cancer. The clinic was using a twenty five machine,
Speaker 1: but there was a problem the machine didn't seem to
Speaker 1: be working. Every time the operator tried to fire the
Speaker 1: radiation beam, the machine paused, produced an error message and
Speaker 1: reported that no dose had been given. The operator hit
Speaker 1: the pea key to proceed, and the same thing happened.
Speaker 1: The operator hit p again, and it happened again. We've
Speaker 1: all been there, clicking an icon on a screen, finding
Speaker 1: that nothing seems to happen, and then clicking it again.
Speaker 1: After four attempts, the operator called a technician who couldn't
Speaker 1: find anything wrong with the Thearact twenty five. Francis Hill
Speaker 1: left and the machine was used successfully on half a
Speaker 1: dozen other patients that afternoon. That sort of thing wasn't
Speaker 1: particularly strange, the operator reflected. The fair Act twenty five
Speaker 1: would often seem to glitch like that, produce mysterious error messages,
Speaker 1: and then suddenly working again for no particular reason. But
Speaker 1: while glitches didn't seem strange to the operator, something seemed
Speaker 1: strange to Francis. She could feel a kind of burning, tingling,
Speaker 1: electric shock kind of sensation in her hip near where
Speaker 1: the therapeutic beam had been aimed. When she came back
Speaker 1: for another round of treatment three days later, her doctors
Speaker 1: immediately diagnosed a radiation burn in her hip, which was
Speaker 1: painfully swollen. They called the machines manufacturers to report a
Speaker 1: suspected radiation overdose. The manufacturers were Atomic Energy of Canada
Speaker 1: Limited AECL of radiation overdose. AECL had never heard of
Speaker 1: anything like that before Strange. They sent a engineer along
Speaker 1: to investigate. The FARACT twenty five could be used in
Speaker 1: two different modes. The electron mode attacked cancer near the
Speaker 1: surface of the patient's body. The machine emitted a beam
Speaker 1: of electrons spread out by an array of magnets. The
Speaker 1: X ray mode attacked cancer deep inside a patient's body.
Speaker 1: The magnet array would be moved aside and replaced by
Speaker 1: a device called a flattener, which focused the X ray
Speaker 1: beam precisely on the cancer. The flattener absorbed a lot
Speaker 1: of energy, which meant the X ray beam had to
Speaker 1: be very powerful. The components which diffused the electron beams
Speaker 1: or focused the X rays on the FAACT twenty five
Speaker 1: were positioned on a turntable. As the machine was programmed
Speaker 1: to fire either electrons or X rays at the patient's tumor,
Speaker 1: the turntail would rotate automatically to fix the right component
Speaker 1: into position. At least that was the idea. Two types
Speaker 1: of radiation beam then one that needs diffusing, one that
Speaker 1: needs focusing. If this sounds like an accident waiting to happen,
Speaker 1: well it was. But machines like the THEAK twenty five
Speaker 1: are expensive, and this dual purpose design meant that hospitals
Speaker 1: got more bang for the buck. As long as the
Speaker 1: right component was in place for the right beam, there
Speaker 1: would be no problem. The THEAK twenty five was fully
Speaker 1: controlled by a computer, unremarkable these days, but radical for
Speaker 1: the mid nineteen eighties. Its predecessors, the THEAK six and
Speaker 1: the THEAK twenty allowed a human operator to physically position
Speaker 1: the magnets or the flattener. On the THEAK twenty five,
Speaker 1: this manual position was replaced by servo motors, computer controlled
Speaker 1: to quickly and precisely put everything in position. When AECL
Speaker 1: investigated the incident with Francis Hill, they weren't actually able
Speaker 1: to reproduce the error, but they suspected that the turntable
Speaker 1: system hadn't worked properly. The turntable had three tiny switches
Speaker 1: designed to measure when it was in position, but it
Speaker 1: emerged that a single bit of error the computer glitching
Speaker 1: and mistaking a zero for a one could produce a
Speaker 1: faulty reading of the turntable's position. So AECL told the
Speaker 1: clinics that used the THEAAK twenty five to visually confirm
Speaker 1: before each procedure that the turntable was in the correct position,
Speaker 1: just as a precaution until further notice. They tightened up
Speaker 1: the software, making it more robust to a small error
Speaker 1: like that. Then they got back in touch with the clinics.
Speaker 1: No need for those visual checks anymore. We've just made
Speaker 1: the machine five orders of magnitude safer, which in plain
Speaker 1: English means it's about one hundred thousand times safer than before.
Speaker 1: And it was safe already. But one thing AECL don't
Speaker 1: seem to have done is to have notified the clinics
Speaker 1: that an accident had happened and that a patient had
Speaker 1: been injured by the machine. The clinics, at least say
Speaker 1: they weren't told of any injuries. When AECL announced that
Speaker 1: they'd fixed the problem, it was September nineteen eighty five.
Speaker 1: A month later, they were sued by a woman named
Speaker 1: Katie Yarborough. AECL had never heard of Katie Yarborough, who
Speaker 1: was Katie Yarborough. Cautionary tales will return after the break.
Speaker 1: Katie Yarborough's injury happened seven weeks before Francis Hills, early
Speaker 1: in the summer of nineteen eighty five. She was being
Speaker 1: treated at the Kenniston Oncology Center in Marietta, Georgia. Katie
Speaker 1: was sixty one. She'd had a malignant tumor removed from
Speaker 1: her breast, and now she needed follow up treatment to
Speaker 1: destroy any secondary tumors which might have spread to the
Speaker 1: lymph nodes under her collar bone. That treatment, of course,
Speaker 1: would be provided by a Therak twenty five machine. But
Speaker 1: when the technician fired up the Therak twenty five, Katie
Speaker 1: felt an agonizing pain, a tremendous force of heat. This
Speaker 1: red hot sensation. You burned me, Katie, exclaimed the technician,
Speaker 1: who was puzzled. That shouldn't be possible. The technician said
Speaker 1: the twenty five was safe, and there wasn't any sign
Speaker 1: of a burn. Perhaps Katie's clavicle was a little warm
Speaker 1: to the touch, but otherwise nothing seemed to be a miss.
Speaker 1: The physicist at the Keniston Center was a man called
Speaker 1: Tim Still. When he was informed, he was just as puzzled.
Speaker 1: Tim knew about the two treatment modes, the powerful X
Speaker 1: ray fire through the flattener and the gentler electron beam
Speaker 1: fired through the magnet array. Katie Yarborough had been treated
Speaker 1: in the electron mode, but Tim still wondered if something
Speaker 1: had gone wrong with the array. He called the manufacturers
Speaker 1: AECL with a question, was there any way the electron
Speaker 1: beam could be fired directly at a patient without the
Speaker 1: magnetarray in position? After three days, AECL replied no. They
Speaker 1: explained that was simply impossible. But somewhere along the line
Speaker 1: AECL didn't seem to get the message that a patient
Speaker 1: had been injured. Maybe Tim still didn't tell them, or
Speaker 1: maybe he did but the message didn't get through to
Speaker 1: the key decision makers. It's quite possible that still didn't
Speaker 1: even realize that Katie Yarborough was injured. After all. At
Speaker 1: first she seemed fine, But Katie Yarborough wasn't recovering from
Speaker 1: that mysterious burn. In fact, her symptoms were getting worse.
Speaker 1: The skin above her left breast had reddened, her shoulder
Speaker 1: would freeze up, and she suffered excruciating spasms. Her doctors
Speaker 1: were baffled. They continued sending her for theac treatment. After all,
Speaker 1: with malignant breast cancer, you can't afford to take risks.
Speaker 1: But when her skin started to fall off, Katie refused
Speaker 1: to continue lying on the treatment table underneath that machine.
Speaker 1: And when Tim Still later examined Katie, he noticed something strange.
Speaker 1: Not only did Katie seem to have a severe burn
Speaker 1: to her upper chest, but her upper back we was
Speaker 1: starting to Redden two. It was as though whatever had
Speaker 1: burned her had passed right through her body and caused
Speaker 1: an exit wound. When Francis Hill was injured, her clinicians
Speaker 1: didn't know about the injury to Katie Yarborough, and as
Speaker 1: hospitals across North America continued to use the THERAK twenty five,
Speaker 1: they didn't know about Francis Hill or Katie Yarborough. They
Speaker 1: only knew that the machine was already safe and had
Speaker 1: just become one hundred thousand times safer. And that makes
Speaker 1: what happened next almost inevitable. In December nineteen eighty five,
Speaker 1: Dora Moss, a patient at the Yakima Valley Memorial Hospital
Speaker 1: in Washington State, complained that her right hip seemed red
Speaker 1: and inflamed in a distinctive striped pattern. Dora's doctors were puzzled.
Speaker 1: It wasn't clear what could have caused the inflammation, although
Speaker 1: possibly it was a perfectly normal reaction to the course
Speaker 1: of radiation therapy she was having on her hip. Which
Speaker 1: device was being used, funny you should ask. It was
Speaker 1: a THERAC twenty five, But because the hospital staff at
Speaker 1: Yakima weren't aware of the history of accidents, they were baffled.
Speaker 1: Some of them wondered whether a slotted component on the
Speaker 1: THERAK twenty five might explain the striped pattern. Others suspected
Speaker 1: that it was a burn caused by Dora's habit of
Speaker 1: sleeping with an electric heating pad. Maybe those heated wires
Speaker 1: had slowly burned her skin, although on closer inspection the
Speaker 1: arrangement of wires in the heating pad didn't actually match
Speaker 1: the sore stripes on Dora's hip. So they contacted AECL
Speaker 1: who responded.
Speaker 2: After careful consideration, we are of the opinion that this
Speaker 2: damage could not have been produced by any malfunction of
Speaker 2: the FARAC twenty five or by any operator error, So
Speaker 2: that was it.
Speaker 1: Then, Officially, the cause of that stripeye burn was cause unknown,
Speaker 1: But if the cause was unclear. The consequences were stark.
Speaker 1: Dora Moss needed surgery and skin grafts to patch up
Speaker 1: her ulcerated skin and treat her chronic pain. Maybe it
Speaker 1: wasn't the FAAC twenty five. It certainly wasn't a burn
Speaker 1: from a heating pad. The Yakima hospital staff were even
Speaker 1: told that there'd been no other incidents with the THERAK
Speaker 1: twenty five. Was anyone putting all these incidents together and
Speaker 1: spotting a pattern, It seems not, although it's hard to
Speaker 1: be sure. Nancy Levison, software safety expert and the author
Speaker 1: of a definitive account of the affair, explains that because
Speaker 1: there was never an official investigation, it's often unclear who
Speaker 1: exactly knew what and when they knew it. At the
Speaker 1: East Texas Cancer Center in March nineteen eighty six, three
Speaker 1: months after the injury to Dora Moss, Mary Beth was puzzled.
Speaker 1: She'd tried twice to administer the treatment to ray Cox,
Speaker 1: apparently without success. Third time, lucky she hit again. Ray
Speaker 1: Cox had been trying to ease himself off the table,
Speaker 1: but when that searing skewer feeling hit him for a
Speaker 1: third time, jabbing through his neck and shoulder. He leapt
Speaker 1: for safety, barged open the door, and ran to the
Speaker 1: nurses station. When Mary Beth emerged, Ray was obviously shaken
Speaker 1: by what had happened. He told her that he felt
Speaker 1: like had been given three separate powerful electric shocks. How strange,
Speaker 1: Mary Beth reassured him that the machine had automatically shut
Speaker 1: down and according to the computer's display panel, Ray had
Speaker 1: only received one tenth of the intended dose. Mary Beth
Speaker 1: informed Ray's doctor and the center's physicist, Fritz Hager about
Speaker 1: the electric shocks. They came to examine the machine and Ray.
Speaker 1: There seemed to be nothing wrong with either of them,
Speaker 1: but that's the nature of a radiation overdose. It's invisible
Speaker 1: and at first the injuries it causes who are invisible too.
Speaker 1: Ray looked fine, but he really wasn't. Hagar called the
Speaker 1: manufacturer of the machine AECL to apport the incident. Then
Speaker 1: he ran through some tests and since everything seemed to
Speaker 1: be in order, pronounced the machine good to go for
Speaker 1: the afternoon patients who were waiting for treatment, and everything
Speaker 1: went smoothly. That's what happened. Remember, the THERAK twenty five
Speaker 1: often produced mysterious error messages. And then suddenly started to
Speaker 1: work again for no obvious reason. Three weeks after the
Speaker 1: strange incident with Ray Cox, Mary Beth had a new patient,
Speaker 1: sixty six year old Vernon Kidd, who had a tumor
Speaker 1: on his ear. As soon as the treatment beam was activated,
Speaker 1: Vernon cried out and started moaning for help. This time,
Speaker 1: the audio link was working. What happened, asked Mary Beth.
Speaker 2: Fire.
Speaker 1: He replied, fire on the side of his face. When
Speaker 1: the physicist Fritz Hager arrived on the scene, Vernon elaborated
Speaker 1: he'd heard a sound like frying eggs and a flash
Speaker 1: of light and then pain. He was confused and upset.
Speaker 1: What happened to me? We'll find out what happened to
Speaker 1: Vernon Kidd after the break. September ninth, nineteen forty seven,
Speaker 1: computer scientists at Harvard University get to the bottom of
Speaker 1: why their fancy computer, the Mark two, is malfunctioning. It's
Speaker 1: a bug, a literal bug, a moth in fact, which
Speaker 1: has crawled into the mass of electrical relays in the
Speaker 1: room sized computing machine and caused a short circuit. The
Speaker 1: logbook tells the tale. Handwritten on blue gridded paper are
Speaker 1: the words relay seventy panel f moth in relay. Next
Speaker 1: to those words, the bug itself is preserved under a
Speaker 1: short length of yellowing sticky tape underneath the dry remark.
Speaker 1: First actual case of bug being found. Software bugs are
Speaker 1: the bane of programmers, although as that punchline implies, this
Speaker 1: wasn't the first time the word bug had been used
Speaker 1: to describe a device malfunctioning. It was just the first
Speaker 1: time that an insect had been obliging enough to turn
Speaker 1: the metaphor into reality. In fact, software bugs are older
Speaker 1: than computers. The first computer program is widely thought to
Speaker 1: have been written by Ada Lovelace, an English mathematician and
Speaker 1: friend of the engineer Charles Babbage. In eighteen forty three,
Speaker 1: Lovelace published an algorithm that would enable Babbage's proto computer,
Speaker 1: the Analytical Engine, to calculate a particular sequence of numbers.
Speaker 1: What's so striking about Lovelace's algorithm is that the Analytical
Speaker 1: Engine had not been built, nor would it ever be.
Speaker 1: Babbage's designs were just too ambitious, and nobody would be
Speaker 1: able to construct a general purpose computer for another century.
Speaker 1: But modern analysis concludes that if Lovelace's program ever had
Speaker 1: been run on an analytical engine. It wouldn't have worked,
Speaker 1: not first time anyway, because there was a typo. Not
Speaker 1: only had Lovelace published the first software she had also
Speaker 1: published the first software bug. Anyone who's had the experience
Speaker 1: of writing computer code, or even for gen x as
Speaker 1: like me, of typing in a computer program printed in
Speaker 1: the pages of a magazine will know what it's like
Speaker 1: to have a bug. In some cases, the bugs are
Speaker 1: easily fixed. You run the program and it doesn't work.
Speaker 1: Maybe the computer even tells you where things went off
Speaker 1: the rails, exactly where the error was, or maybe not,
Speaker 1: because some bugs are more like an unsuspected hole in
Speaker 1: one of James Reasons slices of cheese. They sit there,
Speaker 1: hidden by other slices, causing no trouble until the holes
Speaker 1: and the slices aligne and the computer crashes. You may
Speaker 1: not know why it crashed. You may be able to
Speaker 1: restart the program and find it runs with no trouble,
Speaker 1: but somewhere under the surface, the bug is still there.
Speaker 1: Two of Fritz Hager's patients had apparently been electrocuted, and
Speaker 1: while the Farak twenty five seemed to be working perfectly well,
Speaker 1: he wasn't going to risk a third. He shut the
Speaker 1: machine down, notified AECL, and then tried to figure out
Speaker 1: what had occurred. It wasn't easy. After each incident, the
Speaker 1: machine seemed to be working just fine. Hager and mary
Speaker 1: Beth worked together trying to figure out what had triggered
Speaker 1: the malfunction fifty four message. Eventually they succeeded. Mary Beth
Speaker 1: recalled how she had originally typed X to use the
Speaker 1: THURAC twenty five in X ray mode, then realized she
Speaker 1: should have typed E. Quickly using the cursor keys to
Speaker 1: move to the correct box. She corrected the entry to E,
Speaker 1: repeatedly hit return to accept all the other treatment variables
Speaker 1: which were correct, and awaited the message beam ready. What
Speaker 1: she hadn't known, what nobody had known, is that a
Speaker 1: quick edit like that confused the computer's subroutines, which checked
Speaker 1: the setup only at certain moments. The result, the beam
Speaker 1: was set to full X ray power, but the flattener
Speaker 1: that would absorb most of the radiation wasn't in position.
Speaker 1: To make matters even worse, the Thorax software was confused
Speaker 1: by this dangerous setup and didn't correctly monitor the dose administered.
Speaker 1: It was a particular sequence of keystrokes that caused the
Speaker 1: problem an unlikely sequence, but not an inconceivable one. It
Speaker 1: shouldn't be particularly surprising that experienced operators such as Mary
Speaker 1: Beth might type the wrong mode, notice the error, then
Speaker 1: swiftly correct it. It was the swiftness of that correction
Speaker 1: that bewildered the software. Fritz Hager explained to AECL that
Speaker 1: he could now reproduce the error on demand. The AECL
Speaker 1: engineer couldn't until Hager explained that all the keystrokes had
Speaker 1: to be entered in less than eight seconds. The next day,
Speaker 1: the AECL engineer called back, Yes, he could now replicate
Speaker 1: the error, and he had bad news. If the beam
Speaker 1: was fired in such conditions, the patient would receive a
Speaker 1: dose of twenty five thousand rads, more than one hundred
Speaker 1: times more than intended, which potentially could be fatal. Over
Speaker 1: the course of three weeks, sixty six year old Vernon
Speaker 1: Kid moved from disorientation to a coma to death. The
Speaker 1: autopsy revealed that the section of his brain running from
Speaker 1: under his right temple to behind his right ear had
Speaker 1: been withered by a high dose of radiation. It's natural
Speaker 1: to describe the problem with a Tharak twenty five as
Speaker 1: a software bug, and while that's true, it doesn't really
Speaker 1: help us understand the problem or prevent similar problems in future.
Speaker 1: As Nancy Levison writes, virtually or complex software can be
Speaker 1: made to behave in an unexpected fashion under some conditions.
Speaker 1: Demanding software with no bugs is like demanding a slice
Speaker 1: of Swiss cheese with no holes. It's in the nature
Speaker 1: of Swiss cheese that there will always be holes, and
Speaker 1: it's in the nature of complex software that there will
Speaker 1: always be bugs. The question is what happens when a
Speaker 1: bug appears. Perhaps another part of the software is able
Speaker 1: to spot the problem. A separate slice of software cheese
Speaker 1: that didn't happen With a Therac twenty five. The computer
Speaker 1: would tell the operator what dose the patient had received.
Speaker 1: There was no direct measurement of that dose. The bug
Speaker 1: that led to the overdose also led to the software
Speaker 1: failing to report the overdose, or perhaps there are failsafes
Speaker 1: in the hardware. Again, not with the Therrak twenty five,
Speaker 1: the machine relied on the software being perfect. The shielding
Speaker 1: components were put in place by the software. The decision
Speaker 1: to allow the electron gun to fire or not was
Speaker 1: made by the software, and the dose the patient received
Speaker 1: was reported by the software. If the software was wrong
Speaker 1: about one of these things, it could easily be wrong
Speaker 1: about the others. There is a different approach. The THEAK twenty,
Speaker 1: the predecessor of the THERAK twenty five, was designed to
Speaker 1: operate with or without a computer, and it was built
Speaker 1: with mechanical interlocks. If you tried to fire the beam
Speaker 1: without the right component in place, the machine just wouldn't
Speaker 1: do it. It was only after the THEAK twenty five's
Speaker 1: problems became widely known that something began to dawn on
Speaker 1: THEA'AC twenty users. Sometimes the machine's fuse would blow after
Speaker 1: quick edits I was annoying and a bit mysterious. The
Speaker 1: machine would have to be switched off, the fuse replaced,
Speaker 1: and everything restarted. The fra AC twenty software had been
Speaker 1: built on the same codebase as the fair AC twenty five,
Speaker 1: and on closer examination it became clear that the THEA
Speaker 1: ACT twenty had the same software bug. Because of the
Speaker 1: mechanical interlocks, which would physically prevent the machine from working
Speaker 1: unless it was correctly set up, the bug was never
Speaker 1: anything more than an annoyance. The most precious thing that
Speaker 1: was damaged was a fuse. The Fair Act twenty five
Speaker 1: five needed better software, and it needed better hardware. But
Speaker 1: as I said earlier, that's also the wrong place to focus.
Speaker 1: Another kind of cheese slice was missing. There was no
Speaker 1: proper process for noting anomalous incidents, suspected malfunctions, or possible injuries.
Speaker 1: Hospitals should have been reporting both the strange incidents and
Speaker 1: the later mysterious injuries. Someone either AECL or a regulator
Speaker 1: should have been collecting and analyzing the reports. Because that
Speaker 1: didn't happen, every new incident caused a new wave of
Speaker 1: muddle and bewilderment. Of course, we should try hard to
Speaker 1: eliminate bugs, especially when lives are at stake. But the
Speaker 1: real lesson here is that safety is not a function
Speaker 1: of good software alone. It's the function of the whole system,
Speaker 1: and the system goes beyond the software. In fact, it
Speaker 1: goes beyond the machine. The system includes the network of
Speaker 1: people who make the machine, use the machine, and regulate
Speaker 1: the machine. In this case, they should have been keeping
Speaker 1: each other closely informed. They weren't, and as a result,
Speaker 1: it took months for anyone to assemble the pieces of
Speaker 1: this awful puzzle Voin. Ray Cox was young and strong,
Speaker 1: but he had taken a triple blast of high dose
Speaker 1: radiation to his back, shoulder, and neck. Before long, he
Speaker 1: was starting to spit up blood. The awful radiation burns
Speaker 1: on his back and neck turned into yawning lesions as
Speaker 1: the skin and flesh started to peel off his body.
Speaker 1: While over the weeks that followed, the damage to his
Speaker 1: spinal column paralyzed his left arm, both legs, and his
Speaker 1: vocal cords, ray tried to keep his sense of humor.
Speaker 1: Before he lost his ability to speak, he would joke
Speaker 1: to friends and family Captain Kirk forgot to put the
Speaker 1: machine on stun. Ray Cox died in August nineteen eighty six,
Speaker 1: a few months after the accident, and five months after that,
Speaker 1: Glenn Dodd, a sixty five year old cancer sufferer, was
Speaker 1: given radiation therapy at Yakima Valley Memorial Hospital, where Dora
Speaker 1: Moss had acquired those mysterious striped radiation burns after treatment
Speaker 1: from a THERAK twenty five. Glen Dodd was being treated
Speaker 1: by a THEAK twenty five, he received a fatal overdose.
Speaker 1: Dodd was terminally ill, but doctors concluded that the injuries
Speaker 1: he had suffered from the malfunctioning machine had killed him
Speaker 1: before his own cancer could. Why hadn't aecl fixed the
Speaker 1: software fault? Well? As it happens, Glen do Odd wasn't
Speaker 1: killed by the glitch that had killed ray Cox and
Speaker 1: Vernon Kidd, but by a different software error. You see,
Speaker 1: there's always another bug lying in wait for the moment
Speaker 1: to strike. This isn't the first cautionary tale we've done
Speaker 1: about radiation overdoses. Two of my very favorite episodes of
Speaker 1: the podcast were Glowing Peril, The magical glitter that poisoned
Speaker 1: a City, and How the Radium Girls Fought Back, two
Speaker 1: unforgettable episodes which came out as a pair late in
Speaker 1: twenty twenty three. You may enjoy listening to them. I
Speaker 1: first heard about ray Cox's case from Stephen M. Casey's
Speaker 1: book Set Phases on stun while the definitive authority on
Speaker 1: the THERAK twenty five case is Nancy Levison's investigation, written
Speaker 1: with Clark Turner. For a full list of our sources,
Speaker 1: see the show notes at Timharford dot com. Cautionary Tales
Speaker 1: is written by me Tim Harford with Andrew Wright. The
Speaker 1: show is produced by Alice Fines with Marilyn Rust. The
Speaker 1: sound design and original music are the work of Pascal Wye.
Speaker 1: Sarah Nix edited the script. Cautionary Tales features the voice
Speaker 1: talents of Ben Crow, Melanie Gushridge, Stella Harford, Gemma Saunders
Speaker 1: and Rufus Wright. The show wouldn't have been possible without
Speaker 1: the work of Jacob Weisberg, Ryan Dilly, Greta Cohne, Eric Sandler,
Speaker 1: Carrie Brody, Christina Sullivan, Kira Posey and Owen Miller. Cautionary
Speaker 1: Tales as a production of Pushkin Industries. It's recorded at
Speaker 1: Wardoor Studios in London by Tom Berry. If you like
Speaker 1: the show, please remember to share, rate and review. It
Speaker 1: doesn't really make a difference to us and if you
Speaker 1: want to hear the show ad free, sign up to
Speaker 1: Pushkin Plus on the show page on Apple Podcasts or
Speaker 1: at pushkin dot Fm, slash plus, st