1
00:00:15,370 --> 00:00:26,010
Speaker 1: Pushkin. As the night draws in and the fire blazes

2
00:00:26,050 --> 00:00:30,050
Speaker 1: on the hearth. We warn the children by telling them stories.

3
00:00:30,850 --> 00:00:34,450
Speaker 1: The little Mermaid warns them he's just not worth its, sister.

4
00:00:35,370 --> 00:00:39,370
Speaker 1: But my stories are for the education of the grown ups,

5
00:00:40,170 --> 00:00:45,810
Speaker 1: and my stories are all true. I'm Tim Harford. Gather

6
00:00:45,890 --> 00:01:04,090
Speaker 1: close and listen to my cautionary tales. Two and a

7
00:01:04,170 --> 00:01:08,770
Speaker 1: half thousand years ago, King Croesus of Lydia ruled a

8
00:01:09,170 --> 00:01:13,810
Speaker 1: mighty empire. Lydia had endured for more than six centuries

9
00:01:13,930 --> 00:01:19,210
Speaker 1: in the lands we now call Turkey. Criesus was legendarily wealthy,

10
00:01:19,570 --> 00:01:22,890
Speaker 1: yet he felt threatened by the growing strength of the

11
00:01:22,970 --> 00:01:27,370
Speaker 1: First Persian Empire on his borders. What should Creesus do?

12
00:01:27,930 --> 00:01:30,890
Speaker 1: Should he strike against the Persians in an attempt to

13
00:01:30,930 --> 00:01:34,810
Speaker 1: seize their lands and destroy their power, or should he

14
00:01:34,850 --> 00:01:39,650
Speaker 1: aim for a peaceful trading relationship. King Creesus yearned for

15
00:01:39,730 --> 00:01:43,610
Speaker 1: something we've all wanted from time to time, to see

16
00:01:43,690 --> 00:01:48,370
Speaker 1: into the future, and this being the ancient world, that

17
00:01:48,490 --> 00:01:54,290
Speaker 1: desire could be satisfied. Criesus could consult an oracle. That

18
00:01:54,370 --> 00:01:58,570
Speaker 1: meant traveling to a temple, making an offering, and asking

19
00:01:58,610 --> 00:02:03,530
Speaker 1: the advice of a god. The most famous oracle was

20
00:02:03,570 --> 00:02:07,410
Speaker 1: at Delphi, in the heart of ancient Greece, whether God

21
00:02:07,490 --> 00:02:12,290
Speaker 1: Apollo would possess the temple's appointed priestess and give divine

22
00:02:12,410 --> 00:02:16,650
Speaker 1: answers to the questions asked of her. Criesus had tested

23
00:02:16,810 --> 00:02:21,570
Speaker 1: many oracles. As the Greek historian Herodotus tells it, Creesus

24
00:02:21,610 --> 00:02:26,170
Speaker 1: sent messengers to far flung temples, asking each oracle what

25
00:02:26,330 --> 00:02:31,090
Speaker 1: the king was doing on a particular date. What he

26
00:02:31,170 --> 00:02:35,730
Speaker 1: was actually doing was frankly unguessable, boiling a stew of

27
00:02:36,090 --> 00:02:41,690
Speaker 1: lamb and tortoise in a bronze cauldron. The messengers returned

28
00:02:41,730 --> 00:02:46,490
Speaker 1: from each oracle, and when Criesus heard the pronouncement from Delphi,

29
00:02:46,730 --> 00:02:51,090
Speaker 1: which was correct in every detail, he bowed his head

30
00:02:51,410 --> 00:02:57,130
Speaker 1: in respect. Truly, the priestess at Delphi spoke with the

31
00:02:57,250 --> 00:03:03,450
Speaker 1: voice of Apollo, and so Crisus asked, should I seek

32
00:03:03,530 --> 00:03:09,730
Speaker 1: war with a Persian empire? If King Criesus attacks the Persians,

33
00:03:10,290 --> 00:03:16,410
Speaker 1: he shall destroy a mighty empire. And how long shall

34
00:03:16,450 --> 00:03:20,290
Speaker 1: my kingdom endure till the time shall come when a

35
00:03:20,490 --> 00:03:25,770
Speaker 1: mule is monarch of Persia. The kingdom of Lydia then

36
00:03:25,930 --> 00:03:29,170
Speaker 1: would last forever, since the Persian Empire would never be

37
00:03:29,290 --> 00:03:32,850
Speaker 1: ruled by a half breed animal. The Persian Empire, in

38
00:03:32,930 --> 00:03:37,050
Speaker 1: any case, was destined to be destroyed by King Creesus himself,

39
00:03:38,010 --> 00:03:43,210
Speaker 1: so the oracle foretold, And since the oracle guaranteed victory,

40
00:03:43,690 --> 00:03:51,290
Speaker 1: Crisus attacked and was defeated. Criesus was the last king

41
00:03:51,490 --> 00:03:56,330
Speaker 1: of Lydia. He was vanquished by the Persian king Cyrus,

42
00:03:56,370 --> 00:04:00,930
Speaker 1: whose complex heritage apparently qualified him as a mule in

43
00:04:00,970 --> 00:04:05,770
Speaker 1: the eyes of the oracle. Criesus had indeed destroyed a

44
00:04:05,930 --> 00:04:11,690
Speaker 1: mighty empire his own. It's not easy to see into

45
00:04:11,730 --> 00:04:15,010
Speaker 1: the future, but the tale of King Cresus tells us

46
00:04:15,050 --> 00:04:18,290
Speaker 1: that even when we do see the future, that doesn't

47
00:04:18,290 --> 00:04:22,730
Speaker 1: guarantee will make wise decisions in the present. In the

48
00:04:22,770 --> 00:04:26,210
Speaker 1: modern world, we have our own oracles, and they too

49
00:04:26,450 --> 00:04:29,570
Speaker 1: tell us what the future holds. We call them by

50
00:04:29,650 --> 00:04:35,490
Speaker 1: mysterious names such as Alexa, Google, and Siri, and we

51
00:04:35,530 --> 00:04:39,330
Speaker 1: need to think much harder about what those oracles are

52
00:04:39,370 --> 00:05:15,530
Speaker 1: telling us. Settle back and listen to another cautionary tale.

53
00:05:22,810 --> 00:05:27,850
Speaker 1: August two thousand and nine, Death Valley, California, one of

54
00:05:27,890 --> 00:05:33,170
Speaker 1: the hottest places on Earth. National Park ranger Amburner Trass

55
00:05:33,170 --> 00:05:35,810
Speaker 1: has been following the tracks of a car along a

56
00:05:35,890 --> 00:05:39,450
Speaker 1: rough country trail that's barely a road at all. The

57
00:05:39,530 --> 00:05:43,410
Speaker 1: tracks shouldn't be there. The road has fallen into disrepair

58
00:05:43,610 --> 00:05:46,770
Speaker 1: and been covered by the shifting sands of the Mahave Desert.

59
00:05:47,490 --> 00:05:51,290
Speaker 1: In fact, the route is officially closed, the closure marked

60
00:05:51,330 --> 00:05:54,570
Speaker 1: by small rocks and bushes laid across the road, but

61
00:05:54,690 --> 00:05:59,450
Speaker 1: the tracks go straight through those slight barriers. Then Ranger

62
00:05:59,530 --> 00:06:03,090
Speaker 1: a Trass sees the jeep. It's up to its axles

63
00:06:03,130 --> 00:06:07,690
Speaker 1: in soft sand. It has SOS spelled out in medical

64
00:06:07,730 --> 00:06:11,170
Speaker 1: tape on the windshield, and there's some one lying beside

65
00:06:11,170 --> 00:06:15,290
Speaker 1: it in the hundred and fifteen degrees shade. Are you okay?

66
00:06:15,890 --> 00:06:20,810
Speaker 1: Ranging a Trass asks. The prone figure scrambles to her feet.

67
00:06:21,850 --> 00:06:25,610
Speaker 1: She's alive. Her tongue is swollen, her lips are bleeding

68
00:06:25,650 --> 00:06:29,850
Speaker 1: and blistered. She's not okay. She's not okay at all.

69
00:06:31,330 --> 00:06:34,930
Speaker 1: Alysia Sanchez and her six year old son Carlos had

70
00:06:34,970 --> 00:06:38,210
Speaker 1: been stuck in the sand and the heat for five days.

71
00:06:39,890 --> 00:06:44,810
Speaker 1: Carlos hadn't made it. He'd drifted away, telling his mother

72
00:06:45,090 --> 00:06:50,210
Speaker 1: that had been speaking to his grandfather in heaven. How

73
00:06:50,250 --> 00:06:53,290
Speaker 1: did a young mother and her son end up so

74
00:06:53,770 --> 00:06:59,250
Speaker 1: terribly fatally lost. They'd been relying on the directions of

75
00:06:59,290 --> 00:07:04,050
Speaker 1: a dashboard computer. Rangers in Death Valley National Park have

76
00:07:04,170 --> 00:07:09,690
Speaker 1: a phrase for it, death by GPS. There was probably

77
00:07:09,730 --> 00:07:12,650
Speaker 1: a point where she said, Oh my god, I don't

78
00:07:12,650 --> 00:07:15,690
Speaker 1: know where I am. I'm going to keep going because

79
00:07:15,690 --> 00:07:19,730
Speaker 1: I think I'm going in the right direction. People are

80
00:07:19,770 --> 00:07:22,450
Speaker 1: so reliant on there a GPS that they failed to

81
00:07:22,490 --> 00:07:25,450
Speaker 1: look out the windshield and make wise decisions based on

82
00:07:25,530 --> 00:07:28,250
Speaker 1: what they're seeing. Many of us have done the same

83
00:07:28,330 --> 00:07:32,210
Speaker 1: thing that got Alysia Sanchez stranded in Death Valley. We

84
00:07:32,370 --> 00:07:36,730
Speaker 1: relied on GPS only to find ourselves lost in one

85
00:07:36,770 --> 00:07:40,650
Speaker 1: way or another. And now I have once I typed

86
00:07:40,690 --> 00:07:43,690
Speaker 1: in the wrong address. Another time the route was blocked.

87
00:07:44,250 --> 00:07:47,570
Speaker 1: More than once, I've just lost the signal, And because

88
00:07:47,610 --> 00:07:52,290
Speaker 1: I've relied on the computer's guidance, I've been helpless. I've

89
00:07:52,330 --> 00:07:59,730
Speaker 1: just never been so cruelly punished for my mistake. The

90
00:08:02,970 --> 00:08:06,210
Speaker 1: story of Crisus and the Oracle is the last and

91
00:08:06,410 --> 00:08:10,410
Speaker 1: most ancient cautionary tale in our series, but it holds

92
00:08:10,450 --> 00:08:13,850
Speaker 1: a very modern lesson for us. I think it teaches

93
00:08:13,930 --> 00:08:17,250
Speaker 1: us what might go wrong when we ask computers to

94
00:08:17,250 --> 00:08:21,650
Speaker 1: predict the future for us. Most of the economic forecasts

95
00:08:21,690 --> 00:08:25,570
Speaker 1: we see are made by computers, so are the weather forecasts,

96
00:08:25,810 --> 00:08:28,810
Speaker 1: and so are many computerized decisions that we don't even

97
00:08:28,850 --> 00:08:32,730
Speaker 1: think of as forecasts, such as when an algorithm recommends

98
00:08:32,770 --> 00:08:35,970
Speaker 1: who should get a mortgage approved and who shouldn't, or

99
00:08:36,010 --> 00:08:40,370
Speaker 1: even which criminal suspects should get bail. When you ask

100
00:08:40,410 --> 00:08:43,290
Speaker 1: a GPS system or your phone to plot you a route,

101
00:08:43,690 --> 00:08:47,290
Speaker 1: that's a forecast too. The computer makes a prediction of

102
00:08:47,330 --> 00:08:50,050
Speaker 1: which roads are open based on a map database that

103
00:08:50,290 --> 00:08:53,490
Speaker 1: may or may not be accurate. It then unleashes an

104
00:08:53,530 --> 00:08:57,250
Speaker 1: algorithm to forecast which route through that map will be

105
00:08:57,290 --> 00:09:01,010
Speaker 1: the swiftest. If the map is wrong, the prediction will

106
00:09:01,010 --> 00:09:03,810
Speaker 1: be wrong too. But even when the prediction is right,

107
00:09:04,290 --> 00:09:07,090
Speaker 1: you may still end up far from where you wanted

108
00:09:07,130 --> 00:09:11,410
Speaker 1: to be. Under the predicament of a Swedish couple on

109
00:09:11,530 --> 00:09:14,770
Speaker 1: holiday in Italy, they went to the tourist office in

110
00:09:14,810 --> 00:09:18,570
Speaker 1: the small town of Carpi, near Bologna, in the industrial

111
00:09:18,650 --> 00:09:22,850
Speaker 1: north of Italy and ask for directions. Which way is

112
00:09:22,890 --> 00:09:29,250
Speaker 1: it to the Grotta Adzura Please, Grotta Azura. Sorry, the

113
00:09:29,330 --> 00:09:34,930
Speaker 1: Blue Grotto is sea cave? Yes, the Blue Grotto exactly,

114
00:09:35,810 --> 00:09:40,050
Speaker 1: but the Blue Grotto is in Capri. We are in Carpi.

115
00:09:40,930 --> 00:09:44,570
Speaker 1: Instead of driving four hours south from Rome to the

116
00:09:44,610 --> 00:09:49,130
Speaker 1: beautiful island of Capri, they had driven four hours north

117
00:09:49,330 --> 00:09:52,810
Speaker 1: to Carpi, where they were about as far away from

118
00:09:52,850 --> 00:09:55,730
Speaker 1: the sea as it's possible for a little Italian town

119
00:09:55,810 --> 00:09:59,290
Speaker 1: to be. He is to understand how they managed it.

120
00:09:59,450 --> 00:10:04,250
Speaker 1: I mean, Gabri is an island. Well, yes, but it's

121
00:10:04,290 --> 00:10:07,130
Speaker 1: not hard to understand at all, is it. It was

122
00:10:07,170 --> 00:10:10,530
Speaker 1: a typo. A typo. It meant a long drive in

123
00:10:10,650 --> 00:10:15,370
Speaker 1: exactly the wrong direction. Compared to the suffering of Alicia

124
00:10:15,490 --> 00:10:20,250
Speaker 1: and Carlos Sanchez, that was a small enough misadventure. If

125
00:10:20,290 --> 00:10:23,050
Speaker 1: the Swedish tourists had known anything about which cities in

126
00:10:23,090 --> 00:10:26,290
Speaker 1: Italy and north of Rome and which lie south, or

127
00:10:26,330 --> 00:10:28,890
Speaker 1: had checked a map or a compass, and they would

128
00:10:28,930 --> 00:10:31,730
Speaker 1: never have made the mistake. But why would they have

129
00:10:31,730 --> 00:10:34,890
Speaker 1: done any of that. They'd asked their GPS to take

130
00:10:34,930 --> 00:10:39,170
Speaker 1: them to Carpi, and it did, Just like the Oracle

131
00:10:39,210 --> 00:10:43,850
Speaker 1: of Delphi, it produced exactly the right answer, and just

132
00:10:43,970 --> 00:10:48,650
Speaker 1: like Creesus, they acted on that answer without pausing to

133
00:10:48,690 --> 00:10:56,610
Speaker 1: think the concerns of King Creesus. Two and a half

134
00:10:56,690 --> 00:11:00,650
Speaker 1: thousand years ago may seem very different from our problems

135
00:11:00,690 --> 00:11:03,530
Speaker 1: today as we type an address into a GPS or

136
00:11:03,730 --> 00:11:08,770
Speaker 1: check the weather forecast, But they really aren't. Oracles, like

137
00:11:09,130 --> 00:11:14,330
Speaker 1: computer algorithms, are mysterious black boxes. We ask them questions

138
00:11:14,370 --> 00:11:18,810
Speaker 1: about the future, we receive answers, and then we have

139
00:11:18,890 --> 00:11:25,290
Speaker 1: to work out what those answers mean. There's a fierce

140
00:11:25,370 --> 00:11:29,090
Speaker 1: debate raging about the use of algorithmic predictions. How can

141
00:11:29,130 --> 00:11:31,810
Speaker 1: we trust a computer to decide whether a criminal is

142
00:11:31,850 --> 00:11:34,850
Speaker 1: at high risk of reoffending, or whether a teacher is

143
00:11:34,970 --> 00:11:39,930
Speaker 1: promoted or fired. That debate tends to focus on whether

144
00:11:39,970 --> 00:11:44,610
Speaker 1: the algorithms deliver predictions that are fair and accurate, which,

145
00:11:44,730 --> 00:11:48,330
Speaker 1: of course is an important question, but it leaves out

146
00:11:48,330 --> 00:11:51,170
Speaker 1: a point that we tend to overlook, a point that

147
00:11:51,290 --> 00:11:54,690
Speaker 1: is the lesson of our cautionary tale. Just because you

148
00:11:54,730 --> 00:11:57,890
Speaker 1: get a good forecast doesn't mean you're guaranteed to make

149
00:11:57,890 --> 00:12:02,930
Speaker 1: a good decision. After all, the oracle at Delphi told

150
00:12:03,090 --> 00:12:07,090
Speaker 1: King Creesus the truth about the future. The truth didn't

151
00:12:07,130 --> 00:12:11,170
Speaker 1: help him, and the GPS that delivered the Swedish tourists

152
00:12:11,210 --> 00:12:14,570
Speaker 1: to Carpi also gave the correct answer to the question

153
00:12:14,610 --> 00:12:17,490
Speaker 1: they'd asked, they'd still have been much better off if

154
00:12:17,490 --> 00:12:22,410
Speaker 1: they'd used a roadmap. And as for Alicia Sanchez, we

155
00:12:22,490 --> 00:12:25,730
Speaker 1: don't know whether her GPS had an outdated map or

156
00:12:25,730 --> 00:12:29,570
Speaker 1: an intermittent signal, or maybe it worked fine and she

157
00:12:29,650 --> 00:12:32,210
Speaker 1: made some trivial mistake in the way she used it.

158
00:12:33,010 --> 00:12:35,330
Speaker 1: What we do know is that she didn't have a map,

159
00:12:35,810 --> 00:12:37,890
Speaker 1: and she didn't stop when she got to the barrier

160
00:12:37,930 --> 00:12:40,850
Speaker 1: of stones that was supposed to show the road was closed.

161
00:12:41,810 --> 00:12:47,890
Speaker 1: Trusting the GPS had truly awful consequences. A good forecaster

162
00:12:48,090 --> 00:12:52,130
Speaker 1: can lull you into believing it's infallible, creating a crisis

163
00:12:52,290 --> 00:12:55,770
Speaker 1: when it fails, or it can give you an accurate

164
00:12:55,810 --> 00:12:59,890
Speaker 1: answer that you misunderstand, perhaps because you don't know what

165
00:12:59,930 --> 00:13:04,730
Speaker 1: you've really asked. One of the gurus of futurology was

166
00:13:04,770 --> 00:13:10,090
Speaker 1: a French economist called Pierre vac. He once wrote, forecasts

167
00:13:10,330 --> 00:13:13,810
Speaker 1: are not always wrong. More often than not, it can

168
00:13:13,810 --> 00:13:18,330
Speaker 1: be reasonably accurate, and that is what makes them so dangerous.

169
00:13:35,450 --> 00:13:39,610
Speaker 1: I don't want to suggest that computers always produce good forecasts.

170
00:13:39,770 --> 00:13:43,930
Speaker 1: In fact, behind the scenes, a mathematical algorithm was responsible

171
00:13:43,970 --> 00:13:46,930
Speaker 1: for what I think has a strong claim to being

172
00:13:46,930 --> 00:13:52,250
Speaker 1: the worst forecast anyone has ever made. It's August two

173
00:13:52,330 --> 00:13:56,010
Speaker 1: thousand and seven, the early days of the Great Financial Crisis.

174
00:13:56,490 --> 00:14:00,770
Speaker 1: An insurance executive called Joseph Cassano is trying to reassure

175
00:14:00,810 --> 00:14:04,650
Speaker 1: the world that his company, AIG, is doing just fine.

176
00:14:05,490 --> 00:14:08,770
Speaker 1: You might remember AIG from our episode about giving the

177
00:14:09,090 --> 00:14:12,650
Speaker 1: skirt to the wrong movie. AIG was an insurance company

178
00:14:12,690 --> 00:14:15,970
Speaker 1: at the epicenter of the financial crisis. It had been

179
00:14:15,970 --> 00:14:20,650
Speaker 1: writing contract after contract ensuring other companies against debts not

180
00:14:20,770 --> 00:14:24,530
Speaker 1: being repaid. It's hard for us, without being flippant, to

181
00:14:24,530 --> 00:14:27,970
Speaker 1: see us scenario within any kind of realm of reason

182
00:14:28,410 --> 00:14:31,730
Speaker 1: that would see us losing one dollar in any of

183
00:14:31,730 --> 00:14:38,450
Speaker 1: those transactions, Not a single dollar, not in any conceivable scenario.

184
00:14:39,610 --> 00:14:43,810
Speaker 1: Eighteen months later, AIG announced that it had lost more

185
00:14:43,850 --> 00:14:49,610
Speaker 1: than sixty billion dollars in a single quarter. What on

186
00:14:49,650 --> 00:14:56,370
Speaker 1: earth had happened? Simple, AIG had wagered billions, no trillions,

187
00:14:56,410 --> 00:15:00,610
Speaker 1: on the financial markets equivalent of a GPS, and the

188
00:15:00,690 --> 00:15:05,690
Speaker 1: GPS had led it astray. The financial GPS was a

189
00:15:05,730 --> 00:15:09,490
Speaker 1: mathematical formula that attempted to forecast the risk that two

190
00:15:09,610 --> 00:15:13,650
Speaker 1: bad things happen together. Let me take a moment to explain.

191
00:15:14,330 --> 00:15:17,290
Speaker 1: Let's say I lend money to two businesses. I hope

192
00:15:17,330 --> 00:15:19,450
Speaker 1: that they're both going to pay me back, but they

193
00:15:19,530 --> 00:15:22,610
Speaker 1: might not. One of them might default and not repay

194
00:15:22,650 --> 00:15:27,050
Speaker 1: the loan. Both of them might default. So here's a question,

195
00:15:27,570 --> 00:15:30,690
Speaker 1: what's the chance that the second one defaults given that

196
00:15:30,730 --> 00:15:34,730
Speaker 1: the first one does. It's a subtle question. If one

197
00:15:34,770 --> 00:15:37,450
Speaker 1: business is a food truck in Cancoon and the other

198
00:15:37,530 --> 00:15:41,090
Speaker 1: is a food truck in Amsterdam, their fates are presumably

199
00:15:41,290 --> 00:15:44,250
Speaker 1: totally unrelated. If you're trying to figure out whether the

200
00:15:44,290 --> 00:15:47,370
Speaker 1: food truck in Amsterdam will fail to pay back the loan,

201
00:15:47,890 --> 00:15:51,650
Speaker 1: the fate of the truck in Cancoon is irrelevant. The

202
00:15:51,770 --> 00:15:56,090
Speaker 1: technical term for this is that they're uncorrelated. But if

203
00:15:56,090 --> 00:15:58,730
Speaker 1: the food trucks are in the same city, their fates

204
00:15:58,810 --> 00:16:00,930
Speaker 1: might be linked. They both have to deal with the

205
00:16:00,970 --> 00:16:04,410
Speaker 1: same local economy, the same licenses, and the same weather.

206
00:16:05,210 --> 00:16:08,290
Speaker 1: If a big local business closes, that's bad for both

207
00:16:08,290 --> 00:16:12,850
Speaker 1: of them. If so, their fates are correlated. What happens

208
00:16:12,850 --> 00:16:16,050
Speaker 1: to one is likely to happen to the other. But

209
00:16:16,170 --> 00:16:19,650
Speaker 1: even then the link between the two isn't obvious. If

210
00:16:19,730 --> 00:16:22,610
Speaker 1: one truck gets bad reviews or is shut down for

211
00:16:22,730 --> 00:16:26,170
Speaker 1: hygiene violations. Maybe that's good news for the other truck.

212
00:16:26,570 --> 00:16:30,410
Speaker 1: One truck going bankrupt might make the other truck less

213
00:16:30,410 --> 00:16:35,610
Speaker 1: likely to go bankrupt. If so, their fates are negatively correlated.

214
00:16:36,050 --> 00:16:39,210
Speaker 1: What happens to one is less likely to happen to

215
00:16:39,250 --> 00:16:43,210
Speaker 1: the other. If you're on Wall Street, it's important to

216
00:16:43,250 --> 00:16:48,850
Speaker 1: figure out whether loans are positively correlated, negatively correlated, or uncorrelated.

217
00:16:49,490 --> 00:16:51,730
Speaker 1: One of the things that Wall Street likes to do

218
00:16:52,130 --> 00:16:56,130
Speaker 1: is build big financial structures out of these individual loans.

219
00:16:56,810 --> 00:17:00,170
Speaker 1: How safe the financial structure is depends a lot on

220
00:17:00,210 --> 00:17:04,850
Speaker 1: these correlations. If they're all highly correlated, then either none

221
00:17:04,890 --> 00:17:07,410
Speaker 1: of them are going to go bad, or they all are.

222
00:17:08,130 --> 00:17:11,730
Speaker 1: That's a lot of risk. But if they're uncorrelated or

223
00:17:11,770 --> 00:17:15,810
Speaker 1: even negatively correlated, then I can pretty much guarantee that

224
00:17:15,890 --> 00:17:19,570
Speaker 1: when some go bad, the others will be fine. Predicting

225
00:17:19,570 --> 00:17:22,770
Speaker 1: the correlation tells you all you need to know about

226
00:17:22,770 --> 00:17:27,010
Speaker 1: predicting the risk. Given how useful it is to predict

227
00:17:27,050 --> 00:17:31,930
Speaker 1: the correlation between loans, Wall Street's finest mathematicians, often dubbed

228
00:17:32,050 --> 00:17:35,970
Speaker 1: the QUANTZ, turned to the challenge about two decades ago.

229
00:17:36,450 --> 00:17:39,850
Speaker 1: They deployed a formula known as the Gaussian copular function.

230
00:17:40,690 --> 00:17:43,490
Speaker 1: You don't need to know what the Gaussian copular function

231
00:17:43,530 --> 00:17:48,010
Speaker 1: actually is, just that it's a way of predicting correlations automatically.

232
00:17:48,570 --> 00:17:53,050
Speaker 1: Plug your historical data into a computer and outcomes the prediction.

233
00:17:53,610 --> 00:17:59,090
Speaker 1: It's the financial GPS, and that financial GPS. The Gaussian

234
00:17:59,170 --> 00:18:03,130
Speaker 1: Copular function is now commonly described in the business press

235
00:18:03,490 --> 00:18:06,850
Speaker 1: as the formula that destroyed Wall Street and the Great

236
00:18:06,930 --> 00:18:12,970
Speaker 1: Financial Crisis of two thousand and eight. The problem wasn't

237
00:18:13,010 --> 00:18:16,850
Speaker 1: just that the formula gave the wrong answer, although it did.

238
00:18:17,650 --> 00:18:21,330
Speaker 1: The problem was that AIG and other big Wall Street

239
00:18:21,410 --> 00:18:27,050
Speaker 1: institutions had bet everything, absolutely everything, on the assumption that

240
00:18:27,090 --> 00:18:30,890
Speaker 1: the formula couldn't be wrong. That was like trusting a

241
00:18:31,010 --> 00:18:36,890
Speaker 1: GPS in Death Valley. It's fine until it isn't, and

242
00:18:36,930 --> 00:18:42,890
Speaker 1: then you're in terrible, terrible trouble. When people trusted the

243
00:18:42,930 --> 00:18:46,490
Speaker 1: Gaussian copular function, they didn't realize that they were making

244
00:18:46,530 --> 00:18:49,810
Speaker 1: a knife edge bet. If the copular function produced the

245
00:18:49,890 --> 00:18:52,890
Speaker 1: right prediction, you could bet a billion dollars and be

246
00:18:53,010 --> 00:18:56,530
Speaker 1: absolutely certain that you wouldn't lose anything, not a cent,

247
00:18:57,250 --> 00:19:00,810
Speaker 1: And if the copular function produced the wrong prediction, you

248
00:19:00,850 --> 00:19:05,130
Speaker 1: could lose absolutely everything. It's hard for us, without being flippant,

249
00:19:05,450 --> 00:19:08,570
Speaker 1: to see a scenario within any kind of realm of

250
00:19:08,650 --> 00:19:12,610
Speaker 1: reason that would see us losing one dollar in any

251
00:19:12,650 --> 00:19:20,210
Speaker 1: of those transactions sixty billion dollars, Jocasano sixty billion dollars.

252
00:19:21,210 --> 00:19:23,970
Speaker 1: And if you're confident that this sort of thing isn't

253
00:19:24,010 --> 00:19:27,810
Speaker 1: still happening in the financial system, you're more confident than

254
00:19:27,890 --> 00:19:35,610
Speaker 1: I am before you declare war on the Persian Empire,

255
00:19:36,330 --> 00:19:39,810
Speaker 1: or drive into Death Valley, or bet trillions on a

256
00:19:39,890 --> 00:19:45,090
Speaker 1: mathematical function you don't really understand. How about this? Stop

257
00:19:45,090 --> 00:19:49,450
Speaker 1: and think, think hard. Don't just trust what comes out

258
00:19:49,490 --> 00:19:52,530
Speaker 1: of the black box. Whether that black box is a

259
00:19:52,570 --> 00:19:57,770
Speaker 1: priestess possessed by Apollo, or a GPS enabled device or

260
00:19:58,130 --> 00:20:02,970
Speaker 1: the financial spreadsheet. Stopping and thinking is what most people

261
00:20:03,050 --> 00:20:07,130
Speaker 1: did in ancient Greece. Esther Eidenhau is a professor of

262
00:20:07,250 --> 00:20:10,970
Speaker 1: ancient history who studies what oracles did and what they

263
00:20:11,010 --> 00:20:14,130
Speaker 1: meant to the Greeks. She says that people would prepare

264
00:20:14,250 --> 00:20:18,450
Speaker 1: diligently before they asked for divine advice. They'd phrase their

265
00:20:18,530 --> 00:20:22,850
Speaker 1: questions carefully, they'd think about different possibilities, they'd ponder the

266
00:20:22,930 --> 00:20:28,490
Speaker 1: meaning of the answer. In two thousand and eight, a

267
00:20:28,570 --> 00:20:33,170
Speaker 1: group of Japanese were searchers led by Professor Toro Ishikawa,

268
00:20:33,330 --> 00:20:37,530
Speaker 1: ran an experiment to test what GPS does to our

269
00:20:37,530 --> 00:20:41,530
Speaker 1: capacity to notice the world around us. They directed people

270
00:20:41,610 --> 00:20:44,730
Speaker 1: on a route through Kashua, a small city near Tokyo.

271
00:20:45,450 --> 00:20:49,090
Speaker 1: Some of the experimental subjects, chosen at random, had first

272
00:20:49,130 --> 00:20:51,970
Speaker 1: been taken on the route by a guide, others had

273
00:20:52,010 --> 00:20:54,290
Speaker 1: been asked to follow the route on a paper map,

274
00:20:54,690 --> 00:21:00,650
Speaker 1: and others had GPS guidance instead. After walking the route,

275
00:21:00,650 --> 00:21:03,810
Speaker 1: everyone was asked to do it all again, this time

276
00:21:03,930 --> 00:21:07,690
Speaker 1: without help. The ones who'd followed a guide or used

277
00:21:07,690 --> 00:21:11,810
Speaker 1: a map generally man this task just fine. The ones

278
00:21:11,850 --> 00:21:15,050
Speaker 1: who had used GPS had a hard time of it.

279
00:21:15,410 --> 00:21:19,050
Speaker 1: They stopped more often, walked more slowly, and rated the

280
00:21:19,090 --> 00:21:22,490
Speaker 1: task as more difficult. The GPS may have gotten them

281
00:21:22,490 --> 00:21:25,970
Speaker 1: around first time, but it hadn't got them to engage

282
00:21:25,970 --> 00:21:29,450
Speaker 1: with the world at all. They didn't pay attention because

283
00:21:29,490 --> 00:21:34,250
Speaker 1: they didn't have to, and then they regretted it. An

284
00:21:34,250 --> 00:21:37,730
Speaker 1: automated decision doesn't have to work like that. In the

285
00:21:37,770 --> 00:21:41,090
Speaker 1: mid nineteen eighties, a group of British doctors and medical

286
00:21:41,130 --> 00:21:45,490
Speaker 1: statisticians carried out an intriguing experiment. They wanted to test

287
00:21:45,530 --> 00:21:49,610
Speaker 1: out a computerize the diagnostic system for patients suffering acute

288
00:21:49,650 --> 00:21:52,970
Speaker 1: abdominal pain. Such pain could have a lot of different

289
00:21:53,010 --> 00:21:58,370
Speaker 1: causes an ulcer, a kidney infection, a heart attack, appendicitis,

290
00:21:58,370 --> 00:22:01,850
Speaker 1: even an ectopic pregnancy, and that means a lot of

291
00:22:01,970 --> 00:22:07,330
Speaker 1: different possible treatments, so getting the diagnosis right really matters.

292
00:22:09,330 --> 00:22:12,570
Speaker 1: This being the nineteen eighties, the computers in question were

293
00:22:12,730 --> 00:22:17,370
Speaker 1: old school Apple two ease, big beige plastic bricks with

294
00:22:17,570 --> 00:22:21,010
Speaker 1: sixty four K of memory and software that you loaded

295
00:22:21,250 --> 00:22:24,050
Speaker 1: using a five and a quarter inch floppy disk. A

296
00:22:24,210 --> 00:22:27,730
Speaker 1: cheap cell phone today could easily offer a million times

297
00:22:27,770 --> 00:22:31,970
Speaker 1: more computing power. The doctor or perhaps a medical assistant

298
00:22:32,290 --> 00:22:36,730
Speaker 1: would type away on the clunky keys, laboriously entering data

299
00:22:36,770 --> 00:22:40,770
Speaker 1: into the computer and ticking yes no boxes using the cursor,

300
00:22:41,170 --> 00:22:46,250
Speaker 1: no mouse or touchscreen. Obviously, this diagnostic system wasn't bad.

301
00:22:46,890 --> 00:22:50,370
Speaker 1: It wasn't particularly good either. It took a lot of

302
00:22:50,410 --> 00:22:54,290
Speaker 1: effort to use, and it only gave the correct diagnosis

303
00:22:54,610 --> 00:22:58,490
Speaker 1: two thirds of the time. But yet it was a

304
00:22:58,690 --> 00:23:03,410
Speaker 1: huge success. The proportion of patients who died fell by

305
00:23:03,450 --> 00:23:06,410
Speaker 1: more than twenty percent, the number of cases in which

306
00:23:06,410 --> 00:23:09,690
Speaker 1: a serious medical error was made from about nine in

307
00:23:09,690 --> 00:23:13,050
Speaker 1: a thousand to just two in a thousand. There was

308
00:23:13,090 --> 00:23:17,370
Speaker 1: a huge drop in unnecessary surgeries. Why given that the

309
00:23:17,370 --> 00:23:23,330
Speaker 1: computer diagnosis wasn't all that good simple The computer prompted

310
00:23:23,330 --> 00:23:27,330
Speaker 1: the doctors to stop and think, to work through different

311
00:23:27,330 --> 00:23:30,650
Speaker 1: possibilities rather than to leap to the most obvious answer.

312
00:23:31,090 --> 00:23:34,450
Speaker 1: It was like the opposite of the GPS's used to

313
00:23:34,530 --> 00:23:38,570
Speaker 1: navigate around Cashua City, which were effortless and let people

314
00:23:38,610 --> 00:23:43,730
Speaker 1: switch off. The computer diagnostic was anything but effortless, and

315
00:23:43,810 --> 00:23:58,170
Speaker 1: it got the doctors to switch on. It's easy to

316
00:23:58,250 --> 00:24:02,130
Speaker 1: move through the world without really thinking, and usually that's fine.

317
00:24:02,490 --> 00:24:04,890
Speaker 1: It would be exhausting to have to think about everything

318
00:24:04,890 --> 00:24:08,450
Speaker 1: we do every time we do it. But sometimes being

319
00:24:08,490 --> 00:24:12,610
Speaker 1: prodded to stop and think can make you realize something important.

320
00:24:13,570 --> 00:24:16,490
Speaker 1: I don't want to get two meta here, but let's

321
00:24:16,730 --> 00:24:20,170
Speaker 1: stop and think for a moment about stopping and thinking.

322
00:24:23,010 --> 00:24:25,970
Speaker 1: There's a beautiful little experiment about what we do and

323
00:24:26,130 --> 00:24:31,330
Speaker 1: don't notice. It was conducted by two psychologists, Leonard Rosenblitt

324
00:24:31,330 --> 00:24:35,930
Speaker 1: and Frank Kyle, who gave their experimental subjects a simple task.

325
00:24:36,850 --> 00:24:40,650
Speaker 1: Here's a list of everyday objects. As you'll see, there's

326
00:24:40,690 --> 00:24:45,330
Speaker 1: a flush laboratory, a zip fastener, and several others. I'd

327
00:24:45,370 --> 00:24:48,690
Speaker 1: just like you to rate your understanding of each object

328
00:24:49,010 --> 00:24:53,810
Speaker 1: on a scale of one to seven. But after people

329
00:24:53,810 --> 00:24:57,290
Speaker 1: had written down their ratings, the researchers would launch a

330
00:24:57,370 --> 00:25:01,530
Speaker 1: gentle but devastating ambush. I see you've raided your knowledge

331
00:25:01,530 --> 00:25:05,450
Speaker 1: of the flush laboratory at six out of seven. That's great.

332
00:25:06,370 --> 00:25:09,330
Speaker 1: Here's a pen and a piece of paper. Please, would

333
00:25:09,330 --> 00:25:12,370
Speaker 1: you write out your explanation in as much detail as possible.

334
00:25:13,890 --> 00:25:19,250
Speaker 1: Feel free to use diagrams. That sometimes helps. Ah. Not

335
00:25:19,330 --> 00:25:22,290
Speaker 1: so easy, now, is it. And it wasn't that people

336
00:25:22,330 --> 00:25:26,210
Speaker 1: had been lying to the researchers. They'd been lying to themselves.

337
00:25:26,490 --> 00:25:30,490
Speaker 1: They felt they understood zippers and lavatories, but when invited

338
00:25:30,530 --> 00:25:35,050
Speaker 1: to elaborate, they realized they didn't understand at all. Rosen

339
00:25:35,090 --> 00:25:39,210
Speaker 1: Blitt and Kyle called this the illusion of explanatory depth,

340
00:25:39,810 --> 00:25:43,290
Speaker 1: and when people were asked to reconsider their previous one

341
00:25:43,370 --> 00:25:47,570
Speaker 1: to seven rating, they marked themselves down, acknowledging that their

342
00:25:47,650 --> 00:25:52,450
Speaker 1: knowledge had been shallower than they'd realized. What King Creesus

343
00:25:52,650 --> 00:25:57,010
Speaker 1: really needed was someone to gently ask him what exactly

344
00:25:57,050 --> 00:26:01,610
Speaker 1: did he think the oracle might have meant this research

345
00:26:01,810 --> 00:26:05,010
Speaker 1: is about more than zippers. It might even offer a

346
00:26:05,050 --> 00:26:08,770
Speaker 1: way to make our political discourse less polarized and bitterly

347
00:26:08,810 --> 00:26:13,930
Speaker 1: part As Anne how well, other researchers have adapted the

348
00:26:14,050 --> 00:26:17,130
Speaker 1: flush laboratory question to ask about policies such as a

349
00:26:17,250 --> 00:26:20,970
Speaker 1: cap and trade system for carbon emissions or a proposal

350
00:26:20,970 --> 00:26:26,690
Speaker 1: to impose unilateral sanctions on Iran. The researchers importantly didn't

351
00:26:26,770 --> 00:26:30,370
Speaker 1: ask people whether they were in favor or against these policies.

352
00:26:30,850 --> 00:26:34,370
Speaker 1: They just asked them the same simple question, I'd just

353
00:26:34,490 --> 00:26:37,530
Speaker 1: like you to rate your understanding on a scale of

354
00:26:37,530 --> 00:26:42,130
Speaker 1: one to seven, And the same thing happened. People said, yes,

355
00:26:42,170 --> 00:26:46,890
Speaker 1: they basically understood these policies fairly well. Then, when prompted

356
00:26:46,970 --> 00:26:52,410
Speaker 1: to explain, the illusion faded, they realized that perhaps they

357
00:26:52,450 --> 00:26:58,770
Speaker 1: didn't really understand at all. Another thing that faded political polarization.

358
00:26:59,290 --> 00:27:02,770
Speaker 1: People who would have instinctively described their political opponents as

359
00:27:02,810 --> 00:27:05,290
Speaker 1: wicked and who would have gone to the barricades to

360
00:27:05,330 --> 00:27:09,450
Speaker 1: defend their own ideas, tended to be less strident when

361
00:27:09,530 --> 00:27:13,090
Speaker 1: forced to admit to themselves that they didn't really understand

362
00:27:13,250 --> 00:27:15,490
Speaker 1: what it was that they were so passionate about in

363
00:27:15,490 --> 00:27:19,730
Speaker 1: the first place. It's a rather beautiful discovery. In a

364
00:27:19,810 --> 00:27:23,290
Speaker 1: world where so many people seem to hold extreme views

365
00:27:23,330 --> 00:27:28,050
Speaker 1: with strident certainty. You can deflate someone's over confidence and

366
00:27:28,250 --> 00:27:32,730
Speaker 1: moderate their politics simply by asking them to explain the details.

367
00:27:33,570 --> 00:27:36,010
Speaker 1: Whether we're asking people to walk through a city in

368
00:27:36,130 --> 00:27:40,930
Speaker 1: Japan or talk about their political differences, it really helps

369
00:27:41,010 --> 00:27:49,370
Speaker 1: to call their attention to what they're actually doing. Asking

370
00:27:49,410 --> 00:27:53,410
Speaker 1: the oracle or the computer might give us a better prediction,

371
00:27:53,970 --> 00:27:57,050
Speaker 1: but it discourages us from thinking hard about the world

372
00:27:57,090 --> 00:28:00,290
Speaker 1: around us, and that's not something we should be giving

373
00:28:00,370 --> 00:28:05,530
Speaker 1: up lightly. Sometimes it's not the forecast that matters. It's

374
00:28:05,530 --> 00:28:09,450
Speaker 1: whether that forecast helps you think harder or encourage you

375
00:28:09,770 --> 00:28:21,330
Speaker 1: to stop thinking altogether. Nearly seventy years after the fall

376
00:28:21,410 --> 00:28:26,130
Speaker 1: of King Croesus, another great civilization was at war with

377
00:28:26,210 --> 00:28:30,610
Speaker 1: the Persians. An alliance of Greek city states faced an

378
00:28:30,610 --> 00:28:35,490
Speaker 1: invasion by a vast Persian army and turned once again

379
00:28:36,010 --> 00:28:41,170
Speaker 1: the Delphic Oracle for advice. Ambassadors from Athens received the

380
00:28:41,250 --> 00:28:47,850
Speaker 1: following prophecy, which didn't sound encouraging. Wretches, Why sit ye here,

381
00:28:48,690 --> 00:28:56,090
Speaker 1: fly fly to the ends of creation? All ruined and lost?

382
00:28:56,930 --> 00:29:02,690
Speaker 1: Lo from the high Roof's trickle of black blood, signed

383
00:29:02,770 --> 00:29:08,490
Speaker 1: prophetic of hard distresses, impending get ye away from the

384
00:29:08,570 --> 00:29:13,810
Speaker 1: temple and brood on the ills that await ye. The

385
00:29:13,850 --> 00:29:17,490
Speaker 1: ambassadors were dismayed. They couldn't go back to Athens with that,

386
00:29:18,370 --> 00:29:23,090
Speaker 1: and so they asked the Delphic oracle for another answer.

387
00:29:23,770 --> 00:29:28,410
Speaker 1: The second prophecy was vivid but confusing, and included some

388
00:29:28,530 --> 00:29:33,130
Speaker 1: lines that hinted at safety. Safe, Shall the wooden wall

389
00:29:33,210 --> 00:29:37,890
Speaker 1: continue for thee and thy children? Wait not the tramp

390
00:29:37,930 --> 00:29:41,090
Speaker 1: of the horse, nor the footman mightily moving over the land,

391
00:29:42,450 --> 00:29:45,890
Speaker 1: But turn your back to the foe and retire ye.

392
00:29:46,650 --> 00:29:49,730
Speaker 1: Yet shall a day arrive when ye shall meet him

393
00:29:49,930 --> 00:29:54,050
Speaker 1: in battle. When this message was brought back to Athens,

394
00:29:54,250 --> 00:29:56,970
Speaker 1: it was the subject of heated debate. What did the

395
00:29:57,010 --> 00:30:02,570
Speaker 1: oracle mean? Different factions saw different meanings and made different arguments.

396
00:30:03,010 --> 00:30:06,970
Speaker 1: The oracle mentioned a battle at Holy Salamis, an island

397
00:30:07,050 --> 00:30:10,330
Speaker 1: near the coast of Athens, where any men would perish.

398
00:30:10,770 --> 00:30:14,490
Speaker 1: That didn't sound good, But maybe that was King Creesus's

399
00:30:14,690 --> 00:30:17,770
Speaker 1: error in reverse. Maybe the men who would die would

400
00:30:17,770 --> 00:30:22,810
Speaker 1: be the Persian invaders. I realize this all sounds absurd

401
00:30:22,930 --> 00:30:27,450
Speaker 1: to modern ears, poetic and completely ambiguous. Predictions from a

402
00:30:27,530 --> 00:30:32,930
Speaker 1: Greek god possessing a young priestess. But the Athenians did

403
00:30:33,010 --> 00:30:36,970
Speaker 1: what we should still be doing. When faced with any forecast,

404
00:30:37,410 --> 00:30:41,610
Speaker 1: They discussed and debated. They asked, what does this really mean?

405
00:30:42,050 --> 00:30:45,530
Speaker 1: Are we sure? How seriously should we take it? And

406
00:30:45,690 --> 00:30:48,770
Speaker 1: what are we going to do about it? The Athenian

407
00:30:48,850 --> 00:30:53,970
Speaker 1: general Themisticles successfully argued that the wooden Wall referred to

408
00:30:54,010 --> 00:30:57,490
Speaker 1: the Greek navy, and the Greeks should seek a sea battle.

409
00:30:58,410 --> 00:31:03,050
Speaker 1: They did and destroyed the large Persian fleet at Salamis.

410
00:31:04,090 --> 00:31:07,010
Speaker 1: It's a strange, old story about a very different culture,

411
00:31:07,570 --> 00:31:11,370
Speaker 1: but we could learn from it. While we humans might

412
00:31:11,410 --> 00:31:14,810
Speaker 1: not be very good at seeing into the future, thinking

413
00:31:14,930 --> 00:31:19,130
Speaker 1: seriously about what the future holds might just make us

414
00:31:19,130 --> 00:31:29,370
Speaker 1: slightly better humans. If you've been with me for the

415
00:31:29,570 --> 00:31:34,210
Speaker 1: entire first season of Cautionary Tales, thank you. You'll have

416
00:31:34,290 --> 00:31:37,730
Speaker 1: heard what airships teach us about the downsides of competition,

417
00:31:38,130 --> 00:31:41,610
Speaker 1: what an apocalyptic cult shows us about changing our minds,

418
00:31:41,970 --> 00:31:44,890
Speaker 1: and what a charismatic con artist tells us about the

419
00:31:44,970 --> 00:31:51,170
Speaker 1: power of persuasion, one small step at a time. I

420
00:31:51,290 --> 00:31:54,570
Speaker 1: hope that my strange stories have made you wiser, and

421
00:31:54,690 --> 00:31:57,090
Speaker 1: I hope that They've been fun to listen to. I've

422
00:31:57,130 --> 00:32:00,570
Speaker 1: certainly enjoyed making them. Thanks again for joining me. Please

423
00:32:00,810 --> 00:32:03,090
Speaker 1: tell your friends I hope to be back with a

424
00:32:03,130 --> 00:32:07,810
Speaker 1: second season of Cautionary Tales before long. There is alas

425
00:32:08,490 --> 00:32:12,370
Speaker 1: no sure stage of calamities from which we can all learn.

426
00:32:15,130 --> 00:32:18,730
Speaker 1: Cautionary Tales is written and presented by me Tim Harford.

427
00:32:19,090 --> 00:32:23,050
Speaker 1: Our producers are Ryan Dilley and Marilyn Rust. The sound

428
00:32:23,090 --> 00:32:27,050
Speaker 1: designer and mixer was Pascal Wise, who also composed the

429
00:32:27,090 --> 00:32:34,290
Speaker 1: amazing music. This season stars Alan Cumming, Archie Panchabi, Toby

430
00:32:34,330 --> 00:32:39,650
Speaker 1: Stephens and Russell Tovey, with enso Celenti, Ed Gochen, Melanie Gutteridge,

431
00:32:39,810 --> 00:32:44,730
Speaker 1: Mass Siam and Roe rufus Wright and introducing Malcolm Gladwell.

432
00:32:45,970 --> 00:32:50,290
Speaker 1: Thanks to the team at Pushkin Industries, Julia Barton, Heather Fane,

433
00:32:50,570 --> 00:32:55,010
Speaker 1: Mia LaBelle, Carlie Milliori, Jacob Weisberg and of course the

434
00:32:55,170 --> 00:32:58,450
Speaker 1: mighty Malcolm Gladwell. And thanks to my colleagues at the

435
00:32:58,490 --> 00:32:59,410
Speaker 1: Financial Times