WEBVTT - Special Episode: Adam Kucharski & Proof

0:00:44.240 --> 0:00:47.760
<v Speaker 1>Hi, I'm Aaron Welsh and this is This Podcast will

0:00:47.840 --> 0:00:51.000
<v Speaker 1>Kill You. You're tuning in to the latest episode of

0:00:51.040 --> 0:00:54.000
<v Speaker 1>the tp w K Y Book Club, where I chat

0:00:54.040 --> 0:00:57.120
<v Speaker 1>with authors of popular science and medicine books about their

0:00:57.200 --> 0:01:00.520
<v Speaker 1>latest work. Since starting this series a few years ago,

0:01:00.840 --> 0:01:04.520
<v Speaker 1>I've gotten to cover some amazing books and I appreciate

0:01:04.640 --> 0:01:07.200
<v Speaker 1>so many of you reaching out with your suggestions for

0:01:07.240 --> 0:01:11.240
<v Speaker 1>books to feature. Keep those recommendations coming, please, and if

0:01:11.240 --> 0:01:12.960
<v Speaker 1>you'd like to take a look at the full list

0:01:12.959 --> 0:01:15.240
<v Speaker 1>of books that we've covered in this series, as well

0:01:15.280 --> 0:01:17.240
<v Speaker 1>as get a sneak peek at ones that are coming

0:01:17.360 --> 0:01:20.400
<v Speaker 1>up in future episodes, head on over to our bookshop

0:01:20.440 --> 0:01:23.080
<v Speaker 1>dot org affiliate page, which you can find on our

0:01:23.120 --> 0:01:25.720
<v Speaker 1>website This Podcast will Kill You dot com. Under the

0:01:25.800 --> 0:01:29.680
<v Speaker 1>extras tab on the bookshop page, you'll find several podcast

0:01:29.720 --> 0:01:33.399
<v Speaker 1>related lists, including one for this book Club and the

0:01:33.480 --> 0:01:37.320
<v Speaker 1>TPWKY Kids book Club, which if you're not following us

0:01:37.360 --> 0:01:40.679
<v Speaker 1>on social media you absolutely should be because Aaron Epdyke

0:01:40.720 --> 0:01:44.160
<v Speaker 1>has been putting together videos reviewing children's books. It is

0:01:44.240 --> 0:01:47.000
<v Speaker 1>such a great resource for sciencey kids books for all

0:01:47.040 --> 0:01:49.760
<v Speaker 1>ages and if you want to share your thoughts on

0:01:49.800 --> 0:01:53.600
<v Speaker 1>these episodes, make topic suggestions submit a first hand account.

0:01:53.680 --> 0:01:55.720
<v Speaker 1>You can get in touch with us using the contact

0:01:55.840 --> 0:01:59.360
<v Speaker 1>us form on our website. Two last things before moving

0:01:59.400 --> 0:02:01.200
<v Speaker 1>on to the Book of the Week, and that is

0:02:01.240 --> 0:02:04.360
<v Speaker 1>to please rate, review and subscribe. It really does help

0:02:04.440 --> 0:02:07.559
<v Speaker 1>us out. And second, you can now find full video

0:02:07.640 --> 0:02:10.600
<v Speaker 1>versions of most of our newest episodes on YouTube. Make

0:02:10.639 --> 0:02:13.600
<v Speaker 1>sure you're subscribed to the exactly Right Media YouTube channel

0:02:13.639 --> 0:02:17.600
<v Speaker 1>so you never miss a new episode drop. Belief is

0:02:17.720 --> 0:02:22.040
<v Speaker 1>a powerful force. It shapes every facet of our lives

0:02:22.160 --> 0:02:26.760
<v Speaker 1>and transforms perception into reality. What we believe to be

0:02:26.840 --> 0:02:30.480
<v Speaker 1>true is not always what is actually true, something I'm

0:02:30.520 --> 0:02:33.080
<v Speaker 1>sure we can all relate to. Maybe you've debated with

0:02:33.120 --> 0:02:35.840
<v Speaker 1>a friend over the answer to a trivia question, like

0:02:35.919 --> 0:02:38.359
<v Speaker 1>you both know the right answer, but your answers are

0:02:38.400 --> 0:02:42.320
<v Speaker 1>somehow different. Or maybe you've had a heated exchange with

0:02:42.360 --> 0:02:46.000
<v Speaker 1>a relative who firmly believes that the moon landing was faked.

0:02:46.840 --> 0:02:50.280
<v Speaker 1>How do we decide what we believe? How can we

0:02:50.400 --> 0:02:54.079
<v Speaker 1>know that what we believe is the truth, and how

0:02:54.080 --> 0:02:57.960
<v Speaker 1>can we convince others of that? These are precisely the

0:02:58.040 --> 0:03:01.160
<v Speaker 1>questions that Adam Kucharski, who is professor at the London

0:03:01.200 --> 0:03:05.520
<v Speaker 1>School of hygiene and tropical medicine asks. In his latest book, Proof,

0:03:05.760 --> 0:03:09.200
<v Speaker 1>The Art and Science of Certainty, Kucharski, who is a

0:03:09.240 --> 0:03:13.600
<v Speaker 1>mathematician that works on infectious disease outbreaks, explores how we

0:03:13.639 --> 0:03:18.880
<v Speaker 1>are inundated with information and increasingly misinformation, that we have

0:03:18.960 --> 0:03:22.040
<v Speaker 1>to evaluate to determine whether or not we should incorporate

0:03:22.080 --> 0:03:26.560
<v Speaker 1>it into our decision making. This extends beyond personal decisions

0:03:26.760 --> 0:03:29.000
<v Speaker 1>which root is best to take to work, what to

0:03:29.040 --> 0:03:33.160
<v Speaker 1>make for dinner. Our world is built upon structures of

0:03:33.200 --> 0:03:36.880
<v Speaker 1>proof with varying degrees of support. That car that you

0:03:37.000 --> 0:03:41.520
<v Speaker 1>drive to work is manufactured under rigorous safety testing, meaning

0:03:41.600 --> 0:03:45.360
<v Speaker 1>there are established guidelines for what is considered safe and

0:03:45.400 --> 0:03:48.400
<v Speaker 1>how to test that same thing. With the food we eat,

0:03:48.440 --> 0:03:51.160
<v Speaker 1>the medicines we take, the buildings we spend time in.

0:03:51.960 --> 0:03:55.280
<v Speaker 1>We don't question so many of our beliefs. To do

0:03:55.400 --> 0:03:58.840
<v Speaker 1>so would leave you frozen, uncertain of which direction to

0:03:58.920 --> 0:04:01.720
<v Speaker 1>move in, what to trust. You'd have no time to

0:04:01.840 --> 0:04:05.880
<v Speaker 1>actually live your life. But when we do scrutinize our certainty,

0:04:06.240 --> 0:04:10.160
<v Speaker 1>we might find a gulf between our beliefs and someone else's,

0:04:10.760 --> 0:04:15.320
<v Speaker 1>and those beliefs and the objective truth. Where does that

0:04:15.480 --> 0:04:20.240
<v Speaker 1>incongruity originate. Why are we skeptical about some things and

0:04:20.320 --> 0:04:23.280
<v Speaker 1>not others? What does it take to make up our

0:04:23.320 --> 0:04:26.479
<v Speaker 1>mind and what does it take to change it? That

0:04:26.640 --> 0:04:30.680
<v Speaker 1>answer might not be the same for everyone. An enlightening

0:04:30.760 --> 0:04:37.040
<v Speaker 1>blend of philosophical musings, political commentary, statistical exploration, and personal reflection.

0:04:37.680 --> 0:04:42.560
<v Speaker 1>Proof is a fascinating read, particularly as this unceasing flood

0:04:42.600 --> 0:04:47.040
<v Speaker 1>of information, both good and bad, shows no sign of stopping.

0:04:47.920 --> 0:04:50.799
<v Speaker 1>Let's take a quick break and then get into things.

0:05:15.320 --> 0:05:17.920
<v Speaker 1>Professor Kochowski, thank you so much for joining me today.

0:05:18.279 --> 0:05:19.000
<v Speaker 2>Thanks for having me.

0:05:19.640 --> 0:05:23.480
<v Speaker 1>I am thrilled to talk with you about your latest book, Proof,

0:05:23.680 --> 0:05:26.800
<v Speaker 1>The Art and Science of Certainty. And before we dig

0:05:26.839 --> 0:05:29.800
<v Speaker 1>into the various forms of proof and how we determine

0:05:29.800 --> 0:05:32.760
<v Speaker 1>a threshold for proof or what different types of proof

0:05:32.800 --> 0:05:35.000
<v Speaker 1>exists for certain situations, I want to start at the

0:05:35.120 --> 0:05:39.599
<v Speaker 1>very beginning. What is proof? Is there a standard definition?

0:05:40.480 --> 0:05:40.680
<v Speaker 2>Yes?

0:05:40.760 --> 0:05:42.960
<v Speaker 3>I think that's a great question, and that I think.

0:05:43.560 --> 0:05:46.520
<v Speaker 3>My background's in math, so I think a lot of

0:05:46.560 --> 0:05:49.120
<v Speaker 3>MICA training was around this idea that you can have

0:05:49.920 --> 0:05:54.000
<v Speaker 3>this definitive knowledge that something is true, and I think

0:05:54.000 --> 0:05:56.640
<v Speaker 3>it's something that people grappled with across fields I mean,

0:05:56.640 --> 0:05:59.240
<v Speaker 3>more of the stories that really struck me was Abraham Lincoln,

0:05:59.320 --> 0:06:03.200
<v Speaker 3>when he was training to be a lawyer, came across

0:06:03.200 --> 0:06:06.119
<v Speaker 3>this word demonstrate and yeah, this kind of beyond reasonable doubt,

0:06:06.120 --> 0:06:08.400
<v Speaker 3>this certainty, and he's like, I don't really understand what

0:06:08.560 --> 0:06:11.240
<v Speaker 3>this is as a concept, and he actually went back

0:06:11.279 --> 0:06:15.600
<v Speaker 3>to all of these ancient Greek mathematical texts to understand

0:06:15.680 --> 0:06:19.679
<v Speaker 3>how can we take what the knowledge we have, build

0:06:19.760 --> 0:06:23.560
<v Speaker 3>on that prove new theorems, use that to prove subsequent knowledge.

0:06:23.560 --> 0:06:25.760
<v Speaker 3>But I think one of the things that was really

0:06:25.760 --> 0:06:28.360
<v Speaker 3>the motivation for the book, and something that I think

0:06:28.400 --> 0:06:31.839
<v Speaker 3>anyone who works with information and decision making and evidence

0:06:32.240 --> 0:06:35.880
<v Speaker 3>happens across very often is it can become quite a

0:06:36.600 --> 0:06:39.359
<v Speaker 3>shifting concepts. I mean, even in mathematics, things that people

0:06:39.480 --> 0:06:44.120
<v Speaker 3>thought were proven turned out had some hidden assumptions or

0:06:44.200 --> 0:06:48.320
<v Speaker 3>human judgments that were kind of lurking there and caused

0:06:48.320 --> 0:06:50.000
<v Speaker 3>a lot of that to collapse. So I think it's

0:06:50.320 --> 0:06:52.880
<v Speaker 3>it's a kind of fascinating concept because it's something that's

0:06:52.920 --> 0:06:56.440
<v Speaker 3>so important in life, not just having knowledge that we

0:06:56.480 --> 0:06:58.720
<v Speaker 3>gradually accrue, but from many of the things we care about,

0:06:58.800 --> 0:07:01.960
<v Speaker 3>whether it's dealing with them, emergency, whether it's a legal case,

0:07:02.760 --> 0:07:05.239
<v Speaker 3>whether it's even just a kind of minor business decision

0:07:05.320 --> 0:07:08.080
<v Speaker 3>in our day, we have to work out where we

0:07:08.120 --> 0:07:10.520
<v Speaker 3>set the bar and how we evaluate what we've got.

0:07:10.640 --> 0:07:13.360
<v Speaker 3>And I think for me that was really the launching

0:07:13.400 --> 0:07:16.120
<v Speaker 3>off point to explore this. You know, how do we

0:07:16.200 --> 0:07:19.120
<v Speaker 3>converge on certainty and what happens when it goes wrong?

0:07:20.000 --> 0:07:24.640
<v Speaker 1>Thinking about the difference between proof and certainty and truth, like,

0:07:24.720 --> 0:07:27.280
<v Speaker 1>what is the relationship between those concepts?

0:07:27.680 --> 0:07:30.480
<v Speaker 3>Yeah, I think that's a great question, and without going

0:07:30.520 --> 0:07:33.040
<v Speaker 3>down the kind of philosophical rabbit or it could have

0:07:33.080 --> 0:07:33.800
<v Speaker 3>been a book.

0:07:33.560 --> 0:07:34.720
<v Speaker 2>On you what is reality?

0:07:35.840 --> 0:07:38.080
<v Speaker 3>Yeah, But I think the way that I approached it

0:07:38.120 --> 0:07:40.640
<v Speaker 3>is just to look at how people thought about this

0:07:40.840 --> 0:07:44.880
<v Speaker 3>in different fields. And again even going back to Lincoln

0:07:45.080 --> 0:07:49.480
<v Speaker 3>and much earlier, there was this this appeal of this certainty,

0:07:49.920 --> 0:07:51.640
<v Speaker 3>this idea that there could be this universe truth, and

0:07:51.640 --> 0:07:54.320
<v Speaker 3>it's why a lot of fields ended up borrowing for mathematics.

0:07:54.320 --> 0:07:56.320
<v Speaker 3>You see it in the US Declaration of Independence. Yeah,

0:07:56.320 --> 0:07:58.480
<v Speaker 3>we hold these truths to be self evident. The visional

0:07:58.560 --> 0:08:01.080
<v Speaker 3>draft was we hold these truths to be sacred and undeniable.

0:08:01.160 --> 0:08:03.520
<v Speaker 3>But Benjamin Franklin didn't like that, because it sounded like

0:08:03.520 --> 0:08:07.000
<v Speaker 3>they were kind of appealing to some divine authority and

0:08:07.120 --> 0:08:10.560
<v Speaker 3>self evidence is just borrowed directly for maths, it's just

0:08:10.600 --> 0:08:13.360
<v Speaker 3>a given truth. And unfortunately it turned out a lot

0:08:13.400 --> 0:08:16.480
<v Speaker 3>of these things about equality weren't self evident. But I

0:08:16.480 --> 0:08:19.560
<v Speaker 3>think that that story of how you think about these things,

0:08:19.600 --> 0:08:21.680
<v Speaker 3>and even when we see in the legal world, a

0:08:21.680 --> 0:08:24.520
<v Speaker 3>lot of it was originally derived from concepts around matter,

0:08:24.520 --> 0:08:26.840
<v Speaker 3>around probability. If you talk about, you know, some of

0:08:26.880 --> 0:08:30.200
<v Speaker 3>these thresholds preponderance of evidence, you're saying it's more likely

0:08:30.240 --> 0:08:31.760
<v Speaker 3>than not, and you're kind of borrowing a lot of

0:08:31.760 --> 0:08:34.560
<v Speaker 3>these kind of probability based ideas, and even in the

0:08:34.600 --> 0:08:39.120
<v Speaker 3>world kind of more experimental design as that kind of developed,

0:08:39.240 --> 0:08:41.800
<v Speaker 3>a lot of it was about I mean, actually some

0:08:41.840 --> 0:08:44.920
<v Speaker 3>of these early studies were almost trying to discount some

0:08:45.000 --> 0:08:47.200
<v Speaker 3>of the influences of religion. You're wanting to understand cause

0:08:47.200 --> 0:08:50.440
<v Speaker 3>of effect in the world, rather than just appealing to

0:08:50.520 --> 0:08:51.800
<v Speaker 3>some other influence.

0:08:52.440 --> 0:08:56.040
<v Speaker 2>And then it for a lot of people it became a.

0:08:56.000 --> 0:08:57.920
<v Speaker 3>Discussion of how do you take the evidence you have

0:08:58.000 --> 0:09:00.720
<v Speaker 3>and how do you link that to a conclusion that

0:09:00.760 --> 0:09:02.440
<v Speaker 3>you want to make, and where do you set the

0:09:02.480 --> 0:09:05.120
<v Speaker 3>bar for that do you try and get ever closer

0:09:05.160 --> 0:09:08.360
<v Speaker 3>to certainty? And there's actually a lot of statistical tension

0:09:08.360 --> 0:09:10.559
<v Speaker 3>about one hundred years ago, no statistical debates. Kind of

0:09:10.600 --> 0:09:13.720
<v Speaker 3>sounds a bit boring, but it was actually this real. Yeah,

0:09:13.720 --> 0:09:16.360
<v Speaker 3>people just almost I wouldn't talk to each other because

0:09:16.360 --> 0:09:19.200
<v Speaker 3>there was this tension between do you just try and

0:09:19.400 --> 0:09:22.600
<v Speaker 3>get ever closer to the truth or do you have

0:09:22.640 --> 0:09:25.120
<v Speaker 3>a framework that allows you to make decisions? And I

0:09:25.160 --> 0:09:27.240
<v Speaker 3>think a lot of times in life we don't get

0:09:27.280 --> 0:09:29.040
<v Speaker 3>to do the academic I'm just going to sit on

0:09:29.040 --> 0:09:31.360
<v Speaker 3>the fence yet I just I don't know, and I'm

0:09:31.400 --> 0:09:35.079
<v Speaker 3>just not going to do anything with life or actions

0:09:35.240 --> 0:09:37.520
<v Speaker 3>that often we have to decide we do something or

0:09:37.559 --> 0:09:39.600
<v Speaker 3>we don't do something, or you know, we say someone's

0:09:39.600 --> 0:09:41.360
<v Speaker 3>guilty or we let them go three or there's these

0:09:41.400 --> 0:09:45.040
<v Speaker 3>decisions we have to make. And so that process of

0:09:45.600 --> 0:09:48.120
<v Speaker 3>interacting with evidence is much more pressure and I think

0:09:48.160 --> 0:09:50.080
<v Speaker 3>that that was one of the real big tensions that

0:09:50.640 --> 0:09:51.640
<v Speaker 3>never fully got resolved.

0:09:51.679 --> 0:09:52.760
<v Speaker 2>Actually, even how.

0:09:52.520 --> 0:09:55.760
<v Speaker 3>We teach statistics at school, we kind of smushed together

0:09:55.800 --> 0:09:59.319
<v Speaker 3>these two very different philosophies, one of this ever higher

0:09:59.320 --> 0:10:02.280
<v Speaker 3>bar for evidence and one where we're sort of outlining

0:10:02.280 --> 0:10:04.920
<v Speaker 3>a framework to make a decision based on the knowledge

0:10:04.960 --> 0:10:05.200
<v Speaker 3>we have.

0:10:05.880 --> 0:10:09.840
<v Speaker 1>When it comes to public health and medicine, there's a

0:10:09.840 --> 0:10:13.320
<v Speaker 1>lot more pressing, you know, need to make decisions, and

0:10:13.520 --> 0:10:17.439
<v Speaker 1>yet this decision is often dragged out for long periods

0:10:17.440 --> 0:10:21.600
<v Speaker 1>of time, and sometimes that is at the urging of,

0:10:22.000 --> 0:10:25.640
<v Speaker 1>you know, someone who has incentive to drag out a decision.

0:10:25.720 --> 0:10:28.120
<v Speaker 1>So one of the examples that you talk about is

0:10:28.240 --> 0:10:32.480
<v Speaker 1>Austin Bradford Hill, who is talking about this relationship between

0:10:32.520 --> 0:10:34.720
<v Speaker 1>cigarettes and lung cancer and saying, oh, we have the

0:10:34.960 --> 0:10:38.120
<v Speaker 1>we have some evidence, and there's still a lot of skepticism,

0:10:38.320 --> 0:10:40.440
<v Speaker 1>but we have enough to make a decision. We cannot

0:10:40.600 --> 0:10:44.679
<v Speaker 1>use uncertainty as an excuse for inaction. Do you feel

0:10:44.720 --> 0:10:48.920
<v Speaker 1>like that, like we've ever truly learned that as a society,

0:10:49.040 --> 0:10:53.040
<v Speaker 1>or has it been you know, players like the tobacco

0:10:53.120 --> 0:10:56.199
<v Speaker 1>industry saying oh, no, this uncertainty, you know, we need

0:10:56.240 --> 0:10:58.200
<v Speaker 1>to push for more and more and more evidence.

0:10:59.160 --> 0:10:59.360
<v Speaker 3>Yeah.

0:10:59.400 --> 0:11:01.720
<v Speaker 2>I think that's so, that's a really good question.

0:11:01.760 --> 0:11:04.080
<v Speaker 3>I think that's a really good example of almost kind

0:11:04.080 --> 0:11:09.160
<v Speaker 3>of weaponized certainty, that you can always set the bar higher,

0:11:09.200 --> 0:11:11.000
<v Speaker 3>and any aspects of life, you can set the bar

0:11:11.080 --> 0:11:13.160
<v Speaker 3>high and higher and higher to the point where you

0:11:13.240 --> 0:11:16.480
<v Speaker 3>just won't do anything. And in action, of course, is

0:11:16.840 --> 0:11:19.439
<v Speaker 3>in itself a decision. And I think Bradford Hill's work,

0:11:19.480 --> 0:11:22.679
<v Speaker 3>you know, he was extremely thoughtful in how we approached

0:11:22.679 --> 0:11:25.920
<v Speaker 3>this because something like smoking, you can't really design it

0:11:25.960 --> 0:11:27.800
<v Speaker 3>like a try. You can't get people to randomly take

0:11:27.840 --> 0:11:30.160
<v Speaker 3>up smoking and see if they get cancer. There's obviously

0:11:30.200 --> 0:11:32.719
<v Speaker 3>ethical reasons about there's also just timeline reasons. You know,

0:11:32.760 --> 0:11:35.400
<v Speaker 3>if you look at the timescale of the intervention versus

0:11:35.400 --> 0:11:36.880
<v Speaker 3>what happened, you might have to wait decades to have

0:11:36.960 --> 0:11:39.480
<v Speaker 3>that clear signal. And so he did a lot of

0:11:39.600 --> 0:11:44.040
<v Speaker 3>poning work with others, linking together the various sort of

0:11:44.120 --> 0:11:46.920
<v Speaker 3>non random data sets you had available. Because one of

0:11:47.000 --> 0:11:49.160
<v Speaker 3>the criticisms, of course, as any days, is yes, smoke

0:11:49.240 --> 0:11:50.840
<v Speaker 3>is more like to get cancer, but maybe there's a

0:11:50.840 --> 0:11:52.839
<v Speaker 3>genetic reason that makes them all like to smoke and

0:11:53.520 --> 0:11:55.800
<v Speaker 3>get this. And he outlined a lot of the ways

0:11:55.840 --> 0:11:57.760
<v Speaker 3>we can think about cause and effect, and I think

0:11:57.800 --> 0:12:00.280
<v Speaker 3>that's a very useful set of concept and some of

0:12:00.320 --> 0:12:02.599
<v Speaker 3>it the obvious ones of the cause that needs to

0:12:02.679 --> 0:12:04.840
<v Speaker 3>come before the effect, or that you know, if you

0:12:04.920 --> 0:12:08.360
<v Speaker 3>have the strength of association more of cigarettes makes you

0:12:08.400 --> 0:12:10.240
<v Speaker 3>more likely to get cancer, if you see that across

0:12:10.280 --> 0:12:12.480
<v Speaker 3>multiple countries, or if you can start to think about,

0:12:12.800 --> 0:12:16.600
<v Speaker 3>you know, the biological plausibility. We see carstinogens in other

0:12:16.679 --> 0:12:19.360
<v Speaker 3>kind of situations as well. None of those things on

0:12:19.400 --> 0:12:22.800
<v Speaker 3>their own is conclusive. You can start to build this

0:12:22.920 --> 0:12:25.079
<v Speaker 3>evidence space. And he made this really good point that

0:12:25.920 --> 0:12:28.679
<v Speaker 3>any knowledge we have, even if it's very confident knowledge,

0:12:29.160 --> 0:12:33.400
<v Speaker 3>is always subject to further refinement. But we still have

0:12:33.480 --> 0:12:35.920
<v Speaker 3>that knowledge at that point in time, and we can

0:12:36.480 --> 0:12:40.120
<v Speaker 3>seek further information. There's been lots more studies of smoking

0:12:40.160 --> 0:12:44.040
<v Speaker 3>since they're early ones, but also that's information that we

0:12:44.120 --> 0:12:45.640
<v Speaker 3>have to do something with. And I think we often

0:12:45.720 --> 0:12:49.400
<v Speaker 3>particularly in the situations with emerging threats or kind of

0:12:49.440 --> 0:12:52.800
<v Speaker 3>early concerns about things, whether it's a health intervention we

0:12:52.840 --> 0:12:54.920
<v Speaker 3>think might be harmful. I mean, what of the examples

0:12:55.080 --> 0:12:58.000
<v Speaker 3>are given the book is the work at the FDA

0:12:58.080 --> 0:13:03.000
<v Speaker 3>around flidamide, which was this for sickness in pregnancy, and

0:13:04.640 --> 0:13:06.960
<v Speaker 3>there was actually a lot of concerns about safety for babies,

0:13:06.960 --> 0:13:09.760
<v Speaker 3>and the FDA blocked it as a result. But on

0:13:09.800 --> 0:13:12.640
<v Speaker 3>the other hand, you get things where there might be

0:13:13.080 --> 0:13:16.400
<v Speaker 3>a lot of value. For example, in reducing smoking for

0:13:16.720 --> 0:13:19.559
<v Speaker 3>health outcomes, and even if there's an uncertainty, And Bradford

0:13:19.600 --> 0:13:23.079
<v Speaker 3>Hill made this nice point of actually the standard you

0:13:23.120 --> 0:13:26.040
<v Speaker 3>should apply for taking action kind of depends a bit

0:13:26.080 --> 0:13:27.920
<v Speaker 3>on the situation you're dealing with, if it's a fairly

0:13:28.000 --> 0:13:30.520
<v Speaker 3>cheap action to take, if it's not too disruptive for people.

0:13:31.240 --> 0:13:34.160
<v Speaker 3>But actually, in his argument he said, smoking is something

0:13:34.200 --> 0:13:36.679
<v Speaker 3>people really enjoy, so we need a kind of.

0:13:36.679 --> 0:13:38.160
<v Speaker 2>Higher barrier if you're get it.

0:13:38.160 --> 0:13:40.080
<v Speaker 3>And I think it's a reasonable point if you're going

0:13:40.080 --> 0:13:41.600
<v Speaker 3>to tell a lot of people to change how they

0:13:41.600 --> 0:13:45.920
<v Speaker 3>live their lives, that the evidence you need is perhaps

0:13:45.920 --> 0:13:49.760
<v Speaker 3>different for something where you can take some action and

0:13:49.800 --> 0:13:51.800
<v Speaker 3>you can unwind that. So it is those kind of

0:13:51.880 --> 0:13:54.559
<v Speaker 3>trade offs that you have available that obviously need to

0:13:54.600 --> 0:13:55.240
<v Speaker 3>play in as well.

0:13:56.559 --> 0:13:58.640
<v Speaker 1>Let's take a short break and when we get back,

0:13:58.800 --> 0:14:19.640
<v Speaker 1>there's still so much to discuss. Welcome back everyone. I've

0:14:19.680 --> 0:14:22.760
<v Speaker 1>been chatting with doctor Adam Kucharski about his book Proof,

0:14:23.000 --> 0:14:25.800
<v Speaker 1>The Art and Science of Certainty. Let's get back into

0:14:25.840 --> 0:14:31.720
<v Speaker 1>things right. The thresholds for certainty is it can be

0:14:31.760 --> 0:14:34.120
<v Speaker 1>different depending on the situation, and then there's also these

0:14:34.160 --> 0:14:38.240
<v Speaker 1>personal thresholds for certainty or evidence, you know, how much

0:14:38.280 --> 0:14:40.920
<v Speaker 1>information do we need? And one of the things that

0:14:41.000 --> 0:14:43.000
<v Speaker 1>you discuss in your book as well is sort of

0:14:43.000 --> 0:14:46.440
<v Speaker 1>what happens when evidence flies in the face of our

0:14:46.480 --> 0:14:51.760
<v Speaker 1>personal beliefs, and how sometimes even despite a mountain of evidence,

0:14:51.920 --> 0:14:54.560
<v Speaker 1>we can just still feel like that's not possible. We

0:14:54.600 --> 0:14:57.680
<v Speaker 1>can't reject it. It's not an intuitive truth, you know,

0:14:57.840 --> 0:15:01.240
<v Speaker 1>what happens, Like what is show us about sort of

0:15:01.280 --> 0:15:04.240
<v Speaker 1>the personal nature of proof and certainty.

0:15:04.840 --> 0:15:07.160
<v Speaker 3>Yeah, I think that's one of the things that really

0:15:07.240 --> 0:15:09.760
<v Speaker 3>kind of struck me in researching that. I mean, even

0:15:09.600 --> 0:15:12.640
<v Speaker 3>in some of these kind of mathematical puzzles examples, it's

0:15:12.680 --> 0:15:15.360
<v Speaker 3>things that you know, I'd come across as a kid

0:15:15.720 --> 0:15:17.920
<v Speaker 3>and convince myself, Oh, that's just that's the answer to

0:15:17.960 --> 0:15:18.400
<v Speaker 3>the puzzle.

0:15:18.800 --> 0:15:22.280
<v Speaker 2>And it was only years later when I was explaining it.

0:15:22.200 --> 0:15:24.080
<v Speaker 3>To someone else or someone else had asked me about it,

0:15:24.640 --> 0:15:27.240
<v Speaker 3>and I sort of went through the thing that convinced me,

0:15:27.280 --> 0:15:30.000
<v Speaker 3>and it just didn't convince them at all. And I

0:15:30.040 --> 0:15:32.960
<v Speaker 3>think that's really interesting. I think we focus a lot on,

0:15:33.600 --> 0:15:36.680
<v Speaker 3>you know, how science works, how methods work, what convinces us,

0:15:37.000 --> 0:15:38.400
<v Speaker 3>and you see this in even a lot of studies

0:15:38.400 --> 0:15:40.880
<v Speaker 3>around political beliefs that people will often try and convince

0:15:40.960 --> 0:15:43.640
<v Speaker 3>others with arguments that convince them, and then you get

0:15:43.640 --> 0:15:45.800
<v Speaker 3>this gap and it's almost like that just fails. And

0:15:45.840 --> 0:15:48.920
<v Speaker 3>I think that's a really interesting step to explore, well

0:15:49.000 --> 0:15:51.280
<v Speaker 3>why does that film. One of the things that I

0:15:51.320 --> 0:15:53.280
<v Speaker 3>find even just kind of in some of the modern

0:15:53.320 --> 0:15:54.720
<v Speaker 3>tools we have in the modern there we are quite

0:15:54.720 --> 0:15:59.280
<v Speaker 3>striking is where we have this desire to explain things. So, yeah,

0:15:59.280 --> 0:16:00.760
<v Speaker 3>few years ago to talk to a bunch of people

0:16:00.920 --> 0:16:04.040
<v Speaker 3>working on AI, and there's a lot of concern about

0:16:04.040 --> 0:16:05.680
<v Speaker 3>I think things that self driving cars that we don't

0:16:05.760 --> 0:16:08.240
<v Speaker 3>understand why they make mistakes. We need that explainability. We

0:16:08.240 --> 0:16:10.760
<v Speaker 3>can't have things we don't trust. And actually, in medicine,

0:16:10.800 --> 0:16:13.440
<v Speaker 3>we have all sorts of things that we know work.

0:16:13.520 --> 0:16:16.120
<v Speaker 3>We now often they work, we don't fully understand the

0:16:16.120 --> 0:16:20.280
<v Speaker 3>physics and biology something like anesthesia, for example. We can

0:16:20.320 --> 0:16:22.520
<v Speaker 3>control the effect it's going to have, but actually all

0:16:22.520 --> 0:16:25.080
<v Speaker 3>the underlying biology and kind physics mechanisms is still more

0:16:25.080 --> 0:16:27.760
<v Speaker 3>work to be done. Things like defibrillation. You know, if

0:16:27.760 --> 0:16:29.080
<v Speaker 3>you give a heart a shock, you can kind of

0:16:29.080 --> 0:16:32.360
<v Speaker 3>reset it against Some of that's understood, but there's still

0:16:32.360 --> 0:16:34.160
<v Speaker 3>those kind of gaps in knowledge, but we know that

0:16:34.200 --> 0:16:35.920
<v Speaker 3>these are useful tools. And even yet, if you run

0:16:35.960 --> 0:16:39.520
<v Speaker 3>a clinical trial, you can assess how effective a treatment is,

0:16:40.120 --> 0:16:42.360
<v Speaker 3>but that on its own will just tell you the effect.

0:16:42.440 --> 0:16:44.800
<v Speaker 3>It won't tell you necessarily all the mechanisms that are

0:16:44.840 --> 0:16:47.480
<v Speaker 3>going on to explain it. But there's these tools that

0:16:47.520 --> 0:16:50.080
<v Speaker 3>we've got, and we've got the evidence to take action

0:16:50.160 --> 0:16:50.720
<v Speaker 3>and use these things.

0:16:50.720 --> 0:16:51.400
<v Speaker 2>We're very happy with.

0:16:51.440 --> 0:16:54.480
<v Speaker 3>And there's other areas of life where actually that inability

0:16:54.520 --> 0:16:57.040
<v Speaker 3>to explain something kind of really bothers us. You know,

0:16:57.280 --> 0:17:00.600
<v Speaker 3>even if self driving cars were much safer other than humans,

0:17:00.640 --> 0:17:02.720
<v Speaker 3>as humans we are start looking to a book, humans

0:17:02.720 --> 0:17:04.600
<v Speaker 3>are not good at driving. You know, it's not a

0:17:04.640 --> 0:17:07.520
<v Speaker 3>massively high bar, but I think it would still make

0:17:07.560 --> 0:17:09.879
<v Speaker 3>people uncomfortable. Even if they would say twice as safe

0:17:10.600 --> 0:17:12.439
<v Speaker 3>in cities were as well very well defined, I think

0:17:12.440 --> 0:17:14.439
<v Speaker 3>it still bother people if every now and again there

0:17:14.520 --> 0:17:16.879
<v Speaker 3>was just an accident that we had no real idea

0:17:16.880 --> 0:17:19.720
<v Speaker 3>of what was happening. And I think that's really important

0:17:19.760 --> 0:17:22.480
<v Speaker 3>to bridge because I think that particularly when you get

0:17:22.480 --> 0:17:27.760
<v Speaker 3>that gap in understanding, that's room for other explanations to

0:17:28.119 --> 0:17:29.760
<v Speaker 3>kind of creep in. And I think that's where we

0:17:29.760 --> 0:17:32.960
<v Speaker 3>start see emergence where things like conspiracy theories, whether it's

0:17:32.960 --> 0:17:35.639
<v Speaker 3>things with kind of incorrect logic, Often it is that

0:17:35.720 --> 0:17:38.680
<v Speaker 3>gap between what we're seeing and the understanding of why

0:17:38.720 --> 0:17:39.400
<v Speaker 3>that's happening.

0:17:40.080 --> 0:17:41.240
<v Speaker 2>I think humans have this.

0:17:41.280 --> 0:17:44.199
<v Speaker 3>Very in many ways, very powerful desire to explain what

0:17:44.200 --> 0:17:48.040
<v Speaker 3>they're seeing. But in some cases where the explanation is

0:17:48.240 --> 0:17:50.240
<v Speaker 3>very hard to untangle, it can lead us astray.

0:17:51.000 --> 0:17:54.920
<v Speaker 1>That's fascinating to think of that, the gap between understanding

0:17:55.119 --> 0:17:58.320
<v Speaker 1>and what is happening. We don't understand how anesthesia works,

0:17:58.400 --> 0:18:01.560
<v Speaker 1>or how Thailand are a menif it truly works. But

0:18:01.680 --> 0:18:05.960
<v Speaker 1>we do understand how vaccines work, for instance, and yet

0:18:06.000 --> 0:18:10.080
<v Speaker 1>there's so many conspiracy theories and misinformation surrounding this thing

0:18:10.119 --> 0:18:12.520
<v Speaker 1>that we do know how it works. I guess what

0:18:12.640 --> 0:18:15.520
<v Speaker 1>good does evidence do if we do not take it

0:18:15.520 --> 0:18:17.399
<v Speaker 1>into account and are not open to it.

0:18:18.040 --> 0:18:19.640
<v Speaker 3>Yeah, And I think for me, a lot of it

0:18:19.720 --> 0:18:23.399
<v Speaker 3>is just understanding at what point that breaks down. I mean,

0:18:23.480 --> 0:18:26.119
<v Speaker 3>even if you if you look at some of the

0:18:26.440 --> 0:18:28.879
<v Speaker 3>COVID vaccines for example, you know, or or even some

0:18:28.920 --> 0:18:31.679
<v Speaker 3>of the kind of other debates around climate interventions. I

0:18:31.720 --> 0:18:36.960
<v Speaker 3>think often it gets very into debating some element of

0:18:37.000 --> 0:18:40.479
<v Speaker 3>the technology, and I think often it's actually just people

0:18:40.800 --> 0:18:44.960
<v Speaker 3>disliked some of the control that was exerted over them

0:18:45.000 --> 0:18:47.040
<v Speaker 3>through mandates or for other things. And actually, you know,

0:18:47.040 --> 0:18:50.360
<v Speaker 3>if you've got an intervention you're unhappy with, you can

0:18:50.400 --> 0:18:53.440
<v Speaker 3>disagree with the intervention of saying look, yeah, for example,

0:18:53.600 --> 0:18:55.399
<v Speaker 3>we know that intervention works, but I disagree with how

0:18:55.400 --> 0:18:58.320
<v Speaker 3>you're implementing it, or you can disagree that the intervention

0:18:58.440 --> 0:19:01.600
<v Speaker 3>actually has an effect, even one step down and just

0:19:01.640 --> 0:19:03.919
<v Speaker 3>say you know, actually, I think there's there's sort of

0:19:03.920 --> 0:19:07.359
<v Speaker 3>deeper problems, or maybe the disease isn't a threat. And

0:19:07.440 --> 0:19:11.399
<v Speaker 3>I think often those kind of levels get get tangled up,

0:19:11.440 --> 0:19:13.040
<v Speaker 3>and I think in a lot of conversations I've had

0:19:13.080 --> 0:19:17.199
<v Speaker 3>with people, often they're sort of deep down concern or

0:19:17.200 --> 0:19:20.720
<v Speaker 3>the thing that they're approaching it with isn't necessarily that

0:19:20.760 --> 0:19:23.760
<v Speaker 3>they've just out of nowhere decided that this isn't a threat,

0:19:23.880 --> 0:19:27.040
<v Speaker 3>or that that technology doesn't work. It's actually in some

0:19:27.119 --> 0:19:29.520
<v Speaker 3>of these instances things are a bit more marginal, and

0:19:29.560 --> 0:19:31.720
<v Speaker 3>you could say, you know, you can make an argument

0:19:31.760 --> 0:19:36.760
<v Speaker 3>either way even if the underlying intervention is effective or

0:19:36.840 --> 0:19:38.080
<v Speaker 3>is going to have this You know, you can make

0:19:38.320 --> 0:19:40.280
<v Speaker 3>there's moral and this it's not just about an sort

0:19:40.280 --> 0:19:43.720
<v Speaker 3>of epilogical question, as I think, kind of understanding where

0:19:43.720 --> 0:19:47.040
<v Speaker 3>those drivers are and also just in our own arguments.

0:19:47.520 --> 0:19:49.399
<v Speaker 3>I think sometimes, you know, I have the conversation with

0:19:49.440 --> 0:19:52.040
<v Speaker 3>people and I think I'm just arguing about the kind

0:19:52.080 --> 0:19:53.880
<v Speaker 3>of the nuances of whether intervention is a good idea

0:19:53.960 --> 0:19:56.080
<v Speaker 3>or not, and they're actually arguing whether it's a problem

0:19:56.119 --> 0:19:59.520
<v Speaker 3>in the first place. We see it vaccines, I guess

0:19:59.560 --> 0:20:02.760
<v Speaker 3>examples more polarized, but even something in climate You know,

0:20:02.880 --> 0:20:05.240
<v Speaker 3>you can have a lot of people who just agree

0:20:05.280 --> 0:20:07.040
<v Speaker 3>on the nature of the effect of climate change. They

0:20:07.080 --> 0:20:09.840
<v Speaker 3>agree on the different levers that we probably have available

0:20:09.840 --> 0:20:13.240
<v Speaker 3>a society, but they might strongly disagree about actually how

0:20:13.280 --> 0:20:16.160
<v Speaker 3>we prioritize those and all of the trade offs. And

0:20:16.560 --> 0:20:19.359
<v Speaker 3>I think it's just understanding what level we're on and

0:20:19.400 --> 0:20:21.400
<v Speaker 3>where the evidence might stop and where it might then

0:20:21.480 --> 0:20:23.800
<v Speaker 3>just be other things that are filtering in on a

0:20:23.840 --> 0:20:24.520
<v Speaker 3>personal level.

0:20:25.119 --> 0:20:29.720
<v Speaker 1>This idea of proof and certainty and truth. It seems

0:20:29.800 --> 0:20:32.880
<v Speaker 1>very intuitive in a lot of ways today, but this

0:20:33.040 --> 0:20:36.360
<v Speaker 1>maybe wasn't always the case, Like when did the concepts

0:20:36.359 --> 0:20:40.040
<v Speaker 1>of truth and the need for these self evident truths

0:20:40.200 --> 0:20:43.199
<v Speaker 1>or certainty or proof when did these come to be?

0:20:43.359 --> 0:20:45.840
<v Speaker 1>And then you know in what fields or what areas

0:20:45.880 --> 0:20:47.520
<v Speaker 1>where they initially applied.

0:20:48.400 --> 0:20:50.520
<v Speaker 3>Yeah, I think that's a great It's easier just to

0:20:50.560 --> 0:20:53.240
<v Speaker 3>think of like the world and sort of science and

0:20:53.240 --> 0:20:55.960
<v Speaker 3>evidences always was as it is. I mean even in mathematics,

0:20:56.000 --> 0:20:58.600
<v Speaker 3>it's idea that we had a universal truth wasn't the

0:20:58.600 --> 0:21:01.400
<v Speaker 3>same throughout history. If you go back to the ancient Egyptians,

0:21:01.440 --> 0:21:04.840
<v Speaker 3>ancient Babylonians, they were much more focused on problems solving.

0:21:05.000 --> 0:21:07.280
<v Speaker 3>A lot of their texts are kind of these these

0:21:07.359 --> 0:21:09.320
<v Speaker 3>kind of puzzles and very much things around kind of

0:21:09.320 --> 0:21:12.440
<v Speaker 3>practical everyday problems. And even if you look at their

0:21:12.440 --> 0:21:15.000
<v Speaker 3>formulas for an area of a circle, they're quite approximate,

0:21:15.040 --> 0:21:17.680
<v Speaker 3>and if you're building something that needs quite a large circle,

0:21:17.720 --> 0:21:20.360
<v Speaker 3>you're probably going to be okay using those, but it's

0:21:20.400 --> 0:21:23.080
<v Speaker 3>not going to give you that really precise truth no

0:21:23.119 --> 0:21:25.359
<v Speaker 3>matter what problem you're working on. And that's something where

0:21:25.920 --> 0:21:29.680
<v Speaker 3>the ancient Greeks mathetations like Euclid came in and tried

0:21:29.680 --> 0:21:31.679
<v Speaker 3>to put things on much more solid footing. So you've

0:21:31.720 --> 0:21:34.320
<v Speaker 3>got these concepts like pie that if you want the

0:21:34.359 --> 0:21:37.359
<v Speaker 3>area of a circle that will just be universally true

0:21:37.800 --> 0:21:39.440
<v Speaker 3>and you won't have this issue of your kind of

0:21:39.480 --> 0:21:43.159
<v Speaker 3>approximation breaks down. And it was then, I think that

0:21:43.840 --> 0:21:46.320
<v Speaker 3>as it sort of came into the Enlightenment here, it

0:21:46.359 --> 0:21:48.840
<v Speaker 3>was very appealing for people that you could have these

0:21:49.760 --> 0:21:51.960
<v Speaker 3>undeniable truths about the world. And I think that's where

0:21:51.960 --> 0:21:54.520
<v Speaker 3>a lot of other fields started growing them as well.

0:21:54.520 --> 0:21:57.840
<v Speaker 3>But even in medicine, if you look at this study

0:21:57.880 --> 0:22:00.480
<v Speaker 3>of cause and effect, a lot of that it was

0:22:00.600 --> 0:22:02.600
<v Speaker 3>the sort of medieval Arabic world that a lot of

0:22:02.640 --> 0:22:04.879
<v Speaker 3>that started to emerge. So a lot of the kind

0:22:04.920 --> 0:22:08.639
<v Speaker 3>of superstition, this idea that disease or conditions just kind

0:22:08.680 --> 0:22:11.199
<v Speaker 3>of come out of nowhere and it's bad people or

0:22:11.720 --> 0:22:13.800
<v Speaker 3>you know, someone's a witch. All this kind of stuff

0:22:13.840 --> 0:22:15.879
<v Speaker 3>that was going around in much of Europe at the time.

0:22:16.720 --> 0:22:19.200
<v Speaker 3>There's a lot of early writings inn around sort of

0:22:19.240 --> 0:22:22.760
<v Speaker 3>the eleventh century saying these aren't supernatural, there's natural causes

0:22:23.400 --> 0:22:25.000
<v Speaker 3>and we can study them. Yeah, we can study them,

0:22:25.000 --> 0:22:26.720
<v Speaker 3>we can work out what the cause of effects were

0:22:27.119 --> 0:22:30.240
<v Speaker 3>a lot of early attempts to try and think about

0:22:30.280 --> 0:22:32.639
<v Speaker 3>concepts that we would now call things like having a

0:22:32.680 --> 0:22:35.560
<v Speaker 3>control group or thinking about how we kind of you know,

0:22:35.760 --> 0:22:38.680
<v Speaker 3>would divide and treat some people and not treat some people,

0:22:38.760 --> 0:22:42.800
<v Speaker 3>and then compend the difference. The conclusions didn't always work out.

0:22:42.840 --> 0:22:44.679
<v Speaker 3>I mean, there was I think one of the earlier

0:22:44.720 --> 0:22:48.040
<v Speaker 3>studies was someone who'd identified correctly the symptoms of meningitis,

0:22:48.480 --> 0:22:51.679
<v Speaker 3>but then concluded that blood letting was really effective for it,

0:22:51.720 --> 0:22:54.879
<v Speaker 3>which probably something in their study design had gone astray.

0:22:54.960 --> 0:22:57.199
<v Speaker 3>But again, it just kind of really and it's one

0:22:57.240 --> 0:22:58.880
<v Speaker 3>of the things you look back on and you think

0:22:58.920 --> 0:23:01.480
<v Speaker 3>it's just it's pretty obvious that we should be doing

0:23:01.480 --> 0:23:03.760
<v Speaker 3>it that way. But even coming into the twentieth century,

0:23:03.760 --> 0:23:06.280
<v Speaker 3>if you look at something like analyzing a medical treatment,

0:23:06.720 --> 0:23:09.760
<v Speaker 3>a lot of the early studies did an alternation method,

0:23:09.840 --> 0:23:12.399
<v Speaker 3>because if you think about it, rather than randomized patients,

0:23:12.440 --> 0:23:14.240
<v Speaker 3>you could just say, well, the first patient that comes in,

0:23:14.280 --> 0:23:16.160
<v Speaker 3>I'm going to treat, the second I won't, the third

0:23:16.160 --> 0:23:18.920
<v Speaker 3>I will, forth they won't, and on average you should

0:23:18.920 --> 0:23:22.200
<v Speaker 3>get something that any other sources of variability should balance out,

0:23:22.240 --> 0:23:24.600
<v Speaker 3>and the difference between those groups should be on average

0:23:24.640 --> 0:23:25.600
<v Speaker 3>down to the treatment effect.

0:23:26.040 --> 0:23:27.679
<v Speaker 2>But Bradford Hill actually, who did a.

0:23:27.640 --> 0:23:31.359
<v Speaker 3>Lot of the pilneering work in the early clinical trial space,

0:23:32.119 --> 0:23:35.119
<v Speaker 3>noticed that the groups were often imbalance because what's happening

0:23:35.160 --> 0:23:39.040
<v Speaker 3>as patients were coming in and doctors were maybe subconsciously that, oh,

0:23:39.040 --> 0:23:40.920
<v Speaker 3>maybe that person looks a bit ill, I'll enroll them,

0:23:41.000 --> 0:23:44.000
<v Speaker 3>or maybe they don't meet the diagnosis it And actually

0:23:44.000 --> 0:23:46.840
<v Speaker 3>a lot of the early randomization wasn't statistical. It was

0:23:46.920 --> 0:23:49.440
<v Speaker 3>just it was to sort of keep humans from themselves

0:23:49.480 --> 0:23:52.720
<v Speaker 3>because we couldn't trust subconscious judgment. So a lot of

0:23:52.880 --> 0:23:56.919
<v Speaker 3>the early randomization in medicine wasn't about the statistical properties

0:23:57.560 --> 0:23:59.600
<v Speaker 3>of the trial design. It was just about making sure

0:23:59.680 --> 0:24:02.960
<v Speaker 3>human didn't muck things up basically with their internal biases.

0:24:03.359 --> 0:24:05.560
<v Speaker 1>Well, I mean, we'll find a way, I'm sure somehow

0:24:05.680 --> 0:24:09.600
<v Speaker 1>in some way. It's interesting to think about this idea

0:24:09.640 --> 0:24:12.960
<v Speaker 1>of like self evident truths thinking back to you okay, yes,

0:24:13.080 --> 0:24:15.679
<v Speaker 1>or superstition and this person is a witch based on

0:24:15.760 --> 0:24:19.640
<v Speaker 1>these signs or whatever was that also viewed as proof.

0:24:20.359 --> 0:24:23.240
<v Speaker 3>The story of those trials by old deal is a

0:24:23.240 --> 0:24:25.199
<v Speaker 3>fascinating one as well, because they were used for a

0:24:25.200 --> 0:24:27.720
<v Speaker 3>long time. You could have trial by ord deal like

0:24:27.760 --> 0:24:29.520
<v Speaker 3>by water by fi whatever, and then you could also

0:24:29.560 --> 0:24:32.480
<v Speaker 3>chose trial by duels. So basically the big criminals always

0:24:32.480 --> 0:24:34.840
<v Speaker 3>pick that, and people start to notice like, oh, you know,

0:24:35.080 --> 0:24:37.560
<v Speaker 3>if God is deciding which one's innocent, God tends to

0:24:37.600 --> 0:24:40.240
<v Speaker 3>pick the bigger one like pretty much every time, which.

0:24:40.040 --> 0:24:41.800
<v Speaker 2>Is I think there was that that came.

0:24:41.920 --> 0:24:44.120
<v Speaker 3>But actually one of the reasons they stopped using them

0:24:44.320 --> 0:24:48.040
<v Speaker 3>is a lot of the religious scholars became concerned that

0:24:48.080 --> 0:24:51.639
<v Speaker 3>they were basically trying to by running those trials, you're

0:24:51.720 --> 0:24:53.760
<v Speaker 3>essentially trying to get God to do your work for you,

0:24:53.840 --> 0:24:55.879
<v Speaker 3>and that felt for them a bit awkward because you're

0:24:55.920 --> 0:24:58.520
<v Speaker 3>sort of on demand saying, hey, can you come and

0:24:58.520 --> 0:25:00.320
<v Speaker 3>make a decision for us, which they it sort of

0:25:00.320 --> 0:25:03.280
<v Speaker 3>got quite uncomfortable with. But even those early systems, I

0:25:03.320 --> 0:25:06.720
<v Speaker 3>mean early juries in England were kind of fascinating because

0:25:06.760 --> 0:25:09.040
<v Speaker 3>they they weren't the structure that we had today. They

0:25:09.280 --> 0:25:11.840
<v Speaker 3>kind of did their own investigation. So often someone was

0:25:11.880 --> 0:25:14.040
<v Speaker 3>accused and then they went off and accused someone else

0:25:14.080 --> 0:25:15.440
<v Speaker 3>and kind of did their own thing, and it was

0:25:15.960 --> 0:25:18.440
<v Speaker 3>it was only over time that system kind of evolved

0:25:18.640 --> 0:25:21.720
<v Speaker 3>of having that way and converging something and I think

0:25:21.720 --> 0:25:25.240
<v Speaker 3>that's we talk a lot about the problem of black boxes,

0:25:25.280 --> 0:25:28.959
<v Speaker 3>but to some extent, juries and talking to legal scholars

0:25:28.960 --> 0:25:31.240
<v Speaker 3>was kind of interesting with this that it's not so

0:25:31.359 --> 0:25:33.720
<v Speaker 3>much about getting to the truth. It's having a system

0:25:33.720 --> 0:25:36.840
<v Speaker 3>where you can reach a decision and you've got kind

0:25:36.840 --> 0:25:40.560
<v Speaker 3>of that finality or send me from finality to that decision,

0:25:40.600 --> 0:25:43.879
<v Speaker 3>and having a system that works, rather than you know,

0:25:43.920 --> 0:25:46.440
<v Speaker 3>your one hundred percent convinced of that. And I think

0:25:46.480 --> 0:25:49.720
<v Speaker 3>we see that kind of across different fields, of that

0:25:49.840 --> 0:25:53.320
<v Speaker 3>emergence of truth. And you said, what's kind of obvious

0:25:53.920 --> 0:25:55.199
<v Speaker 3>and what self evident? I mean, one of the other

0:25:55.240 --> 0:25:58.400
<v Speaker 3>things that I found kind of interesting was how many

0:25:58.480 --> 0:26:02.680
<v Speaker 3>mathematicians were deeply influenced by religion. So Newton, for example,

0:26:03.040 --> 0:26:06.960
<v Speaker 3>Isaac Newton, driving all these equations and theories about planets

0:26:07.000 --> 0:26:10.160
<v Speaker 3>and planetary emotion, he saw that it was God keeping

0:26:10.200 --> 0:26:13.359
<v Speaker 3>the universe in balance, and he was essentially just observing

0:26:13.440 --> 0:26:15.879
<v Speaker 3>divine influence. So for him, although he was doing a

0:26:15.880 --> 0:26:17.560
<v Speaker 3>lot of this scientific work, he saw that there was

0:26:17.600 --> 0:26:21.640
<v Speaker 3>this external influence keeping it all in place along the way.

0:26:21.680 --> 0:26:24.880
<v Speaker 3>So even quite far through history, you had these kind

0:26:24.920 --> 0:26:29.159
<v Speaker 3>of other baseline explanations going on. I think even in

0:26:29.200 --> 0:26:32.360
<v Speaker 3>the modern era. I think the way sometimes we tell

0:26:32.400 --> 0:26:35.280
<v Speaker 3>the story of science I think is sometimes almost a

0:26:35.280 --> 0:26:37.760
<v Speaker 3>bit disingenuous. If you read a scientific paper, it's kind

0:26:37.800 --> 0:26:39.719
<v Speaker 3>of yet there's this problem, and I decided to run

0:26:39.800 --> 0:26:42.600
<v Speaker 3>this experiment and I got these results. But I think

0:26:42.600 --> 0:26:45.520
<v Speaker 3>there's also just that element of what was the hunch

0:26:45.600 --> 0:26:48.000
<v Speaker 3>that made you think that that might be an interesting

0:26:48.080 --> 0:26:50.920
<v Speaker 3>thing to investigate? What was that spark of inspiration? I think,

0:26:50.960 --> 0:26:54.240
<v Speaker 3>even in this era of AI, it's a really interesting question,

0:26:54.320 --> 0:26:58.360
<v Speaker 3>because AI can kind of process and mimic human decisions

0:26:58.400 --> 0:27:00.639
<v Speaker 3>as we write them down. But I think there's often

0:27:00.680 --> 0:27:04.320
<v Speaker 3>that kind of spark or that idea that would lead

0:27:04.359 --> 0:27:06.840
<v Speaker 3>you to do something that just people wouldn't have tried before.

0:27:07.480 --> 0:27:09.840
<v Speaker 3>And that's much much harder to articulate. So it's not

0:27:09.880 --> 0:27:13.280
<v Speaker 3>necessarily that kind of obviousness that we might have had

0:27:13.320 --> 0:27:15.560
<v Speaker 3>in another era. But I think there still are those

0:27:15.600 --> 0:27:18.040
<v Speaker 3>things which are quite hard to explain in where that

0:27:19.080 --> 0:27:20.840
<v Speaker 3>evidence might have initially sparked from.

0:27:21.760 --> 0:27:24.960
<v Speaker 1>One of the things that you mentioned was the use

0:27:25.000 --> 0:27:28.919
<v Speaker 1>of proof and evidence in the legal system, and I

0:27:28.960 --> 0:27:31.320
<v Speaker 1>feel like this was a really fascinating discussion in your

0:27:31.359 --> 0:27:34.560
<v Speaker 1>book as well, where this is employed as like, you know,

0:27:34.680 --> 0:27:39.119
<v Speaker 1>proof beyond a reasonable doubt or innocent until proven guilty.

0:27:39.240 --> 0:27:42.119
<v Speaker 1>What does this show us about like the variable level

0:27:42.240 --> 0:27:45.640
<v Speaker 1>of evidence needed to make a decision, and I guess

0:27:45.720 --> 0:27:49.760
<v Speaker 1>like the different forms that proof can take in this setting.

0:27:50.040 --> 0:27:53.600
<v Speaker 3>It's a really interesting question about how different societies have

0:27:53.640 --> 0:27:55.840
<v Speaker 3>even said that that balance me Essentially, in a legal case,

0:27:55.840 --> 0:27:58.080
<v Speaker 3>there's two main errors you can make that someone can

0:27:58.119 --> 0:27:59.520
<v Speaker 3>be guilty and you can let them go three or

0:27:59.560 --> 0:28:03.360
<v Speaker 3>they can be in you can convict them. And William Blackstone,

0:28:03.400 --> 0:28:06.600
<v Speaker 3>who is a legal scholar, in the seventeen sixties came

0:28:06.680 --> 0:28:08.600
<v Speaker 3>up with wats known as Blackstone's ratio. He said it's

0:28:08.640 --> 0:28:10.960
<v Speaker 3>better for ten guilty people to go free than one

0:28:10.960 --> 0:28:14.920
<v Speaker 3>innocent to be convicted. And Benjamin Franklin actually and even

0:28:14.960 --> 0:28:17.119
<v Speaker 3>accortiately said it's better for one hundred guilty people to

0:28:17.160 --> 0:28:19.440
<v Speaker 3>go free of one isn't to be convicted.

0:28:19.680 --> 0:28:21.240
<v Speaker 2>Seeing that as a kind of balance.

0:28:21.200 --> 0:28:24.760
<v Speaker 3>Other cultures, particularly some communist regimes in twentieth century, set

0:28:24.760 --> 0:28:26.199
<v Speaker 3>it the other way. It's like it was better for

0:28:26.280 --> 0:28:29.199
<v Speaker 3>ten innocents to be in prison than warner guilty to

0:28:29.240 --> 0:28:32.200
<v Speaker 3>go free, because there's this kind of trade off and

0:28:31.920 --> 0:28:34.600
<v Speaker 3>where they're seeing it as the worst error. And actually

0:28:34.800 --> 0:28:38.080
<v Speaker 3>some analysis looking at US legal cases, obviously they don't

0:28:38.080 --> 0:28:40.360
<v Speaker 3>try and target these error rates, but you can sort

0:28:40.360 --> 0:28:43.920
<v Speaker 3>of infer how people are valuing this. A lot of

0:28:43.920 --> 0:28:46.280
<v Speaker 3>them seem to land between that kind of Blackstone and

0:28:46.320 --> 0:28:50.480
<v Speaker 3>Franklin ratio of error. But then, of course, yes, the

0:28:50.520 --> 0:28:53.000
<v Speaker 3>different evidence and how it makes it its way into

0:28:53.000 --> 0:28:56.120
<v Speaker 3>the court room, particularly some of the examples historically of

0:28:56.480 --> 0:28:59.160
<v Speaker 3>kind of things like early probability. And again, one of

0:28:59.160 --> 0:29:01.720
<v Speaker 3>the challenges here is what What's god I talked to

0:29:02.000 --> 0:29:04.080
<v Speaker 3>called the weak evidence problem, and I think a lot

0:29:04.120 --> 0:29:08.720
<v Speaker 3>of how we navigate life is around probabilities that are

0:29:08.800 --> 0:29:11.040
<v Speaker 3>quite likely. You know, a lot of probability theory was

0:29:11.080 --> 0:29:13.200
<v Speaker 3>originally developed around like dice games and things. You know,

0:29:13.240 --> 0:29:17.080
<v Speaker 3>you can study and you can quantify. But in legal cases,

0:29:17.120 --> 0:29:20.800
<v Speaker 3>we often have this weak evidence problem where someone ends

0:29:20.880 --> 0:29:24.480
<v Speaker 3>up in some extremely bad looking situation from a guilt

0:29:24.480 --> 0:29:26.360
<v Speaker 3>point of view, and you're like, well, it's extremely unlikely

0:29:26.440 --> 0:29:28.400
<v Speaker 3>this is just a coincidence. But then if you think

0:29:28.400 --> 0:29:30.880
<v Speaker 3>about it, like, well, this person might just be a normal,

0:29:31.120 --> 0:29:34.960
<v Speaker 3>everyday person. That what, it's extremely unlikely too that they're guilty.

0:29:35.400 --> 0:29:37.840
<v Speaker 3>So you have these two extremely unlikely events, and a

0:29:37.880 --> 0:29:40.560
<v Speaker 3>lot of statistics just isn't equipped to handle that. And

0:29:40.600 --> 0:29:44.080
<v Speaker 3>so there's this notion it's called the prosecutor's fallacy where

0:29:44.080 --> 0:29:47.120
<v Speaker 3>people say, well, this is the probability that that will

0:29:47.120 --> 0:29:50.360
<v Speaker 3>all be a coincidence, and therefore that's the probability they're innocent,

0:29:50.600 --> 0:29:52.480
<v Speaker 3>But of course you've got to weigh it against the

0:29:52.520 --> 0:29:54.920
<v Speaker 3>fact that it's extremely unlikely they're guilty as well. And

0:29:54.960 --> 0:29:58.440
<v Speaker 3>we see this even in other areas. So the work

0:29:58.440 --> 0:30:01.840
<v Speaker 3>we do dealing with like emerging threats and you know,

0:30:01.880 --> 0:30:04.120
<v Speaker 3>pre COVID, there were some studies and actually we did

0:30:04.160 --> 0:30:06.320
<v Speaker 3>a TV show where you sort of say, oh, it's

0:30:06.360 --> 0:30:08.480
<v Speaker 3>you know, a pandemic could just be round the corner.

0:30:08.600 --> 0:30:10.360
<v Speaker 3>Or there's another study that the World Bank I think

0:30:10.400 --> 0:30:12.880
<v Speaker 3>put it at one percent, and you're like, well, what

0:30:13.080 --> 0:30:13.560
<v Speaker 3>is that?

0:30:13.560 --> 0:30:17.120
<v Speaker 2>That's is that right? We was that a good prediction?

0:30:17.320 --> 0:30:19.640
<v Speaker 2>Was that bad? And it's these these very unlikely events.

0:30:19.680 --> 0:30:23.400
<v Speaker 3>I think in legal cases again, for that weak evidence problem,

0:30:23.600 --> 0:30:28.080
<v Speaker 3>it's less about do we definitively work out with high probability,

0:30:28.080 --> 0:30:30.520
<v Speaker 3>which is true, and it's more just we have to

0:30:30.520 --> 0:30:34.120
<v Speaker 3>converge on the best explanation for what we've seen given

0:30:34.200 --> 0:30:37.600
<v Speaker 3>those two possibilities, and in reality we may never have

0:30:38.160 --> 0:30:41.760
<v Speaker 3>certainty about where we are. And I think it's something

0:30:41.760 --> 0:30:43.800
<v Speaker 3>that kind of struck me both think about that, and

0:30:43.840 --> 0:30:46.320
<v Speaker 3>then also thinking about a lot of people who have

0:30:46.400 --> 0:30:49.400
<v Speaker 3>to plan for emergencies and very unlikely events. Thinking a

0:30:49.400 --> 0:30:52.440
<v Speaker 3>lot of the way we traditionally think about probability can

0:30:52.520 --> 0:30:54.479
<v Speaker 3>very quickly lead us astray because I think we're so

0:30:54.680 --> 0:30:56.760
<v Speaker 3>used to having this idea. Well, I can be ninety

0:30:56.800 --> 0:30:59.840
<v Speaker 3>nine percent sure that this happened, But actually it's much

0:30:59.880 --> 0:31:02.440
<v Speaker 3>more about that balancing app that we have to perform.

0:31:03.520 --> 0:31:05.840
<v Speaker 1>Let's take a quick break here, We'll be back before

0:31:05.840 --> 0:31:22.560
<v Speaker 1>you know it. Welcome back, everyone, I'm here chatting with

0:31:22.600 --> 0:31:25.880
<v Speaker 1>doctor Adam Kucharski about his book Proof. Let's get into

0:31:25.920 --> 0:31:30.400
<v Speaker 1>some more questions thinking about this in the context of COVID,

0:31:30.520 --> 0:31:34.200
<v Speaker 1>when you know, things were evolving very rapidly, the situation

0:31:34.280 --> 0:31:38.160
<v Speaker 1>was evolving rapidly, and the general public and you know,

0:31:38.240 --> 0:31:42.320
<v Speaker 1>of course government officials wanted answers and wanted decisions. You know,

0:31:42.360 --> 0:31:44.280
<v Speaker 1>what is the best thing to do, wear masks, not

0:31:44.360 --> 0:31:47.080
<v Speaker 1>wear masks, sanitized groceries, all these things that were just

0:31:47.680 --> 0:31:52.200
<v Speaker 1>constant questions and neat and people wanting hard answers like

0:31:52.440 --> 0:31:55.840
<v Speaker 1>just yes, period. And of as someone who was on

0:31:55.960 --> 0:31:59.680
<v Speaker 1>the informational front lines of the COVID pandemic, what was

0:31:59.680 --> 0:32:02.680
<v Speaker 1>your relationship with uncertainty, Like at that time, did you

0:32:02.920 --> 0:32:06.920
<v Speaker 1>struggle with feeling like we don't have enough information yet?

0:32:06.960 --> 0:32:10.280
<v Speaker 1>You know, how did that feel? I guess in your position?

0:32:11.080 --> 0:32:14.320
<v Speaker 3>Yeah, I think I mean those those kinds of situations

0:32:14.360 --> 0:32:16.880
<v Speaker 3>were normously chanted, both in terms of evidence generation and

0:32:16.920 --> 0:32:20.400
<v Speaker 3>communication and then obviously the political decision making that comes

0:32:20.400 --> 0:32:20.720
<v Speaker 3>off the.

0:32:20.640 --> 0:32:21.040
<v Speaker 2>Back of it.

0:32:21.960 --> 0:32:23.800
<v Speaker 3>I think in many of those situations, I found it

0:32:23.880 --> 0:32:28.640
<v Speaker 3>useful to kind of convert in some cases uncertainty around

0:32:28.640 --> 0:32:32.080
<v Speaker 3>the exact estimate to just just to kind of poorly

0:32:32.120 --> 0:32:34.640
<v Speaker 3>what situation we're in. So for example, when I think

0:32:34.680 --> 0:32:36.800
<v Speaker 3>it was the delta variant emerge and we did a

0:32:36.840 --> 0:32:39.320
<v Speaker 3>lot of the work identifying the early advantage it had,

0:32:39.760 --> 0:32:40.840
<v Speaker 3>and it really wasn't.

0:32:40.960 --> 0:32:42.880
<v Speaker 2>Was it for thirty percent? Was it forty percent? Was

0:32:42.920 --> 0:32:43.600
<v Speaker 2>it sixty percent?

0:32:44.120 --> 0:32:46.760
<v Speaker 3>But essentially all of those were a big problem and

0:32:46.760 --> 0:32:48.600
<v Speaker 3>it's kind of arguing like is this you know, is

0:32:48.640 --> 0:32:51.040
<v Speaker 3>this a disaster or just a catastrophe or just very

0:32:51.120 --> 0:32:54.400
<v Speaker 3>very bad And it's like right, right, Yeah, from a policy,

0:32:54.400 --> 0:32:56.800
<v Speaker 3>you don't need to kind of necessarily commune out as that.

0:32:56.840 --> 0:32:59.440
<v Speaker 3>You just say, like we're very confident that it's going

0:32:59.480 --> 0:33:01.240
<v Speaker 3>to take off a couple of things I think that

0:33:01.560 --> 0:33:05.200
<v Speaker 3>jumped out for me. I think one was the need

0:33:05.240 --> 0:33:08.320
<v Speaker 3>to triangulate across data sources. I think sometimes people have

0:33:08.360 --> 0:33:09.960
<v Speaker 3>this idea of science that you go out and you

0:33:10.000 --> 0:33:11.960
<v Speaker 3>run a study, and that study gives you the answer.

0:33:11.960 --> 0:33:15.000
<v Speaker 3>It doesn't give you the answer. And there were quite

0:33:15.000 --> 0:33:17.840
<v Speaker 3>a few of the early skepticism were saying, well, actually,

0:33:17.880 --> 0:33:21.280
<v Speaker 3>this study wasn't definitive, and this study wasn't definitive. But

0:33:21.440 --> 0:33:23.680
<v Speaker 3>once you start to look at all of those, you know,

0:33:23.720 --> 0:33:25.600
<v Speaker 3>you start to look at the evacuations flights, you start

0:33:25.600 --> 0:33:27.680
<v Speaker 3>to look at the testing data and the contact tracing

0:33:27.760 --> 0:33:29.720
<v Speaker 3>and the big testing of you know, some of the

0:33:29.760 --> 0:33:32.560
<v Speaker 3>cruise ships. You start to look at the clinical data.

0:33:32.840 --> 0:33:35.040
<v Speaker 3>All of those signals start to drag you in the

0:33:35.080 --> 0:33:37.400
<v Speaker 3>same way. And again, each bits of those evidence on

0:33:37.440 --> 0:33:39.640
<v Speaker 3>their own might have problems, but you can start to

0:33:39.640 --> 0:33:42.680
<v Speaker 3>bring together and draw that into a conclusion. I think

0:33:42.720 --> 0:33:45.160
<v Speaker 3>we saw that across the pandemic that if you view

0:33:45.200 --> 0:33:46.440
<v Speaker 3>it very much as like I'm going to get the

0:33:46.440 --> 0:33:49.320
<v Speaker 3>perfect study it's going to give me the answer, you'll struggle.

0:33:49.400 --> 0:33:52.280
<v Speaker 3>But often you can actually find a lot of complementary

0:33:52.360 --> 0:33:55.160
<v Speaker 3>data sources that all for variants or a lot of

0:33:55.160 --> 0:33:57.480
<v Speaker 3>that early severity were all pointing in the same direction.

0:33:57.600 --> 0:34:00.480
<v Speaker 3>I think it's harder, obviously when they're pointing in different directions,

0:34:01.040 --> 0:34:03.640
<v Speaker 3>as we saw with some of the interventions, where it

0:34:03.680 --> 0:34:07.200
<v Speaker 3>was less clear because different countries, different economies, certain things

0:34:07.320 --> 0:34:10.480
<v Speaker 3>did affect behavior and other things in different ways.

0:34:11.000 --> 0:34:13.040
<v Speaker 2>But I think the other challenge.

0:34:12.640 --> 0:34:14.000
<v Speaker 3>That kind of jumped out, and I think a lot

0:34:14.000 --> 0:34:16.600
<v Speaker 3>of the health issues we deal with in the US,

0:34:16.760 --> 0:34:19.120
<v Speaker 3>UK and the modern era are non contagious, so they're

0:34:19.200 --> 0:34:21.280
<v Speaker 3>very much kind of individual you know, things like cancer,

0:34:21.360 --> 0:34:24.280
<v Speaker 3>things like heart disease. It's just very much individual focus.

0:34:24.360 --> 0:34:26.480
<v Speaker 3>So you have someone who's ill, do you treat them,

0:34:26.520 --> 0:34:28.279
<v Speaker 3>do not treat them. If you don't treat them, that's

0:34:28.719 --> 0:34:29.919
<v Speaker 3>someone who's one person who's going.

0:34:29.840 --> 0:34:30.279
<v Speaker 2>To get worse.

0:34:30.960 --> 0:34:35.040
<v Speaker 3>But contagious health threats have this dependence where you know

0:34:35.080 --> 0:34:37.680
<v Speaker 3>a problem can get worse and that problem can then

0:34:37.719 --> 0:34:40.440
<v Speaker 3>accelerate in very different ways. I think that was something

0:34:40.440 --> 0:34:42.960
<v Speaker 3>that was quite a challenge to communicate, cause I think

0:34:42.960 --> 0:34:45.040
<v Speaker 3>a lot of people had this notion of you've got

0:34:45.080 --> 0:34:48.040
<v Speaker 3>normal life and then you could do something else that's

0:34:48.080 --> 0:34:49.880
<v Speaker 3>not normal life. And I would obviously just prefer it

0:34:49.920 --> 0:34:52.360
<v Speaker 3>to be normal. But I think as we saw globally,

0:34:53.800 --> 0:34:56.200
<v Speaker 3>you didn't get that status quote. I mean, that was

0:34:56.239 --> 0:34:59.280
<v Speaker 3>that was gone, and no country had They had varying

0:34:59.320 --> 0:35:01.759
<v Speaker 3>levels of normality, but no country had like just you know,

0:35:01.920 --> 0:35:05.680
<v Speaker 3>pretend absolutely nothing happened. You either had in varying degrees

0:35:05.719 --> 0:35:08.600
<v Speaker 3>depending on the structured society and advantages they had in

0:35:09.000 --> 0:35:11.680
<v Speaker 3>terms of demography and healthcare and other stuff, big changes

0:35:11.719 --> 0:35:15.080
<v Speaker 3>in behavior or borders, whatever, or you saw a.

0:35:15.040 --> 0:35:15.799
<v Speaker 2>Huge amount of death.

0:35:15.840 --> 0:35:19.200
<v Speaker 3>And I think that's something that can be from an

0:35:19.200 --> 0:35:21.640
<v Speaker 3>evidence point of view, much more challenging, because I think

0:35:21.719 --> 0:35:23.719
<v Speaker 3>just in life, we're much more used to those kind

0:35:23.719 --> 0:35:26.879
<v Speaker 3>of linear problems where you know, like with cancer or something,

0:35:27.000 --> 0:35:30.160
<v Speaker 3>these are event tragic events that happen sort of distributed

0:35:30.200 --> 0:35:34.160
<v Speaker 3>across the population, rather than something that the worse it gets,

0:35:34.200 --> 0:35:36.239
<v Speaker 3>the worse that worseness accelerates.

0:35:37.000 --> 0:35:40.799
<v Speaker 1>You mentioned how we have these different data sources, these

0:35:40.800 --> 0:35:43.160
<v Speaker 1>different you know, studies that are all leading us in

0:35:43.200 --> 0:35:46.480
<v Speaker 1>a certain direction, and we have by this point in

0:35:46.520 --> 0:35:51.800
<v Speaker 1>time developed ways to measure both the quantity and quality

0:35:51.920 --> 0:35:55.480
<v Speaker 1>of evidence. I really enjoyed your discussion on un randomized

0:35:55.520 --> 0:35:59.880
<v Speaker 1>controlled trials because this quote unquote gold standard of medical

0:35:59.840 --> 0:36:03.080
<v Speaker 1>studies that might not always be the gold standard, And

0:36:03.560 --> 0:36:05.239
<v Speaker 1>I was hoping you could tell me a little bit

0:36:05.239 --> 0:36:08.640
<v Speaker 1>about the times when the true gold standard might not be,

0:36:08.920 --> 0:36:12.360
<v Speaker 1>for instance, an RCT, but it might be something else entirely,

0:36:12.480 --> 0:36:14.640
<v Speaker 1>or it might be unethical to do a randomized control

0:36:14.719 --> 0:36:15.760
<v Speaker 1>trial in that situation.

0:36:16.200 --> 0:36:18.920
<v Speaker 3>Yeah, I think we've seen quite a lot of examples

0:36:19.200 --> 0:36:22.360
<v Speaker 3>where treating it as that kind of cookie cutter, this

0:36:22.840 --> 0:36:25.400
<v Speaker 3>is the only method we can use coleadisant problems. I mean,

0:36:25.400 --> 0:36:28.480
<v Speaker 3>smoking cancers are very well known one that we couldn't

0:36:28.800 --> 0:36:31.239
<v Speaker 3>just have in action because you can't get that level

0:36:31.239 --> 0:36:31.640
<v Speaker 3>of perfection.

0:36:31.640 --> 0:36:33.279
<v Speaker 2>I mean actually even the first.

0:36:33.040 --> 0:36:36.839
<v Speaker 3>Randomized control trial in modern medicine, which is ninety forty seven,

0:36:36.880 --> 0:36:38.919
<v Speaker 3>so stretcht to mycin, a trial for TB.

0:36:39.680 --> 0:36:41.520
<v Speaker 2>Austin Bradford Hill, who led.

0:36:41.400 --> 0:36:44.759
<v Speaker 3>That, made the point that actually Stretto mycin had some

0:36:44.880 --> 0:36:48.160
<v Speaker 3>very promising looking lab data and kind of early signals,

0:36:48.680 --> 0:36:51.800
<v Speaker 3>and he suggested it would have been unethical to withhold

0:36:51.840 --> 0:36:54.400
<v Speaker 3>it from patients if it was available. But actually as

0:36:54.400 --> 0:36:57.000
<v Speaker 3>a ninety forty seven there were currency controls. The UK

0:36:57.239 --> 0:36:59.560
<v Speaker 3>and its post war state couldn't get enough dollars to

0:36:59.560 --> 0:37:01.800
<v Speaker 3>buy uptimize it. So there wasn't enough to go around.

0:37:02.640 --> 0:37:04.800
<v Speaker 3>So in that situation, they said it would be ethical

0:37:04.840 --> 0:37:06.839
<v Speaker 3>to randomize because there's not enough of it. So there's

0:37:06.840 --> 0:37:08.640
<v Speaker 3>not enough of it, you might as well randomize and

0:37:08.680 --> 0:37:11.839
<v Speaker 3>just learn something along the way. And I think we've

0:37:11.840 --> 0:37:14.560
<v Speaker 3>seen that in other situations. I mean other sort of

0:37:14.600 --> 0:37:17.600
<v Speaker 3>examples that you see where things are very difficult to randomize.

0:37:17.640 --> 0:37:19.560
<v Speaker 3>You can think about natural experiments, a lot of the

0:37:20.120 --> 0:37:23.360
<v Speaker 3>well known moms the Vietnam Draft, where people essentially randomly

0:37:23.360 --> 0:37:25.640
<v Speaker 3>assigned to go to war based on their birthdays. A

0:37:25.680 --> 0:37:28.320
<v Speaker 3>lot of economists have done in Nobrock pires winning work

0:37:29.200 --> 0:37:32.520
<v Speaker 3>using that to understand the effects of war on subsequent

0:37:32.840 --> 0:37:36.680
<v Speaker 3>life outcomes, because it's not something where you can fully

0:37:36.719 --> 0:37:40.239
<v Speaker 3>design that experiment, but you can then make use of

0:37:40.239 --> 0:37:42.719
<v Speaker 3>what you have available. So I think a lot of

0:37:42.719 --> 0:37:45.160
<v Speaker 3>it just comes down to this issue we want to

0:37:45.239 --> 0:37:48.320
<v Speaker 3>understand called effect, and the benefit of randomization is a

0:37:48.360 --> 0:37:50.840
<v Speaker 3>lot of the other things that would influence whether or

0:37:50.880 --> 0:37:52.880
<v Speaker 3>not you know, someone's getting a vaccine and someone's getting

0:37:52.880 --> 0:37:55.960
<v Speaker 3>a disease, because you're randomizing on the vaccine, on average,

0:37:56.000 --> 0:37:57.960
<v Speaker 3>those will cancel out as effects, so it gives you

0:37:57.960 --> 0:38:00.799
<v Speaker 3>that quite neat benefit. But of course, you've also got

0:38:00.800 --> 0:38:03.520
<v Speaker 3>the challenge that you might run a population in one

0:38:03.520 --> 0:38:06.000
<v Speaker 3>group when one population that doesn't generalize to someone else.

0:38:06.040 --> 0:38:07.320
<v Speaker 2>You've also got the time issue.

0:38:07.400 --> 0:38:10.239
<v Speaker 3>So for diseases that evolve, you know, you might run

0:38:10.280 --> 0:38:13.400
<v Speaker 3>a trial now against flu or COVID or something a

0:38:13.480 --> 0:38:15.360
<v Speaker 3>year later that's going to be a different variant, and

0:38:15.560 --> 0:38:18.360
<v Speaker 3>to what extent can you carry over those conclusions. I

0:38:18.400 --> 0:38:21.719
<v Speaker 3>think we see a lot of examples in the literature where,

0:38:21.760 --> 0:38:25.000
<v Speaker 3>for instance, someone might run a trial in one population

0:38:25.760 --> 0:38:28.480
<v Speaker 3>for one disease, for flu, for example, and then see

0:38:28.480 --> 0:38:31.600
<v Speaker 3>a very different result when people look at population patterns elsewhere,

0:38:31.680 --> 0:38:35.040
<v Speaker 3>because it's a different immunse structure, it's a different strain,

0:38:35.280 --> 0:38:38.560
<v Speaker 3>it's a different time period. And yeah, I think we

0:38:38.600 --> 0:38:40.560
<v Speaker 3>can't just say, well that study from a few years

0:38:40.600 --> 0:38:42.120
<v Speaker 3>ago as the gold standard, we're only going to use

0:38:42.160 --> 0:38:44.000
<v Speaker 3>that one. We have to think about how these things

0:38:44.400 --> 0:38:46.080
<v Speaker 3>move along. I mean that being said, though, I think

0:38:46.080 --> 0:38:49.360
<v Speaker 3>in COVID there were missed opportunities I think to gather

0:38:49.440 --> 0:38:53.080
<v Speaker 3>much stronger data. I think it's very hard to justify

0:38:53.960 --> 0:38:57.160
<v Speaker 3>running those kinds of studies As a threat increases. I

0:38:57.200 --> 0:39:00.840
<v Speaker 3>think when epidemic's going up, your time to kind of

0:39:00.840 --> 0:39:02.920
<v Speaker 3>try and randomize it. I mean, I think essentially countries

0:39:03.000 --> 0:39:05.520
<v Speaker 3>have to take that threat, as the evidence suggests, but

0:39:05.560 --> 0:39:08.960
<v Speaker 3>I think particularly as countries lifted measures, that was often

0:39:09.000 --> 0:39:10.640
<v Speaker 3>just done in quite an ad hoc way, and we

0:39:10.680 --> 0:39:13.480
<v Speaker 3>could have done much more kind of staging. In the UK,

0:39:13.600 --> 0:39:16.480
<v Speaker 3>there were some early studies, for example, of can we

0:39:16.560 --> 0:39:19.640
<v Speaker 3>use rapid tests so people test themselves every day rather

0:39:19.680 --> 0:39:21.680
<v Speaker 3>than quarantining for like a week or two, and then

0:39:21.800 --> 0:39:24.239
<v Speaker 3>in practice a lot of people just didn't bother But

0:39:24.280 --> 0:39:26.319
<v Speaker 3>apart from that, you know, I think there's a lot

0:39:26.360 --> 0:39:29.480
<v Speaker 3>of these debates we're still having, and we probably could

0:39:29.480 --> 0:39:32.720
<v Speaker 3>have got better answers for that with some higher quality studies,

0:39:32.760 --> 0:39:35.520
<v Speaker 3>so not necessarily even an RCT, just just making use

0:39:36.080 --> 0:39:37.680
<v Speaker 3>of what we had with more observations.

0:39:38.719 --> 0:39:42.279
<v Speaker 1>One thing that I feel like during the COVID pandemic,

0:39:42.400 --> 0:39:46.760
<v Speaker 1>especially the early months, was this desire from the public

0:39:46.800 --> 0:39:49.440
<v Speaker 1>to have the answers. And I feel like there's a

0:39:49.440 --> 0:39:52.440
<v Speaker 1>lot of variation in how willing someone is to say

0:39:52.520 --> 0:39:55.520
<v Speaker 1>I don't know. And I'm wondering your feelings on this.

0:39:55.680 --> 0:39:59.480
<v Speaker 1>Do you feel that scientists in particular have a difficult

0:39:59.480 --> 0:40:02.239
<v Speaker 1>time so saying that they don't know the answer to

0:40:02.280 --> 0:40:05.640
<v Speaker 1>something like do we need to embrace uncertainty more in

0:40:05.920 --> 0:40:08.880
<v Speaker 1>as scientists or do you feel like there's that we

0:40:08.960 --> 0:40:12.080
<v Speaker 1>are embracing it but just not communicating it. Well, Yeah,

0:40:12.080 --> 0:40:12.399
<v Speaker 1>I think.

0:40:12.280 --> 0:40:14.360
<v Speaker 3>That's a really good question. It's kind of how I

0:40:14.360 --> 0:40:16.800
<v Speaker 3>guess how personality in politics and all these things go.

0:40:17.040 --> 0:40:20.080
<v Speaker 3>So and I think, I mean, there's been good reviews

0:40:20.080 --> 0:40:23.680
<v Speaker 3>of evidence showing that the overstated certainty just just undermines

0:40:23.719 --> 0:40:26.680
<v Speaker 3>trust and confidence, whether it's kind vaccine to its other things.

0:40:27.120 --> 0:40:29.600
<v Speaker 3>So saying yeah, this is one hundred percent, say there's

0:40:29.600 --> 0:40:31.839
<v Speaker 3>absolutely no risk, and if there's even a tiny risk,

0:40:31.880 --> 0:40:33.160
<v Speaker 3>you then kind of undermined that.

0:40:33.719 --> 0:40:35.640
<v Speaker 2>Yeah. One of the challenges with kind of that over.

0:40:36.280 --> 0:40:39.400
<v Speaker 3>Certainty, I think, particularly once you make that public statement,

0:40:39.640 --> 0:40:41.360
<v Speaker 3>it's very hard to back that, and we saw that

0:40:41.400 --> 0:40:44.280
<v Speaker 3>with some of the airborne trying some even health organizations

0:40:44.280 --> 0:40:47.600
<v Speaker 3>say it's not airborne fact. It's very difficult to then

0:40:48.000 --> 0:40:50.239
<v Speaker 3>walk that back. But I think it's fine line because

0:40:50.239 --> 0:40:51.960
<v Speaker 3>you don't just want to say we have no idea.

0:40:52.440 --> 0:40:54.880
<v Speaker 3>You want to try and communicate the way that evidence.

0:40:54.880 --> 0:40:57.360
<v Speaker 3>I think some countries did that better than others, particularly

0:40:57.360 --> 0:41:00.000
<v Speaker 3>them Mark Singapore s pling to mind on their reopening

0:41:00.000 --> 0:41:02.399
<v Speaker 3>where they said, this is the data we're looking at

0:41:02.400 --> 0:41:05.799
<v Speaker 3>to do this, that might change and this is kind

0:41:05.800 --> 0:41:07.560
<v Speaker 3>of how we have to work things through.

0:41:07.680 --> 0:41:10.520
<v Speaker 2>But I think one of the one of the.

0:41:10.440 --> 0:41:12.880
<v Speaker 3>Difficulty I think because any emergency it goes on for

0:41:12.920 --> 0:41:15.080
<v Speaker 3>that long, is you know, you have some people who

0:41:15.200 --> 0:41:18.080
<v Speaker 3>have very loudly said something's one hundred less, one hundred

0:41:18.120 --> 0:41:20.640
<v Speaker 3>times less severe than it is, and then they're kind

0:41:20.640 --> 0:41:24.160
<v Speaker 3>of very nailed onto to having to keep promoting that.

0:41:24.239 --> 0:41:26.680
<v Speaker 3>And I think it is There was one that one

0:41:26.680 --> 0:41:28.960
<v Speaker 3>of the government advisory committees I sat on. You know,

0:41:29.000 --> 0:41:30.520
<v Speaker 3>so a lot of the early alpha very and early

0:41:30.600 --> 0:41:32.359
<v Speaker 3>delta very and a lot of this early severity came

0:41:32.360 --> 0:41:34.960
<v Speaker 3>out of this group. And there was a phrase that

0:41:35.040 --> 0:41:37.080
<v Speaker 3>became used quite a lot, which was tell me why

0:41:37.120 --> 0:41:39.960
<v Speaker 3>I'm wrong. If you have that discussion where you want

0:41:40.000 --> 0:41:43.239
<v Speaker 3>to get criticism, if you present stuff and especially people

0:41:43.360 --> 0:41:46.000
<v Speaker 3>more senior and say is this is this correct, it's

0:41:46.080 --> 0:41:49.080
<v Speaker 3>very hard for people to kind of come in and say, oh, yeah,

0:41:49.120 --> 0:41:51.960
<v Speaker 3>actually I've sported a problem, especially if there's a power

0:41:52.000 --> 0:41:55.400
<v Speaker 3>dynamics or seniority and other things. So I think there

0:41:55.440 --> 0:41:56.880
<v Speaker 3>was a lot of that thing where people present work

0:41:56.920 --> 0:41:58.640
<v Speaker 3>and right, tell me why I'm wrong, tell me what

0:41:58.719 --> 0:42:01.799
<v Speaker 3>I'm missing. And I think that's quite a healthy attitude

0:42:01.800 --> 0:42:05.680
<v Speaker 3>in that kind of environment to be much more, you know,

0:42:05.760 --> 0:42:07.480
<v Speaker 3>looking for weaknesses and being able to.

0:42:07.480 --> 0:42:08.440
<v Speaker 2>Kind of lay out.

0:42:08.920 --> 0:42:11.360
<v Speaker 3>And I remember, actually I think it was when was

0:42:11.360 --> 0:42:13.480
<v Speaker 3>it the gamma variant is sort of emerging in Latin

0:42:13.520 --> 0:42:16.239
<v Speaker 3>America And I gave immediate interview, and when they write

0:42:16.239 --> 0:42:19.640
<v Speaker 3>it up, it was basically, you know, Dotowski doesn't really know.

0:42:19.760 --> 0:42:21.560
<v Speaker 2>It was the kind of open but in that situation,

0:42:22.400 --> 0:42:23.280
<v Speaker 2>we didn't.

0:42:23.360 --> 0:42:25.560
<v Speaker 3>And it is hard to do because I think, you know,

0:42:25.640 --> 0:42:28.600
<v Speaker 3>especially people asking you questions around your area of expertise,

0:42:29.320 --> 0:42:31.359
<v Speaker 3>I think in terms of how to balance that not

0:42:31.400 --> 0:42:33.239
<v Speaker 3>just saying I don't know, but saying, well, we do

0:42:33.360 --> 0:42:35.799
<v Speaker 3>know this, and we can make some judgment. And there

0:42:35.840 --> 0:42:38.840
<v Speaker 3>was this wonderful study in the nineteen fifty one is

0:42:38.880 --> 0:42:42.080
<v Speaker 3>by the CIA analyst, and it was about words we

0:42:42.200 --> 0:42:46.200
<v Speaker 3>use when we're unsure and words about judgment, and it

0:42:46.360 --> 0:42:49.239
<v Speaker 3>basically realized that people used probable and possible to meet

0:42:49.280 --> 0:42:51.279
<v Speaker 3>all sorts of things, and they all know had kind

0:42:51.280 --> 0:42:53.799
<v Speaker 3>of different notions. And he said, yeah, humans will go

0:42:53.920 --> 0:42:57.000
<v Speaker 3>out of their way to making a judgment about something

0:42:57.920 --> 0:43:00.200
<v Speaker 3>that we'll often you know, the risk is you get

0:43:00.239 --> 0:43:02.839
<v Speaker 3>the uncertainty where we're like very hazy, like, oh it's

0:43:02.880 --> 0:43:06.880
<v Speaker 3>you know, it's a definite possibility. And actually, in some cases,

0:43:07.200 --> 0:43:09.200
<v Speaker 3>like with you know, if you've got an emerging threat

0:43:09.520 --> 0:43:11.360
<v Speaker 3>and you've got experts, you do actually want them to

0:43:11.360 --> 0:43:13.520
<v Speaker 3>put a number on it, you know, even if there's uncertainty,

0:43:13.560 --> 0:43:16.520
<v Speaker 3>you want them to say, I am sixty percent sure

0:43:16.960 --> 0:43:19.000
<v Speaker 3>that this is the case. And there's been a lot

0:43:19.000 --> 0:43:21.600
<v Speaker 3>of nice work, you know, even around things like super forecasting,

0:43:21.680 --> 0:43:25.719
<v Speaker 3>where people make those predictions and you can go back

0:43:25.719 --> 0:43:27.400
<v Speaker 3>and then look because you know, if people are well

0:43:27.560 --> 0:43:31.160
<v Speaker 3>calibrated in their uncertainty. Yeah, if you say you're fifty

0:43:31.160 --> 0:43:34.399
<v Speaker 3>percent sure about a list of things, about fifty percent

0:43:34.400 --> 0:43:36.080
<v Speaker 3>at the time those things should happen. So about half

0:43:36.120 --> 0:43:38.520
<v Speaker 3>the things on that list should occur. So there are

0:43:38.520 --> 0:43:41.640
<v Speaker 3>these situations where I think we can get better just

0:43:41.680 --> 0:43:43.840
<v Speaker 3>about thinking about our own uncertainty. And one of the

0:43:43.840 --> 0:43:45.880
<v Speaker 3>things that I actually tried to do, I've tried to

0:43:45.920 --> 0:43:47.640
<v Speaker 3>do a lot of kind of emerging threats is even

0:43:47.719 --> 0:43:49.600
<v Speaker 3>just just writing down what you think is going to happen.

0:43:49.600 --> 0:43:53.600
<v Speaker 3>Because I think we're great, you know, the human mind

0:43:53.680 --> 0:43:56.279
<v Speaker 3>at like kind of rationalizing. Oh yeah, maybe I did

0:43:56.280 --> 0:43:58.800
<v Speaker 3>think that, And so yeah, I did quite quite a

0:43:58.800 --> 0:44:02.440
<v Speaker 3>lot of like where where you could state I actually

0:44:02.480 --> 0:44:04.520
<v Speaker 3>think the vaccine is going to be pretty good, or

0:44:04.560 --> 0:44:07.000
<v Speaker 3>you know, I think this, and like this is where

0:44:07.040 --> 0:44:09.320
<v Speaker 3>social media, when it was maybe slightly less polarized, is

0:44:09.360 --> 0:44:11.239
<v Speaker 3>quite helpful because you could just put a post out.

0:44:11.600 --> 0:44:13.359
<v Speaker 3>And I think I was always very careful. I didn't

0:44:13.360 --> 0:44:15.279
<v Speaker 3>delete any of my tweets during COVID because I was like,

0:44:15.280 --> 0:44:18.040
<v Speaker 3>I actually want that record, and there were time I

0:44:18.080 --> 0:44:20.439
<v Speaker 3>got one. You know, I was in Singapore in feb

0:44:20.520 --> 0:44:23.239
<v Speaker 3>twenty twenty and their policy was don't wear a mask

0:44:23.320 --> 0:44:26.080
<v Speaker 3>unless you have symptoms, and I think I tweeted I

0:44:26.080 --> 0:44:27.880
<v Speaker 3>was like, yeah, that seems like a sensible policy, and

0:44:27.880 --> 0:44:30.840
<v Speaker 3>that seems quite at evidence space. And now we'd probably

0:44:31.320 --> 0:44:33.600
<v Speaker 3>with some of the studies not look back on that

0:44:33.680 --> 0:44:36.359
<v Speaker 3>as being the best post. But so yeah, I think

0:44:36.360 --> 0:44:39.239
<v Speaker 3>it's almost that as well as overstated certainty. I think

0:44:39.280 --> 0:44:42.239
<v Speaker 3>it's also holding ourselves to account, even IF's just you know,

0:44:42.360 --> 0:44:46.080
<v Speaker 3>privately about how confident we were and what played out.

0:44:46.960 --> 0:44:50.480
<v Speaker 1>I want to close out by asking you about the

0:44:50.520 --> 0:44:53.040
<v Speaker 1>subtitle of your book, which is the art and science

0:44:53.080 --> 0:44:55.719
<v Speaker 1>of certainty? And I want to know about the art

0:44:56.120 --> 0:45:00.040
<v Speaker 1>part of this, what is the art s?

0:45:01.200 --> 0:45:04.640
<v Speaker 3>So I think for me it was the more I

0:45:04.719 --> 0:45:08.480
<v Speaker 3>dug into this, the more I saw these other elements

0:45:08.520 --> 0:45:12.399
<v Speaker 3>beyond kind of pure logic, pure observation coming in. I mean,

0:45:12.440 --> 0:45:15.400
<v Speaker 3>even if you look at what was essentially a bit

0:45:15.440 --> 0:45:18.400
<v Speaker 3>of a mathematical civil war in the late nineteenth century,

0:45:18.480 --> 0:45:22.239
<v Speaker 3>where a lot of these ancient Greek theorems, you know,

0:45:22.480 --> 0:45:25.759
<v Speaker 3>things about the properties of triangles started to break down

0:45:25.800 --> 0:45:28.399
<v Speaker 3>because people started to draw shapes on spheres and other

0:45:28.600 --> 0:45:32.080
<v Speaker 3>structures and come up with functions that these supposedly proven

0:45:32.120 --> 0:45:35.280
<v Speaker 3>theorems didn't hold. And I think one of the reasons

0:45:35.280 --> 0:45:38.040
<v Speaker 3>that was really controversial was there was this idea that

0:45:38.040 --> 0:45:41.080
<v Speaker 3>there's a universal truth out there about the world, and

0:45:41.600 --> 0:45:44.640
<v Speaker 3>actually in this situation, it kind of depended on what

0:45:44.719 --> 0:45:47.440
<v Speaker 3>assumptions humans were making and what we were willing to

0:45:48.280 --> 0:45:49.040
<v Speaker 3>kind of define.

0:45:49.480 --> 0:45:51.120
<v Speaker 2>And even in this.

0:45:51.120 --> 0:45:56.040
<v Speaker 3>Supposedly pure subjects, there's still these debates around well, it

0:45:56.120 --> 0:45:58.160
<v Speaker 3>kind of depends on which one you want to pick,

0:45:58.200 --> 0:46:01.800
<v Speaker 3>and that will change the answer. I think even in science,

0:46:01.840 --> 0:46:04.319
<v Speaker 3>it's a lot of these situations where you know, we

0:46:04.320 --> 0:46:07.960
<v Speaker 3>can accumulate the evidence, but then you have disagreement about

0:46:07.960 --> 0:46:09.759
<v Speaker 3>where you set the freshal. I mean that this kind

0:46:09.760 --> 0:46:12.239
<v Speaker 3>of five percent cut off has become very popular, this

0:46:12.560 --> 0:46:14.680
<v Speaker 3>sort of p value that the chance that you'd get

0:46:14.719 --> 0:46:17.520
<v Speaker 3>a result that extreme if there was nothing going on

0:46:17.960 --> 0:46:20.719
<v Speaker 3>or you're no, no hypothesis was wrong. But that was

0:46:20.760 --> 0:46:22.400
<v Speaker 3>kind of arbitrary. I mean it was partly picked just

0:46:22.480 --> 0:46:24.600
<v Speaker 3>for convenience, that this was you one hundred years ago.

0:46:24.600 --> 0:46:26.200
<v Speaker 3>The calculations just a bit easier if they picked a

0:46:26.280 --> 0:46:28.880
<v Speaker 3>value one fished a lot to work, just easier to

0:46:28.880 --> 0:46:32.600
<v Speaker 3>pick a value around point zero five. And others who

0:46:32.640 --> 0:46:35.520
<v Speaker 3>were more pragmatic, you know, working in business on something

0:46:35.560 --> 0:46:38.280
<v Speaker 3>and thinking, well, actually the evidence is a bit weaker,

0:46:39.200 --> 0:46:41.279
<v Speaker 3>but that's still useful to it. So there's this kind

0:46:41.280 --> 0:46:45.319
<v Speaker 3>of human balancing act. And yeah, we saw it again,

0:46:45.480 --> 0:46:48.960
<v Speaker 3>things like legal cases where how much you value different

0:46:49.000 --> 0:46:52.080
<v Speaker 3>types of errors depends a lot on the individuals. I mean,

0:46:52.120 --> 0:46:55.120
<v Speaker 3>one of the examples that that fan fascinating in the

0:46:55.120 --> 0:46:57.480
<v Speaker 3>book was Einstein, when you moved to the US, got

0:46:57.560 --> 0:47:00.200
<v Speaker 3>very angry about peer review because he sent some to

0:47:00.560 --> 0:47:02.160
<v Speaker 3>send something to a journal and it came back as like, oh,

0:47:02.160 --> 0:47:03.680
<v Speaker 3>we've got another opinion on it, and he was like

0:47:03.680 --> 0:47:05.440
<v Speaker 3>whoaah whoa, Like, why haven't you.

0:47:05.440 --> 0:47:07.000
<v Speaker 2>Just accepted my work?

0:47:07.320 --> 0:47:09.600
<v Speaker 3>And actually Max Max Plank, who published some of his

0:47:09.719 --> 0:47:13.120
<v Speaker 3>like amazing early papers, Plank made that point that actually,

0:47:13.280 --> 0:47:15.920
<v Speaker 3>I would rather kind of publish a few things that

0:47:15.960 --> 0:47:18.719
<v Speaker 3>are a bit you know, nonsense, than this is me

0:47:18.760 --> 0:47:22.040
<v Speaker 3>paraphrasing the miss a really important idea. So for him,

0:47:22.080 --> 0:47:24.680
<v Speaker 3>his threshold was like, I want to set the threshold low.

0:47:25.520 --> 0:47:29.640
<v Speaker 3>Admittedly mainly amongst kind of physicists he knew, because I

0:47:29.640 --> 0:47:31.440
<v Speaker 3>don't want to set it too high and miss a

0:47:31.440 --> 0:47:32.680
<v Speaker 3>good idea, and.

0:47:32.640 --> 0:47:34.080
<v Speaker 2>I think we all we all have those.

0:47:34.160 --> 0:47:35.920
<v Speaker 3>This kind of that's where the art, I think creeps

0:47:35.920 --> 0:47:39.360
<v Speaker 3>in that that kind of subjectivity in not just the evidence,

0:47:39.360 --> 0:47:41.319
<v Speaker 3>I think one of For me, the real difference was

0:47:41.360 --> 0:47:44.080
<v Speaker 3>something like proof is it's not just generating data. It's

0:47:44.120 --> 0:47:46.480
<v Speaker 3>how that data interacts with the world and the decisions

0:47:46.520 --> 0:47:49.240
<v Speaker 3>we make. And I think that's where things get really interesting.

0:47:49.280 --> 0:47:50.680
<v Speaker 3>It's like where do we actually set the bar for

0:47:50.760 --> 0:47:54.160
<v Speaker 3>evidence and then both to convince ourselves but then go

0:47:54.200 --> 0:47:56.960
<v Speaker 3>out and convince others too well.

0:47:56.960 --> 0:47:59.799
<v Speaker 1>Professor Kocharski, thank you so much for joining me today.

0:47:59.800 --> 0:48:03.400
<v Speaker 1>This This was such an enlightening conversation, and I really did.

0:48:03.520 --> 0:48:06.680
<v Speaker 1>I loved your book Proof, so I appreciate you coming

0:48:06.680 --> 0:48:07.280
<v Speaker 1>onto the show.

0:48:07.800 --> 0:48:08.759
<v Speaker 2>Thanks great to talk.

0:48:27.800 --> 0:48:30.319
<v Speaker 1>A big thank you again to doctor Adam Kocharski for

0:48:30.400 --> 0:48:32.640
<v Speaker 1>taking the time to chat with me. If you enjoyed

0:48:32.640 --> 0:48:35.520
<v Speaker 1>today's episode and would like to learn more, check out

0:48:35.520 --> 0:48:38.080
<v Speaker 1>our website This podcast will Kill You dot com, where

0:48:38.080 --> 0:48:40.360
<v Speaker 1>I'll post a link to where you can find Proof,

0:48:40.480 --> 0:48:42.840
<v Speaker 1>the Art and Science of Certainty, as well as a

0:48:42.880 --> 0:48:45.600
<v Speaker 1>link to DODR. Kocharski's website where you can also find

0:48:45.640 --> 0:48:49.520
<v Speaker 1>his other book, The Rules of Contagion, Why Things Spread

0:48:49.719 --> 0:48:52.640
<v Speaker 1>and Why They Stop and Don't Forget. You can also

0:48:52.760 --> 0:48:55.640
<v Speaker 1>check out our website for all sorts of other cool things,

0:48:56.040 --> 0:49:00.719
<v Speaker 1>including but not limited to, transcripts, quarantine recipes, show notes,

0:49:00.719 --> 0:49:03.680
<v Speaker 1>and references for all of our episodes, links to merch

0:49:03.760 --> 0:49:06.840
<v Speaker 1>our bookshop dot Org, affiliate account, our Goodreads list, a

0:49:06.880 --> 0:49:11.200
<v Speaker 1>first hand account, form, and music by Bloodmobile. Speaking of which,

0:49:11.320 --> 0:49:13.799
<v Speaker 1>thank you to Bloodmobile for providing the music for this

0:49:13.840 --> 0:49:16.760
<v Speaker 1>episode and all of our episodes. Thank you to Leona

0:49:16.760 --> 0:49:20.240
<v Speaker 1>Squilacci and Tom Bryfocal for our audio mixing, and thanks

0:49:20.239 --> 0:49:22.759
<v Speaker 1>to you listeners for listening. I hope you liked this

0:49:22.880 --> 0:49:26.680
<v Speaker 1>episode and our loving still being part of the TPWKY

0:49:26.760 --> 0:49:30.080
<v Speaker 1>book Club. A special thank you, as always to our

0:49:30.200 --> 0:49:35.320
<v Speaker 1>fantastic patrons. We appreciate your support so very much. Well,

0:49:35.440 --> 0:50:02.359
<v Speaker 1>until next time, keep washing those hands, FU