WEBVTT - Modeling the Coronavirus 0:00:15.396 --> 0:00:22.076 Pushkin from Pushkin Industries. This is Deep Background, the show 0:00:22.116 --> 0:00:25.756 where we explore the stories behind the stories in the news. 0:00:26.276 --> 0:00:30.436 I'm Noah Feldman. These days, it seems we live and 0:00:30.516 --> 0:00:34.236 die by the model. You can't turn on your phone, 0:00:34.316 --> 0:00:37.276 or open the newspaper or watch the television without being 0:00:37.476 --> 0:00:40.436 hit in the face by some sort of model or 0:00:40.516 --> 0:00:43.596 graph or chart that purports to show you how fast 0:00:43.636 --> 0:00:46.316 the coronavirus is likely to spread, when it will peak, 0:00:46.396 --> 0:00:49.076 whether it will plateau, how many people will die, you 0:00:49.236 --> 0:00:53.396 name it. These graphs and charts and models are constantly changing, 0:00:53.796 --> 0:00:57.516 and sometimes they are in conflict with one another. So 0:00:57.836 --> 0:00:59.636 how should we be making sense of all of this? 0:01:00.676 --> 0:01:05.036 Here to help is Carl Bergstrup. He's a computational biologist 0:01:05.076 --> 0:01:08.316 at the University of Washington who's got a deep background 0:01:08.436 --> 0:01:12.996 both in epidemiology and in model building and in model analysis. 0:01:13.276 --> 0:01:16.636 He's also an expert on the spread of misinformation. With 0:01:16.676 --> 0:01:20.476 his co author Jevin D. West, he's written a forthcoming book, 0:01:20.996 --> 0:01:24.716 Calling Bullshit, The Art of Skepticism in a Data Driven World. 0:01:25.196 --> 0:01:28.356 I spoke to Carl on Monday afternoon. Carl, you are, 0:01:28.676 --> 0:01:33.316 by profession some combination of a modeler and an explainer 0:01:33.396 --> 0:01:37.556 of models and a debunker of bad models, and we 0:01:37.636 --> 0:01:42.196 now live in model world twenty four seven. So my 0:01:42.236 --> 0:01:44.036 first question before I ask you to do all three 0:01:44.076 --> 0:01:47.556 of those things simultaneously is how weird has this time 0:01:47.636 --> 0:01:50.316 been for you? That's a good question. I think it's 0:01:50.356 --> 0:01:52.156 hard when you're in the middle of it. You don't 0:01:52.196 --> 0:01:54.916 even really pause to think about whether it's weird or normal. 0:01:54.996 --> 0:01:59.036 It just is. The thing that strikes me as the 0:01:59.076 --> 0:02:02.636 strangest in a sense, is that I spent many years 0:02:02.636 --> 0:02:07.396 doing confectious disease epidemiology, and then, by various convoluted paths, 0:02:07.716 --> 0:02:09.716 ended up over the last few years studying the spread 0:02:09.756 --> 0:02:13.476 of misinformation on social networks. And to have those two 0:02:13.516 --> 0:02:15.756 things come together the way that they're coming together right 0:02:15.756 --> 0:02:18.796 now has been really striking. Let's start with a model 0:02:18.836 --> 0:02:21.516 that you've spoken in favor of, in fact strongly in 0:02:21.556 --> 0:02:25.876 favor of, namely the simple model that produced the expression 0:02:25.916 --> 0:02:28.516 which has now entered the lingo of flattening the curve. 0:02:29.236 --> 0:02:32.196 Say a word about why that basic curve with the 0:02:32.236 --> 0:02:35.836 flattening was so effective. Well, I think it's the notion 0:02:35.876 --> 0:02:39.916 of a very simple idea made concrete with a nice picture, 0:02:39.956 --> 0:02:42.436 and so you know, the most important thing about that 0:02:42.516 --> 0:02:46.236 picture was that it is worth a thousand words, and 0:02:46.316 --> 0:02:49.196 those thousand words are worth thousands of lives, I believe, 0:02:49.556 --> 0:02:53.156 because they showed people at the time why one argument 0:02:53.196 --> 0:02:55.476 that was floating around was not a good argument. And 0:02:55.476 --> 0:02:57.276 what had been floating around at the time was, look, 0:02:57.636 --> 0:02:59.716 we're all going to you know, people were saying, we're 0:02:59.716 --> 0:03:01.556 all going to get this or a large fraction unless 0:03:01.556 --> 0:03:04.276 they're going to get this, and there's nothing we can 0:03:04.316 --> 0:03:06.276 do about it. So let's get this over with as 0:03:06.316 --> 0:03:08.476 quick as possible. So they must talk about taking it 0:03:08.516 --> 0:03:11.596 on the chin, talk about you know, why would you 0:03:11.636 --> 0:03:16.436 prolong the economic disruption and so forth. And one thing 0:03:16.436 --> 0:03:19.636 that people hadn't been thinking about was the sort of 0:03:20.356 --> 0:03:22.836 way that if we didn't do anything to control the pandemic, 0:03:22.916 --> 0:03:25.756 then we would badly exceed hospital capacity. And so what 0:03:25.876 --> 0:03:30.436 this one little picture did was stressed that you have 0:03:30.516 --> 0:03:33.596 to worry not only about the area under the pandemic 0:03:33.676 --> 0:03:36.276 curve where the epidemic curve, which also have to worry 0:03:36.396 --> 0:03:38.676 about the height of that epidemic curve at any given 0:03:38.716 --> 0:03:40.796 time because we have a limited hospital capacity and for 0:03:40.836 --> 0:03:44.836 this particular disease, I see you care saves lives. So 0:03:44.876 --> 0:03:47.156 I just shifted the framework of people's thinking in a 0:03:47.236 --> 0:03:49.316 very simple way, and I think, you know, people definitely 0:03:49.316 --> 0:03:51.676 started rallying to this notion of flattening the curve, and 0:03:52.396 --> 0:03:55.756 people now take that for granted. It's time to start 0:03:55.756 --> 0:03:58.036 thinking about new models and new new pictures now that 0:03:58.076 --> 0:04:00.956 this is something we all take for granted. But I 0:04:00.956 --> 0:04:03.276 think certainly some of the reason we haven't exceeded healthcare 0:04:03.276 --> 0:04:06.716 capacity any worse than we have is that people kind 0:04:06.716 --> 0:04:08.916 of took this message to heart and realize that there 0:04:08.996 --> 0:04:13.556 is an important public health role that'll be played by 0:04:13.956 --> 0:04:17.196 trying to slow down the spread of this early on, Carl, 0:04:17.236 --> 0:04:20.396 I want to ask you a sort of philosophical question 0:04:20.476 --> 0:04:23.756 about models that's been really very much front of mine 0:04:23.836 --> 0:04:27.876 for me, and that is, broadly speaking, the relationship between 0:04:27.916 --> 0:04:30.156 a model that's meant to tell you what you should 0:04:30.196 --> 0:04:34.276 do in life, a kind of normative model like the 0:04:34.316 --> 0:04:37.196 model that you were talking about, the flattening the curve model, 0:04:37.916 --> 0:04:41.636 and a descriptive model, a model that's meant to describe 0:04:41.756 --> 0:04:43.556 the world as it is or as you think there's 0:04:43.596 --> 0:04:47.156 some probability of it being. This is a very hard 0:04:47.156 --> 0:04:49.756 line to draw. But I think it's a distinction that 0:04:49.796 --> 0:04:54.076 really matters because we're in this delicate moment now where 0:04:54.156 --> 0:04:57.756 people are observing that models that were meant to say, hey, 0:04:57.836 --> 0:05:00.476 if you don't do anything, you'll have results X and 0:05:00.676 --> 0:05:03.756 Y and Z are now giving way to new models 0:05:03.756 --> 0:05:05.436 that say, well, we've updated the models in light of 0:05:05.476 --> 0:05:08.556 the social distancing that we've done. And that's leading some 0:05:08.596 --> 0:05:11.316 part of the public, including even the somewhat educated public, 0:05:11.356 --> 0:05:13.796 I think, to say, well, wait a minute, maybe those 0:05:13.836 --> 0:05:16.956 initial models were grossly overstated. And it seems to me 0:05:16.996 --> 0:05:19.196 that part of the distinction here is the difference between 0:05:19.516 --> 0:05:22.716 a model that is meant to say, take action and 0:05:22.836 --> 0:05:25.436 a model that's meant to say, here's the description of 0:05:25.436 --> 0:05:28.316 the world, updating for what we have done. Could you 0:05:28.356 --> 0:05:30.676 say a word about that distinction. Yeah, I mean, it's 0:05:30.676 --> 0:05:33.636 not actually a distinction I had been thinking along about. 0:05:33.716 --> 0:05:36.436 I mean, I would have thought, you know, more about 0:05:36.996 --> 0:05:41.156 the fact that the models have a feedback in terms 0:05:41.156 --> 0:05:44.516 of the models influence our behavior, which influences the data 0:05:44.556 --> 0:05:46.036 that go into the model that you know as you 0:05:46.076 --> 0:05:48.396 go through this loop. And so what's happening, of course, 0:05:48.516 --> 0:05:50.636 is that early models, including the flat and the curve 0:05:51.236 --> 0:05:55.876 just a conceptual model influence people's behavior, which then changes 0:05:55.916 --> 0:05:59.036 the situation that we're in. And so now people can say, oh, well, 0:05:59.036 --> 0:06:00.956 you know that was stupid. Why didn't we flatten the curve. 0:06:00.996 --> 0:06:03.996 Hospitals haven't been overrun. It's like, well, yeah, hospitals haven't 0:06:04.036 --> 0:06:08.876 been overrun precisely because we locked down places and flatten 0:06:08.916 --> 0:06:11.236 the cur and that's why. You know, except for a 0:06:11.276 --> 0:06:14.796 New York City, essentially most places are managing pretty recently 0:06:14.796 --> 0:06:17.676 in terms of hospital capacity. I'd kind of, rather than 0:06:17.716 --> 0:06:20.636 saying there are models that are normative and models that 0:06:20.636 --> 0:06:23.556 are descriptive, I would say, there's the models can be 0:06:24.036 --> 0:06:26.636 used in both of those directions, and so you can 0:06:26.676 --> 0:06:29.716 come up with a model likely fight the curve model, 0:06:29.756 --> 0:06:32.156 and that sort of describes two different things that could happen, 0:06:32.156 --> 0:06:34.556 and then you have to draw your own normative conclusions 0:06:34.596 --> 0:06:37.236 from that. So, you know, we could have this big 0:06:37.276 --> 0:06:39.036 fast peak, we'd be done with it by the middle 0:06:39.036 --> 0:06:40.556 of the summer. We'd all be back, you know, those 0:06:40.556 --> 0:06:43.276 of us who are still alive, to our normal lives. 0:06:43.676 --> 0:06:45.956 But we would have had to live through this period 0:06:45.956 --> 0:06:49.836 where the hospitals are massively over capacity. We would have 0:06:49.876 --> 0:06:52.116 lost friends and relatives and neighbors and so on. Or 0:06:52.276 --> 0:06:55.076 we can try this other approach, which we don't even 0:06:55.116 --> 0:06:57.756 know exactly how we're going to manage to keep this 0:06:57.796 --> 0:06:59.356 thing down run in the future, but we can at 0:06:59.396 --> 0:07:01.236 least get ourselves onto that track so that we can 0:07:01.276 --> 0:07:03.836 solve the problem. And then if we do that, we're 0:07:03.836 --> 0:07:06.156 probably going to be looking at a more protracted period 0:07:06.196 --> 0:07:10.476 of life being different, but we're going to avoid these 0:07:11.196 --> 0:07:15.756 really catastrophic periods of exceeding health capacity. So I guess 0:07:15.796 --> 0:07:18.476 that would be sort of my distinction, which is perhaps 0:07:18.516 --> 0:07:20.556 a little different than the one you're driving. So if 0:07:20.596 --> 0:07:23.156 I understand you correctly, what you're saying is the model 0:07:23.196 --> 0:07:27.556 itself is at least arguably neutral, and then it can 0:07:27.596 --> 0:07:30.916 be used either to make a descriptive point about how 0:07:30.916 --> 0:07:33.596 the world is or a normative point. And I guess 0:07:33.676 --> 0:07:37.036 my pushback on that, if I'm understanding you correctly, would 0:07:37.036 --> 0:07:41.356 be that because as you say, you're modeling either taking 0:07:41.356 --> 0:07:45.716 into account probable effects on the world or not taking 0:07:45.756 --> 0:07:49.036 those things into account. It's tricky, maybe not impossible, but 0:07:49.076 --> 0:07:53.196 tricky to separate out a model that says, hey, this 0:07:53.276 --> 0:07:54.476 is the way the world, you know, take it or 0:07:54.516 --> 0:07:55.876 leave it. But this is the world way the world 0:07:55.876 --> 0:07:58.636 would be if you do nothing, versus here's a model 0:07:58.836 --> 0:08:01.076 based on what we think will happen if you do 0:08:01.196 --> 0:08:04.196 take these steps. Yeah, I think that's right, and I mean, 0:08:04.316 --> 0:08:06.676 but I agree with you about that, and I also 0:08:06.916 --> 0:08:10.236 with respect pushback. I mean, I pushback. I'm making you 0:08:10.236 --> 0:08:13.356 know this claim that they're absolutely neutral myself, because I mean, 0:08:13.396 --> 0:08:15.836 models or tools, models are designed for purposes, and so 0:08:15.916 --> 0:08:18.756 people are making each of these models for some particular reason. 0:08:19.076 --> 0:08:20.756 You know, the flat and the curve was made to 0:08:20.796 --> 0:08:23.436 try to help people see that it's not just the 0:08:23.476 --> 0:08:26.276 total number of cases, it's also the timing of those cases, 0:08:26.316 --> 0:08:27.996 whether they all happen at the same time or not. 0:08:28.316 --> 0:08:31.076 It had a purpose. I think you can kind you 0:08:31.116 --> 0:08:32.436 can kind of see them that way when I look 0:08:32.436 --> 0:08:33.996 at some of the models that people are doing now, 0:08:34.156 --> 0:08:37.676 For example, the there's this IGMME model that's being used 0:08:37.676 --> 0:08:39.916 a lot that predicts hospital needs by state and also 0:08:39.996 --> 0:08:42.836 death rates by state and so on, and so that's 0:08:42.836 --> 0:08:46.916 a model that's been designed to be used precisely for 0:08:47.036 --> 0:08:49.276 thinking about, you know, what kind of equipment, how many 0:08:49.276 --> 0:08:51.636 beds do we need and so on, So you know, 0:08:51.636 --> 0:08:53.476 even though you can say, well it's just a model 0:08:53.516 --> 0:08:55.716 of what happens, it's still designed to this purpose. And 0:08:55.716 --> 0:08:58.356 then once that purpose comes in, then you know that 0:08:58.396 --> 0:09:00.396 purpose is usually to make a decision, and so the 0:09:00.436 --> 0:09:04.356 whole normative aspect of modeling flows into that through that channel. 0:09:04.356 --> 0:09:08.756 I think if we're going to try to reopen the economy, 0:09:08.836 --> 0:09:13.796 cause before we have the capacity, assuming we ever get 0:09:13.836 --> 0:09:17.796 the capacity to do millions and millions of tests, then 0:09:17.876 --> 0:09:19.796 there are going to have to be, as it were, 0:09:20.276 --> 0:09:24.476 many spikes in the curve. We just want those spikes 0:09:24.476 --> 0:09:27.716 to be below the point where they overwhelm the hospitals. 0:09:28.236 --> 0:09:30.676 What's a model that sort of pictures that? I mean? 0:09:30.796 --> 0:09:32.596 Is it a kind of sign curve model where the 0:09:33.116 --> 0:09:35.516 top of the curve is just beneath the number of 0:09:35.516 --> 0:09:37.716 people who will flood the hospitals? Is that what we're 0:09:37.716 --> 0:09:40.876 realistically talking about until such time as we have either 0:09:40.876 --> 0:09:43.396 a vaccine or very extensive testing. Well, I'm not very 0:09:43.436 --> 0:09:47.276 optimistic about that approach, because you don't get something like 0:09:47.276 --> 0:09:49.396 a nice smooth signed curve, but you usually get as 0:09:49.436 --> 0:09:52.756 a pretty rapid ramp up to being at hospital capacity 0:09:52.796 --> 0:09:54.756 and a fairly slow drop off and then a rapid 0:09:54.836 --> 0:09:56.836 ramp up at a fairly slow drop off, so you 0:09:56.836 --> 0:10:00.036 actually spend most of your time with social distancing still 0:10:00.116 --> 0:10:03.836 han and only these little gaps with social distancing off. 0:10:03.876 --> 0:10:07.436 That's what came out of the Imperial College model at 0:10:07.436 --> 0:10:10.236 the scenario. And then there are been other plans along 0:10:10.276 --> 0:10:12.156 these lines, a two day work week plan that our 0:10:12.316 --> 0:10:14.196 lines groups proposed and so on, and they have this 0:10:14.316 --> 0:10:17.436 general form. So that's one problem. The other problem is 0:10:17.476 --> 0:10:21.436 you're dealing in these cases with exponential growth and imperfect measurement, 0:10:21.516 --> 0:10:25.356 and so when you try to manage exponential growth to 0:10:25.436 --> 0:10:28.156 sort of top it out right below hospital capacity, that's 0:10:28.156 --> 0:10:30.836 a very difficult call to make given the amount of 0:10:30.836 --> 0:10:32.636 information you have. And so what's going to happen is 0:10:32.636 --> 0:10:34.716 you're going to miss, and you're going to miss by 0:10:34.756 --> 0:10:37.516 a lot because it's exponential growth. So I think the 0:10:37.516 --> 0:10:39.876 other thing, you know, there was a nice paper that 0:10:40.556 --> 0:10:44.516 you looked at that sort of optimal management of a 0:10:44.556 --> 0:10:47.356 pandemic like this and keeping it under hospital capacity. But 0:10:47.476 --> 0:10:52.236 the conclusion, informally stated, you know, was essentially, don't try 0:10:52.236 --> 0:10:55.516 to get cute with epidemic growth. And I think that's 0:10:55.636 --> 0:10:57.596 kind of a key message that comes out of modeling 0:10:57.636 --> 0:10:59.556 there is. It's you know, as soon as you start 0:10:59.596 --> 0:11:02.996 thinking about the uncertainties that are in place, you recognize 0:11:03.076 --> 0:11:07.796 that actually keeping it under capacity is an almost impossible challenge. 0:11:09.116 --> 0:11:12.596 And making a really powerful point about the likelihood of 0:11:12.796 --> 0:11:16.316 best case and worst case scenarios, which I think is 0:11:16.436 --> 0:11:20.116 quirky and totally non obvious to the labors of whom 0:11:20.116 --> 0:11:22.876 I am a good example. So we often think, well, 0:11:22.916 --> 0:11:24.476 there are two models. There's a best case, they're in 0:11:24.476 --> 0:11:26.956 a worst case scenario, and in life things will probably 0:11:26.996 --> 0:11:29.516 be somewhere in the middle. That's how we often think. 0:11:29.556 --> 0:11:32.916 It's a kind of Goldilocks principle. Yeah, we go through life, 0:11:32.956 --> 0:11:35.596 and you have been arguing that that's exactly wrong. In 0:11:35.716 --> 0:11:39.036 thinking about an epidemics say more for us, Well, epidemics 0:11:38.756 --> 0:11:42.036 are an interesting kind of dynamical process. There are sort 0:11:42.036 --> 0:11:46.756 of attractors in an epidemic. You can have a dynamic 0:11:46.836 --> 0:11:49.756 where you relatively quickly suppress an epidemic, and you go 0:11:49.836 --> 0:11:51.836 to a relatively small number of cases where you can 0:11:51.876 --> 0:11:53.516 have a dynamic where it sort of blows up and 0:11:53.636 --> 0:11:56.796 racist through the population. And whether you have this former 0:11:56.836 --> 0:11:59.316 case where you suppress it or the latter case where 0:11:59.316 --> 0:12:02.476 you fail depends basically on this number or not that 0:12:02.516 --> 0:12:05.636 everyone's talking about basic reproductive numbers. So it's how many 0:12:05.676 --> 0:12:10.676 cases does an initial infected case generate downstream, And of 0:12:10.676 --> 0:12:13.156 course that number depends not only in the properties the virus, 0:12:13.196 --> 0:12:15.156 but on the amounts of social distancing we're doing and 0:12:15.196 --> 0:12:17.716 so forth. And so as we've gone into this lockdown 0:12:17.756 --> 0:12:19.956 period in the social distancing, we've tried to knock our 0:12:20.076 --> 0:12:24.196 not down pretty low, and we've got it down around one, 0:12:24.316 --> 0:12:26.236 but we don't know exactly where it is, and so 0:12:26.276 --> 0:12:28.436 if it's a little bit below one, then each case 0:12:28.516 --> 0:12:31.116 generates fewer than one additional cases, and we get this 0:12:31.156 --> 0:12:33.716 exponential fall off in the number of cases, and we 0:12:33.836 --> 0:12:36.316 might end up with say about three percent of the 0:12:36.396 --> 0:12:39.876 US having been infected by the virus after the first way. 0:12:40.196 --> 0:12:42.436 If we're just a little bit above one, then each 0:12:42.476 --> 0:12:45.516 case generates more than one new cases, and it grows 0:12:45.516 --> 0:12:47.676 and grows, and it grows exponentially, and so it races 0:12:47.716 --> 0:12:49.956 through the whole population. And if that happens, you're going 0:12:49.956 --> 0:12:52.276 to end up with you somewhere in the range of 0:12:52.316 --> 0:12:55.236 thirty to seventy percent of the population infected in the 0:12:55.276 --> 0:13:00.036 absence of continued intervention by people changing their behavior so 0:13:00.076 --> 0:13:03.196 that this are zero are not is adjusted over time, 0:13:03.476 --> 0:13:05.476 you really end up on one of these two trajectories 0:13:05.556 --> 0:13:08.516 or not. Compared to rolling a ball down a ridge 0:13:08.596 --> 0:13:11.076 or a fence, I mean, Paul may stay on the 0:13:11.196 --> 0:13:13.076 ridge line for a while, but sooner or later it's 0:13:13.116 --> 0:13:14.916 going to drop off of one side or the other 0:13:15.516 --> 0:13:19.196 just because your trajectory is just slightly in one direction 0:13:19.276 --> 0:13:21.716 or the other. And so sort of the what happens 0:13:21.716 --> 0:13:25.236 in the pandemic is a bit like that. What's really 0:13:25.276 --> 0:13:27.476 interesting is something that people have suggested to me that 0:13:27.516 --> 0:13:30.156 we may have this element of what physicists call self 0:13:30.276 --> 0:13:34.076 organized criticality where actually, even though you've got this unstable 0:13:34.116 --> 0:13:36.316 point where or not is one where you're kind of 0:13:36.356 --> 0:13:39.076 rolling down the ridge line, the pandemic starts to take 0:13:39.156 --> 0:13:41.396 off and really accelerate, Then people get scared and they 0:13:41.436 --> 0:13:43.556 do more social distancing, and they push it back down. 0:13:43.956 --> 0:13:46.596 If the pandemic starts to get shut down, then people 0:13:46.596 --> 0:13:49.316 start to relax and they do less. And so you've 0:13:49.316 --> 0:13:51.436 got on one hand, you've got the fundamental dynamics of 0:13:51.476 --> 0:13:54.396 the pandemic. If everyone's behaviors helped constant we'd either have 0:13:54.436 --> 0:13:56.516 a great, big one or we'd shut it down pretty quickly. 0:13:56.756 --> 0:13:59.036 People are modulating their behavior, and they may actually be 0:13:59.076 --> 0:14:02.556 pushing us along this along the ridge, yeah, or on 0:14:02.596 --> 0:14:05.596 the ridge exactly. So that's an interesting sort of second 0:14:05.636 --> 0:14:07.996 layer on all of this. I'm glad you raised that issue, 0:14:08.036 --> 0:14:09.716 because that's exactly what I was and to ask about it. 0:14:09.716 --> 0:14:12.356 I mean, in a dynamic picture where we're always reacting 0:14:12.396 --> 0:14:15.436 to whatever we see happening out there, you could imagine 0:14:15.436 --> 0:14:18.556 a government at least trying very hard to get its 0:14:18.596 --> 0:14:21.916 people to type trate going to work and then coming 0:14:21.916 --> 0:14:23.636 home from work, going back and forth, and then getting 0:14:23.676 --> 0:14:25.396 some of a thing in the middle. I thought, in 0:14:25.396 --> 0:14:28.036 that context of the Swedish example, I mean, Sweden is 0:14:28.036 --> 0:14:30.476 sort of functioning as a kind of experiment where they're 0:14:30.516 --> 0:14:32.756 doing a fair amount of social distancing, but in otherwise 0:14:32.836 --> 0:14:35.076 way as they aren't, it'll be really interesting to see 0:14:35.156 --> 0:14:37.796 maybe they will turn out somewhere in the middle. Yeah, 0:14:37.996 --> 0:14:39.876 I mean it's certainly possible. I think even there, you're 0:14:40.236 --> 0:14:43.116 kind of unlikely too, because not only do you have 0:14:43.196 --> 0:14:46.316 this system that's you know, with exponential growth, but then 0:14:46.316 --> 0:14:49.636 you've got long delays, so there are big delays between 0:14:49.676 --> 0:14:51.436 when people are infected and when we start to really 0:14:51.476 --> 0:14:54.356 observe the consequences of those infections. You know, with full 0:14:54.356 --> 0:14:56.676 on testing, it would be five days to a week. 0:14:57.196 --> 0:14:59.516 If we're waiting for people to end up in the hospital, 0:14:59.556 --> 0:15:02.356 we're looking at more like ten to fifteen days. And 0:15:02.436 --> 0:15:05.476 so whenever you're trying to do control on a system 0:15:05.556 --> 0:15:09.516 that has exponential growth, implification had major delay place that 0:15:09.596 --> 0:15:12.996 control becomes extremely hard and you're overshooting. You you're fish 0:15:12.996 --> 0:15:16.156 tailing and overcorrecting, and you're going to end up off 0:15:16.156 --> 0:15:28.236 the road. We'll be back in just a moment. Can 0:15:28.276 --> 0:15:31.156 I ask you to put on your misinformation science has tad? 0:15:31.916 --> 0:15:34.916 What are the most egregious examples that you've seen thus 0:15:34.956 --> 0:15:39.916 far in this pandemic of misinformation with observable real world consequences? 0:15:40.276 --> 0:15:43.316 You know. Unfortunately, I think some of the most egregious 0:15:43.356 --> 0:15:47.276 misinformation in terms of the magnitude of the consequences has 0:15:47.276 --> 0:15:49.516 been coming out of the White House. And it was 0:15:49.516 --> 0:15:53.036 the protracted period where there was a serious effort to 0:15:53.076 --> 0:15:57.396 downplay the magnitude of what was going on, and that 0:15:57.996 --> 0:16:01.476 delayed the national response in a whole bunch of ways. 0:16:01.916 --> 0:16:04.796 And I think it also has the consequence of really 0:16:05.316 --> 0:16:08.916 hemorrhaging the trust that people need in order to comply 0:16:09.476 --> 0:16:12.436 over the long term with public health measures, which are 0:16:12.516 --> 0:16:14.716 more or less our only way of controlling the pandemic 0:16:14.796 --> 0:16:18.436 right now. So, if you have a story that is 0:16:18.516 --> 0:16:24.236 changing all the time, where you've got different agencies presenting 0:16:24.316 --> 0:16:27.716 different version of the story, You're hearing from some that 0:16:28.276 --> 0:16:31.076 this is going to go away on its own, that 0:16:31.116 --> 0:16:33.476 it's on the way down, it's going to disappear in April, 0:16:33.476 --> 0:16:35.836 and whatever the case is, and then the same government 0:16:35.876 --> 0:16:38.036 has other agencies telling you this is a serious threat, 0:16:38.076 --> 0:16:39.916 there's going to be protracted to a human to human 0:16:39.956 --> 0:16:42.796 spread in the United States, etc. People start to not 0:16:42.916 --> 0:16:45.196 be sure who they can trust. And when that happens, 0:16:45.716 --> 0:16:49.876 then it becomes harder and harder to manifest the political 0:16:49.876 --> 0:16:53.116 will that you need to take unpopular measures like closing schools, 0:16:53.116 --> 0:16:55.556 closing non essential businesses and like so. I think, but 0:16:55.916 --> 0:16:58.596 in terms of like the measurable consequences, one of the 0:16:58.676 --> 0:17:01.756 worst things that's happened in terms of misinformation has been 0:17:01.756 --> 0:17:05.956 this initial attempt to downplay the seriousness of a pandemic 0:17:05.996 --> 0:17:08.516 in order to I think, prop up the stock market 0:17:08.556 --> 0:17:11.396 on a first or you have a book coming out 0:17:11.436 --> 0:17:15.676 soon with the wonderful title calling Bullshit. Presumably when you 0:17:15.716 --> 0:17:18.756 were writing the book was before the pandemic broke out. 0:17:19.356 --> 0:17:22.236 What were the things that at the time you wanted 0:17:22.276 --> 0:17:25.436 to call bullshit on? And then maybe from there we'll 0:17:25.476 --> 0:17:27.996 go to how that's relevant in the present moment. So 0:17:28.116 --> 0:17:29.996 when we wrote the book, we were really thinking about 0:17:29.996 --> 0:17:34.316 the ways that quantitative information is used to mislead people. 0:17:34.556 --> 0:17:37.716 So what we thought was very, very important was to 0:17:37.956 --> 0:17:41.076 teach people that you are not at the mercy of 0:17:41.116 --> 0:17:45.316 the person who's bringing data and statistical analyses and machine 0:17:45.396 --> 0:17:47.556 learning or whatever it is to the table. You don't 0:17:47.556 --> 0:17:50.276 have to just accept those because you are not a 0:17:50.316 --> 0:17:53.436 PhD satistician, or you're not a computer scientist or whatever. 0:17:53.756 --> 0:17:55.956 The basic way that we think about this is that 0:17:56.076 --> 0:17:58.836 with any of these systems you've got, you know, when 0:17:58.836 --> 0:18:01.556 people are drawing conclusions based on data, they collect a 0:18:01.556 --> 0:18:03.556 bunch of data. That data then or put into some 0:18:03.636 --> 0:18:05.956 kind of machinery which you can think of as a 0:18:05.996 --> 0:18:08.956 black box out the other end, and comes some results, 0:18:09.276 --> 0:18:13.556 and then people draw conclusions from the results, and the 0:18:13.636 --> 0:18:16.196 bullshit is rarely in the black box where we find, 0:18:16.196 --> 0:18:18.116 so the bullshit is rarely in hard effect of the 0:18:18.116 --> 0:18:20.756 technical construction of the model. Of course, there are cases 0:18:20.756 --> 0:18:23.836 where that is what happens, but that's a strong minority 0:18:23.876 --> 0:18:26.116 of the bullshit that we see out there. Almost all 0:18:26.116 --> 0:18:28.196 of it is because people have data that are not 0:18:28.236 --> 0:18:32.036 necessarily representative or appropriate for the question that they're asking. 0:18:32.036 --> 0:18:34.556 You're comparing apples and oranges. They've got a biased sample. 0:18:34.596 --> 0:18:36.236 There are all these kinds of things that can go wrong, 0:18:36.676 --> 0:18:39.236 or people get results out and then they draw unjustified 0:18:39.276 --> 0:18:44.116 conclusions from the results. They overgeneralize, they infer causality where 0:18:44.116 --> 0:18:47.236 they've only got an observational study and only know about correlation, 0:18:47.556 --> 0:18:49.196 these kinds of things. And so what we wanted to 0:18:49.196 --> 0:18:51.516 really stress with the book was that you don't have 0:18:51.556 --> 0:18:53.596 to be sort of held hostage by people that have 0:18:53.676 --> 0:18:56.156 the numbers, because you can use the basic critical thinking 0:18:56.156 --> 0:18:58.956 skills that anybody has in order to see through this 0:18:58.996 --> 0:19:00.956 sort of stuff. You say, well, these data appropriate for 0:19:00.996 --> 0:19:05.196 answering that question, to these conclusions actually follow from those results. 0:19:05.836 --> 0:19:09.996 I love the idea that careful critical thinking and analyzing 0:19:09.996 --> 0:19:14.156 the premises can really help you identify bullshit when it 0:19:14.276 --> 0:19:16.716 is being dealt to you. I want to ask you 0:19:16.916 --> 0:19:20.076 though about a variant on that which I think may 0:19:20.076 --> 0:19:23.276 be pretty different that we've been seeing throughout the current pandemic, 0:19:23.356 --> 0:19:26.196 which is that it's not just that non experts in 0:19:26.316 --> 0:19:30.276 fields are calling bullshit on models, they also think that 0:19:30.316 --> 0:19:34.516 they can build their own models better. And at first 0:19:34.596 --> 0:19:36.716 I thought to myself, this problem is mostly out there 0:19:36.756 --> 0:19:39.676 on medium, or it's you know, the co worker in 0:19:39.716 --> 0:19:42.276 the office. And I actually wrote a column for Bloomberg 0:19:42.356 --> 0:19:45.396 where I write a column saying, well, the amateur epidemiologists 0:19:45.396 --> 0:19:48.156 please just sit down and shut up. So this phenomenon 0:19:48.156 --> 0:19:51.316 seems to be kind of everywhere right now, and I'm 0:19:51.356 --> 0:19:53.716 wondering if there's, like, is there a cognitive thing going 0:19:53.716 --> 0:19:55.876 on here? If people think they can find a problem, 0:19:56.556 --> 0:19:59.516 which is true by your view, they also think that 0:19:59.556 --> 0:20:01.996 they can then devise a better model, and at least 0:20:02.036 --> 0:20:04.196 as far as I can tell, that's not true. They 0:20:04.276 --> 0:20:07.836 usually cannot devise a better model. Well, I think, you know, 0:20:08.156 --> 0:20:11.276 on one hand, we only need input from a lot 0:20:11.316 --> 0:20:14.756 of sources, and so I think that it's really important 0:20:15.116 --> 0:20:18.916 to recognize that there are going to be good ideas 0:20:18.916 --> 0:20:22.316 coming from outside of the epidemiology community at the same time, 0:20:22.916 --> 0:20:25.956 one wants to be aware of the dunning Krueger effect. 0:20:25.956 --> 0:20:28.396 You know, the basic idea of the dunning Kruger effect 0:20:28.436 --> 0:20:30.476 is you don't know enough to know you're wrong, essentially, 0:20:30.476 --> 0:20:33.036 and so you think that you have a very good understanding. 0:20:33.036 --> 0:20:35.356 And so the Dunning Kruger effect typically has this sort 0:20:35.356 --> 0:20:39.876