WEBVTT - Episode 4: Respect the Polygon 0:00:15.276 --> 0:00:25.956 Pushkin, what's your biggest fear. I'd say the biggest fear 0:00:26.276 --> 0:00:30.036 is something a mistake that I would make that would 0:00:30.116 --> 0:00:33.236 damage my credibility to where people would not listen to 0:00:33.276 --> 0:00:39.116 me when there's a tornado down. James Span meteorologist, maybe 0:00:39.156 --> 0:00:43.236 Alabama's best known person aside from some football coaches. He's 0:00:43.276 --> 0:00:46.516 all over TV talking about the weather, especially when the 0:00:46.556 --> 0:00:49.476 weather might kill you. So this is a tornado emergency, 0:00:49.636 --> 0:00:52.116 but the city's at Tuscaloosa and North Fort and the 0:00:52.156 --> 0:00:56.516 campus of the University of Alabama. James is one of 0:00:56.556 --> 0:00:59.316 those people who's never really had a job because he 0:00:59.396 --> 0:01:02.236 found his calling. He once stayed on the air as 0:01:02.236 --> 0:01:04.636 he watched a tornado make straight for his own home, 0:01:05.356 --> 0:01:08.396 pleading with people to see the risk. If you're just 0:01:08.516 --> 0:01:11.996 joining us, This is James Span with Taylor Serrallo mainly 0:01:12.076 --> 0:01:16.076 chicken on my wife's she's okay and she's in the 0:01:16.116 --> 0:01:20.076 tornado shelter. Okay, go ahead, Taylor. I'm sorry. I was 0:01:20.116 --> 0:01:22.236 put on this planet to mitigate loss of life when 0:01:22.276 --> 0:01:25.956 their tornadoes flying around here, and I have to be 0:01:26.116 --> 0:01:29.316 very careful in what I say and what I do, 0:01:29.796 --> 0:01:34.196 not just on the air, but on social media. And 0:01:34.436 --> 0:01:39.676 in real life. To build trust with his audience, James 0:01:39.716 --> 0:01:43.556 goes to incredible links. He's published a children's book called 0:01:43.996 --> 0:01:47.836 Benny and Chipper Prepared Not Scared. He spends time in 0:01:47.876 --> 0:01:51.076 dollar stores talking to people because the people who shop 0:01:51.076 --> 0:01:53.156 in dollar stores are also the people who live in 0:01:53.156 --> 0:01:57.156 trailer homes, the sort of homes that tornadoes obliterate. He 0:01:57.196 --> 0:02:01.316 memorizes the names of Alabamians who've died in storms, people 0:02:01.356 --> 0:02:04.636 he might have saved. There's lots of them. On a 0:02:04.676 --> 0:02:07.636 single day back in April twenty eleven, a line of 0:02:07.676 --> 0:02:10.796 tornadoes in Alabama killed two hundred and fifty three people. 0:02:11.676 --> 0:02:15.316 I know their stories, I know their family members. I've 0:02:15.316 --> 0:02:18.036 talked to many of them, and it's very motivating for me. 0:02:18.116 --> 0:02:20.796 And that's my main job in life. It's to make 0:02:20.796 --> 0:02:25.276 the warning process better with severe weather. He's doing all 0:02:25.356 --> 0:02:29.956 he can to warn people, yet people still don't understand 0:02:29.996 --> 0:02:36.676 what he's saying. I'm Michael Lewis. Welcome back to Against 0:02:36.756 --> 0:02:41.476 the Rules, where we explore unfairness in American life by 0:02:41.516 --> 0:02:44.356 looking at what's happened to various characters in American life. 0:02:45.276 --> 0:02:49.956 This season is all about experts. Today, we're going to 0:02:49.996 --> 0:02:53.596 explore the strange thing that's happened to experts. Not all experts, 0:02:53.596 --> 0:02:56.796 a certain percentage of them, the experts who think and 0:02:56.956 --> 0:03:01.516 speak in probabilities, who use data to forecast the likelihood 0:03:01.516 --> 0:03:05.076 of this or that coming to pass. The experts who 0:03:05.116 --> 0:03:07.956 can never be perfectly certain, and who risk our wrath 0:03:08.116 --> 0:03:23.476 because we love thinking in absolutes. James Span has been 0:03:23.476 --> 0:03:26.036 making and explaining weather forecast for the better part of 0:03:26.076 --> 0:03:29.396 half a century. In that time, it's kind of incredible 0:03:29.436 --> 0:03:32.876 how much has changed. So here's in nineteen seventy eight 0:03:32.916 --> 0:03:36.116 forecast partly sunny tomorrow with a chance of showers in 0:03:36.156 --> 0:03:39.836 the high of eighty. That's it. So today, under the 0:03:39.876 --> 0:03:43.396 same circumstances, I'd say we'll have a pretty good bit 0:03:43.396 --> 0:03:46.236 of sunshine between nine and eleven o'clock. After eleven o'clock 0:03:46.996 --> 0:03:49.276 rain is likely between eleven and one. The chance of 0:03:49.316 --> 0:03:51.676 any one spot getting wet during that two hour window 0:03:51.676 --> 0:03:53.996 it's about seventy five percent. It's going to rain about 0:03:53.996 --> 0:03:56.236 a half inch in most places. There could be some thunder. 0:03:56.556 --> 0:03:58.356 Most of that should be out of here by two thirty. 0:03:58.356 --> 0:04:00.356 After three o'clock, you're good to go. The sun breaks 0:04:00.356 --> 0:04:03.036 back out a temperature should peek around eighty at one o'clock, 0:04:03.036 --> 0:04:05.836 then falling back into the seventies by four o'clock. That's 0:04:05.836 --> 0:04:07.676 the difference in what we can do now compared to 0:04:07.756 --> 0:04:10.676 nineteen seventy eight. It's the prince between daylight and darkness. 0:04:10.956 --> 0:04:13.116 If you go back to the beginning of your career, 0:04:13.436 --> 0:04:16.236 were you encouraged to speak to the audience that way, like, 0:04:16.516 --> 0:04:18.876 we don't know that much about this, this could be wrong. 0:04:19.476 --> 0:04:21.436 Oh no, no, no, they didn't want you to say that. 0:04:21.516 --> 0:04:24.236 I mean, coodn't you know. Back in the seventies, this 0:04:24.316 --> 0:04:26.316 was when TV news was coming of age and I 0:04:26.516 --> 0:04:29.836 witness news, you know, and they wanted to be this 0:04:30.076 --> 0:04:33.876 godlike figure, you know on television. I was scared to 0:04:33.876 --> 0:04:37.916 communicate uncertainty because that wasn't encouraged. We were the news, 0:04:38.036 --> 0:04:42.596 the evening news, the Ron Burgundy newscast. Weather forecasts are 0:04:42.596 --> 0:04:47.036 inherently uncertain, the where, the when, the how much. With 0:04:47.116 --> 0:04:49.156 the current data we have, the best you can do 0:04:49.236 --> 0:04:52.596 is judge the odds. But the odds have gotten much 0:04:52.676 --> 0:04:56.076 more accurate over time. Back when James span was a 0:04:56.076 --> 0:04:59.476 young meteorologist, he knew very little but tried to sound 0:04:59.476 --> 0:05:02.316 like he knew a lot. Now that he knows a lot, 0:05:02.676 --> 0:05:06.316 he works hard to explain what he doesn't know. You're 0:05:06.396 --> 0:05:09.796 giving the audience more information and more new, honest information. 0:05:09.876 --> 0:05:13.916 So it's more demanding on the audience, right it is. 0:05:14.236 --> 0:05:17.556 And you know I hear this all the time. I 0:05:17.596 --> 0:05:19.076 just want to know if it's going to rain tomorrow, 0:05:19.716 --> 0:05:23.156 and they want a yes orno. They want that deterministic forecast, 0:05:24.196 --> 0:05:28.916 deterministic as imperfectly predictable, which is something the weather still isn't. 0:05:30.076 --> 0:05:32.716 When James Span started out, the ten day forecast was 0:05:32.796 --> 0:05:35.556 no better than just guessing. Now it's a lot better. 0:05:36.396 --> 0:05:39.516 But maybe the most obvious improvement, the one people really 0:05:39.556 --> 0:05:42.836 should notice, has been in forecasters understanding of the kind 0:05:42.836 --> 0:05:46.076 of weather that kills people. In nineteen seventy eight, we 0:05:46.156 --> 0:05:50.796 were using nineteen fifty seven era radar and the old 0:05:51.156 --> 0:05:53.516 black and white printouts of radar. It looked like somebody 0:05:53.516 --> 0:05:55.716 barsed on a piece of paper, and so warnings. In 0:05:55.756 --> 0:05:58.356 nineteen seventy eight, let's say we had a tornado down. 0:05:58.916 --> 0:06:01.116 We didn't really know where it was, we had an idea, 0:06:01.196 --> 0:06:04.796 so warnings were issued by an entire county. Tornadoes, even 0:06:04.836 --> 0:06:08.196 the big tornadoes are small and counties are huge. So 0:06:08.316 --> 0:06:11.436 here you are warning an entire county to get into 0:06:11.476 --> 0:06:14.716 your safe place and do something where most people didn't 0:06:14.716 --> 0:06:18.276 need to do anything. We're today we know literally within 0:06:18.356 --> 0:06:21.236 maybe a few city blocks of where the tornado is located. Well, 0:06:21.276 --> 0:06:25.836 so if I'm a consumer of tornado warnings, I get 0:06:25.876 --> 0:06:28.716 a much more precise warning, and I do I get 0:06:29.036 --> 0:06:31.276 a more advanced warning. Am I likely to get it? 0:06:31.356 --> 0:06:34.556 Get more more time to prepare for this thing? Yes? 0:06:34.596 --> 0:06:37.316 They have. Average lead time here is about twelve to 0:06:37.316 --> 0:06:39.796 fifteen minutes, and the average lead time back in the 0:06:39.836 --> 0:06:43.676 seventies was zero to three minutes. So we've come a 0:06:43.716 --> 0:06:46.436 long way, and we don't use counties anymore. We use small, small, 0:06:46.476 --> 0:06:50.356 small segments of counties. Geometric shapes, polygons. Anybody that knows 0:06:50.436 --> 0:06:53.516 James span I've said this over and over. Respect the polygon, 0:06:53.556 --> 0:06:56.756 and if you're in it, you do something. Respect the polygon. 0:06:57.156 --> 0:06:59.436 And if you're in the polygon, you respect the polygon. 0:06:59.596 --> 0:07:05.236 Respect polygon. Every storm today will mean business. Respect A 0:07:05.396 --> 0:07:08.396 James Spans superfan did a remix of his famous frame 0:07:09.796 --> 0:07:17.116 a polygon. I love this, of course, but it also 0:07:17.196 --> 0:07:21.756 raises a question, why respect the Polygon instead of just 0:07:22.116 --> 0:07:27.556 respect what I say. It's weird. If the James Span 0:07:27.716 --> 0:07:30.836 back in nineteen seventy eight had been as accurate as 0:07:30.956 --> 0:07:35.676 James Span is now, he'd have endured hail storms of gratitude, 0:07:36.476 --> 0:07:41.836 hurricanes of appreciation, tornadoes of awe. But that's not the 0:07:41.836 --> 0:07:46.836 wather he now lives with. Hello friends, this is James Span. 0:07:46.956 --> 0:07:50.476 It's time to read some mean tweets. And thanks to 0:07:50.516 --> 0:07:52.396 all of you for sending in the mean tweets. I 0:07:52.436 --> 0:07:55.716 really appreciate from from my heart. You cost the people 0:07:55.756 --> 0:07:59.876 in the state millions of dollars by your boop poor 0:08:00.236 --> 0:08:05.996 boot forecasts. I woke up today expecting snow. I blame you, James. 0:08:06.076 --> 0:08:10.596 I got my dogs all excited for nothing. James, either 0:08:10.636 --> 0:08:15.356 you're the worst meteorologist I've ever layer my eyes on, 0:08:16.516 --> 0:08:18.676 or you have the worst luck at predicting the weather. 0:08:18.756 --> 0:08:22.756 I think it's time to step down. Brother. The only 0:08:22.796 --> 0:08:26.636 difference between James Span and every other meteorologist is that 0:08:26.756 --> 0:08:30.036 James reads his mean tweets on the air. Just to 0:08:30.076 --> 0:08:32.876 show you where we stand, my producer called up some 0:08:32.916 --> 0:08:36.116 weather tweeters. Here's the kinds of things that people have 0:08:36.196 --> 0:08:41.316 to say weather forecasting is the only job you can 0:08:41.396 --> 0:08:43.716 have where you can be wrong fifty percent of the 0:08:43.756 --> 0:08:47.276 time and still make thousands of dollars. If we were 0:08:47.276 --> 0:08:51.076 wrong fifty percent in our jobs, we probably would be fired. 0:08:51.556 --> 0:08:55.076 I know nothing about meteorologists, but I know that you 0:08:55.116 --> 0:08:57.636 know they always wrong. I'm one of those people that 0:08:57.716 --> 0:09:01.396 actually vainly looks at the weather forecast because nine times 0:09:01.436 --> 0:09:06.476 attend it's different from the forecast. As technology improves, they 0:09:06.476 --> 0:09:13.116 don't improve. The continue to be it's smine boggling. You're 0:09:13.156 --> 0:09:15.756 going to get the hate, not necessarily because of your 0:09:15.836 --> 0:09:18.476 missed weather forecast, but just because of who you are. 0:09:19.156 --> 0:09:22.156 You're a you're a weather person, and you know you're 0:09:22.196 --> 0:09:25.796 a stooge. You're a you don't deserve to be on 0:09:25.836 --> 0:09:29.316 the planet. You shouldn't be breathing air. People have that 0:09:29.356 --> 0:09:32.676 attitude towards weather people. Oh listen. So I cut off 0:09:32.676 --> 0:09:36.156 a basketball game on Christmas Day in twenty fifteen, and 0:09:36.236 --> 0:09:38.636 we had a tornado coming up on the southwest part 0:09:38.636 --> 0:09:40.196 of the city here. It could have killed a lot 0:09:40.196 --> 0:09:42.956 of people, So we had to cover up about twenty 0:09:42.996 --> 0:09:45.996 twenty five minutes of that game, and nobody lost their life. 0:09:46.036 --> 0:09:49.236 The warning system worked beautifully but this is Christmas Day. 0:09:49.356 --> 0:09:51.996 Joy to the world, peace on earth, goodwill toward men. 0:09:52.436 --> 0:09:54.476 The first email I got, you know what it said. 0:09:54.476 --> 0:09:57.876 It said you should have been aborted by a coat hanger. 0:09:59.156 --> 0:10:01.836 So this is the stuff I deal with. I mean, 0:10:01.876 --> 0:10:05.396 I'm amazed he still goes on the air. His forecast 0:10:05.676 --> 0:10:08.076 just keep getting better and better, but the job of 0:10:08.116 --> 0:10:11.156 being a meteorology just keeps getting worse and worse. But 0:10:11.196 --> 0:10:13.236 I till these young people, you know, you better have 0:10:13.276 --> 0:10:14.996 a thick skin when you get out of here and 0:10:15.036 --> 0:10:16.996 you get your first job, because they're going to come 0:10:17.036 --> 0:10:22.476 after you when you found that first forecast up. Back 0:10:22.516 --> 0:10:24.716 in an earlier season of this show, I talked about 0:10:24.716 --> 0:10:28.756 the problem of referees and a strange phenomenon. A lot 0:10:28.756 --> 0:10:31.316 of refs are getting better at their jobs. They have 0:10:31.396 --> 0:10:34.436 new tools, they're better train they get better feedbacks, they 0:10:34.436 --> 0:10:37.556 are less likely to repeat mistakes. I mean, there's just 0:10:37.756 --> 0:10:40.076 no way that the refs in pro sports are less 0:10:40.116 --> 0:10:42.596 accurate than they were forty years ago when there was 0:10:42.596 --> 0:10:45.436 no replay, less training and all the refs got hired 0:10:45.476 --> 0:10:48.236 from the same old boys club, But they didn't used 0:10:48.236 --> 0:10:51.956 to need police escorts from the arenas. Now they do. 0:10:54.676 --> 0:10:58.796 In December of twenty twenty one, tornadoes ripped through Kentucky. 0:10:59.596 --> 0:11:02.996 Weather experts gave people lots of warning, So this is 0:11:03.036 --> 0:11:07.076 just an explosive severe weather set up, and that's the 0:11:07.116 --> 0:11:10.156 outlook that we have heading our way, especially after midnight 0:11:10.196 --> 0:11:12.876 through about eight o'clock in the morning. We definitely need 0:11:12.876 --> 0:11:16.276 to stay aware of the weather game. Meteorologists like this 0:11:16.316 --> 0:11:20.476 guy on WHASTV and Louisville were better than they'd ever been. 0:11:20.956 --> 0:11:23.196 Make sure you have a way to wake up if 0:11:23.236 --> 0:11:25.796 a warning is issued like this one that we have. 0:11:26.236 --> 0:11:29.596 That night in Kentucky, at least seventy seven people were killed, 0:11:29.996 --> 0:11:32.356 more people than have ever died from a weather event 0:11:32.396 --> 0:11:35.236 in the state's history. All people had to do to 0:11:35.276 --> 0:11:38.956 survive was listened to the experts, and still a lot 0:11:38.996 --> 0:11:50.436 of them didn't. I think that having data is a 0:11:50.516 --> 0:11:56.476 really recent phenomenon, Rebecca Golden, math professor at George Mason University. 0:11:56.636 --> 0:11:59.876 We didn't have data about how things were, we didn't 0:11:59.916 --> 0:12:03.636 record what happened previously. Then it's only really recently that 0:12:03.716 --> 0:12:08.076 we think maybe our lived experiences could be in part 0:12:08.116 --> 0:12:11.636 based on something probabilistic, Like a lot of people who 0:12:11.676 --> 0:12:15.556 are good at math Rebecca noticed the confusion and wrongheadedness 0:12:15.596 --> 0:12:18.716 of people who weren't. She also noticed that even when 0:12:18.716 --> 0:12:23.116 statistics and these new big piles of data were properly explained, 0:12:23.876 --> 0:12:26.716 people didn't really grasp their meaning. People have a hard 0:12:26.716 --> 0:12:29.676 time being convinced by data. It's just that they don't 0:12:29.756 --> 0:12:33.436 think that their experiences is in line with that data, 0:12:33.636 --> 0:12:37.476 and so they dismiss it, or they have other experiences 0:12:37.516 --> 0:12:39.996 that tell them that there are reasons to be skeptical 0:12:40.036 --> 0:12:42.476 of the source of that data or the source of 0:12:42.516 --> 0:12:45.276 the statements that are relying on the data. The problem 0:12:45.396 --> 0:12:47.596 isn't just in the quality of the information we have 0:12:47.676 --> 0:12:50.356 access to. It's in the way we make sense of 0:12:50.356 --> 0:12:54.676 the world. There's a very large segment of the population 0:12:54.996 --> 0:12:59.316 who are really struggle with basic mathematics. So people are 0:12:59.356 --> 0:13:02.836 making mistakes because they don't think in probabilities. I think 0:13:02.836 --> 0:13:07.156 that's right. Rebecca actually helped to start an organization called 0:13:07.276 --> 0:13:12.236 stats to expose the statistical mistakes made by journalists. She 0:13:12.356 --> 0:13:15.356 thought that if statistics were conveyed more accurately to the public, 0:13:15.756 --> 0:13:19.156 the public would see the world more clearly. Eventually, she 0:13:19.236 --> 0:13:22.396 decided she was wasting her time because there was this 0:13:22.596 --> 0:13:27.756 bigger problem how people comprehend statistics, even when they're accurate. 0:13:29.036 --> 0:13:32.196 Why is it that people don't think in probabilities, like 0:13:32.276 --> 0:13:38.236 the world's probabilistic. Why are our minds so deterministic? It's 0:13:38.276 --> 0:13:41.756 kind of a philosophical question. I think we're hardwired to believe. 0:13:41.796 --> 0:13:45.716 I think it helps us make decisions without being stressed 0:13:45.716 --> 0:13:49.956 about those decisions. It helps us act with certainty and 0:13:50.196 --> 0:13:55.556 make decisions so we don't hesitate too much and think 0:13:55.596 --> 0:13:59.236 too hard. In the savannah, we don't say that's probably 0:13:59.276 --> 0:14:05.916 a lion, right, we just run. But to just run 0:14:06.516 --> 0:14:08.796 is less and less a viable way to move through 0:14:08.836 --> 0:14:13.156 the world world because this relatively new thing called data 0:14:13.276 --> 0:14:17.196 has given us a far shrewder alternative. Everywhere you turn, 0:14:17.276 --> 0:14:20.516 you find someone analyzing data to generate the same sort 0:14:20.516 --> 0:14:23.676 of probabilistic understanding of the world that weather people do. 0:14:25.556 --> 0:14:29.356 I kind of come from this world of like, you know, 0:14:29.476 --> 0:14:33.636 kind of quantz and like baseball geeks and like poker players. 0:14:34.396 --> 0:14:37.196 That's Nate Silver. He got swept up in the nineteen 0:14:37.276 --> 0:14:41.076 nineties by the statistical revolution in baseball. And I think 0:14:41.076 --> 0:14:43.716 I'm kind of like one of the relatively people who's 0:14:43.796 --> 0:14:46.116 kind of escaped, so to speak, from that world in 0:14:46.196 --> 0:14:49.596 the like mainstream society. Back in two thousand and seven, 0:14:50.196 --> 0:14:54.076 Nate quit forecasting the future of young baseball players. He 0:14:54.196 --> 0:14:57.596 began to forecast elections instead. It's sort of what you're 0:14:57.596 --> 0:14:59.916 doing is actually accepting the possibility that maybe you can 0:14:59.996 --> 0:15:03.516 predict something that's right. But yeah, it's like kind of 0:15:03.516 --> 0:15:06.476 like saying, hey, look, we built an audience for this 0:15:06.596 --> 0:15:11.836 in in baseball, and so politics is still in the 0:15:11.876 --> 0:15:14.716 Stone Age, and so there must be kind of an 0:15:14.716 --> 0:15:17.876 audience for some politics too. When you turn your attention 0:15:17.916 --> 0:15:22.876 to politics, at what point are you aware that the 0:15:22.956 --> 0:15:28.436 expertise in political forecasting is sort of limited, that there's 0:15:28.476 --> 0:15:32.596 kind of an opportunity. I mean I had an intuition 0:15:33.476 --> 0:15:36.196 from that from the very beginning in politics, I mean, 0:15:36.196 --> 0:15:39.196 the campaigns have to be fairly smart and data driven 0:15:39.236 --> 0:15:41.596 about they were targeting. But like, but the media was 0:15:41.636 --> 0:15:46.596 all about kind of narratives. It was really quite bad 0:15:46.636 --> 0:15:48.636 in two thousand and eight. Right. It's really like a 0:15:48.676 --> 0:15:51.436 bunch of like, you know, old white men getting together 0:15:51.476 --> 0:15:54.836 and kind of deciding based on, you know, what their 0:15:54.876 --> 0:15:59.836 friends think, kind of what the narratives should be in 0:15:59.876 --> 0:16:03.156 the presidential primaries of two thousand and eight, Nate Silver 0:16:03.276 --> 0:16:07.156 gave an upstart senator named Barack Obama a much better 0:16:07.236 --> 0:16:10.316 chance than most everyone else did. In the general election. 0:16:10.356 --> 0:16:13.196 He nailed not just the outcome, but the result in 0:16:13.276 --> 0:16:17.356 every state, plus the precise number of votes Obama received 0:16:17.396 --> 0:16:21.116 in the Electoral College. People paid more attention to what 0:16:21.236 --> 0:16:24.276 Nate had done than how he had done it. He'd 0:16:24.276 --> 0:16:26.956 simply use polling data rather than his gut or some 0:16:27.036 --> 0:16:31.396 anecdote about some Iowa farmer. The polling data might not 0:16:31.476 --> 0:16:34.276 be perfect, but it was better than every other source 0:16:34.276 --> 0:16:38.836 of information, and they never made outright predictions. He issued 0:16:38.876 --> 0:16:44.236 political forecasts like weather forecasts, with probabilities attached to them. 0:16:44.276 --> 0:16:46.756 Going into election day of two thousand and eight, he'd 0:16:46.796 --> 0:16:49.876 given Obama a ninety point nine percent chance of winning. 0:16:50.756 --> 0:16:53.236 I mean, the irather thing about it is like like 0:16:53.276 --> 0:16:55.716 there was always a chance that we would be wrong, 0:16:56.036 --> 0:16:59.036 you know what I mean, and probably never heard from 0:16:59.036 --> 0:17:05.596 politics again, potentially. Instead, Nate became basically overnight the country's 0:17:05.676 --> 0:17:10.476 leading political forecaster because his expert piece was superior to 0:17:10.516 --> 0:17:16.836 the storytelling it replaced. Nate Silver is his name, fortune 0:17:16.836 --> 0:17:23.276 telling is his game. He's a celebrity statistician. Please welcome 0:17:23.356 --> 0:17:28.276 Nate Silver. That's right, Nate Silver's the good will hunting 0:17:28.316 --> 0:17:32.836 of political prognosticasia. There's a difference between weather forecasting or 0:17:32.916 --> 0:17:37.276 sports statistics and politics, a difference more of degree than kind, 0:17:37.636 --> 0:17:40.956 but still a difference. The people who celebrated Nate Silver 0:17:41.516 --> 0:17:45.516 really really didn't understand how to judge him. His better 0:17:45.556 --> 0:17:47.836 insights into pulling data had allowed him to see that 0:17:47.876 --> 0:17:51.316 Obama was basically always doing better than political pundits thought 0:17:51.316 --> 0:17:54.236 he was doing, but there was still no law that 0:17:54.276 --> 0:17:57.596 said Obama had to win. Polling data is a bit 0:17:57.636 --> 0:18:00.876 like the data that card counters get in blackjack. It's 0:18:00.916 --> 0:18:03.836 a lot better than having no data at all. It 0:18:03.876 --> 0:18:07.236 helps you to predict what comes next, but even card 0:18:07.276 --> 0:18:10.956 counters lose lots of hands. And here we go, ladies 0:18:10.996 --> 0:18:14.036 and gentlemen, welcome to Decision Night in America. Here at 0:18:14.156 --> 0:18:17.516 NBC's Democracy Point, which brings us to two thou sixteen. 0:18:20.676 --> 0:18:24.636 Nate Silver now had an enormous following. Once again, the 0:18:24.676 --> 0:18:28.436 pundits gave Hillary Clinton better odds than the polls. On 0:18:28.556 --> 0:18:32.116 election day, Nate gave Donald Trump a roughly thirty percent 0:18:32.196 --> 0:18:35.636 chance of winning at the time that was a radical call. 0:18:35.996 --> 0:18:38.436 A few traditional pundits thought Trump had that much of 0:18:38.436 --> 0:18:41.356 a shot. Yeah, I guess question, guys, are we post 0:18:41.476 --> 0:18:44.836 Nate Silver, are we pulled out? Well? They've been wrong, 0:18:45.116 --> 0:18:48.036 not only just wrong, they're just they're superfluous. And at 0:18:48.076 --> 0:18:49.596 the point where they just that's when you kind of 0:18:49.596 --> 0:18:54.116 begin to realize that, like, the way you define success 0:18:54.156 --> 0:18:56.076 and the way other people look at your forecast as 0:18:56.116 --> 0:18:59.596 being successful are very different. And also because it wasn't 0:18:59.636 --> 0:19:02.636 just that, like we got criticized after twenty sixteen for 0:19:02.756 --> 0:19:05.876 having quote unquote been wrong, it was also in the 0:19:05.956 --> 0:19:07.916 roup twentyteen people were actually mad at us for not 0:19:08.396 --> 0:19:13.516 being confident enough in Clinton's chances. Right, Nate never claimed 0:19:13.516 --> 0:19:16.436 to have some mystical ability to call a presidential election, 0:19:17.276 --> 0:19:20.716 and assigning probabilities is not the same as taking sides. 0:19:21.636 --> 0:19:23.836 Yelling at him for saying that Donald Trump had a 0:19:23.876 --> 0:19:26.636 thirty percent chance of winning was like being mad at 0:19:26.636 --> 0:19:28.796 the weather man for saying there was a thirty percent 0:19:28.876 --> 0:19:32.476 chance of rain and then getting mad all over again 0:19:32.796 --> 0:19:35.396 after it rains. I'm gonna get myself in a little 0:19:35.396 --> 0:19:38.356 bit of trouble for saying this, right, But like people 0:19:38.596 --> 0:19:45.716 like me really care about being right quote unquote for 0:19:45.756 --> 0:19:48.596 the intrinsic value of like making a good forecast as 0:19:48.596 --> 0:19:56.476 opposed to like influencing the narrative, if you will, Okay, 0:19:56.556 --> 0:19:59.996 so I would love because it would educate me. How 0:20:00.036 --> 0:20:05.356 do you evaluate a probabilistic a forecast? What's the right 0:20:05.396 --> 0:20:10.156 way for people to judge Nate's Silver Expert. The right 0:20:10.196 --> 0:20:13.636 way is if you take a whole bunch of forecasts 0:20:13.636 --> 0:20:17.316 that we've made and look at how they've done collectively. Right, So, 0:20:17.556 --> 0:20:20.476 let's say you made one hundred forecasts where the favorite 0:20:20.516 --> 0:20:22.756 had a seventy percent chance of winning. Look at that 0:20:22.796 --> 0:20:26.636 group of forecasts, and was it true that the favorite 0:20:26.716 --> 0:20:29.836 actually won about seventy percent at a time? Right? The 0:20:29.836 --> 0:20:33.316 slip side of that is that like it does mean that, like, 0:20:33.996 --> 0:20:37.636 you can tell very little from anyone prediction. I mean, 0:20:37.676 --> 0:20:41.356 unless you're like, unless you're very very close to one 0:20:41.396 --> 0:20:46.156 hundred zero percent, right, then one prediction alone won't tell 0:20:46.196 --> 0:20:51.076 you that much. Experts have gotten better, but they've also 0:20:51.156 --> 0:20:54.156 gotten harder to judge, so hard that you need an 0:20:54.196 --> 0:20:57.356 expert to judge them. And that's a problem, right, I Mean, 0:20:57.636 --> 0:21:00.116 who's going to go to the trouble of evaluating hundreds 0:21:00.116 --> 0:21:03.076 of Nate Silver's forecasts. And while it's true that he's 0:21:03.076 --> 0:21:06.436 made thousands of election forecasts, he hasn't made thousands of 0:21:06.476 --> 0:21:10.396 forecasts for presidential elections. Most people don't even think about 0:21:10.396 --> 0:21:13.556 elections or forecasts or anything else the way Nate Silver does. 0:21:14.276 --> 0:21:17.716 Most people don't even speak his language. I actually think 0:21:17.756 --> 0:21:21.596 that the word uncertainty is used in English in a 0:21:21.716 --> 0:21:27.276 very different way than uncertainty is used in statistics. Rebecca 0:21:27.316 --> 0:21:30.996 Golden again, So when we talk about uncertainty and statistics, 0:21:31.636 --> 0:21:34.436 we might say something about a confidence interval, or we 0:21:34.516 --> 0:21:37.756 might use a pee value. I'm not really sure you 0:21:37.796 --> 0:21:40.076 want this on your podcast, Like, maybe that's a little 0:21:40.116 --> 0:21:47.236 bit too technical. It might be better to trying to 0:21:47.276 --> 0:21:49.596 think of how it might be better to talk about 0:21:49.676 --> 0:21:53.356 uncertainty for your Well, this is the root of the matter. So, 0:21:53.516 --> 0:21:56.436 because it's not just my podcast listeners who are cut 0:21:56.476 --> 0:22:00.116 above the average human beings, it's like, how the American 0:22:00.236 --> 0:22:05.116 public understands uncertainty? How do you convey it? I think 0:22:05.116 --> 0:22:07.476 the best way to talk about it is to actually 0:22:08.356 --> 0:22:13.236 put it with specific numbers, Like instead of talking percentages, 0:22:13.636 --> 0:22:17.996 let's talk about numbers instead of ten percent, say one 0:22:18.076 --> 0:22:21.236 in ten, that kind of thing. But there's a much 0:22:21.356 --> 0:22:26.796 bigger problem behind all this, an emotional problem. It comes 0:22:26.796 --> 0:22:30.676 from us wanting certainty in situations where certainty just doesn't exist. 0:22:31.636 --> 0:22:34.196 If the weatherman says is an eighty percent chance of 0:22:34.276 --> 0:22:37.996 rain and it doesn't rain, the people on the receiving 0:22:38.076 --> 0:22:40.116 end of the forecast don't say, oh, that was one 0:22:40.116 --> 0:22:41.956 of the twenty percent of that was one of the 0:22:41.996 --> 0:22:43.436 times when it wasn't going to right. They say the 0:22:43.476 --> 0:22:46.756 expert doesn't know what he's talking about. So the inability 0:22:46.796 --> 0:22:51.476 to think in terms of probabilities also becomes an inability 0:22:51.516 --> 0:22:57.596 to evaluate the experts. There's a huge amount of inability 0:22:57.636 --> 0:23:01.996 to evaluate who is an expert, and yeah, it costs lives, 0:23:02.076 --> 0:23:06.236 it really does. A new kind of expert appears on 0:23:06.276 --> 0:23:09.356 the scene, an expert who works with these new big 0:23:09.356 --> 0:23:14.356 piles of data, an expert who thinks improbabilities, an expert 0:23:14.396 --> 0:23:19.036 who admits to being uncertain. These new experts are clearly 0:23:19.076 --> 0:23:22.876 better than the experts they replaced, and yet people treat 0:23:22.916 --> 0:23:26.716 them as if they're worse and neglect their advice, even 0:23:26.796 --> 0:23:30.276 when their lives depend on it. We who depend on 0:23:30.316 --> 0:23:32.996 the experts still want them to have a definitive answer. 0:23:33.716 --> 0:23:37.516 Either it will reign or it won't. Trump either will 0:23:37.596 --> 0:23:40.876 win or he won't. But that's not the nature of 0:23:40.916 --> 0:23:44.116 the world we live in, and we're having some trouble 0:23:44.196 --> 0:23:53.956 accepting that fact. The more you look for it, the 0:23:53.956 --> 0:23:56.876 more you see this problem. We've been talking about the 0:23:56.956 --> 0:23:59.596 problem of the experts getting better yet being treated as 0:23:59.636 --> 0:24:02.476 if they've gotten worse, a problem that leads to a 0:24:02.476 --> 0:24:05.636 lot of mystifying behavior, like what's gone on in the 0:24:05.676 --> 0:24:11.356 past two years inside the American healthcare system. I definitely 0:24:11.476 --> 0:24:14.876 saw a lot of this coming. Alison Fearing is a 0:24:14.956 --> 0:24:19.036 nurse at Rush Copley Medical Center in Aurora, Illinois. I 0:24:19.076 --> 0:24:21.796 can't tell you how many times I've had somebody coming 0:24:21.836 --> 0:24:25.396 and say I have this because I read X, Y 0:24:25.476 --> 0:24:28.236 and Z on WebMD, so I know that that's what's 0:24:28.236 --> 0:24:30.036 going on, and it's like, well, there's a lot more 0:24:30.076 --> 0:24:32.876 that goes into it that we need to work up further, 0:24:33.076 --> 0:24:36.156 because there are also other things that this could be 0:24:36.396 --> 0:24:39.036 and we won't know this until we diagnose it with 0:24:39.836 --> 0:24:42.836 lab work or a cat scan or whatever. Has this 0:24:42.916 --> 0:24:46.916 been going on your whole career? I would say it's 0:24:46.916 --> 0:24:51.276 definitely gotten worse over the past five years or so. 0:24:51.436 --> 0:24:54.676 I think prior to that there was a bit of it, 0:24:54.756 --> 0:24:58.036 but definitely not to the extent that there is now. 0:24:59.276 --> 0:25:02.116 Modern medicine is one of the great miracles of our age. 0:25:02.276 --> 0:25:04.356