WEBVTT - How Election Polling Works and Doesn't Work 0:00:01.120 --> 0:00:03.520 Welcome to Stuff You Should Know, a production of five 0:00:03.600 --> 0:00:12.760 Heart Radios How Stuff Works. Hey, and welcome to the podcast. 0:00:12.800 --> 0:00:16.239 I'm Josh Clark, and there's Charles W. Chuck Bryant over there, 0:00:16.239 --> 0:00:21.159 and we've got the scoop. Jerry's around here somewhere, and 0:00:21.239 --> 0:00:24.080 this is Stuff you Should Know. Off to a great start. 0:00:24.320 --> 0:00:27.800 She's in hers, she is she's got this remote thing 0:00:27.880 --> 0:00:34.680 going on. Uh, it's like the COVID special, that's right. Uh. 0:00:34.720 --> 0:00:37.680 And this was this has been what I've been wanting 0:00:37.720 --> 0:00:43.240 to do since two thousand sixteen, and why it seems 0:00:43.280 --> 0:00:46.839 like the fire kind of went down on it and 0:00:46.880 --> 0:00:49.560 now it's the fires back up again. In election season, 0:00:50.040 --> 0:00:53.479 I thought no better time than to talk election polling 0:00:54.360 --> 0:00:58.200 and this weird sort of black magic which is really 0:00:58.240 --> 0:01:01.360 not black magic at all. And it was, um, the 0:01:01.400 --> 0:01:05.960 polling wasn't even really that off. No, it was great. 0:01:06.000 --> 0:01:08.800 There was a furious fear we'll talk about in a second, 0:01:08.800 --> 0:01:12.199 but there's a furious reaction by the media just left 0:01:12.319 --> 0:01:16.960 polling and polsters out to dry, saying like you you're terrible. 0:01:17.080 --> 0:01:22.440 Your whole craft is is useless. As the pollsters went 0:01:22.480 --> 0:01:25.119 back after election night on two thousand and sixteen, which, 0:01:25.280 --> 0:01:26.760 by the way, is a bit of a surprise to 0:01:26.840 --> 0:01:32.120 everybody involved. Um. Yeah, when the pollsters went back and 0:01:32.160 --> 0:01:33.800 looked at their stuff, they said, wait a minute, no, 0:01:33.959 --> 0:01:36.800 this is this is all fine. It was you guys, media, 0:01:37.240 --> 0:01:40.400 You screwed up. You don't know what polling is or 0:01:40.440 --> 0:01:43.480 what it does, or how to talk about it. Most importantly, Yeah, 0:01:43.520 --> 0:01:46.400 and then you public, you have no idea what's going on. 0:01:46.480 --> 0:01:49.040 You just see some percentages and you automatically leaked to 0:01:49.080 --> 0:01:51.400 some conclusions and this is way off. So it's impartant 0:01:51.400 --> 0:01:55.120 that the media was misrepresenting it. Some polls weren't very good. 0:01:55.480 --> 0:01:58.400 And then, um, the public in general just needs to 0:01:58.440 --> 0:02:01.360 be a bit more educated on statistics to understand what 0:02:01.400 --> 0:02:03.880 they're hearing. And that's what we're here for. Because I 0:02:03.880 --> 0:02:08.120 I took statistics three times in college, the same course 0:02:08.320 --> 0:02:11.399 at Georgia. At Georgia, I took one of those classes. 0:02:11.600 --> 0:02:14.359 I hated it, intro to statistics, right, yeah, boy, I 0:02:14.440 --> 0:02:17.640 hated the class. The professor. Finally I walked up to 0:02:17.680 --> 0:02:19.520 her on the last day of the third time, was 0:02:19.560 --> 0:02:22.480 like please, and she bumped my d up to a 0:02:22.600 --> 0:02:25.519 seat and I was I never looked back. She say 0:02:25.560 --> 0:02:27.760 you have a one in four chance, and you're like, 0:02:27.840 --> 0:02:31.600 but what does that mean? Right? What is four? But 0:02:31.760 --> 0:02:34.320 so if I can understand this after doing some research, 0:02:34.360 --> 0:02:36.720 then anybody can understand at least the gist of it 0:02:36.880 --> 0:02:40.280 enough to understand polling and not be taken in by 0:02:40.639 --> 0:02:44.600 bad representation of what poll results are. Yeah. So, if 0:02:44.639 --> 0:02:48.880 you remember, in there were polsters saying or I'm sorry 0:02:49.320 --> 0:02:51.360 and I'm gonna say that wrong over and over again. 0:02:51.400 --> 0:02:54.880 You had media saying that Hillary Clinton is going to 0:02:54.960 --> 0:02:58.880 win in a landslide. Um, she's got a chance to 0:02:58.880 --> 0:03:01.880 win some set as high is ninety five, She's gonna 0:03:01.880 --> 0:03:05.600 win the popular vote by three percentage points. Um, all 0:03:05.639 --> 0:03:08.919 the all the battleground states in the Midwest, Um, she's 0:03:08.960 --> 0:03:12.639 gonna win those narrowly. And it did not work out 0:03:12.639 --> 0:03:14.600 that way. And like you said, there was a furor 0:03:14.639 --> 0:03:17.560 over how could everyone be this wrong with the polling. 0:03:17.720 --> 0:03:20.839 And there's man named Nate Silver, who everyone probably knows 0:03:20.840 --> 0:03:23.760 at this point, who has made his name as a 0:03:23.880 --> 0:03:29.320 data specialist and runs the five thirty eight blog and said, 0:03:29.400 --> 0:03:33.360 you know what, um pulling is flawed. And that's probably 0:03:33.400 --> 0:03:36.360 the first thing that everyone should understand is all polling 0:03:36.400 --> 0:03:39.360 is a little bit flawed. Um, state polling is is 0:03:39.400 --> 0:03:42.760 definitely a little more flawed than national polling. But here's 0:03:42.760 --> 0:03:47.560 the deal, everybody, these polls from we're not only not 0:03:48.520 --> 0:03:51.840 so far off, but historically, dating back to since nineteen 0:03:51.880 --> 0:03:54.880 seventy two, they actually performed a little better than a 0:03:54.960 --> 0:03:58.480 lot of elections. Yeah, and the state polling, while worse 0:03:58.560 --> 0:04:02.480 than average, wasn't a far off from the average error rate. 0:04:02.640 --> 0:04:05.200 So what do you what do you want? So there's 0:04:05.200 --> 0:04:07.080 a lot of stuff, Like we said that there was 0:04:07.120 --> 0:04:08.880 a lot of post mortem that was done on the 0:04:08.880 --> 0:04:11.640 two thousand and sixteen polls and what what was gotten 0:04:11.640 --> 0:04:13.520 wrong and what was gotten right, And we'll talk about 0:04:13.560 --> 0:04:17.720 that later. But um, the point is is that overall 0:04:18.240 --> 0:04:21.760 it wasn't that far off. And so the the idea 0:04:21.920 --> 0:04:24.960 isn't that the polls failure, that there's something inherently flawed 0:04:24.960 --> 0:04:27.760 with polling, or that there's even something inherently wrong with 0:04:27.800 --> 0:04:30.520 the media. Like I want to go on record here, 0:04:30.640 --> 0:04:34.080 especially in this climate, the media is not our enemy. 0:04:34.320 --> 0:04:38.960 Like any healthy democracy needs a vital, robust, independent media 0:04:39.520 --> 0:04:43.680 is free from bias as an objective to reality and 0:04:43.760 --> 0:04:47.720 good injustice as possible. But there's also such a thing 0:04:47.720 --> 0:04:49.680 as a twenty four hour news cycle, and you've got 0:04:49.680 --> 0:04:52.440 to fill that. And that's given the rise of opinion 0:04:52.520 --> 0:04:56.240 news and pundits and um and basically trying to capture 0:04:56.240 --> 0:04:59.240 as much market share as possible, which is definitely the 0:04:59.279 --> 0:05:02.560 wrong track for media in general. But I just want 0:05:02.560 --> 0:05:04.400 to go on record, while we're gonna be kind of 0:05:04.600 --> 0:05:06.440 beating the media up a little bit, that does not 0:05:06.560 --> 0:05:09.440 mean that the media has inherently flawed or evil or 0:05:09.800 --> 0:05:12.600 or seeks to um to to kill you and your 0:05:12.640 --> 0:05:16.960 family and your family dog. So Silver goes back and 0:05:17.240 --> 0:05:19.200 a bunch of people go back and look at um 0:05:19.440 --> 0:05:22.400 history and kind of what went wrong here in as 0:05:22.440 --> 0:05:24.479 far as the polling goes. He says, you know what, 0:05:24.560 --> 0:05:28.840 we went back for the past twelve presidential cycles since 0:05:28.920 --> 0:05:31.440 nineteen seventy two, and he said the polling air was 0:05:31.480 --> 0:05:35.600 four point one. He said in that national polling air 0:05:35.839 --> 0:05:38.760 was three point one, so technically by a full point 0:05:39.200 --> 0:05:42.279 it was. It was a full point better. He said, Uh, 0:05:42.480 --> 0:05:44.800 we predicted that she would win the popular vote by 0:05:44.839 --> 0:05:47.839 three percentage points. She actually did win the popular vote 0:05:47.880 --> 0:05:52.320 by two percentage points. Um. The state polls were the 0:05:52.360 --> 0:05:55.200 real difference maker. They actually did underperform at a five 0:05:55.240 --> 0:05:59.120 point to error rate, and that doesn't sound like that much. 0:05:59.160 --> 0:06:01.560 I think the overall or rate for state poll since 0:06:01.640 --> 0:06:05.240 nineteen four point eight, so four point eight five point 0:06:05.240 --> 0:06:08.120 two doesn't sound like much. But if you're talking about 0:06:08.480 --> 0:06:11.279 a percentage of error and just a handful of swing states, 0:06:11.960 --> 0:06:14.280 that can make something look like a landslide even though 0:06:14.320 --> 0:06:17.400 you lose a popular vote. That's exactly what happened, right, 0:06:17.440 --> 0:06:20.280 That's exactly what happened, because you gotta remember Trump didn't 0:06:20.279 --> 0:06:23.240 win the popular vote, he won the electoral college, and 0:06:23.279 --> 0:06:25.640 it came down to those swing states. But the fact 0:06:25.680 --> 0:06:31.560 that they were off just by point four points from 0:06:31.560 --> 0:06:34.719 the average for the error rate um goes to show 0:06:34.760 --> 0:06:38.039 you just how close that race actually was, which again 0:06:38.279 --> 0:06:41.560 is the opposite of how it was being broadcast throughout 0:06:41.600 --> 0:06:44.000 the election. It was supposed to be a landslide, like 0:06:44.160 --> 0:06:47.479 Hillary Clinton might as well just be like taking measurements 0:06:47.520 --> 0:06:50.440 for curtains in the oval office right now, Like it 0:06:50.480 --> 0:06:53.600 was just that set So it was presented one way, 0:06:53.640 --> 0:06:56.280 when in reality, if you really looked at the polls 0:06:56.279 --> 0:06:58.920 and the polling results, if you looked at them with 0:06:59.160 --> 0:07:02.560 a sober face, it was a much closer race than 0:07:02.600 --> 0:07:06.200 it appeared or than it was being broadcast. I haven't 0:07:06.240 --> 0:07:11.600 had a sober face since that night. So, uh, we 0:07:11.600 --> 0:07:14.160 should talk about the margin of error. Um, in polling. 0:07:14.200 --> 0:07:17.440 Anytime you see a poll, it's you. They talk about 0:07:17.480 --> 0:07:19.720 the margin of air. It's usually plus plus or minus 0:07:19.760 --> 0:07:23.480 three or four, and that is on each side. So 0:07:23.600 --> 0:07:26.720 for each candidate's poll, uh. In other words, it could 0:07:26.760 --> 0:07:29.480 be a potential like seven to eight point swing and 0:07:29.600 --> 0:07:32.480 still be within that margin of error. So when Trump 0:07:32.600 --> 0:07:36.840 is winning States by a point two percent margin or 0:07:36.880 --> 0:07:40.760 a point five or a point seven percent margin, that's well, 0:07:40.840 --> 0:07:44.160 well well well within the margin of error, right right, 0:07:44.240 --> 0:07:47.560 So um, that margin of error, by the way, is 0:07:47.640 --> 0:07:49.600 just bill tim. We'll talk about it a little more 0:07:49.640 --> 0:07:51.360 and a little bit, but it's like there's just no 0:07:51.440 --> 0:07:54.080 way around it. If to to get around any margin 0:07:54.080 --> 0:07:56.760 of air, you would have to literally go through and 0:07:57.000 --> 0:08:01.280 interview every single voter in America and then compile the 0:08:01.360 --> 0:08:06.480 evidence or their their data perfectly without any miss keys 0:08:06.600 --> 0:08:09.280 or anything like that. And it's just impossible. So everyone 0:08:09.320 --> 0:08:12.000 accepts that any poll is going to have a margin error, 0:08:12.080 --> 0:08:14.960 but you want to keep it within plus or minus 0:08:15.000 --> 0:08:19.560 three points. Right be four. Yeah, So a little history 0:08:19.600 --> 0:08:25.000 of polling. Um, we've always been, um, pretty spell bound 0:08:25.040 --> 0:08:26.960 by poles in this country. We put a lot of 0:08:26.960 --> 0:08:30.760 stock and polls, especially the presidential race. Um. The word 0:08:30.840 --> 0:08:32.839 straw pole, if you've ever heard that, that comes from 0:08:32.880 --> 0:08:35.280 the idea that you hold up a piece of straw 0:08:35.360 --> 0:08:37.400 to see which way the wind is blowing. So a 0:08:37.440 --> 0:08:40.360 straw pole is kind of like, here's how things stand 0:08:40.400 --> 0:08:43.240 today on something like this is why the way the 0:08:43.240 --> 0:08:45.480 wind is blowing today on this matter. Yeah, and they're 0:08:45.480 --> 0:08:47.560 just kind of informal. They used to take them like 0:08:47.640 --> 0:08:50.360 on train cars, as journalists would ask people who they 0:08:50.360 --> 0:08:52.319 were on the train with, who they're going to vote for. 0:08:52.640 --> 0:08:55.199 Nothing like formal or anything, but it it was. It 0:08:55.520 --> 0:08:58.480 does kind of reveal how long standing our fascination with 0:08:58.559 --> 0:09:01.800 poles really really is. Yeah, it got pretty serious in 0:09:01.840 --> 0:09:06.040 the nineteen thirties, specifically the nineteen thirty six election, where 0:09:07.000 --> 0:09:09.199 a literary digest it was, it was a pretty big 0:09:09.240 --> 0:09:13.800 magazine at the time, pulled it's subscribers and it's just 0:09:13.880 --> 0:09:16.960 kind of funny even seeing this sence they predicted a 0:09:17.040 --> 0:09:22.520 landslide wind for Republican Republican alf landon over fd R. 0:09:23.400 --> 0:09:27.400 So if you've never heard of alf Landon, uh, you 0:09:27.480 --> 0:09:31.040 know I because alf Landon did not beat f DR. Uh. 0:09:31.080 --> 0:09:34.040 And the magazine's editor said, you know what, we didn't 0:09:34.040 --> 0:09:36.120 even think about the fact that we just pulled our 0:09:36.200 --> 0:09:40.360 subscribers and that they're wealthy people are at least wealthier 0:09:40.440 --> 0:09:44.319 on average, and they're probably going to vote Republicans. So um, 0:09:44.360 --> 0:09:47.040 alf Landon was their man. Right. So if you go 0:09:47.120 --> 0:09:51.640 out it's even today and just interview Republicans and say, hey, 0:09:52.200 --> 0:09:54.600 who you're going to vote for, and then take that 0:09:54.679 --> 0:09:57.560 results and apply it to the entire population of the nation, 0:09:57.760 --> 0:10:00.720 you've got a flawed pole. And that's what Literary digested. 0:10:01.120 --> 0:10:04.319 But in doing so, they established this kind of they 0:10:04.360 --> 0:10:06.800 pointed out a real design flaw that now is just 0:10:07.200 --> 0:10:09.959 one of the first basic things that anybody conducting a 0:10:10.040 --> 0:10:13.760 pole gets rid of. That's right. Gallup came on the scene. 0:10:13.760 --> 0:10:18.319 They galloped onto the scene. Uh so sorry, And they 0:10:18.320 --> 0:10:21.200 were one of the first big polling companies to say, 0:10:21.240 --> 0:10:22.760 all right, we gotta get this right. We gotta get 0:10:22.760 --> 0:10:25.720 a representation of all of America here. So we're gonna 0:10:25.760 --> 0:10:27.880 send our people door to door. We're gonna go to 0:10:27.920 --> 0:10:30.520 every zip code in America, and they did that from 0:10:30.600 --> 0:10:35.120 n to nineteen four UH and got basically within about 0:10:35.120 --> 0:10:38.560 three percentage points, doing a pretty good job. UM, but 0:10:38.640 --> 0:10:40.839 it was really expensive. So in the eighties, in the 0:10:40.880 --> 0:10:45.400 mid eighties, they switched to calling people on telephone, which 0:10:45.640 --> 0:10:48.240 which I mean that that's still today. That is the 0:10:48.320 --> 0:10:51.480 gold standard is for a human being to dial up 0:10:51.520 --> 0:10:55.280 another human being and ask them some questions. And we'll 0:10:55.320 --> 0:10:58.120 talk a little more about it. But what what Gallup 0:10:58.280 --> 0:11:01.880 does and what you does, and what a few others 0:11:02.160 --> 0:11:06.920 UM do is it's called randomized sampling or probability sampling, 0:11:07.280 --> 0:11:10.520 which is where you basically leave it to chance that 0:11:10.640 --> 0:11:14.840 any voter registered voter in America is going to get 0:11:14.880 --> 0:11:18.320 a phone call from you, So that what what Gallup 0:11:18.400 --> 0:11:21.720 is doing and what Pew does is called randomized sampling 0:11:21.800 --> 0:11:26.080 or probability sampling, where the any voter in America has 0:11:26.120 --> 0:11:29.920 an equal chance of receiving a phone call from Gallop 0:11:30.000 --> 0:11:33.120 or from Pew and being asked these questions. And it 0:11:33.200 --> 0:11:36.800 worked pretty well for a while when they moved from 0:11:37.080 --> 0:11:39.960 UH in person over to the phone, because they were 0:11:39.960 --> 0:11:43.520 still asking people questions and they could still UM get 0:11:43.600 --> 0:11:45.559 their answers and harass them, which is a big thing 0:11:45.640 --> 0:11:50.080 is we'll see about the this type of sampling. Um, 0:11:50.240 --> 0:11:54.480 the problem is when people started to use caller I D, 0:11:54.640 --> 0:11:57.600 they stopped picking up the phone as much, and so 0:11:57.720 --> 0:12:01.520 the response rate went down dramatic. Yeah, so they would 0:12:01.520 --> 0:12:04.679 call people using random digit dialing, which is a computer 0:12:04.800 --> 0:12:07.880 system where it fed in an area code and then 0:12:07.960 --> 0:12:11.439 the first three digits and then randomly dialed the last four. 0:12:12.040 --> 0:12:13.640 So you've got a pretty good start there on the 0:12:13.720 --> 0:12:16.840 random sampling. But even then they said, you know what, 0:12:16.960 --> 0:12:19.280 women tend to answer the phone more than men. So 0:12:19.360 --> 0:12:22.480 to truly randomize that whenever whoever picks up the phone, 0:12:22.960 --> 0:12:24.600 we have to then follow up and say we want 0:12:24.600 --> 0:12:27.040 to talk to the person in the house who's had 0:12:27.040 --> 0:12:31.400 the most recent birthday, further randomizing. Um. I got it 0:12:31.480 --> 0:12:34.040 kind of a laugh about this because I don't know 0:12:34.120 --> 0:12:37.240 that I've ever, literally ever seen my father pick up 0:12:37.280 --> 0:12:40.760 a telephone in his life, or at least growing up 0:12:40.800 --> 0:12:42.560 for the first eighteen years of my life, I don't 0:12:42.559 --> 0:12:44.800 think I ever saw him answer the phone. It's all 0:12:44.880 --> 0:12:47.520 ham radio, huh. Not not he went into that, but 0:12:48.160 --> 0:12:50.400 just literally not not one time. He would just let 0:12:50.400 --> 0:12:52.880 it ring. If no one was around if my mom 0:12:52.920 --> 0:12:55.720 wasn't around to answer it, and granted it was usually 0:12:55.720 --> 0:12:57.640 never for him, no one ever called to talk to him. 0:12:57.679 --> 0:13:00.240 But I picked up on that and my friends used 0:13:00.280 --> 0:13:02.800 to get really frustrated back before texting that I would 0:13:02.800 --> 0:13:06.600 just never answer my phone. And I always just thought 0:13:06.600 --> 0:13:08.240 it was an option, Like when the phone rings, it 0:13:08.240 --> 0:13:10.720 doesn't mean you're obligated. It just means now you have 0:13:10.760 --> 0:13:13.160 an option you can answer it or not. Well, technically 0:13:13.240 --> 0:13:16.480 that's true. I mean like it depends no, you don't 0:13:16.520 --> 0:13:20.240 have to answer the phone, but it depends on you know, 0:13:20.360 --> 0:13:23.360 who in your life could possibly be calling. I didn't 0:13:23.360 --> 0:13:25.200 think it was rude or anything. I just thought it 0:13:25.240 --> 0:13:28.959 was literally, like, you know, I'm gonna hedge my beds here, 0:13:29.000 --> 0:13:30.880 that one of my friends isn't stuck on the side 0:13:30.880 --> 0:13:33.080 of the road. They can leave a message and if 0:13:33.080 --> 0:13:36.080 they are, I'll go get them. So, UM, what you're 0:13:36.120 --> 0:13:40.079 talking about, Chuck, is what's called a non response, and 0:13:40.400 --> 0:13:45.240 that's factored into the response rate, which with phone pulling 0:13:45.320 --> 0:13:49.240 from nine the nine eight until the nineteen nineties, um 0:13:49.240 --> 0:13:52.120 it was manageable. You. I think the response rate peaked 0:13:52.160 --> 0:13:56.960 at thirty six percent in n which is good. Now 0:13:57.000 --> 0:13:59.080 it's down to like nine percent, because, like I said, 0:13:59.120 --> 0:14:01.840 people have callorad D and if some unknown numbers calling, 0:14:01.880 --> 0:14:05.280 you typically don't answer. And that actually affects things because 0:14:05.320 --> 0:14:07.679 there is a certain kind of person who answers the 0:14:07.720 --> 0:14:12.000 phone no matter what, and they are not like every 0:14:12.000 --> 0:14:16.080 single American, and that actually factors into the kind of 0:14:16.120 --> 0:14:19.560 pole you're connecting. Plus also you want like a certain 0:14:19.600 --> 0:14:24.120 amount of responses. I think out of a sample size, 0:14:24.160 --> 0:14:28.200 you want a minimum of eight hundred survey responses. And 0:14:28.360 --> 0:14:30.040 back in the day, when you got a thirty six 0:14:30.040 --> 0:14:32.880 percent response rate, meaning thirty six percent of those people 0:14:32.920 --> 0:14:35.200 you called would answer the phone and go through all 0:14:35.240 --> 0:14:38.600 of the questions and answer them fully and complete the survey. 0:14:39.080 --> 0:14:41.480 Um since it was down to nine percent, you went 0:14:41.480 --> 0:14:45.240 from having to call between two thousand people to to 0:14:45.560 --> 0:14:48.200 up to nine thousand people now just to get eight 0:14:48.280 --> 0:14:51.080 hundred surveys completed. And that made the whole thing a 0:14:51.200 --> 0:14:55.640 lot more expensive. On the one hand, because it was expensive, 0:14:55.680 --> 0:14:57.760 it meant that there were fewer and fewer companies that 0:14:57.800 --> 0:15:00.880 could conduct these polls, which meant the polls you were 0:15:00.880 --> 0:15:04.120 seeing were more and more legitimate. But on the other hand, 0:15:04.480 --> 0:15:08.000 it also UM usually decreased sample size a little bit, because, 0:15:08.040 --> 0:15:10.680 as as Gallup pointed out, like you can kind of 0:15:10.720 --> 0:15:13.120 fiddle with the numbers a little bit with a smaller 0:15:13.200 --> 0:15:16.880 response rate and smaller sample size. Yeah, and it also 0:15:16.960 --> 0:15:20.080 led to robo calls because of expense, because of people 0:15:20.120 --> 0:15:23.440 not answering their phone as much, and those systems. Uh. 0:15:23.600 --> 0:15:25.200 I mean, I love how Dave Ruce put it. He 0:15:25.240 --> 0:15:28.320 said they they range from okay to terrible, UM, and 0:15:28.360 --> 0:15:31.840 how well they work online polls in these other new techniques. 0:15:32.480 --> 0:15:35.480 But I think we should take a break and then 0:15:35.520 --> 0:15:39.800 talk about, uh what I found the very interesting UM 0:15:39.920 --> 0:15:42.520 way that they further randomized this thing from this point 0:15:42.520 --> 0:16:10.520 forward right after this. All right, so we've already talked 0:16:10.520 --> 0:16:12.800 about the fact that they randomly called someone, and then 0:16:12.800 --> 0:16:15.600 they take one further step on that that call by saying, 0:16:15.640 --> 0:16:17.880 let me speak with whoever had the most recent birthday, 0:16:18.800 --> 0:16:21.360 even if it's I guess you're your three year old, right, 0:16:21.920 --> 0:16:23.880 And and one other thing I kind of made mention 0:16:23.960 --> 0:16:28.000 to it that I have to interject, dude, like harassing people, 0:16:28.320 --> 0:16:31.360 like if you've been picked by this computer, if your 0:16:31.360 --> 0:16:33.800 phone number has been picked, they're going to keep calling 0:16:33.840 --> 0:16:37.000 you and calling you. And that is because, as a 0:16:37.040 --> 0:16:41.240 person who doesn't normally participate in phone surveys, you are 0:16:41.280 --> 0:16:43.960 a specific kind of person that you you can't be 0:16:44.080 --> 0:16:47.440 left out of the population because you represent a large 0:16:47.560 --> 0:16:50.440 number of people and they want your opinion. So part 0:16:50.480 --> 0:16:53.160 of this phone standard of calling people is to call 0:16:53.240 --> 0:16:57.120 them over and over and over again to basically harass 0:16:57.200 --> 0:16:59.920 them into participating to get their answers for this serve 0:17:00.360 --> 0:17:03.760 because it's as important, if not more important, sometimes than 0:17:03.840 --> 0:17:05.639 the people who are like, oh, yeah, I'd love to 0:17:05.680 --> 0:17:10.000 answer this phone survey. Two totally different kinds of people. Yeah. Absolutely. 0:17:10.280 --> 0:17:12.399 And I was totally kidding, by the way, to the 0:17:12.400 --> 0:17:14.200 listener when I said they will speak to a three 0:17:14.280 --> 0:17:16.639 year old. They they asked the most recent birthday of 0:17:16.640 --> 0:17:19.879 someone a voting age obviously, right. Um, So then you've 0:17:19.920 --> 0:17:22.520 got a pretty pretty decent random sampling to begin with. 0:17:22.960 --> 0:17:27.200 And then you have to start uh the process of waiting, uh, 0:17:27.280 --> 0:17:29.639 which comes in a lot of different forms. Um. If 0:17:29.680 --> 0:17:32.560 you want an example of like a really good political poll, 0:17:33.400 --> 0:17:36.280 it's gonna be paid for by a neutral source. It's 0:17:36.320 --> 0:17:39.480 not gonna be um like you know, a CNN pole 0:17:39.560 --> 0:17:41.840 or a Fox News poll or a superpack or anything 0:17:41.880 --> 0:17:44.320 like that. Um, you're gonna have a random sample of 0:17:44.359 --> 0:17:47.199 the public, which we just talked about. You're gonna be 0:17:47.320 --> 0:17:50.960 dialing cell phones and landlines these days. That's a big one. Also, 0:17:51.040 --> 0:17:53.199 they'll ask you if you have a cell phone and 0:17:53.280 --> 0:17:55.480 a landline, and if you say yes, I have both, 0:17:55.800 --> 0:17:59.399 they're going to adjust your response based on the fact 0:17:59.440 --> 0:18:02.720 that you had a higher chance of being selected because 0:18:03.000 --> 0:18:06.400 you have two numbers that the computer could have picked right. 0:18:06.560 --> 0:18:08.960 And another thing is, like you mentioned, they're gonna keep 0:18:08.960 --> 0:18:13.800 calling you the best ones, use live interviewers still. And 0:18:13.840 --> 0:18:15.640 then what they want to do, and this last one 0:18:15.680 --> 0:18:18.520 is really important, is you're they're gonna try and improve 0:18:18.560 --> 0:18:21.960 the accuracy of the results by waiting the response to match. 0:18:22.000 --> 0:18:23.560 What they want to do is just match a real 0:18:23.600 --> 0:18:29.200 world demographic, age race, your income level, your education level, 0:18:29.680 --> 0:18:32.360 and all of that stuff is factored in, and all 0:18:32.400 --> 0:18:36.720 the stuff is weighed out because, um, well we'll talk 0:18:36.760 --> 0:18:40.320 about it, but you know, there are many different kinds 0:18:40.320 --> 0:18:42.400 of Americans, and if you want a really good sampling 0:18:42.400 --> 0:18:45.240 of different kinds of Americans, you're gonna, like, like you said, 0:18:45.280 --> 0:18:47.199 I have to fidget with the numbers to make it 0:18:47.280 --> 0:18:53.280 a true representative population right. So, um, because even if 0:18:53.280 --> 0:18:57.120 you just get it exactly right demographically and waited, which, 0:18:57.200 --> 0:18:58.760 like you said, we'll talk about some more in a second, 0:18:58.960 --> 0:19:01.200 you still have that marge and of error. And again 0:19:01.240 --> 0:19:05.000 that's that um, you know, plus or minus three points, 0:19:05.000 --> 0:19:07.320 and that means that it could be cent or it 0:19:07.320 --> 0:19:11.920 could be they don't know, but somewhere between that, most 0:19:11.960 --> 0:19:13.920 of your answers are going to be the like, the 0:19:13.960 --> 0:19:17.240 correct answers somewhere in there. That's what that means with 0:19:17.359 --> 0:19:20.080 that that margin of error, And the reason that that's 0:19:20.119 --> 0:19:24.600 built in is because it is basically impossible to perfectly 0:19:25.000 --> 0:19:29.400 represent the larger population through random sampling. You're just not 0:19:29.480 --> 0:19:33.320 going to pick everybody correctly just by the fact that 0:19:33.359 --> 0:19:36.840 it's random and it's a sample. Yeah, and that's important 0:19:36.880 --> 0:19:39.560 because um, like that's why you hear so much hay 0:19:39.560 --> 0:19:42.600 being made over a double digit lead in a poll um, 0:19:42.640 --> 0:19:45.239 which Biden had sort of semi recently. I know it's 0:19:45.280 --> 0:19:47.560 it's gotten a lot tighter since then, but you know, 0:19:47.600 --> 0:19:50.240 when Biden was up by I think like ten percentage points, 0:19:50.600 --> 0:19:54.280 people were flipping out because you know, like we said, 0:19:54.320 --> 0:19:56.920 it's plus or minus four for each candidate. So that's 0:19:56.960 --> 0:19:59.960 a total of eight. And so basically the press start 0:20:00.119 --> 0:20:04.359 screaming like he's outside of the margin of error. Everybody 0:20:05.480 --> 0:20:08.439 like nothing can beat him, right right, yeah, But now 0:20:08.560 --> 0:20:11.480 things are back within that margin. I saw PBS News 0:20:11.520 --> 0:20:15.040 Our UM they interview Mark Shields and David Brooks. Um. 0:20:15.160 --> 0:20:18.160 Brooks is a New York Times columnists and I think 0:20:18.160 --> 0:20:21.919 Mark Shields is an independent columnist. And one of them 0:20:22.200 --> 0:20:25.480 actually said, and this is in July, America has clearly 0:20:25.520 --> 0:20:27.960 made up its mind on who's going to be the 0:20:28.000 --> 0:20:31.400 next president. And I was like, this is July. Did 0:20:31.400 --> 0:20:34.040 you not learn anything from two thousand and sixteen or 0:20:34.080 --> 0:20:37.760 you I couldn't believe that those words, And there's a 0:20:38.280 --> 0:20:41.520 matter of factly, yeah, it's irresponsible. And there's been studies 0:20:41.520 --> 0:20:45.960 about this too that have suggested that that words like that, 0:20:45.960 --> 0:20:50.040 that um polls that say chance of winning, that this 0:20:50.160 --> 0:20:54.639 kind of stuff like actually has a negative impact on 0:20:54.720 --> 0:20:56.760 the leader because it makes people think, well, I don't 0:20:56.760 --> 0:20:58.119 need to go out and vote. Everybody else is going 0:20:58.160 --> 0:21:01.280 to go vote, and the turnout might be lower than otherwise. 0:21:01.320 --> 0:21:03.880 There's also people that just who well, there's people who 0:21:03.920 --> 0:21:07.919 dispute that. They say, yes, it makes sense intuitively and anecdotally, 0:21:08.160 --> 0:21:11.240 but we've yet to actually see genuine data that that 0:21:11.280 --> 0:21:14.720 says clearly that this has this effect. But it's something 0:21:14.760 --> 0:21:17.680 that's still being studied right now whether it actually does 0:21:17.800 --> 0:21:20.000 or not well. And I also saw an article the 0:21:20.000 --> 0:21:23.520 other day about the the quote unquote silent majority, and 0:21:23.560 --> 0:21:25.840 that another reason those polls were so wrong back then 0:21:25.960 --> 0:21:29.240 and they're saying are probably wrong now is because there 0:21:29.280 --> 0:21:33.240 are there they They say that there's a substantial block 0:21:33.280 --> 0:21:37.520 of voters who vary privately and secretly vote for Trump. Yeah, 0:21:37.560 --> 0:21:41.320 they're the term for them among pollsters as shy Trump voters. 0:21:42.160 --> 0:21:44.440 They won't admit that they're going to vote for Trump, 0:21:44.520 --> 0:21:46.280 but they're going to vote for Trump, and that that 0:21:46.359 --> 0:21:50.160 affects polls. I saw that that's actually not been proven 0:21:50.200 --> 0:21:53.880 to actually exist. Um, but I think it was a Pew. 0:21:54.160 --> 0:21:56.359 There's a really great Pew article. If this stuff is 0:21:56.400 --> 0:22:00.720 speaking to you at all, go check out Pews Key 0:22:00.800 --> 0:22:03.800 Things to Know about Election Polling in the US and 0:22:03.800 --> 0:22:05.520 Now has a bunch of great links that you should 0:22:05.560 --> 0:22:09.720 follow in there. And there's also Sideline UM. They have 0:22:10.040 --> 0:22:13.760 Surveys and Polling, which is a guide for journalists to polling. 0:22:13.800 --> 0:22:15.760