WEBVTT - Big Take DC: Economists May Be Using Bad Data to Make Big Decisions 0:00:00.200 --> 0:00:03.920 Hello, odd last listeners, you'll be enjoying an episode of 0:00:03.960 --> 0:00:07.800 one of our fellow podcasts on the Bloomberg Network, Big 0:00:07.840 --> 0:00:08.800 Take DC. 0:00:08.880 --> 0:00:13.160 YEP, hosted by Soliah Mosen. This episode features one of 0:00:13.160 --> 0:00:16.799 our favorite guest economist, Claudia Salm. You might remember that 0:00:16.840 --> 0:00:19.680 we spoke to her back in November about the outlook 0:00:19.720 --> 0:00:23.680 for the US economy and Soleia digs even deeper into 0:00:23.720 --> 0:00:28.479 what's driving things like sentiment consumer surveys versus the hard data. 0:00:28.680 --> 0:00:29.840 Right now, take a listen. 0:00:31.120 --> 0:00:33.520 When the government is trying to get a handle on inflation, 0:00:34.040 --> 0:00:37.919 it's the Federal Reserve that has the biggest lever to pull. 0:00:38.080 --> 0:00:40.879 Think of the FED like a traffic cop instead of 0:00:40.880 --> 0:00:44.160 a whistle and cone. The Central Bank uses interest rates 0:00:44.159 --> 0:00:48.040 to try and contain inflation. When rates go up, money 0:00:48.080 --> 0:00:51.320 becomes expensive and people tend to spend and borrow less. 0:00:52.040 --> 0:00:56.280 That slows the economy down. When rates go down, people 0:00:56.320 --> 0:00:59.080 are more willing to spend since everything from credit card 0:00:59.080 --> 0:01:03.040 fees to mortgage rates are cheaper. Unlike the traffic build 0:01:03.080 --> 0:01:06.120 up on a road which anyone can see. The Fed 0:01:06.160 --> 0:01:08.600 has to get creative in order to manage the economy, 0:01:09.000 --> 0:01:12.399 so it uses data to decide when and how to intervene. 0:01:13.080 --> 0:01:16.400 But last year, when economists everywhere were expecting a full 0:01:16.440 --> 0:01:19.560 blown recession, the FED was raising interest rates. 0:01:19.280 --> 0:01:20.360 Over and over again. 0:01:21.040 --> 0:01:23.800 They needed to rain in inflation, and the man in 0:01:23.920 --> 0:01:27.319 charge chared your own Powell. He kept pointing to one 0:01:27.440 --> 0:01:30.560 category of data that was guiding the Fed's decision, the 0:01:30.640 --> 0:01:31.120 labor market. 0:01:31.160 --> 0:01:33.240 Of labor market, A labor market remains very tight. 0:01:34.760 --> 0:01:37.480 All this talk about the tight labor market made Claudia 0:01:37.560 --> 0:01:40.880 Sam's ears perk up. She's an economist and a Bloomberg 0:01:40.959 --> 0:01:43.959 Opinion contributor. She worked in the Obama White House and 0:01:44.000 --> 0:01:47.480 spent twelve years at the FED. She'd been looking into 0:01:47.480 --> 0:01:50.680 the labor market numbers herself, and the Fed's decisions left 0:01:50.680 --> 0:01:51.800 her scratching her head. 0:01:52.040 --> 0:01:54.760 They are making big decisions about the interest rates, the 0:01:54.840 --> 0:01:58.960 mortgage rates we pay, the credit card interest rates, auto loans. 0:01:59.200 --> 0:02:01.880 So we want them to be data driven, but they 0:02:01.880 --> 0:02:04.320 can only do as good a job as the data. 0:02:04.120 --> 0:02:07.520 They have, and that data they've been focused on, she's 0:02:07.560 --> 0:02:09.400 had some serious questions about it. 0:02:13.639 --> 0:02:15.600 This morning, the government. 0:02:15.240 --> 0:02:19.000 Released new GDP data that shows the US successfully avoided 0:02:19.040 --> 0:02:23.080 a recession, even though almost every economist was predicting one, 0:02:23.639 --> 0:02:26.200 but the data that the Federal Reserve examined as it 0:02:26.280 --> 0:02:29.040 made policy decisions is complicated. 0:02:29.800 --> 0:02:33.519 On today's show, have policymakers trusted data that might have 0:02:33.560 --> 0:02:37.160 been faulty? I talked to Claudia Sam about her findings, 0:02:37.720 --> 0:02:40.680 and I sit down with Tracy Alloway and Joe Wisenthal 0:02:40.720 --> 0:02:44.400 from Bloomberg's Odd Lots podcast. We talk about what's behind 0:02:44.440 --> 0:02:47.320 the numbers and why it's important in an election year. 0:02:48.400 --> 0:02:52.480 From Bloomberg's Washington Bureau. This is the Big Take DC podcast. 0:02:53.080 --> 0:03:02.679 I'm your host Seleiah Mosen. Claudia Sam decided her concerns 0:03:02.680 --> 0:03:06.440 about the Federal Reserves data were worth voicing, so in 0:03:06.480 --> 0:03:09.840 November she wrote an article for Bloomberg Opinion. It had 0:03:09.880 --> 0:03:13.560 an eye catching headline, economists may have been flying blind 0:03:13.760 --> 0:03:14.440 all along. 0:03:15.080 --> 0:03:18.080 So the argument I was making when I said economists 0:03:18.120 --> 0:03:21.720 or flying blind is the awareness that we need to 0:03:21.760 --> 0:03:25.480 have in terms of the measures like how we try 0:03:25.520 --> 0:03:28.760 and measure quote unquote reality, and then in our giving 0:03:28.800 --> 0:03:30.040 policy advice. 0:03:30.400 --> 0:03:32.600 How we measure quote unquote reality. 0:03:33.280 --> 0:03:36.120 I know that sounds dense, but her point is that 0:03:36.240 --> 0:03:38.480 as much as we'd love to think that the FED 0:03:38.640 --> 0:03:43.040 is making its decisions based on hard numbers, you know, objective, 0:03:43.240 --> 0:03:44.160 unbiased data. 0:03:44.960 --> 0:03:46.400 Often it's not. 0:03:46.720 --> 0:03:49.000 Data doesn't doesn't come down from heaven. 0:03:49.680 --> 0:03:52.840 For example, let's look at that tight labor market that 0:03:52.880 --> 0:03:53.600 FED Chair J. 0:03:53.840 --> 0:03:55.240 Powell kept mentioning. 0:03:55.880 --> 0:03:59.080 He said that the labor market was tight, meaning more 0:03:59.200 --> 0:04:02.760 job openings than workers. He cited numbers from the Job 0:04:02.800 --> 0:04:07.200 Openings and Labor Turnover Survey JOLTS for short. Now that 0:04:07.280 --> 0:04:10.760 might sound straightforward, right, measuring the number of open jobs 0:04:11.520 --> 0:04:12.280 not so fast. 0:04:12.520 --> 0:04:15.400 Now, there was a lot of conversation those of us 0:04:15.600 --> 0:04:18.479 who have nothing better to do than study data. What 0:04:18.560 --> 0:04:22.200 a job opening is could be changing over time. 0:04:22.240 --> 0:04:25.680 Because of the pandemic. The way employers list jobs is 0:04:25.720 --> 0:04:27.480 just different than it was before. 0:04:27.480 --> 0:04:29.719 Especially from work from home. You can put up multiple 0:04:29.720 --> 0:04:33.520 ones for different geographies because it doesn't matter, So. 0:04:33.440 --> 0:04:36.800 A company might list the same job in several different cities. 0:04:37.200 --> 0:04:39.960 It doesn't cost them anything. But it does mean that 0:04:40.000 --> 0:04:41.480 the numbers are getting inflated. 0:04:41.920 --> 0:04:43.240 So when economists at the. 0:04:43.160 --> 0:04:45.839 FED were looking at the number of open jobs and 0:04:45.920 --> 0:04:49.400 basing their assumptions off of what was typical, they were 0:04:49.400 --> 0:04:51.520 at risk of ignoring one key factor. 0:04:52.200 --> 0:04:53.360 The world wasn't typical. 0:04:54.200 --> 0:04:56.960 I wanted to understand just what's going on here and 0:04:57.000 --> 0:04:59.760 whether it was an issue beyond this one job survey. 0:05:00.160 --> 0:05:02.120 So I sat down with two of my colleagues. 0:05:02.640 --> 0:05:04.919 I'm Tracy Alloway, I am the co host of the 0:05:04.960 --> 0:05:06.440 auth Lots podcast. 0:05:06.480 --> 0:05:08.839 And I'm Jill Wasenthal, also the co host of the 0:05:08.839 --> 0:05:09.800 aud Lots podcast. 0:05:10.440 --> 0:05:13.880 Joe and Tracy read Sam's article and they agreed with her. 0:05:14.200 --> 0:05:16.479 They do not trust that Jolts data. 0:05:17.320 --> 0:05:24.120 Pre COVID, Jolts was a bottom shelf economic indicator. It 0:05:24.160 --> 0:05:26.320 was the well drinks of you know, it's like some 0:05:26.480 --> 0:05:29.320 nerds like to pour over it because there is information 0:05:29.440 --> 0:05:31.560 on it, but it was not a market mover. 0:05:32.120 --> 0:05:34.719 If Jolts was a bottom shelf well drink to them, 0:05:34.800 --> 0:05:38.840 pre COVID, it was basically a cheap shot of bad tequila. 0:05:39.000 --> 0:05:40.120 Once the pandemic hit. 0:05:40.640 --> 0:05:44.560 You just don't know that the patterns of history related 0:05:44.600 --> 0:05:47.760 to things like job openings, related to things like claims 0:05:48.000 --> 0:05:52.600 quit really mean the same thing in this environment as 0:05:52.640 --> 0:05:54.480 they might have in past cycles. 0:05:54.760 --> 0:05:57.159 If it was a business cycle, it was the weirdest 0:05:57.320 --> 0:06:01.440 business cycle ever. Companies are behaving differently to how they 0:06:01.560 --> 0:06:04.200 used to. There's the idea of labor hoarding. People are 0:06:04.200 --> 0:06:07.440 so scarred from the pandemic period that they just want 0:06:07.480 --> 0:06:10.160 to make sure they're not caught out again with a 0:06:10.240 --> 0:06:13.200 labor shortage. So they're just hiring who they can, or 0:06:13.200 --> 0:06:16.440 they're putting out ads to see who responds. I mean, 0:06:16.480 --> 0:06:20.000 it's pretty easy to place an ad on some digital 0:06:20.120 --> 0:06:22.880 job site nowadays. It doesn't really cost that much, so 0:06:22.920 --> 0:06:24.600 why not try and see who you get? 0:06:24.960 --> 0:06:28.000 So the pandemic through all our old markers of normal 0:06:28.160 --> 0:06:31.440 out the window. That left the Jolt survey and pretty 0:06:31.440 --> 0:06:35.560 and steady ground. But COVID didn't just mess with jolts. 0:06:35.920 --> 0:06:39.160 It also did another thing that influences all sorts of 0:06:39.200 --> 0:06:43.880 important data points that fed economists rely on survey responses. 0:06:44.000 --> 0:06:47.479 We know they have declined in recent years, so I 0:06:47.520 --> 0:06:52.039 think something like the Housing Survey gets like half of 0:06:52.080 --> 0:06:56.320 the people its surveys actually responding nowadays, and that's down 0:06:56.520 --> 0:06:58.280 from two thirds. 0:06:58.680 --> 0:07:01.120 We reached out to the Bureau of Labor Statistics and 0:07:01.160 --> 0:07:03.960 the Census Bureau for comment for this episode, and they 0:07:04.040 --> 0:07:07.719 both acknowledge declining response rates as a critical problem that 0:07:07.720 --> 0:07:10.760 they're trying to address. It's a problem that only got 0:07:10.800 --> 0:07:14.760 worse during the pandemic. All This matters because if your 0:07:14.800 --> 0:07:18.280 survey only captures half of the people you contact, you're. 0:07:18.080 --> 0:07:21.400 Gonna have to question whether or not that fifty percent 0:07:21.680 --> 0:07:27.520 is reflective of the actual American experience. And of course 0:07:27.560 --> 0:07:32.720 the irony is that most advanced economies are collecting more 0:07:32.880 --> 0:07:37.280 data than ever. We're doing more soft surveys than ever, 0:07:37.840 --> 0:07:40.960 but the response rates are trending down and the quality 0:07:40.960 --> 0:07:42.880 of that data is questionable. 0:07:43.680 --> 0:07:46.880 We'll get to why Americans are getting survey shy, what 0:07:46.920 --> 0:07:49.200 the FED is doing to fix it, and what it 0:07:49.240 --> 0:07:51.120 all means with a twenty twenty four election. 0:07:51.880 --> 0:08:02.160 After the break, we're back. Part of what. 0:08:02.040 --> 0:08:05.440 Made Claudia Sam argue that economists may have been flying 0:08:05.520 --> 0:08:09.520 blind is lower response rates to government surveys, and that 0:08:09.560 --> 0:08:12.320 decline is actually a symptom of a much bigger problem. 0:08:12.720 --> 0:08:16.480 We've seen a growing distrust in government, and I can 0:08:16.640 --> 0:08:18.880 understand if you don't trust the government if they show 0:08:18.960 --> 0:08:20.600 up and be like, hey, tell us all about your 0:08:20.640 --> 0:08:22.960 wealth and your dead and how much income you make. 0:08:23.040 --> 0:08:25.520 Much for a lot, these are very sensitive topics. 0:08:25.960 --> 0:08:28.720 Peer research found that two thirds of adults think the 0:08:28.840 --> 0:08:33.240 risk of responding to a survey outweighs the benefits they're 0:08:33.280 --> 0:08:36.800 concerned about things like privacy and not so concerned about 0:08:36.800 --> 0:08:39.400 the consequences of low response rate data. 0:08:39.920 --> 0:08:42.160 Some says they should be, Well, you need. 0:08:42.040 --> 0:08:46.400 To rebuild that relationship and help people understand know what 0:08:46.480 --> 0:08:50.360 you tell us. Like policymakers, this is important and if 0:08:50.400 --> 0:08:52.680 we don't know what's going on in your life, then 0:08:52.800 --> 0:08:57.800 it's almost guaranteed that the policy just can't address the issues. 0:08:58.240 --> 0:09:01.160 We're in an election year, and when Americans list the 0:09:01.200 --> 0:09:05.520 economy as the top issue driving their votes, so flawed 0:09:05.520 --> 0:09:09.400 assumptions about the economy based on sketchy data carry a 0:09:09.480 --> 0:09:13.080 risk as people decide who they want as president, and 0:09:13.160 --> 0:09:16.800 so does data that doesn't fully capture voters lived experiences. 0:09:17.480 --> 0:09:20.080 I asked Joe and Tracy from Odd Lots about all 0:09:20.080 --> 0:09:22.920 that it kind of feels like the worst timing to 0:09:23.000 --> 0:09:26.360 have bad data or questionable data when there's such a 0:09:26.400 --> 0:09:29.800 consequential election at hand. So I'm curious what you guys think. 0:09:29.960 --> 0:09:32.679 How do you think potentially flawed data is going to. 0:09:32.600 --> 0:09:33.280 Affect all of this. 0:09:34.080 --> 0:09:38.200 One of my favorite surveys to read through is the 0:09:38.320 --> 0:09:42.960 NFIB Small Business Optimism Survey, and there's one chart that 0:09:43.040 --> 0:09:48.200 really catches my eye in which the NFIB itself disambiguates 0:09:48.480 --> 0:09:50.960 between what they call the hard data and the soft data. 0:09:51.040 --> 0:09:54.000 So the hard data is like, we're your sales higher 0:09:54.080 --> 0:09:57.079 or lower in the last three months, it's not really 0:09:57.120 --> 0:09:59.880 an opinion question either was or what was itenc you? 0:10:00.480 --> 0:10:02.720 And then there's the soft data, it's like, do you 0:10:02.760 --> 0:10:06.960 feel confident enough to invest in this environment. What's really 0:10:07.040 --> 0:10:11.360 interesting is that the hard data and soft data really 0:10:11.400 --> 0:10:16.600 do converge during past Republican administrations and really do diverge 0:10:16.800 --> 0:10:20.400 during democratic administration. So there's a huge gap right now 0:10:20.520 --> 0:10:23.720 within the NFIB between their soft and hard data. So 0:10:23.760 --> 0:10:25.920 I do think that there is a split in sort 0:10:25.960 --> 0:10:30.360 of how people perceive the economy versus how people perceive 0:10:30.400 --> 0:10:33.800 their own household finances. That is sort of interesting. How 0:10:33.800 --> 0:10:37.320 do people vote on this? You know, it's hard to say. 0:10:37.720 --> 0:10:38.440 To that point. 0:10:38.480 --> 0:10:40.120 I kind of think about it on a sort of 0:10:40.280 --> 0:10:43.960 personal versus like absolute basis, which is you do see 0:10:44.080 --> 0:10:47.320 a lot of self reporting, so people talking about their 0:10:47.360 --> 0:10:51.679 own financial circumstances, or to Joe's point about small businesses, 0:10:51.760 --> 0:10:55.640 their own small business circumstance, they will say it's going 0:10:56.040 --> 0:10:59.040 relatively well, and you can see some of that born 0:10:59.080 --> 0:11:01.360 out in the hard data, but when they talk about 0:11:01.400 --> 0:11:04.960 the economy in aggregate, that's when you tend to see 0:11:05.120 --> 0:11:08.520 a lot more negative sentiment, and there is a sort 0:11:08.559 --> 0:11:12.040 of weird cognitive dissonance there. We can talk about whether 0:11:12.120 --> 0:11:14.880 that might be down to partisanship, down to the media 0:11:14.920 --> 0:11:18.040 of things like that, But I do think the interesting 0:11:18.120 --> 0:11:21.280 question is if everyone keeps saying they think the economy 0:11:21.360 --> 0:11:24.640 is doing terribly, is that actually going to manifest in 0:11:25.000 --> 0:11:27.880 a slowdown in growth or even a contraction at some point. 0:11:27.960 --> 0:11:29.480 We haven't seen that yet. 0:11:30.559 --> 0:11:33.760 That's significant a lot of people this time last year, 0:11:33.760 --> 0:11:36.959 we're looking at government data and saying we're headed straight 0:11:37.040 --> 0:11:40.400 for a recession. But it turns out all this flawed 0:11:40.520 --> 0:11:44.040 data isn't just affecting the fed's decisions. It also goes 0:11:44.080 --> 0:11:47.760 the other way, as in FED decisions like raising interest 0:11:47.840 --> 0:11:51.960 rates also shape the narratives that economists construct about the 0:11:52.000 --> 0:11:53.079 state of the economy. 0:11:53.280 --> 0:11:57.000 The consensus position going into twenty twenty three was that 0:11:57.040 --> 0:11:59.280 we work on to see a recession, that it was 0:11:59.320 --> 0:12:02.000 impossible to have the extent of the rate hikes that 0:12:02.040 --> 0:12:05.720 we had seen without having some sort of slowing or 0:12:05.760 --> 0:12:07.920 negative effect on the economy. 0:12:08.360 --> 0:12:12.960 The story is you make money more expensive, that decreases 0:12:13.000 --> 0:12:16.800 the ability to invest in borrow. That causes people to 0:12:16.840 --> 0:12:20.800 lose their jobs. Lost jobs mean less demand. Less demand 0:12:20.840 --> 0:12:24.880 means lower prices. That is the basic causal chain between 0:12:25.000 --> 0:12:29.040 how higher rates causes low inflation. It's sort of the 0:12:29.080 --> 0:12:31.480 standard popular telling of how economics work. 0:12:31.800 --> 0:12:36.320 The idea that prices could come down without spiking unemployment 0:12:36.800 --> 0:12:40.480 was just absolutely outrageous sort of this time last year, 0:12:40.679 --> 0:12:43.280 and yet what we've seen is exactly that. 0:12:44.000 --> 0:12:47.599 Janet Yellen, who serves as President Joe Biden's Treasury secretary, 0:12:47.880 --> 0:12:51.960 caught it a soft landing, no pun intended. In other words, 0:12:52.240 --> 0:12:57.760 if the economy is a plane, it didn't crash. So 0:12:57.840 --> 0:13:00.760 what does all this mean about Palm's argument? How can 0:13:00.760 --> 0:13:02.720 we make sense of the data we have and the 0:13:02.760 --> 0:13:04.920 stories economists are telling us about it. 0:13:05.400 --> 0:13:07.120 Maybe one way to think about it is, if you're 0:13:07.120 --> 0:13:10.280 going to extend the flying analogy, it's terrible weather and 0:13:10.280 --> 0:13:14.520 it's cloudy, and it's raining, and there's wind from multiple directions, 0:13:14.520 --> 0:13:17.320 and they're landing in an area with a lot of 0:13:17.400 --> 0:13:20.400 snow and a valley. It's really tough to know what's 0:13:20.440 --> 0:13:23.960 going on and what's striking is the degree of narratives 0:13:24.000 --> 0:13:25.680 that I could tell you right now, but what's happening 0:13:25.679 --> 0:13:27.600 with the economy. I could tell you a story about 0:13:27.600 --> 0:13:30.640 how inflation is coming down and the labor market is 0:13:30.679 --> 0:13:33.240 still robust and we're on pace for self landing. I 0:13:33.280 --> 0:13:36.680 could say there are certain measures of inflation that aren't 0:13:36.720 --> 0:13:38.720 coming down as much, and there are signs that the 0:13:38.840 --> 0:13:42.520 labor market is actually weakening. I could say, look at 0:13:42.520 --> 0:13:46.160 what's going on with financial market speculation and say, look, actually, 0:13:46.200 --> 0:13:50.160 we haven't extinguished the inflationary embers at all in this economy. 0:13:50.440 --> 0:13:54.880 And so any one of those narratives, someone could convincingly 0:13:54.920 --> 0:13:57.800 make the case it is extremely hard for the FED 0:13:57.840 --> 0:13:59.120 to really know what's going on. 0:13:59.480 --> 0:14:02.960 Yeah, there was no uncertainty, there would be no market basically, 0:14:03.040 --> 0:14:06.280 and not to labor the flying analogy, but I think 0:14:06.320 --> 0:14:09.640 the trick is that, yes, it's stormy outside, but you're 0:14:09.640 --> 0:14:12.280 flying a plane. You have all these different indicators. You know, 0:14:12.320 --> 0:14:15.720 you can look out the windscreen and see what the 0:14:15.760 --> 0:14:18.480 weather actually looks like. You can look at your instruments 0:14:18.520 --> 0:14:21.320 and measure a wind shear or whatever. You sort of 0:14:21.320 --> 0:14:24.320 have to figure out which of your instruments to listen 0:14:24.400 --> 0:14:28.320 to at this moment in time, and it's tricky because 0:14:28.360 --> 0:14:32.000 it's not the usual flying environment. Gosh, I'm getting sick 0:14:32.000 --> 0:14:35.400 of this analogy, but it is a weird business cycle. 0:14:35.720 --> 0:14:37.040 Going back to what we were saying. 0:14:36.880 --> 0:14:41.160 Earlier, Some was very clear in her article the Federal 0:14:41.200 --> 0:14:42.960 Reserve is doing the best it can. 0:14:44.080 --> 0:14:47.680 We're trying to get a sense on a twenty trillion 0:14:47.760 --> 0:14:50.680 dollar plus economy with you know, one hundreds of millions 0:14:50.680 --> 0:14:53.920 of people working, and we're like trying to measure a 0:14:53.960 --> 0:14:54.720 moving target. 0:14:55.200 --> 0:14:57.880 But she does think the government overall could do more 0:14:57.920 --> 0:15:00.440 to restore trust so that people are more willing to 0:15:00.480 --> 0:15:04.040 respond to surveys, and she's also been involved in efforts 0:15:04.040 --> 0:15:06.800 to bridge the gap between that hard and soft data 0:15:06.840 --> 0:15:09.840 that Joe mentioned by relying on both. 0:15:10.280 --> 0:15:14.520 There are ways to use administrative data where you could 0:15:14.600 --> 0:15:17.000 put together like surveys where we ask people things. It 0:15:17.000 --> 0:15:19.800 would be really hard to go measure somewhere else, But 0:15:19.880 --> 0:15:23.880 then maybe from the Internal Revenue Service, we know they're 0:15:24.000 --> 0:15:27.440 income and it's definitely easier than figuring out how to 0:15:27.480 --> 0:15:30.320 get people to trust the government more. 0:15:31.040 --> 0:15:34.119 Some says there's an urgent need to address these problems 0:15:34.480 --> 0:15:35.840 before they get worse. 0:15:36.400 --> 0:15:39.000 Statisticians have looked at this, and you know people that 0:15:39.080 --> 0:15:42.280 research in this area, and they still feel comfortable with 0:15:42.400 --> 0:15:46.360 the degree of quality accuracy, Like there are ways to 0:15:46.400 --> 0:15:49.640 get a sense of the reliability, and they're still in 0:15:49.640 --> 0:15:52.960 a place where it's like, Okay, we feel comfortable with these, 0:15:53.160 --> 0:15:57.160 and yet survey response rates that continue to go down. Right, 0:15:57.160 --> 0:16:00.640 at some point, you cross a threshold of being reliable. 0:16:06.720 --> 0:16:09.320 Thanks for listening to The Big Take DC podcast from 0:16:09.360 --> 0:16:14.080 Bloomberg News. I'm Seleiah Mosen. This episode was produced by 0:16:14.080 --> 0:16:18.080 Alex Suguiera, Julia Press, and Naomi Shaven. It was fact 0:16:18.160 --> 0:16:21.760 checked by Stacy Renee. A special thanks to Kate Davidson 0:16:21.840 --> 0:16:25.120 and Matt Bosler. Blake Maples is our mixed engineer, and 0:16:25.200 --> 0:16:29.080 our story editors are Michael Shepherd and Wendy Benjaminson. Nicole 0:16:29.120 --> 0:16:32.360 Beemster Bower is our executive producer. Sage Bauman is our 0:16:32.400 --> 0:16:35.440 head of podcasts. If you like what you heard, please 0:16:35.480 --> 0:16:38.360 be sure to subscribe, rate, and review the show. It'll 0:16:38.360 --> 0:16:39.920 help other listeners find us. 0:16:40.320 --> 0:16:42.520 Thanks for tuning in. I'll be back next week.