WEBVTT - The internet sleuths fighting fake research 0:00:00.960 --> 0:00:06.000 First of all, I'm so curious about your alias smart Clyde. 0:00:06.360 --> 0:00:08.560 To work out your name as a porn star, you 0:00:08.640 --> 0:00:13.360 take the name of your first pet and the street 0:00:13.400 --> 0:00:14.840 on which you first lived. 0:00:16.360 --> 0:00:19.119 So this is Sivanae producer Shane Anderson talking to a 0:00:19.160 --> 0:00:23.079 guy who goes by the moniker of smut Clyde. He's 0:00:23.079 --> 0:00:24.800 not an actual porn star, by the way. 0:00:24.960 --> 0:00:26.680 So you had a pet called Smut. 0:00:27.200 --> 0:00:32.080 Yes, she was a tortoise shell who was covered all 0:00:32.120 --> 0:00:35.320 over on black smuts, as if she'd fallen down the chimney. 0:00:35.920 --> 0:00:39.720 I guess my crimate fighting activity is usually associated with 0:00:39.800 --> 0:00:42.599 the smut Clyde name, so it might be easier if 0:00:42.920 --> 0:00:44.720 that becomes the main I tinderfire. 0:00:45.280 --> 0:00:48.560 So smut Clyde is a science sleuth and his crime 0:00:48.600 --> 0:00:53.760 fighting activity is combating a growing phenomenon of fake science papers. 0:00:54.040 --> 0:00:56.639 And how much of your time do you think it takes. 0:00:56.480 --> 0:00:59.279 Up tier week? Maybe the here week it's getting gone 0:00:59.280 --> 0:01:01.120 to find them time unpaid. 0:01:00.840 --> 0:01:10.480 Job from Shawes Media. I'm Rick Morton and this is 0:01:10.480 --> 0:01:19.959 seven AM. Academic journals pretty much around the world and 0:01:20.000 --> 0:01:24.040 across all disciplines are being flooded with fake submissions. They're 0:01:24.040 --> 0:01:27.520 typically generated with the help of artificial intelligence by paper mills, 0:01:27.840 --> 0:01:31.680 a cottage industry relying on overworked and desperate researchers to 0:01:31.800 --> 0:01:34.679 fuel their business model, and then more than just a usance. 0:01:35.000 --> 0:01:38.480 The infiltration of fake papers is so bad it's undermining 0:01:38.520 --> 0:01:41.679 the work of scientists and critical fields like cancer research. 0:01:42.760 --> 0:01:45.959 Today seven AM producer Shane Anderson on the cost of 0:01:46.000 --> 0:01:49.360 this research fraud and the science detectives who are trying 0:01:49.360 --> 0:02:06.760 to fight it. That's coming up after the break. Shane, 0:02:06.840 --> 0:02:09.880 you've been looking into the issue of fake scientific papers. 0:02:10.720 --> 0:02:12.520 I have to ask, how did you get interested in 0:02:12.560 --> 0:02:13.200 it to begin with? 0:02:13.639 --> 0:02:16.919 I guess it started when I was sick a couple 0:02:16.919 --> 0:02:19.600 of weeks ago, and so, you know, in the process 0:02:19.639 --> 0:02:23.160 of spending way too much time on the internet looking 0:02:23.200 --> 0:02:27.360 at science articles, I started to notice that there were 0:02:27.600 --> 0:02:32.040 a lot of scientists online who were trying to sound 0:02:32.040 --> 0:02:35.200 the alarm about something strange happening in the field of 0:02:35.240 --> 0:02:41.080 science research. They were noticing a huge uptick essentially in 0:02:41.440 --> 0:02:46.520 bunk research. So fake science papers were proliferating academia. 0:02:46.600 --> 0:02:50.200 There is a scientific fraud epidemic. Every week. There seems 0:02:50.200 --> 0:02:52.600 to be so many retractions, There seems to be so 0:02:52.680 --> 0:02:54.160 many scandals, fraudum. 0:02:54.480 --> 0:02:57.840 And so that was how I stumbled onto this community 0:02:58.000 --> 0:03:02.000 of people who are trying to science. And I guess 0:03:02.080 --> 0:03:05.320 that story starts with a guy called smut Clyde. 0:03:05.760 --> 0:03:09.080 You can basically pay someone to make you a never 0:03:09.160 --> 0:03:12.560 plane out of bamboo and sticks and twine, and it's 0:03:12.560 --> 0:03:15.919 not a real ever plane, but you still get promoted 0:03:15.960 --> 0:03:19.120 for it. It still counts on your academic record. 0:03:19.560 --> 0:03:23.000 Smut Clyde is the pseudonym of a guy called David Bimler. 0:03:23.720 --> 0:03:28.679 He's a retired psychologist previously associated with Massive University in Wellington, 0:03:28.760 --> 0:03:31.680 which is where he lives. He's a really lovely guy. 0:03:31.760 --> 0:03:35.200 He's got you know, long long haired, long beard, lives 0:03:35.200 --> 0:03:39.080 with his cats, and he is one of these what 0:03:39.120 --> 0:03:43.520 they call sleuths. He spends a couple of hours every 0:03:43.640 --> 0:03:48.560 day on his computer at home scouring science journals looking 0:03:48.680 --> 0:03:52.119 for signs of suspicious looking research. 0:03:52.640 --> 0:03:55.360 Okay, so I'm getting Scooby Doo vibes. They've got a 0:03:55.360 --> 0:03:59.320 guy called Smutt Clyde and he's looking for clues. What 0:03:59.480 --> 0:04:02.400 is he looking for exactly? What does suspicious research look like? 0:04:02.920 --> 0:04:06.720 Yeah, so Smart gave me a couple of examples of 0:04:06.760 --> 0:04:08.760 the kinds of things that he's looking for. 0:04:12.200 --> 0:04:16.680 First of all, they were really cycling the occasional image. 0:04:17.160 --> 0:04:20.160 So, for example, there are these things called Western blots, 0:04:20.240 --> 0:04:25.880 which are essentially this technique that's used to interpret bands 0:04:25.880 --> 0:04:30.000 of protein, so they images. A lot of the time, 0:04:30.160 --> 0:04:34.640 he'll be looking at these Western blots and he'll notice 0:04:34.680 --> 0:04:38.239 signs that they've been photoshopped, or he'll notice that they're 0:04:38.440 --> 0:04:41.960 identical to a Western blot that is in a paper 0:04:42.000 --> 0:04:45.560 that's from a totally different author, supposedly on a totally 0:04:45.560 --> 0:04:50.240 different subject. Another thing he looks out for is experiments 0:04:50.240 --> 0:04:51.960 that have odd features. 0:04:52.839 --> 0:05:00.120 One paper talked about keeping the mice understandard thick non celsius. 0:04:59.480 --> 0:05:02.680 Condition, which was an experiment that claimed to have treated 0:05:02.760 --> 0:05:06.880 cancer in lab mice kept at thirty nine degrees celsius. 0:05:07.920 --> 0:05:11.839 It's fine if you're keeping cells in the incubator. But 0:05:11.920 --> 0:05:14.640 I put mice at thirty nine degrees, they will not 0:05:14.720 --> 0:05:15.400 live very long. 0:05:16.120 --> 0:05:21.760 So when SMART finds one of these suspicious papers, often 0:05:21.800 --> 0:05:24.760 what they'll do is theyll alert the author to it, 0:05:24.960 --> 0:05:28.640 or they'll alert the publisher, and if it is as 0:05:28.680 --> 0:05:32.640 they suspect, fake science, then ideally the article is then 0:05:32.680 --> 0:05:36.680 retracted and SMART isn't the only person doing this. He's 0:05:37.080 --> 0:05:39.800 part of a tight knit community of sleuths who are 0:05:39.839 --> 0:05:44.000 all themselves scientifically literate. A lot of them are working researchers, 0:05:44.440 --> 0:05:47.240 and they all band together to draw attention to this 0:05:47.440 --> 0:05:50.560 growing problem of essentially fake research. 0:05:52.440 --> 0:05:56.320 Do you feel that science needs more of an immune system? 0:05:56.520 --> 0:05:59.680 In their view, science is sick. There's an epidemic of 0:05:59.720 --> 0:06:04.080 fake research infecting the scientific record, and they see themselves 0:06:04.120 --> 0:06:11.920 as the unofficial, unpaid immune system, and they're really good 0:06:11.920 --> 0:06:15.080 at it. In fact, Smart and a few other sleuths 0:06:15.480 --> 0:06:19.320 managed to uncover this network of three hundred fake papers 0:06:19.320 --> 0:06:22.760 that were all reusing the same data sets, and these 0:06:22.800 --> 0:06:25.720 papers had all been published by one of the biggest 0:06:25.760 --> 0:06:30.120 and most reputable academic publishers, Wiley. So John Wiley and 0:06:30.200 --> 0:06:34.400 Sons this is one of the leading academic publishers. That 0:06:34.440 --> 0:06:38.560 network has since grown to twelve hundred retractions all from 0:06:38.600 --> 0:06:42.320 these special issues, and just last month Wiley announced that 0:06:42.360 --> 0:06:45.479 they were actually going to shudder nineteen of the journals 0:06:45.760 --> 0:06:48.520 as part of the fallout from this discovery, and they 0:06:48.680 --> 0:06:52.839 estimated the closures will cost them around forty million US dollars. 0:06:53.480 --> 0:06:55.240 I mean, honestly, the scale of those blowing my mind. 0:06:55.240 --> 0:06:58.920 It's a huge problem obviously, but how big are we 0:06:59.000 --> 0:06:59.800 talking That. 0:07:00.520 --> 0:07:02.560 Part of what people are trying to figure out. So 0:07:02.640 --> 0:07:06.240 depending on what estimates you use, there are around kind 0:07:06.279 --> 0:07:10.040 of three to four million science papers published each year, 0:07:10.680 --> 0:07:15.120 and the vast, vast majority are good, sound science. But 0:07:15.440 --> 0:07:19.160 against that, the rate of retractions of papers has never 0:07:19.200 --> 0:07:23.040 been higher. So Nature, which is kind of the world's 0:07:23.120 --> 0:07:28.400 leading science journal, they have calculated that last year alone 0:07:28.840 --> 0:07:32.360 there were ten thousand retractions, which is more than double 0:07:32.680 --> 0:07:35.480 the amount for the year before. The thing is that 0:07:35.600 --> 0:07:39.480 bad science is by no means a new problem. You're 0:07:39.600 --> 0:07:42.280 probably aware of some of the more kind of notorious 0:07:42.320 --> 0:07:46.640 examples of bad science, people like Andrew Wakefield, who's the 0:07:47.160 --> 0:07:52.440 discredited doctor who was pushing now very retracted research into 0:07:52.760 --> 0:07:55.320 links between vaccines and autism. 0:07:55.000 --> 0:07:57.200 That was published in the British Medical Journal as well, 0:07:57.360 --> 0:07:59.040 like one of the most esteemed in the world. 0:07:59.280 --> 0:08:02.480 Exactly, this was a bad actor who was specifically using 0:08:02.560 --> 0:08:08.440 this and to essentially launder their political views. But it's 0:08:08.480 --> 0:08:11.000 a little bit different to what we're experiencing now with 0:08:11.120 --> 0:08:15.560 this wave of retractions, and many sluice feel that that 0:08:15.720 --> 0:08:18.480 number of ten thousand is really only the tip of 0:08:18.480 --> 0:08:22.280 the iceberg, because what we're seeing now is this kind 0:08:22.320 --> 0:08:25.480 of rise of mass produced papers of just straight up 0:08:25.520 --> 0:08:28.720 fake science. And so unlike Andrew Wakefield, where you've got 0:08:28.720 --> 0:08:31.280 this clear villain that you can point to and say, 0:08:31.960 --> 0:08:35.240 you know, this research is causing people harm, the brunt 0:08:35.240 --> 0:08:39.280 of this infection is much more quiet and more insidious. 0:08:40.160 --> 0:08:43.560 All it takes is one piece of this fake science 0:08:44.240 --> 0:08:48.000 to derail real research. And so a lot of Smart's 0:08:48.040 --> 0:08:52.200 work is looking at fake science in biomedicine, which is 0:08:52.400 --> 0:08:54.440 a lot of it has to do with cancer research. 0:08:54.600 --> 0:08:57.160 So this is kind of areas where human health is 0:08:57.200 --> 0:09:00.800 at stake. And he explained to me that sometimes these 0:09:00.840 --> 0:09:05.160 fake papers get mixed in with the real and they 0:09:05.200 --> 0:09:09.880 create this framework of knowledge that doesn't actually exist. So 0:09:10.520 --> 0:09:14.400 anyone down the line who reads that and bases real 0:09:14.440 --> 0:09:22.160 clinical research off it is essentially bound to fail. This 0:09:22.320 --> 0:09:26.000 is not only incredibly frustrating for scientists to have their 0:09:26.040 --> 0:09:29.360 experiment fail and not know why, but it's also really 0:09:29.440 --> 0:09:32.960 costly and time consuming in a field where cancer research 0:09:33.120 --> 0:09:36.600 is already very costly and time consuming. So you can 0:09:36.640 --> 0:09:39.679 see how one tiny piece of fake research can have 0:09:39.760 --> 0:09:43.760 an enormous ripple effect. But this isn't just one paper. 0:09:44.040 --> 0:09:47.439 It's tens of thousands that we know of, and many 0:09:47.480 --> 0:09:50.280 more out there that haven't been detected yet. And so 0:09:50.400 --> 0:09:54.520 many people I spoke to said that the accumulated weight 0:09:54.640 --> 0:09:58.320 of these fake papers can threaten the integrity of all 0:09:58.400 --> 0:10:01.840 scientific research time time when we really need to trust 0:10:01.880 --> 0:10:02.480 it the most. 0:10:06.880 --> 0:10:23.080 Coming up after break, what's driving this epidemic of ferny research, Shane, 0:10:23.120 --> 0:10:25.920 We've been talking about the explosion of retractions in the 0:10:25.960 --> 0:10:29.360 scientific literature. Where exactly are these dodgy papers coming from. 0:10:29.760 --> 0:10:34.679 They're not from nothing, iicm no, And to better understand 0:10:35.000 --> 0:10:37.600 where these papers are coming from, I went and spoke 0:10:37.640 --> 0:10:41.640 to a guy called Ivan Ransky. He's Distinguished Journalist in 0:10:41.679 --> 0:10:45.000 Residence at the Arthur el Kada Journalism Institute at New 0:10:45.080 --> 0:10:48.880 York University. But more importantly, he's also the co founder 0:10:48.920 --> 0:10:50.679 of a website called retraction Watch. 0:10:50.960 --> 0:10:55.280 The sort of reason why retraction Watch exists is because 0:10:55.320 --> 0:10:58.520 there was a particularly bad case here in the States 0:10:59.200 --> 0:11:02.560 that person was making up the data and clinical trinds 0:11:03.040 --> 0:11:03.800 of the painkiller. 0:11:04.240 --> 0:11:05.800 One of the main things they do is keep a 0:11:05.920 --> 0:11:09.720 database of retracted articles, which is currently sitting at fifty thousand, 0:11:09.760 --> 0:11:10.959 which is pretty incredible. 0:11:11.480 --> 0:11:15.440 So it's not a new problem. It's definitely more visible now, 0:11:15.960 --> 0:11:17.719 which is a good thing, but it also means it's 0:11:17.720 --> 0:11:21.000 a very painful time for science. It's a painful time 0:11:21.080 --> 0:11:23.839 for publishers, a painful time for universities that are dealing 0:11:23.840 --> 0:11:24.520 with these issues. 0:11:25.000 --> 0:11:29.280 He says there's a few factors driving this epidemic of 0:11:29.320 --> 0:11:34.199 fake papers. One is the real epidemic, the COVID nineteen pandemic. 0:11:34.640 --> 0:11:38.439 Cases like ivermectin and hydroxychloroquine made us a lot more 0:11:38.480 --> 0:11:42.760 aware of how easily misinformation can spread. But he says 0:11:42.800 --> 0:11:46.840 that there's another factor, which is the growth of this 0:11:47.400 --> 0:11:51.080 entire black market of companies who are behind a lot 0:11:51.120 --> 0:11:54.240 of these fake papers. And these are the paper mills. 0:11:54.200 --> 0:11:57.920 These operations, these shady operations that sell papers to people 0:11:57.960 --> 0:12:01.079 that make things up to make fake papers, that sell authorship, 0:12:01.200 --> 0:12:03.560 you know, all the things you need in order to 0:12:03.559 --> 0:12:06.280 succeed as an academic, in order to get a job 0:12:06.360 --> 0:12:08.439 and grants and promotions and everything like that. 0:12:08.840 --> 0:12:13.520 They're essentially companies where for a fee, they will generate 0:12:13.559 --> 0:12:16.160 a science paper and get it published in a journal 0:12:16.240 --> 0:12:19.880 on your behalf and there's this whole economy around this, 0:12:20.000 --> 0:12:22.120 where you pay more to be listed as an author 0:12:22.200 --> 0:12:24.920 higher on the article, or you pay more to be 0:12:25.000 --> 0:12:28.600 published in a high quality journal, for example. And these 0:12:28.679 --> 0:12:35.160 mills also take advantage of the way journals vet papers. Basically, 0:12:35.600 --> 0:12:38.920 if you submit a paper, your research won't get published 0:12:39.000 --> 0:12:42.280 until it passes the peer review process. The problem is 0:12:42.480 --> 0:12:45.640 these peer reviewers are often working for free and they're 0:12:45.679 --> 0:12:46.880 often very busy. 0:12:47.280 --> 0:12:51.920 When you have been told for decades that this is 0:12:52.000 --> 0:12:55.480 the way that peer review like a good housekeeping seal 0:12:55.520 --> 0:12:59.520 of approval, then you find out that in fact it misses, 0:13:00.240 --> 0:13:04.120 maybe more often than it actually finds the problems and 0:13:04.240 --> 0:13:08.679 screens the problems properly, you start to i think justifiably 0:13:09.200 --> 0:13:10.520 lose trust in that system. 0:13:10.960 --> 0:13:13.360 And one study that came out a couple of years ago, 0:13:13.640 --> 0:13:16.679 they actually worked with a bunch of journals and they 0:13:16.720 --> 0:13:20.400 found that on average, around two percent of the papers 0:13:20.400 --> 0:13:24.160 submitted will be junk science. But if a paper slips 0:13:24.160 --> 0:13:27.520 through peer review and then gets published, the number of 0:13:27.679 --> 0:13:30.520 fake science that will be submitted to that journal can 0:13:30.600 --> 0:13:33.640 rise to up to forty six percent. And what these 0:13:33.720 --> 0:13:37.160 loose have done that's incredibly clever is that They've recognized 0:13:37.240 --> 0:13:40.560 shared features in a lot of these fake science articles, 0:13:40.880 --> 0:13:44.920 and through that have actually been able to identify networks 0:13:44.960 --> 0:13:49.000 of hundreds of papers linked to common features and basically 0:13:49.240 --> 0:13:51.920 unmask where some of these paper mills are coming from. 0:13:52.120 --> 0:13:55.920 That's wild. I mean, what can we actually do about it? 0:13:55.960 --> 0:13:59.760 I mean, this is a global phenomenon. Surely do we 0:13:59.760 --> 0:14:02.120 have a role to play here in Australia about beoliefing it. 0:14:02.600 --> 0:14:06.160 There's a bunch of solutions that people have been growing about. 0:14:06.360 --> 0:14:11.560 So last year, South Australia's Independent Commissioner against Corruption, Bruce Lander, 0:14:12.120 --> 0:14:15.920 he recommended actually having a body that could deal with 0:14:15.960 --> 0:14:20.440 specific cases and also equipping the Australian Research Council with 0:14:20.560 --> 0:14:23.480 kind of legislative powers so it could be easier to 0:14:24.240 --> 0:14:28.040 try cases of extreme misconduct in the court system, for example. 0:14:28.440 --> 0:14:32.240 But I mean, ultimately that's not really going to be 0:14:32.280 --> 0:14:35.840 equipped to deal with the problem of paper mills that 0:14:35.880 --> 0:14:40.400 are based internationally infiltrating research that happens in Australia. Like, 0:14:40.440 --> 0:14:43.160 it's just not set up to deal with that. Everyone 0:14:43.240 --> 0:14:45.920 I spoke to agreed that the best way to fix 0:14:46.000 --> 0:14:49.320 this was to change the conditions that allowed these mills 0:14:49.360 --> 0:14:54.840 to flourish. So solving this problem of fake science really 0:14:54.880 --> 0:14:57.960 is about looking at the bigger picture about why this 0:14:58.120 --> 0:15:02.880 market exists. It's because of the way academia is set up, 0:15:03.000 --> 0:15:07.680 so academics have to publish articles to get grants, promotions, 0:15:08.200 --> 0:15:09.960 all of the things you need not just to kind 0:15:10.000 --> 0:15:13.000 of have a good career, but to have any career 0:15:13.160 --> 0:15:16.160 in science. You can't get grant money so you can 0:15:16.240 --> 0:15:19.720 chase a cure for cancer. None of that really happens 0:15:20.080 --> 0:15:25.840 without publications. So this is a system that's incentivizing at best, 0:15:26.000 --> 0:15:30.120 science that's rushed and under pressure, and at worst, it's 0:15:30.360 --> 0:15:32.120 creating the market for paper mills. 0:15:32.560 --> 0:15:35.440 To borrow another kind of science analogy, it's survival of 0:15:35.480 --> 0:15:39.080 the fittest, isn't it's natural selection. We've got these pressures 0:15:39.120 --> 0:15:43.600 existing in academia that are forcing, or at least seem 0:15:43.640 --> 0:15:46.160 to be forcing. Research is down a certain path, which 0:15:46.200 --> 0:15:51.480 is publisher perish. And I guess publisher perish is defining 0:15:51.520 --> 0:15:53.040 the whole system, right it. 0:15:52.960 --> 0:15:55.560 Is, And there is something dull, winny and about it, absolutely, 0:15:55.680 --> 0:15:58.960 And the people I spoke to said that there are 0:15:59.440 --> 0:16:03.000 slowly steps being taken to deal with this, but on 0:16:03.080 --> 0:16:07.760 the whole publishers and universities have been really slow to 0:16:07.880 --> 0:16:11.520 respond to the problem. The more that we ignore it, 0:16:12.200 --> 0:16:15.640 the more damage we're doing down the line, where essentially 0:16:15.920 --> 0:16:20.840 trust in the entire scientific project is being eroded. And 0:16:20.880 --> 0:16:23.560 when we don't have trust in the scientific project, then 0:16:23.800 --> 0:16:27.080 it's much easier for people to weaponize the distrust, and 0:16:27.240 --> 0:16:31.040 we kind of come back to people who actively weaponize science, 0:16:31.240 --> 0:16:34.760 like you know, the vaccine myths and the COVID misinformation. 0:16:37.600 --> 0:16:40.200 At the end of the day, you know, science isn't perfect, 0:16:40.360 --> 0:16:44.200 and it's also never set out to obtain an absolute truth. 0:16:44.240 --> 0:16:46.560 You know, mistakes are made, and a lot of them 0:16:46.560 --> 0:16:49.800 are genuine. But science is a process and it's set 0:16:49.880 --> 0:16:53.360 up to check and to correct itself. But right now 0:16:53.680 --> 0:16:56.960 those systems of correction aren't quite holding up the way 0:16:56.960 --> 0:16:57.360 they used to. 0:17:01.920 --> 0:17:03.720 Seane, thank you so much for taking the time to 0:17:03.760 --> 0:17:04.480 talk to us today. 0:17:04.680 --> 0:17:05.000 Thank you. 0:17:05.080 --> 0:17:22.240 Rick. Also in the news today, first term Senator Fatima 0:17:22.280 --> 0:17:24.960 Payman has now she's leaving the Labor Party to sit 0:17:25.040 --> 0:17:28.400 as an independent. Speaking to the media, Senator payment said 0:17:28.440 --> 0:17:31.600 she was deeply torn over the decision and still supported 0:17:31.640 --> 0:17:33.960 the values of the Labor Party. They could see no 0:17:34.040 --> 0:17:37.000 middle ground when it came to conforming to caucous solidarity 0:17:37.560 --> 0:17:41.480 over the recognition of Palestinian statehood. The senator confirmed she 0:17:41.480 --> 0:17:45.760 will not be joining any Muslim community linked party and 0:17:46.359 --> 0:17:49.440 police have arrested four people after a pro Palestinian demonstration 0:17:49.520 --> 0:17:52.840 on the roof of Parliament House in Canberra. Protesters from 0:17:52.840 --> 0:17:55.760 the group Renegade Activists climbed onto the roof of Parliament 0:17:55.760 --> 0:17:59.720 House to unfurl banners accusing Australia of complicity in war 0:17:59.760 --> 0:18:04.480 crime and jenocide. Seven Am is a daily show from 0:18:04.480 --> 0:18:08.960 Schwartz Media and The Saturday Paper. It's produced by Kara Jensen, McKinnon, 0:18:09.040 --> 0:18:12.440 Shane Anderson and Zalton Fetcher. Our senior producer is Chris 0:18:12.520 --> 0:18:17.440 dan Gaden. Our technical producer Zaticus Bastow. Edited by Scott Mitchell, 0:18:17.520 --> 0:18:21.240 Chris Dangate and Sarah mcveee. Our head of audio, Eric Jensen, 0:18:21.280 --> 0:18:24.359 is our editor in chief. Mixing by Travis Evans and 0:18:24.359 --> 0:18:27.960 Atticus Bastow. Our theme music is by Ned Beckley and 0:18:28.040 --> 0:18:32.840 Josh Hogan of Envelope Audio. I'm Rick Morton, thanks for listening.