WEBVTT - The Powerball Revolution 0:00:15.356 --> 0:00:15.796 Pushkin. 0:00:20.276 --> 0:00:23.076 One day, not long ago, I went to visit the 0:00:23.156 --> 0:00:28.916 Lawrenceville School just outside Princeton, New Jersey, a very old, 0:00:29.036 --> 0:00:35.076 prestigious private school, leafy campus, Gothic this and that. I 0:00:35.116 --> 0:00:37.476 asked one of their deans could I come and meet 0:00:37.476 --> 0:00:41.876 with some students? She said yes, Does everyone have a 0:00:41.876 --> 0:00:45.116 piece of paper and pen? The dean put together a 0:00:45.116 --> 0:00:48.156 group of twenty kids. We met in a conference room 0:00:48.196 --> 0:00:50.676 looking out on the campus lawn. On a scale of 0:00:50.676 --> 0:00:52.836 one at ten, don't put your name on it. I 0:00:52.836 --> 0:00:56.836 would like you to rank your happiness with the Lawrenceville 0:00:56.876 --> 0:00:59.556 School's current electoral process. 0:01:00.076 --> 0:01:01.636 Ten is you think it's perfect? 0:01:02.636 --> 0:01:05.876 One is you think it's maybe time to bring in 0:01:05.956 --> 0:01:13.076 Adam Adam Adam Cronkwright. He was the reason I was there. Okay, 0:01:14.156 --> 0:01:18.236 hi everyone, thanks for coming. My name is Malcolm Babo, 0:01:18.636 --> 0:01:21.636 so you as you know, I am a writer and 0:01:21.716 --> 0:01:29.476 I have a podcast called Revision's History. That's how the 0:01:29.516 --> 0:01:32.716 afternoon began. I said, I wanted to run an idea 0:01:32.796 --> 0:01:35.836 by them, an idea field tested by this person. I 0:01:35.996 --> 0:01:40.476 just met Adam kronkright about talent and chance and how 0:01:40.516 --> 0:01:44.316 we choose others to lead us. And I would encourage 0:01:44.356 --> 0:01:47.876 you all to be. If you think I'm crazy, you 0:01:47.876 --> 0:01:52.196 should say, Malcolm, you're crazy. Maybe this idea could change 0:01:52.196 --> 0:01:54.116 a lot about what's wrong with the world right now, 0:01:54.356 --> 0:01:59.356 or maybe not. Revisionist history is about the overlooked and misunderstood. 0:01:59.876 --> 0:02:03.036 Usually I decide what belongs in either of those categories. 0:02:03.956 --> 0:02:06.476 This time, I thought I'd just run Adam's idea by 0:02:06.516 --> 0:02:14.556 the students of Lawrenceville and that them decide. All this 0:02:14.676 --> 0:02:17.396 had to do with the way they elected their student council. 0:02:18.116 --> 0:02:21.876 I thought they should call a constitutional convention and start over. 0:02:22.796 --> 0:02:25.076 There is no right answer or wrong answer, but I 0:02:25.116 --> 0:02:27.276 want to make a case for something and see how 0:02:27.756 --> 0:02:32.116 that possibly relates to the way you guys do things 0:02:32.156 --> 0:02:36.396 here at Lawrenceville. They listened to Adam's idea argued about 0:02:36.436 --> 0:02:39.116 it for close to two hours. I'll be honest, I 0:02:39.156 --> 0:02:42.156 didn't think I'd get anywhere. The whole student council was 0:02:42.156 --> 0:02:44.636 there as well, all people who had profited from the 0:02:44.716 --> 0:02:48.636 Lawrenceville electoral status quo. And what was wrong with the 0:02:48.636 --> 0:02:54.076 status quo in theory, not much. Lawrenceville elected its leaders 0:02:54.276 --> 0:02:58.796 through the perfect democratic system, campaign speeches, open elections. Their 0:02:58.796 --> 0:03:00.716 constitution might as well have been drawn up by the 0:03:00.756 --> 0:03:03.436 founding Fathers. For all I know, one of the Founding 0:03:03.436 --> 0:03:08.876 fathers went to Lawrenceville. But then at the end of 0:03:08.876 --> 0:03:13.436 the day afternoon, I said, okay, now that I've given 0:03:13.436 --> 0:03:16.676 you an alternative, how happy are you with what you have? 0:03:17.716 --> 0:03:20.796 Scale of one to ten. One is totally miserable with 0:03:20.836 --> 0:03:24.276 the status quo, and ten is delighted. Everything's perfect. 0:03:25.196 --> 0:03:27.276 I want to see your number, rip it off and 0:03:27.676 --> 0:03:29.316 hand it to me. Open to day. 0:03:36.036 --> 0:03:38.476 As the students handed in their little slips of paper, 0:03:38.996 --> 0:03:42.876 I glanced at a few. I was taken aback. Lawrenceville 0:03:42.956 --> 0:03:47.276 had the perfect democratic system, and upon reflection, upon exposure 0:03:47.316 --> 0:03:50.476 to a little of Adam kronk Write's subversion, there were 0:03:50.516 --> 0:03:52.516 a lot of students in that room who decided they 0:03:52.556 --> 0:03:55.316 didn't like it anymore. I told them I'd do an 0:03:55.316 --> 0:04:02.516 official tally later after I got home. But there's some 0:04:02.716 --> 0:04:08.036 unhappiness in this room, and if they seem discontented with 0:04:08.116 --> 0:04:12.116 democracy in the gilded corridors of the Lawrenceville School, I 0:04:12.116 --> 0:04:21.476 can only imagine what it's like elsewhere. I first ran 0:04:21.516 --> 0:04:25.836 across Adam Kronkwrte online. I was just rummaging around Google 0:04:25.876 --> 0:04:28.236 one day, and what he was doing. Struck me is 0:04:28.276 --> 0:04:31.836 so interesting that I tracked him down cold called him. 0:04:32.476 --> 0:04:35.476 Are you ever in New York City? This was November. 0:04:35.916 --> 0:04:38.916 He said yes, he'd be there in February. So he 0:04:38.996 --> 0:04:41.556 settled on a date and he came to my apartment. 0:04:42.236 --> 0:04:47.156 Young guy, beard, big backpack, hiking boots. So tell me, actually, 0:04:47.196 --> 0:04:50.236 before we get into the question, where are you from. 0:04:50.716 --> 0:04:53.396 I'm from just outside of Syracuse, New York. Yeah, so 0:04:53.476 --> 0:04:54.236 central New York. 0:04:55.076 --> 0:04:56.956 How did you end up in Bolivia? 0:04:57.156 --> 0:05:00.076 Coach of Bamba to be more precise, a mid size 0:05:00.116 --> 0:05:03.476 city in the mountains of central Bolivia. It turned out 0:05:03.636 --> 0:05:07.676 to be a long story that starts in Canada, across Lake, 0:05:07.756 --> 0:05:09.476 Ontario from his home town. 0:05:09.876 --> 0:05:11.676 I remember where I was when I thought of the idea, 0:05:11.796 --> 0:05:15.836 Like you know, where were you I was. I was 0:05:15.876 --> 0:05:22.516 outside Victoria Hall, walking on on Queen's campus and it 0:05:22.596 --> 0:05:23.116 was spring. 0:05:23.716 --> 0:05:26.796 It was two thousand and nine. Adam was finishing college 0:05:26.796 --> 0:05:30.396 at Queen's University in Kingston, Ontario. He wanted to be 0:05:30.436 --> 0:05:33.876 a politician. He had, as he puts it, a pretty 0:05:33.876 --> 0:05:35.156 big hero complex. 0:05:35.596 --> 0:05:38.956 He was save the world and coming to terms with 0:05:39.076 --> 0:05:42.916 this idea that, like you know, be difficult to get 0:05:42.956 --> 0:05:45.276 to the heights of power without haveting to make some 0:05:45.356 --> 0:05:50.676 serious compromises externally internally, and you know, so then I 0:05:50.716 --> 0:05:52.436 was kind of like, Okay, well what am I going 0:05:52.476 --> 0:05:53.516 to do? What are we going to do? 0:05:54.116 --> 0:05:57.716 He began to read history, political science, turned it over 0:05:57.756 --> 0:05:59.876 in his mind, looking for a big idea. 0:06:00.436 --> 0:06:03.316 This is going to you know, right, all the wrongs. 0:06:03.796 --> 0:06:06.276 But I wasn't that involved in anything. 0:06:06.836 --> 0:06:10.356 He was more of a dreamer than a doer. Adam graduated. 0:06:10.596 --> 0:06:13.716 The world was in a financial crisis. Occupy Wall Street 0:06:13.756 --> 0:06:16.036 was one of the protest movements that grew up in response. 0:06:21.996 --> 0:06:24.396 There was a tense city of activists in Szubcatti Park 0:06:24.436 --> 0:06:27.116 in Lower Manhattan. Adam decided to go down to New 0:06:27.196 --> 0:06:31.036 York and join them. There were working panels and panel discussions, 0:06:31.316 --> 0:06:34.876 and one speaker in particular stuck with him, an activist 0:06:34.876 --> 0:06:38.676 from Bolivia who talked about an epic struggle there over water. 0:06:39.796 --> 0:06:43.276 Adam lived in Subcatti Park for five weeks until everyone 0:06:43.356 --> 0:06:48.436 was evicted by the NYPD. His experience with Occupy left 0:06:48.516 --> 0:06:53.636 him frustrated. He understood what they were fighting for, but 0:06:53.756 --> 0:06:57.636 not how. There was no how in Occupy. They didn't 0:06:57.676 --> 0:06:59.836 have a plan that seemed like it could actually reform 0:06:59.916 --> 0:07:03.916 the systems that had failed during the financial crisis. Adam 0:07:03.956 --> 0:07:07.076 moved to Brooklyn, slept on someone's couch, and all the 0:07:07.116 --> 0:07:10.596 while kept skyping with his new friend Bolivia, the water 0:07:10.716 --> 0:07:12.516 activist Marcella Olivella. 0:07:13.716 --> 0:07:15.316 I kind of shared with her that I was feeling 0:07:15.316 --> 0:07:17.596 a bit burnt out, disillusion from what I was seeing, 0:07:17.596 --> 0:07:19.236 and she said, well, yeah, I come down to Bolivia 0:07:19.276 --> 0:07:21.356 and check out social moments there. 0:07:21.996 --> 0:07:25.876 So he did Syracuse to Kingston, to Zuccotti Park to Bolivia. 0:07:26.236 --> 0:07:28.356 He made his way to coach A Bomba and there 0:07:28.396 --> 0:07:31.596 he met Marcella's brother, Oscar, who runs a foundation. 0:07:32.156 --> 0:07:35.196 So I was working with them for a while, and 0:07:35.236 --> 0:07:37.916 then Oscar one day he was in the car and 0:07:37.916 --> 0:07:40.076 he's asking me, what are you into, you know? And 0:07:40.116 --> 0:07:44.516 so I started telling him about democratic lotteries and he 0:07:44.556 --> 0:07:45.516 had never heard of the. 0:07:45.436 --> 0:07:53.076 Concept democratic lotteries. That's Adam's idea, replace elections with lotteries. 0:07:53.676 --> 0:07:57.036 Adam tells Oscar about it. Oscar says, we could test 0:07:57.076 --> 0:08:10.756 it in the schools. They sign up a rural elementary 0:08:10.756 --> 0:08:13.716 school and two high schools, a night school and a 0:08:13.756 --> 0:08:17.116 regular high school, and Adam sells them on this new 0:08:17.116 --> 0:08:23.636 way of picking the student council. Anyone who wanted to 0:08:23.636 --> 0:08:26.156 be in the lottery put their name forward, and then 0:08:26.196 --> 0:08:28.956 the whole school gathered in the gym to hear the results. 0:08:29.396 --> 0:08:33.316 We wanted something that was very visual that could generate buzz, 0:08:34.156 --> 0:08:39.036 and so they had some clay pots and we used 0:08:39.076 --> 0:08:41.756 faba beans as a crop that they grow there. They're 0:08:41.956 --> 0:08:44.356 kind of the biggest beans you can find, and so 0:08:44.556 --> 0:08:47.156 we had two colors, green and purple. The students would 0:08:47.156 --> 0:08:48.956 put fava beans in the pot and they'd cover it 0:08:48.996 --> 0:08:53.236 with an aguayoun, a traditional cloth, so that it's from 0:08:53.276 --> 0:08:55.716 there and it was also visually beautiful. 0:08:56.716 --> 0:08:59.316 If there were two hundred students running for eight student 0:08:59.396 --> 0:09:02.396 council positions, they would put one hundred and ninety two 0:09:02.516 --> 0:09:05.636 green fava beans in the pot and eight purple ones. 0:09:06.356 --> 0:09:12.556 Then all the candidates would line up. The kid reaches 0:09:12.596 --> 0:09:14.996 in reaches under the cloth. Yeah, grabs a bean. 0:09:15.116 --> 0:09:15.636 Yeah. 0:09:15.836 --> 0:09:16.596 Do they hold it up? 0:09:16.636 --> 0:09:18.716 If it's purple, they hold it up. If it's purple 0:09:18.796 --> 0:09:20.836 or green, they hold They're supposed to hold it up 0:09:20.916 --> 0:09:27.516 so the school can see. 0:09:28.436 --> 0:09:32.676 No campaigning, no speeches or posters, no glad handing, no 0:09:32.836 --> 0:09:36.756 ballots no recounts. The whole selection was over in twenty minutes. 0:09:37.036 --> 0:09:39.796 Those who picked out the purple fava beans were on 0:09:39.956 --> 0:09:46.796 student council. Done. Now, I know that sounds like a 0:09:46.836 --> 0:09:49.356 crazy idea. You would never go for that, would you? 0:09:51.516 --> 0:10:07.916 Or would you? Adam KRONKWRIGHTE learned three important things from 0:10:07.956 --> 0:10:11.116 his work on school lottery in Bolivia. I'm going to 0:10:11.196 --> 0:10:13.996 call them kronk Right's laws. None of them I think 0:10:14.276 --> 0:10:20.316 are obvious beforehand, at least they were not obvious to him. First, numbers, 0:10:20.836 --> 0:10:25.196 as in number of candidates, typically what percentage of the 0:10:25.276 --> 0:10:27.916 kids want to be part of the lottery. 0:10:28.036 --> 0:10:30.516 It varies by the school. Yeah, it varies by the school. 0:10:30.556 --> 0:10:34.196 And so at the night high school, you know, these 0:10:34.276 --> 0:10:37.196 kids often work even on Saturdays, like they got a 0:10:37.196 --> 0:10:40.876 full schedule, So we would usually get between twenty and 0:10:40.876 --> 0:10:45.316 thirty students who are volunteering for like twelve spots. Right, 0:10:45.356 --> 0:10:49.756 So it wasn't a huge, huge turnout. And then at 0:10:49.836 --> 0:10:53.436 the like rural K through eight school, it was literally 0:10:53.516 --> 0:10:57.996 like every student that could participate participated in every lottery. 0:10:58.116 --> 0:11:01.796 And then you know, at another school we worked out 0:11:01.876 --> 0:11:04.596 later on. It was an urban high school, but in 0:11:04.636 --> 0:11:09.636 the morning you know there you had at least half 0:11:09.676 --> 0:11:11.036 of the school. 0:11:10.836 --> 0:11:13.956 After the student council members finished their time in office. 0:11:14.236 --> 0:11:17.836 Adam always asks a simple question, would you have run 0:11:18.356 --> 0:11:20.916 if we had done things the old way with campaigns 0:11:20.956 --> 0:11:21.556 and speeches. 0:11:22.476 --> 0:11:26.156 It's about three quarters who say no right away? 0:11:26.276 --> 0:11:27.076 Why do they say no? 0:11:28.796 --> 0:11:30.916 Well, let's say that nobody would have voted for me. 0:11:31.356 --> 0:11:33.356 I was new in the school, didn't have a lot 0:11:33.396 --> 0:11:36.796 of friends. You know, I would not want to run 0:11:36.796 --> 0:11:39.196 a campaign, you know, and ask people to vote for me. 0:11:39.916 --> 0:11:40.116 You know. 0:11:40.316 --> 0:11:43.916 It's the things that you can kind of guess are impediments. 0:11:44.236 --> 0:11:47.596 They liked serving in office. They just didn't want to 0:11:47.676 --> 0:11:52.156 run for office. The demands of the campaign were filtering 0:11:52.156 --> 0:11:54.756 out all kinds of people who would otherwise have been 0:11:54.876 --> 0:11:59.756 very interested in student council politics. Overwhelmingly those who win 0:11:59.836 --> 0:12:02.916 the lottery one year in fact run again the next. 0:12:03.276 --> 0:12:05.556 So yeah, they tend to line back up. But when 0:12:05.596 --> 0:12:07.996 we ask them, you know, it'd be normal elections, would 0:12:07.996 --> 0:12:10.116 you have done it? The majority of saying now. 0:12:10.476 --> 0:12:15.796 What's interesting is the extent to which running for an 0:12:15.796 --> 0:12:22.636 office and running an office are two very very different things, 0:12:23.236 --> 0:12:27.796 and someone may be both capable and interested in running 0:12:27.836 --> 0:12:33.236 an office, but have no interest in running for That's 0:12:33.556 --> 0:12:38.596 the first law. In a democracy. Elections are supposed to 0:12:38.676 --> 0:12:48.476 encourage participation, but they don't. They discourage it. Lotteries encourage participation. Okay, 0:12:48.916 --> 0:12:52.756 crank write's second law. When you choose your student government 0:12:52.796 --> 0:12:55.516 by lottery, you see a change in the kinds of 0:12:55.556 --> 0:12:59.356 things the government actually does. In my high school, the 0:12:59.356 --> 0:13:03.716 student council basically organized dances, not at Adams schools. In 0:13:03.756 --> 0:13:07.036 coach of Bamba, at the Night School, for example, the 0:13:07.076 --> 0:13:09.636 student government tackled all kinds of issues. 0:13:10.156 --> 0:13:12.676 Human trafficking is a big problem there, and so they 0:13:12.836 --> 0:13:16.676 organized a workshop on human trafficking that had like you know, 0:13:16.756 --> 0:13:21.876 theatrical production and breakout groups and stuff. They've gotten First 0:13:21.876 --> 0:13:24.316 aid kits for their schools. The schools don't tend to 0:13:24.316 --> 0:13:27.756 have them. They've organized educational field trips. 0:13:28.716 --> 0:13:32.836 The student council put together soccer tournaments and poetry competitions. 0:13:33.236 --> 0:13:34.316 They opened a library. 0:13:34.636 --> 0:13:38.276 School didn't have one, and so they solicited book donations 0:13:38.276 --> 0:13:42.276 from institutions and DVDs and they were able to get 0:13:42.316 --> 0:13:45.796 a computer donated from the Ministry of Education and they 0:13:45.796 --> 0:13:49.676 set it up and they ran it. They also created 0:13:49.676 --> 0:13:52.356 their own student ID cards because that school didn't have 0:13:52.516 --> 0:13:56.156 ID cards and didn't have uniforms, and since they went 0:13:56.196 --> 0:14:00.476 to school at seven pm, the transportation workers didn't believe 0:14:00.516 --> 0:14:03.116 that there were students since they charged them double. So 0:14:03.196 --> 0:14:04.796 that was the first thing that they worked on that 0:14:04.876 --> 0:14:07.156 was like already number one for all the students was 0:14:07.196 --> 0:14:09.276 we want student ID cards, and so they made their 0:14:09.316 --> 0:14:14.396 own and distributed those to Adam. 0:14:14.676 --> 0:14:17.516 The crucial thing about that long list of accomplishments was 0:14:17.956 --> 0:14:20.476 the range of things the students were interested in. 0:14:21.356 --> 0:14:23.356 What do you mean by that? That's a more diverse group. 0:14:24.836 --> 0:14:28.916 They come from different social circles versus typically, at least 0:14:28.956 --> 0:14:31.756 in Bolivia. You know, it'll be like a group of 0:14:31.756 --> 0:14:35.356 friends who runs for election, right, and so have similar 0:14:35.556 --> 0:14:37.836 perspective on the school and what the school might need 0:14:38.716 --> 0:14:41.796 and or what they're going to offer to win people's votes, 0:14:41.836 --> 0:14:45.036 which is typically more the case. And so a group 0:14:45.076 --> 0:14:47.636 that's brought the book together through lottery. None of the 0:14:47.676 --> 0:14:49.716 students in the student government know each other prior to 0:14:50.396 --> 0:14:54.156 that first meeting, and they often have very divergent interests. 0:14:54.196 --> 0:14:58.116 You know, you'll have you know, jocks and artists and 0:14:58.196 --> 0:15:00.516 you know, like just kind of the whole gamut. 0:15:00.356 --> 0:15:03.996 The whole gamut. Democracies are supposed to be the best 0:15:04.036 --> 0:15:07.076 system for ensuring that a whole gamut of interests are 0:15:07.156 --> 0:15:11.876 represented in government. Kronk Right's second law says that's not true, 0:15:12.516 --> 0:15:14.596 not if the government is drawn from the same narrow 0:15:14.676 --> 0:15:18.196 band of society year after year. It's the lottery that 0:15:18.316 --> 0:15:22.116 gives you a truly representative sample of leaders. After the 0:15:22.156 --> 0:15:25.116 success of a few experiments, Adam helps set up a 0:15:25.156 --> 0:15:29.276 nonprofit called Democracy in Practice to work with student governments 0:15:29.276 --> 0:15:29.916 in Bolivia. 0:15:30.796 --> 0:15:34.356 I can't tell you the number of times that, you know, 0:15:35.316 --> 0:15:38.356 fresh off of a lottery selection, we have the first 0:15:38.396 --> 0:15:41.836 couple orientation meetings and my co founder, Raoul and I 0:15:41.876 --> 0:15:45.436 will come out and we'll be walking together and will 0:15:45.476 --> 0:15:49.156 be saying, oh, you know, so and so they're going 0:15:49.236 --> 0:15:51.796 to be a really important part of this team. You know. 0:15:52.436 --> 0:15:56.396 They present themselves well, they speak well, they're confident, and 0:15:57.476 --> 0:16:00.116 they don't even finish the term of office. 0:16:00.516 --> 0:16:03.516 This brings us to kronk Right's third law, maybe the 0:16:03.516 --> 0:16:07.516 most important of all. Nobody knows anything. 0:16:09.876 --> 0:16:11.516 The number of times that we've walked out of those 0:16:11.516 --> 0:16:14.036 meetings and been worried about a particular student who seemed 0:16:14.116 --> 0:16:20.236 really unplugged, you know, just kind of close body language, uncomfortable, 0:16:20.316 --> 0:16:23.236 didn't participate much, and and we say, we've got to 0:16:23.316 --> 0:16:26.316 work cut out with that student. And it, you know, 0:16:26.396 --> 0:16:29.396 they within a couple of weeks they figure out what's 0:16:29.436 --> 0:16:31.956 expected of them, how they can plug in, and by 0:16:31.996 --> 0:16:35.316 the end of the term they're even voted by their peers. 0:16:35.356 --> 0:16:37.476 It's like one of the most important members of the team. 0:16:38.036 --> 0:16:41.396 And so that's an experience we've had over and over 0:16:41.476 --> 0:16:45.276 again and still kind of get fooled by that. Now, 0:16:45.316 --> 0:16:47.916 the charisma that that that certain students will have and 0:16:47.956 --> 0:16:49.076 the confidence that they'll have. 0:16:49.956 --> 0:16:54.396 So the prediction, the mechanism we're using for predicting who 0:16:54.476 --> 0:17:00.196 will be a good leader, it's just flawed. So you're saying, yeah, yeah, 0:17:00.396 --> 0:17:04.516 even you, even you sometimes fall into this trap of 0:17:04.516 --> 0:17:09.596 of thinking that because a kid presents herself well and 0:17:09.636 --> 0:17:12.556 it's both spoken and charismatic, that she's going to be 0:17:12.556 --> 0:17:13.116 a good leader. 0:17:13.356 --> 0:17:18.836 Yeah. Yeah, it's really difficult to counter that bias, right, 0:17:19.196 --> 0:17:23.236 And obviously not every student who's confident charismatic, you know, 0:17:23.756 --> 0:17:25.796 has has a big ego and can't work well with 0:17:25.876 --> 0:17:27.796 others and doesn't show up, you know, we've had. We've 0:17:27.796 --> 0:17:32.716 had several students who fit that mold, and we're great teammates. 0:17:33.236 --> 0:17:39.316 But generally it's a lot easier to teach a Shire 0:17:39.356 --> 0:17:44.116 student or student who's not super popular. It's a lot 0:17:44.156 --> 0:17:46.356 easier to teach them to overcome those fears and kind 0:17:46.356 --> 0:17:51.436 of add a few more leadership skills to their skill set. 0:17:51.716 --> 0:17:55.916 Then it is to to teach some of the more confident, 0:17:55.996 --> 0:17:58.676 charismatic students to kind of set aside their ego and 0:17:58.756 --> 0:18:00.716 like be a good member of a team and not 0:18:00.796 --> 0:18:04.236 try to steal the limelight and listen to others and 0:18:04.236 --> 0:18:05.636 not dominate conversations. 0:18:06.596 --> 0:18:09.636 Democratic elections are based on the idea that voters are 0:18:09.636 --> 0:18:12.316 good predictors, that they can look at a slate of 0:18:12.356 --> 0:18:16.116 candidates and accurately predict who will be the most effective leader. 0:18:16.796 --> 0:18:17.516 That's why we. 0:18:17.436 --> 0:18:20.676 Have campaigns and speeches and debates to help us make 0:18:20.716 --> 0:18:24.916 that prediction. But in Bolivia, Adam and his team discovered 0:18:24.956 --> 0:18:26.836 that people are lousy predictors. 0:18:27.476 --> 0:18:30.636 The teachers who often aren't in the meetings are just 0:18:30.796 --> 0:18:33.636 constantly fooled by, you know, when they come up and say, oh, 0:18:33.716 --> 0:18:35.916 you know that student, what a great leader you know, 0:18:35.916 --> 0:18:38.516 and they're not even aware that the rest of the 0:18:38.516 --> 0:18:40.716 student government's about to kick them out. You know, they've 0:18:40.716 --> 0:18:44.756 had enough of some of their behavior. And it's one 0:18:44.756 --> 0:18:47.956 student who got picked in the lottery. And during the 0:18:47.996 --> 0:18:50.356 lottery I had a teacher turned to me under his 0:18:50.436 --> 0:18:53.356 breath say, you know that students a dud. You're not 0:18:53.396 --> 0:18:56.236 going to work out, you know. I found out afterwards 0:18:56.236 --> 0:18:59.116 he was the one student from his class of thirty 0:18:59.116 --> 0:19:02.436 five students who had failed the previous year, and he 0:19:02.636 --> 0:19:04.956 was one of the one of the more kind of 0:19:05.116 --> 0:19:10.276 outstanding examples that we've had. He just you really loved 0:19:10.516 --> 0:19:13.756 to work on meaningful projects and be a part of 0:19:13.796 --> 0:19:14.796 a team. 0:19:15.476 --> 0:19:19.836 Now I know what you're thinking. It's Bolivia. What works 0:19:19.876 --> 0:19:22.956 for teenagers and coach of Bamba doesn't necessarily work for 0:19:23.036 --> 0:19:25.356 the rest of us Apples and oranges and all that. 0:19:26.236 --> 0:19:29.756 Except after being schooled by Adam, I began to see 0:19:29.836 --> 0:19:31.876 versions of his logic everywhere. 0:19:35.636 --> 0:19:36.956 Hello, this is Michael Loward. 0:19:37.076 --> 0:19:40.156 Nih delighted that you found time to chat with me. 0:19:40.556 --> 0:19:46.076 I just ran across your papers on peer review and 0:19:46.236 --> 0:19:49.756 citations and found them absolutely fascinating and they fit very 0:19:49.836 --> 0:19:53.836 much in a theme I'm exploring in my podcast. This season, 0:19:53.916 --> 0:19:57.996 so I wanted to chat. Michael Lawer is Deputy Director 0:19:58.036 --> 0:20:02.836 of Extramural Research at the National Institutes of Health in Bethesda, Maryland. 0:20:03.356 --> 0:20:04.836 If I had to make a list of the most 0:20:04.876 --> 0:20:08.556 important Americans you've never heard of, Flower's in the top ten. 0:20:09.236 --> 0:20:12.036 He's way way up in what you might call, and 0:20:12.076 --> 0:20:15.276 I mean this in the nicest possible way, the biomedical 0:20:15.476 --> 0:20:19.916 deep state. His office hands out around thirty billion dollars 0:20:19.956 --> 0:20:24.756 a year in research grants to scientists across the United States. Basically, 0:20:24.836 --> 0:20:28.076 if you walk through any biology or chemistry department, or 0:20:28.116 --> 0:20:32.436 any medical school or any medical research institute, Lawer's office 0:20:32.636 --> 0:20:36.716 is what's paying for it. So every year, his office 0:20:36.716 --> 0:20:40.956 gets eighty thousand grant applications from around the country. They 0:20:40.996 --> 0:20:43.676 only have money to find about twenty percent of those, 0:20:44.276 --> 0:20:47.516 so they hold the equivalent of an election. Each of 0:20:47.516 --> 0:20:50.956 those applications gets assigned to a small group of reviewers. 0:20:51.436 --> 0:20:53.836 The reviewers vote for the ones they like and throw 0:20:53.876 --> 0:20:56.716 out the rest. Then those finalists are turned over to 0:20:56.796 --> 0:20:58.116 a second group of voters. 0:20:58.716 --> 0:21:01.596 The reviewers who are assigned to grant will give a 0:21:01.636 --> 0:21:05.196 brief presentation about the grant itself and what they thought 0:21:05.236 --> 0:21:08.196 were the strengths and weaknesses. This is then followed by 0:21:08.596 --> 0:21:12.756 an open discussion. After the discussion is complete, there is 0:21:12.796 --> 0:21:14.516 then a scoring. 0:21:14.916 --> 0:21:18.036 So first round is you're throwing out the bottom the 0:21:18.036 --> 0:21:18.716 bottom half. 0:21:18.836 --> 0:21:20.196 That's right, that's right. 0:21:20.356 --> 0:21:24.916 Second round is you're scoring the remainder. And how what 0:21:25.036 --> 0:21:27.356 is the the scores on a scale of what to water? 0:21:27.356 --> 0:21:28.516 How is the score represented? 0:21:28.836 --> 0:21:31.436 This is like golf, So a lower score is better. 0:21:32.516 --> 0:21:35.996 A score of ten would be perfect. That means that 0:21:36.036 --> 0:21:38.676 as far as the reviewers are concerned, they can't think 0:21:38.716 --> 0:21:42.796 of anything better. A score of ninety would represent the 0:21:42.916 --> 0:21:44.316 absolute worst. 0:21:44.836 --> 0:21:48.716 Those scores are predictions about how good a scientist's proposal is, 0:21:49.116 --> 0:21:52.156 in exactly the same way that the choices voters make 0:21:52.436 --> 0:21:55.236 are predictions about how good a leader a candidate will be. 0:21:56.196 --> 0:21:59.076 Kronkrite's third law says that voters aren't very good. 0:21:58.956 --> 0:21:59.676 At that prediction. 0:22:00.636 --> 0:22:04.636 So Lower wonders, what about the NIH's grant reviewers? Are 0:22:04.676 --> 0:22:07.596 we any good at it? Does the proposal that got 0:22:07.596 --> 0:22:11.196 a perfect ten from the voters grant approval time actually 0:22:11.316 --> 0:22:14.916 end up being the most influential or innovative research. Once 0:22:14.956 --> 0:22:19.756 it's finished, lawer looks around. He can't find any definitive 0:22:19.756 --> 0:22:22.716 answer to that question. This peer review process that you've 0:22:22.756 --> 0:22:25.036 just described has been in place at NIH. 0:22:24.676 --> 0:22:27.236 For how long seventy five years? 0:22:27.676 --> 0:22:30.716 Twenty five years. So we had this situation where we 0:22:30.796 --> 0:22:33.636 have a process in place for many, many years, seventy 0:22:33.636 --> 0:22:37.636 five years, which is from an empirical perspective, unexamined. 0:22:39.236 --> 0:22:41.356 Well, that's right, so. 0:22:41.436 --> 0:22:46.916 Lauer says, let's take a look. His method is simple. 0:22:47.556 --> 0:22:50.156 The standard way that science measures the value of a 0:22:50.196 --> 0:22:53.556 bit of research is how often it's cited by other 0:22:53.676 --> 0:22:56.956 scientists in their work. So if I spend five years 0:22:56.996 --> 0:23:00.596 working on an experiment and no one mentions that experiment again, 0:23:01.036 --> 0:23:04.036 it would be considered a failure. And if it's mentioned 0:23:04.316 --> 0:23:08.396 a thousand times, it's considered a home run. So what 0:23:08.516 --> 0:23:12.716 is there between the score of your grant application and 0:23:12.796 --> 0:23:16.156 the number of citations your work gets once it's finished. 0:23:17.196 --> 0:23:18.636 I think you know where this is going. 0:23:21.796 --> 0:23:24.916 We did not find a strong correlation between the peer 0:23:24.956 --> 0:23:31.476 review score and the citation metrics of the subsequently funded grant. 0:23:32.236 --> 0:23:36.076 Now it's worth pointing something out. Flower doesn't go into 0:23:36.116 --> 0:23:39.036 this thinking that the NAH is going to be subject 0:23:39.116 --> 0:23:43.956 to Chronkrit's third law. Nobody knows anything after all, lower's 0:23:44.036 --> 0:23:48.516 not talking about adolescents judging other adolescents. He's talking about 0:23:48.596 --> 0:23:53.036 distinguished scientists judging the potential of other distinguished scientists. 0:23:53.796 --> 0:23:58.356 We had assumed that we would find a reasonably strong 0:23:58.396 --> 0:24:02.636 correlation between the peer review score and the grant productivity. 0:24:03.156 --> 0:24:09.156 Really, I had assumed that you went in skeptical No. 0:24:10.596 --> 0:24:13.596 But then again, exactly as I said, if we knew 0:24:13.636 --> 0:24:15.396 what we were doing, we wouldn't call it research. 0:24:17.636 --> 0:24:21.516 Kronk Wrt's third law applies outside of coach of Bamba. 0:24:21.636 --> 0:24:24.916 It applies even at the National Institutes of Health, to 0:24:24.996 --> 0:24:28.596 the process by which billions of dollars of research money 0:24:28.636 --> 0:24:32.556 is distributed. It's not kronk Wrt's third law. It's kronk 0:24:32.636 --> 0:24:42.356 Right's universal third law. When Lauer publishes his research, lots 0:24:42.396 --> 0:24:45.476 of other scientists look at his results and say, if 0:24:45.476 --> 0:24:50.076