WEBVTT - The Tortoise and the Hare 0:00:15.356 --> 0:00:25.076 Pushkin. April twenty fourth, two thousand and nine, the late 0:00:25.116 --> 0:00:29.276 Supreme Court Justice Antonin Scalia is speaking at American University's 0:00:29.436 --> 0:00:30.556 Washington College of Law. 0:00:32.516 --> 0:00:35.316 Thank you, thank you very much, Professor Marcus Dean Grossman, 0:00:35.636 --> 0:00:42.916 Ladies and gentlemen, I began one of my either talks 0:00:42.996 --> 0:00:43.796 or law of you WARDI. 0:00:43.916 --> 0:00:47.076 The students are all dressed up for the occasion. Sea 0:00:47.116 --> 0:00:50.276 Span is recording. There's a big stage hung with blue 0:00:50.316 --> 0:00:55.436 polyester drapes. Scalia holds forth, his black hair swept back 0:00:55.476 --> 0:00:59.516 from his forehead, glasses on his nose, strong and square, 0:00:59.556 --> 0:01:03.916 all intellectual heft and force, gripping the podium like it's 0:01:03.916 --> 0:01:04.676 a slab. 0:01:04.356 --> 0:01:07.196 Of beef, and that administrative law is not for sissies. 0:01:08.436 --> 0:01:10.956 It is. It is a very difficult course to teach, 0:01:10.996 --> 0:01:14.956 and I assume it certainly wasn't my day a hard 0:01:15.076 --> 0:01:15.916 course to master. 0:01:16.796 --> 0:01:20.996 It's vintage Scalia. The audience hangs on his every word. 0:01:21.396 --> 0:01:26.476 He finishes triumphantly than hands shoot in the air. 0:01:27.316 --> 0:01:29.636 Good afternoon. My name is Christina Stutt. I'm a one 0:01:29.716 --> 0:01:30.996 l student here at w CL. 0:01:31.156 --> 0:01:33.996 Christina studt First, your student have. 0:01:33.996 --> 0:01:37.076 A more general question, and that is the part of 0:01:37.196 --> 0:01:40.076 American the American ethos is that our society as a 0:01:40.076 --> 0:01:42.956 meritocracy were hard work and talantly to success. But there 0:01:42.956 --> 0:01:45.996 are other important factors like connections and elite degrees. And 0:01:46.036 --> 0:01:49.036 I'm wondering, other than grades of journal what do smart, 0:01:49.356 --> 0:01:52.196 hardworking WSL students with strong writing skills need to do 0:01:52.276 --> 0:01:54.196 to be outrageously successful in the law. 0:01:54.676 --> 0:01:59.236 What does it take to be outrageously successful in the law? 0:02:00.556 --> 0:02:08.356 A just work hard and be very good. I tell 0:02:08.356 --> 0:02:09.236 you a story. 0:02:11.956 --> 0:02:15.236 My name is Malcolm Gladwell. You're listening to Revisionist History, 0:02:15.596 --> 0:02:21.556 my podcast about things overlooked and misunderstood. This episode is 0:02:21.596 --> 0:02:25.276 part two of my examination of the bizarre things the 0:02:25.316 --> 0:02:33.916 legal profession does to pick its best and brightest. In 0:02:33.996 --> 0:02:37.436 part one, which if you haven't listened to you probably should, 0:02:37.836 --> 0:02:40.836 I took the law school admissions test along with my 0:02:40.876 --> 0:02:44.516 assistant Camille and couldn't understand why they made me rush 0:02:44.596 --> 0:02:47.556 through all the questions. But now in part two we 0:02:47.676 --> 0:02:50.356 have bigger fish to fry. I'm going to serve up 0:02:50.476 --> 0:02:54.476 Malcolm Gladwell's grand unified theory of how to fix American 0:02:54.556 --> 0:02:58.676 legal education. No, make that my grand unified theory for 0:02:58.756 --> 0:03:06.676 fixing all American higher education. And what is our text 0:03:06.716 --> 0:03:10.916 for this discussion of Gladwell's grand unified theory. It's the 0:03:10.956 --> 0:03:15.236 answer Justice Scalia gave to the unfortunate Christina Studd. 0:03:16.476 --> 0:03:21.316 You know, by and large, unless I have a professor 0:03:21.996 --> 0:03:26.156 on the faculty who's a good friend and preferably a 0:03:26.196 --> 0:03:29.116 former law clerk of mine whose judgment I can trust, 0:03:30.356 --> 0:03:32.756 I'm going to be picking, you know, for Supreme Court 0:03:32.796 --> 0:03:36.756 law clerks. I can't afford a miss I just can't. 0:03:37.036 --> 0:03:39.356 So I'm going to be picking from the law schools 0:03:39.396 --> 0:03:43.716 that basically there are the hardest to get into. They 0:03:43.756 --> 0:03:45.556 admit the best in the brightest, and they may not 0:03:45.636 --> 0:03:49.316 teach very well, but you can't make You can't make 0:03:50.516 --> 0:03:53.396 a sow's ear out of a silk purse. And if 0:03:53.436 --> 0:03:55.356 they come in the best in the brightest, they're probably 0:03:55.356 --> 0:03:56.796 going to leave the best in the brightest. 0:03:56.836 --> 0:04:00.036 Okay, let's pretend to be fine legal minds for a 0:04:00.076 --> 0:04:05.076 moment and closely parts the meaning and implication of Scalia's statement. 0:04:06.556 --> 0:04:10.236 A student at American University's Washington College of Law, a 0:04:10.316 --> 0:04:13.636 law school that US News and Will Report ranked seventy 0:04:13.676 --> 0:04:17.356 seventh among all American law schools, is asking a question 0:04:17.436 --> 0:04:20.636 of a sitting justice of the US Supreme Court who 0:04:20.676 --> 0:04:25.836 graduated from Harvard Law School. She's basically asking him would 0:04:25.836 --> 0:04:28.236 it be possible to be one of his clerks, and 0:04:28.276 --> 0:04:32.036 he answers, you go to American University's Washington College of Law, 0:04:32.236 --> 0:04:34.676 you have no chance of becoming one of my clerks. 0:04:35.156 --> 0:04:37.276 I only hire people who went to Harvard, like I did. 0:04:38.196 --> 0:04:42.076 But then he goes on and he says this, which 0:04:42.116 --> 0:04:46.956 is my favorite part because it sums up absolutely everything 0:04:47.276 --> 0:04:52.076 I want to talk about in this episode. I mean everything. 0:04:53.316 --> 0:04:57.276 Now I started. The reason I tell the story is 0:04:57.476 --> 0:04:59.596 one of my former clerks who I am the most 0:04:59.636 --> 0:05:03.716 proud of and now sits on the Sixth Circuit Court 0:05:03.756 --> 0:05:08.036 of Appeals, Jeff Sutton. I always referred him as one 0:05:08.036 --> 0:05:10.396 of my former law He wasn't one of my former 0:05:10.476 --> 0:05:12.596 law He was Lewis Powell's clerk. 0:05:12.876 --> 0:05:15.716 At the time. Lewis Powell was semi retired from the 0:05:15.716 --> 0:05:19.636 Supreme Court. He had what's called senior status, so his 0:05:19.756 --> 0:05:22.436 law clerks worked mostly for other justices. 0:05:22.996 --> 0:05:25.436 But I wouldn't have hired Jeff Sutton for God's sake. 0:05:25.476 --> 0:05:34.036 He went to Ohio State and he's one of the 0:05:34.156 --> 0:05:36.276 very best law clerks I ever had, and he's just 0:05:36.316 --> 0:05:38.556 a brilliant guy. So don't tell me this stuff about 0:05:38.636 --> 0:05:40.396 you know, what do you have to do to be successful? 0:05:40.796 --> 0:05:42.916 You have to be good, simple as that. 0:05:44.436 --> 0:05:46.916 Okay, we're not thank you very much, thank you. 0:05:49.156 --> 0:05:53.476 Oh we're not done. We've only just begun. 0:05:59.356 --> 0:06:02.276 Tell me why you decided to go to law school? 0:06:03.316 --> 0:06:07.196 WHOA So? Law school was a third choice. First choice 0:06:07.316 --> 0:06:10.156 was teaching. I was a teacher and coach for several years, 0:06:10.196 --> 0:06:13.916 both middle school, high school, soccer, baseball, little track. 0:06:14.436 --> 0:06:17.996 This is Judge Jeffrey Sutton, the guy who somehow slipped 0:06:17.996 --> 0:06:20.476 through the cracks to become the best clerk and in 0:06:20.556 --> 0:06:21.956 in Scalia ever had. 0:06:22.316 --> 0:06:26.356 Foreign service was choice number two. No lawyers in my family, 0:06:26.436 --> 0:06:28.556 and when I finally went to law school, I wouldn't 0:06:28.556 --> 0:06:32.476 say my parents were beaming with pride. I came from 0:06:32.476 --> 0:06:35.996 a family of kind of service driven folks who were 0:06:36.036 --> 0:06:38.156 either in education some missionaries. 0:06:38.716 --> 0:06:41.476 And why did you decide to go to Ohio State 0:06:41.516 --> 0:06:41.996 law school? 0:06:42.916 --> 0:06:46.076 Well, it was a pretty complicated decision. I applied to 0:06:46.116 --> 0:06:49.916 two law schools, Ohio State and Michigan. I got into 0:06:49.956 --> 0:06:53.356 one of them, and I ended up enrolling at the 0:06:53.356 --> 0:06:57.116 one I got into. Oh I see it was I 0:06:57.236 --> 0:06:59.316 very much would have liked to have gone to Michigan, 0:06:59.356 --> 0:07:01.076 and I was the fact that my father in law 0:07:01.076 --> 0:07:03.196 had gone there and his son in law couldn't get 0:07:03.236 --> 0:07:07.436 in was a little humbling, but we got over it. 0:07:08.276 --> 0:07:11.396 I didn't ask Judge Sutton what his lsat's score was, 0:07:11.956 --> 0:07:16.316 but we can do the math. Michigan is part of 0:07:16.316 --> 0:07:19.196 the elite group of law schools known as the T fourteen, 0:07:19.596 --> 0:07:24.636 the top fourteen Yale, Stanford, Harvard, University of Chicago, Columbia, 0:07:24.716 --> 0:07:28.236 all the big ones. Ohio State is not among the 0:07:28.276 --> 0:07:32.036 T fourteen. The median LSAT score of someone who goes 0:07:32.076 --> 0:07:35.436 to Ohio State these days is eight points lower than 0:07:35.436 --> 0:07:39.556 the median score of someone at Michigan. Now what does 0:07:39.596 --> 0:07:43.156 that fact mean. Well, as you may recall from the 0:07:43.156 --> 0:07:46.196 previous episode, the l set is not a test of 0:07:46.196 --> 0:07:50.356 someone's ability to solve difficult problems. It's a test of 0:07:50.436 --> 0:07:54.596 someone's ability to solve difficult problems quickly. It is five 0:07:54.636 --> 0:07:57.396 sections of twenty to twenty five questions, and you have 0:07:57.436 --> 0:07:59.876 a hard limit of thirty five minutes for each section. 0:08:00.396 --> 0:08:03.356 You have to rush. As one el SAT tutor told me, 0:08:03.596 --> 0:08:08.476 the test favors those capable of processing without understanding it 0:08:08.516 --> 0:08:15.236 favors not tortoises. So what's Jeff Sutton. Well, he's clearly brilliant. 0:08:15.436 --> 0:08:18.276 He was the Ohio State Solicitor in the nineteen nineties 0:08:18.396 --> 0:08:20.836 and while the Supreme Court with his arguments on a 0:08:20.916 --> 0:08:24.676 number of cases. His most recent work of legal scholarship 0:08:24.916 --> 0:08:29.516 is titled fifty one Imperfect Solutions, States and the Making 0:08:29.556 --> 0:08:32.316 of Constitutional Law. The New York Review of Books felt 0:08:32.316 --> 0:08:34.756 they had to get a retired Supreme Court justice to 0:08:34.836 --> 0:08:37.916 review it. There are lots of very serious people, in fact, 0:08:38.116 --> 0:08:40.756 who think Sutton deserves to be a Supreme Court justice 0:08:40.876 --> 0:08:44.876 himself one day. So Sutton is in the category of 0:08:45.036 --> 0:08:47.596 brilliant person who didn't do all that well on the 0:08:47.756 --> 0:08:51.076 lsat what does that make him? It makes him a tortoise, 0:08:51.796 --> 0:08:54.876 and not just any tortoise, a giant tortoise. He's one 0:08:54.876 --> 0:08:57.516 of those tortoises from the Galapagos that's five feet long. 0:08:59.036 --> 0:09:02.876 So Sutton graduates from Ohio State, gets a job clerking 0:09:02.916 --> 0:09:05.356 for a federal judge, then a job cloking on the 0:09:05.356 --> 0:09:09.396 Supreme Court, and in his year as clerk for Scalia, he. 0:09:09.436 --> 0:09:14.516 Thrives the thing that really affects everybody who works with him. 0:09:14.676 --> 0:09:19.396 Is within weeks you just get a sense of this 0:09:19.556 --> 0:09:24.036 incredible passion for the law, and that is just intoxicating. 0:09:24.156 --> 0:09:28.676 And that is what really changed things. And that year. 0:09:28.756 --> 0:09:32.476 I can't emphasize enough how much I got out of 0:09:32.476 --> 0:09:32.956 that year. 0:09:33.716 --> 0:09:36.516 Not long before, Jeffrey Sutton had been a middle school 0:09:36.556 --> 0:09:39.956 teacher and track coaching Columbus. Now he's working with one 0:09:39.996 --> 0:09:42.636 of the greatest legal minds in the country, and he 0:09:42.716 --> 0:09:45.676 does such a good job that seventeen years later, at 0:09:45.676 --> 0:09:49.956 some random speech at American University, Scalia singles him out. 0:09:50.316 --> 0:09:53.876 Scalia had well over one hundred clerks, jeff Sutton is 0:09:53.876 --> 0:09:57.476 the one he's proudest of. So why does a tortoise 0:09:57.516 --> 0:10:01.476 do so well working for the Supreme Court? I asked 0:10:01.476 --> 0:10:04.756 my fanciest legal friend, Tally for Haddien, who is a 0:10:04.796 --> 0:10:07.636 clerk on the Court a few years after Sutton for 0:10:07.876 --> 0:10:10.036 Justice Sandrada O'Connor, if. 0:10:09.916 --> 0:10:13.236 You were working on a case. What what does working 0:10:13.236 --> 0:10:14.036 on a case mean? 0:10:15.116 --> 0:10:18.036 Well, we worked on two kinds of cases. 0:10:18.716 --> 0:10:22.116 The first kind is what's called the sert pool, the 0:10:22.196 --> 0:10:25.796 thousands of petitions sent to Washington every year by people 0:10:25.796 --> 0:10:28.316 who want their cases heard by the Supreme Court. 0:10:28.436 --> 0:10:30.636 We would each get a stack of petitions I think 0:10:30.676 --> 0:10:32.956 on a Wednesday, and we had a week to get 0:10:33.916 --> 0:10:34.636 into the pool. 0:10:34.996 --> 0:10:36.796 Yeah, so that in that case there would be a 0:10:36.796 --> 0:10:37.196 lot to. 0:10:37.156 --> 0:10:38.116 Read, a lot to read. 0:10:38.436 --> 0:10:41.236 When you read those kinds of things, How do you read? 0:10:42.716 --> 0:10:44.796 How do you read? I don't know what that question. 0:10:44.956 --> 0:10:47.316 Do you read the same way you read a work 0:10:47.356 --> 0:10:51.076 of nonfiction or a New Yorker article or well? 0:10:51.116 --> 0:10:54.756 I always read them, and I continue to read similar 0:10:54.796 --> 0:10:57.876 documents with a pencil in my hand, which is not 0:10:57.956 --> 0:11:01.076 how I would read for pleasure, whether nonfiction or fiction. 0:11:01.356 --> 0:11:07.476 Slower or faster? Much slower, much slower, yes, how much slower. 0:11:11.156 --> 0:11:14.756 Personally, I feel I can read very quickly, and I 0:11:14.796 --> 0:11:17.516 can read very slowly, and I get different things out 0:11:17.556 --> 0:11:17.796 of it. 0:11:17.876 --> 0:11:19.836 But this is definitely slow reading territory. 0:11:20.036 --> 0:11:21.516 It is slow reading territory. 0:11:21.676 --> 0:11:23.636 And why is it so important to read slowly? 0:11:26.156 --> 0:11:32.076 Because the details matter and because the arguments are intricate, Yeah, 0:11:32.116 --> 0:11:35.996 and because the solutions are difficult. I mean, everybody will 0:11:35.996 --> 0:11:38.436 tell you this. When a case comes to the Supreme Court. 0:11:38.476 --> 0:11:40.876 You know a case that's really ready for a review 0:11:40.916 --> 0:11:44.396 at the Supreme Court, it's hard. The reason the circuit 0:11:44.396 --> 0:11:47.596 courts have disagreed about it is because it's really hard. 0:11:47.836 --> 0:11:51.716 Like the answer is not obvious. You're kidding yourself if 0:11:51.756 --> 0:11:54.596 you think that it is. So you have to think 0:11:54.676 --> 0:11:55.396 while you read. 0:11:55.796 --> 0:12:00.076 Yeah, yeah, you can't just process. You have to understand. Yes, yeah, you. 0:12:00.036 --> 0:12:03.996 Have to think while you read. This is the primary 0:12:04.036 --> 0:12:06.876 requirement of one of the most prestigious jobs in the 0:12:06.956 --> 0:12:10.516 legal profession. The other part of the job, the main 0:12:10.556 --> 0:12:13.796 part of the job, is researching and analyzing the actual 0:12:13.836 --> 0:12:16.876 cases that come before the court. For Hardian was one 0:12:16.916 --> 0:12:19.756 of four clerks working for O'Connor, so she would get 0:12:19.756 --> 0:12:21.716 assigned a quarter of those cases. 0:12:21.876 --> 0:12:23.396 And how much time would you spend on them? 0:12:24.036 --> 0:12:28.356 I don't think I ever stopped thinking about the cases 0:12:28.396 --> 0:12:29.956 that I was working on. 0:12:30.276 --> 0:12:33.716 Yeah, but what with the time that would elapse. What's 0:12:33.716 --> 0:12:35.916 the time that would elapse from when you were given 0:12:35.956 --> 0:12:38.756 the case to when you when you were finished with 0:12:39.036 --> 0:12:40.436 your contribution was finished? 0:12:43.436 --> 0:12:46.636 I don't remember, I want to say a couple months. 0:12:47.676 --> 0:12:51.196 Being a Supreme Court clerk is a job for a tortoise. 0:12:51.796 --> 0:12:54.756 You can't hurry. You have to work slowly and carefully 0:12:54.876 --> 0:12:58.356 because if you miss something, that's a problem. I didn't 0:12:58.396 --> 0:13:01.756 even have to mention tortoises to Judge Sutton. He brought 0:13:01.796 --> 0:13:02.156 them up. 0:13:02.476 --> 0:13:05.196 You know, law is very much a tortoise. The tortoise 0:13:05.196 --> 0:13:08.076 went beats the hair, and so the hairs that go 0:13:08.116 --> 0:13:12.556 to the elite schools, they better slide into tortoise mode 0:13:12.916 --> 0:13:14.876 or it's not going to work out well for them. 0:13:15.556 --> 0:13:20.396 And the tortoises that go to the state school's better 0:13:20.516 --> 0:13:22.316 stay being a tortoise than stick with it. 0:13:24.236 --> 0:13:28.956 So let us recap. A sitting Supreme Court justice explains 0:13:28.996 --> 0:13:31.356 to a group of law school students that he will 0:13:31.396 --> 0:13:33.916 not consider them for a job that involves being a 0:13:33.956 --> 0:13:36.916 tortoise because they have failed to shine at a test 0:13:36.916 --> 0:13:40.236 that measures their ability to be a hare. And even 0:13:40.236 --> 0:13:42.316 as he says that, he concedes that one of the 0:13:42.316 --> 0:13:45.156 best of his former clerks was a tortoise who also 0:13:45.276 --> 0:13:47.796 did not shine at a test that measured his ability 0:13:47.916 --> 0:13:51.396 to be a hare. And when he presents this confounding 0:13:51.516 --> 0:13:55.436 bit of reasoning that manages both to stigmatize and disparage 0:13:55.676 --> 0:14:08.596 the entire audience, what does the audience do Listen? I mean, 0:14:09.196 --> 0:14:14.116 this is bananas. This is like prisoners cheering award. I 0:14:14.156 --> 0:14:16.676 think you can see why we are in need of 0:14:16.676 --> 0:14:19.836 a grand, unified theory to fix legal education. 0:14:25.956 --> 0:14:26.676 At the. 0:14:29.996 --> 0:14:33.396 Monday, after my assistant, Camille, and I took the l SAT, 0:14:33.676 --> 0:14:37.436 we took the train to Newtown, Pennsylvania, to the headquarters 0:14:37.476 --> 0:14:40.916 of the Law School Admissions Council. This is the group 0:14:40.996 --> 0:14:44.636 that for the past seventy years has created and administered 0:14:44.636 --> 0:14:48.156 the ALSAT. They operate out of a two story red 0:14:48.196 --> 0:14:52.556 brick building in an office park, big atrium, very eighties. 0:14:53.156 --> 0:14:55.516 We were ushered into a conference room on the second 0:14:55.556 --> 0:15:00.436 floor where a row of test experts psychometricians were waiting 0:15:00.476 --> 0:15:00.796 for us. 0:15:00.876 --> 0:15:02.596 What time did you have to arrive at the test center? 0:15:02.636 --> 0:15:07.036 Eight They began with a tutorial on how to make 0:15:07.036 --> 0:15:10.636 a standardized test, which I I have to say was fascinating. 0:15:11.196 --> 0:15:13.036 It turns out a single item on a test like 0:15:13.116 --> 0:15:16.876 the ELSAT takes thirty six months to develop. They don't 0:15:16.956 --> 0:15:20.956 just dream up hard questions, They test the questions over 0:15:21.036 --> 0:15:23.756 and again to make sure they're the right kind of hard. 0:15:23.956 --> 0:15:26.716 So what I've done here is I've identified a question 0:15:27.876 --> 0:15:31.596 that was actually rejected because it was not performing similarly 0:15:31.956 --> 0:15:34.636 for two subgroups of interest. Those were males and females. 0:15:34.756 --> 0:15:35.996 This is Alex Weisman. 0:15:36.316 --> 0:15:40.276 The question text is actually on the second page. It 0:15:40.356 --> 0:15:44.476 starts off Thomas Tompkins, Renaissance English composer wrote in a 0:15:44.516 --> 0:15:47.716 musical style that in his time had already become outdated, 0:15:48.116 --> 0:15:48.836 and so forth. 0:15:49.596 --> 0:15:52.596 This is a passage designed to test the reading comprehension 0:15:52.636 --> 0:15:55.676 skills of would be lawyers. But the results of the 0:15:55.756 --> 0:16:00.036 question came out weird. Women who are otherwise doing really 0:16:00.036 --> 0:16:03.156 well on that section were somehow tripping up on this 0:16:03.316 --> 0:16:07.836 particular question, and the equivalent group of male high scorers 0:16:08.156 --> 0:16:09.956 were overwhelming getting it right. 0:16:10.516 --> 0:16:15.276 So here we have almost two x male versus not 0:16:15.396 --> 0:16:17.956 all well, well, almost twice as many males as women 0:16:18.156 --> 0:16:19.916 as females got this question. 0:16:19.676 --> 0:16:23.676 Correct right, So if you that is, that is already 0:16:24.116 --> 0:16:27.556 indicative of a problem with this question. So why the 0:16:27.636 --> 0:16:30.796 question of why is not always easy to answer it. 0:16:31.276 --> 0:16:35.476 And for a question like this, what we determine is 0:16:36.876 --> 0:16:40.796 even if we can't determine why this is happening, we 0:16:40.876 --> 0:16:43.116 don't take the chance in keeping it on the test. 0:16:43.996 --> 0:16:46.956 In this case, the l SAT wasn't functioning as a 0:16:46.996 --> 0:16:49.876 test of ability, which is what it's supposed to be. 0:16:50.316 --> 0:16:52.716 It seemed like it was a test of gender. Which 0:16:52.756 --> 0:16:55.676 it's not supposed to be, so they threw the question out. 0:16:56.596 --> 0:17:00.116 When I talked to psychometricians outside the legal world, they 0:17:00.116 --> 0:17:02.836 were unanimous in their praise of the l SAT. It 0:17:02.876 --> 0:17:06.236 was like talking to auto mechanics about a Porsche. Mechanics 0:17:06.276 --> 0:17:09.356 love Porsches, and if I had let them, I feel 0:17:09.396 --> 0:17:11.916 those three on the panel would have happily talked about 0:17:11.916 --> 0:17:15.916 their sports car for hours, the engine, the steering, the acceleration. 0:17:16.916 --> 0:17:20.116 But Camille and I had just two days earlier taken 0:17:20.196 --> 0:17:23.836 Nielsat and what I really wanted to know was why 0:17:23.836 --> 0:17:27.196 would these guys building a sports car? I mean, why 0:17:27.276 --> 0:17:30.076 go so fast? Why not just build a really good minivan. 0:17:30.316 --> 0:17:35.996 So we know in law school that doing the work efficiently, 0:17:36.156 --> 0:17:39.356 being able to handle the reading load and handle the 0:17:39.396 --> 0:17:43.476 amount of analysis that's required in a certain amount of 0:17:43.476 --> 0:17:44.476 time is relevant. 0:17:44.916 --> 0:17:47.036 Lily Nissovich takes up the cause. 0:17:47.316 --> 0:17:51.036 Research requirements on any redesign of the test is to 0:17:51.116 --> 0:17:53.556 make sure that if you're changing the timing or the 0:17:53.636 --> 0:17:56.396 number of questions that you're asking the given amount of time, 0:17:56.916 --> 0:17:59.556 that you go back to square one and make sure 0:17:59.636 --> 0:18:00.716 it predicts. 0:18:01.076 --> 0:18:04.436 We're now one hour and fourteen minutes into the presentation. 0:18:05.036 --> 0:18:06.476 I can't hold back any longer. 0:18:06.516 --> 0:18:09.796 So you've been talking about efficiency, Bret. I was trying 0:18:09.836 --> 0:18:12.436 to be more You guys stop me from being efficient. 0:18:14.676 --> 0:18:17.156 I had just been through the experience of finishing the 0:18:17.196 --> 0:18:19.196 first section of the el SAT with time to spare, 0:18:19.596 --> 0:18:21.516 and then running away out of time on the last 0:18:21.556 --> 0:18:22.236 logic section. 0:18:22.636 --> 0:18:24.676 The efficient way for me to take the test would 0:18:24.676 --> 0:18:27.076 be to speed up on the things that I was 0:18:27.116 --> 0:18:30.276 really good at and then use that time on the 0:18:30.316 --> 0:18:33.796 things I needed more time on. That's how efficient people work, right, 0:18:34.956 --> 0:18:37.236 But you wouldn't let me be efficient. I was told 0:18:37.276 --> 0:18:41.276 seventeen times you cannot look ahead at the next section. 0:18:41.836 --> 0:18:44.436 Why I sat there for ten minutes after the first one? 0:18:44.836 --> 0:18:47.476 Okay, so I'm getting a little bit worked up, But 0:18:47.636 --> 0:18:51.476 remember I'm under tremendous pressure to beat Camille on the 0:18:51.596 --> 0:18:54.476 l SAT and all this time she's sitting right next 0:18:54.516 --> 0:18:57.596 to me, all smug and complacent, like she was doing 0:18:57.636 --> 0:18:59.796 logic games in her head just for fun. 0:19:00.156 --> 0:19:01.636 I was like, why can't I look at the next one? 0:19:01.836 --> 0:19:04.156 I'm trying to be efficient. You're not letting me, I mean. 0:19:04.116 --> 0:19:06.596 Because then you'd be giving get more time for that 0:19:06.636 --> 0:19:08.676 next section than the first. And next day or the 0:19:08.676 --> 0:19:12.156 first t who was the one the people who took 0:19:12.316 --> 0:19:15.356 the question when it was gone through all these levels. 0:19:14.996 --> 0:19:19.236 Of being efficient in law school is about time management, right. 0:19:19.276 --> 0:19:22.316 It is about doing things you can do it really 0:19:22.436 --> 0:19:25.316 quickly quickly and using that extra time to If I'm 0:19:25.356 --> 0:19:30.836 a a fast reader but a slow writer, then it 0:19:31.076 --> 0:19:33.996 can you know, I have a different balance. And if 0:19:33.996 --> 0:19:36.276 I'm a fast writer and a slow reader. 0:19:36.636 --> 0:19:39.316 I don't get the sense that I'm making any headway. 0:19:39.476 --> 0:19:41.476 So my question is, why are you forcing us all 0:19:41.476 --> 0:19:44.396 to do every skill in thirty five minutes? If human 0:19:44.396 --> 0:19:46.676 beings are everyone in that room I took it with 0:19:46.836 --> 0:19:50.156 had a different set of skills, but you're why are 0:19:50.156 --> 0:19:59.356 you pushing us all into the same cookie cutter. 0:19:59.916 --> 0:20:01.036 It's a standardized test. 0:20:01.076 --> 0:20:04.716 I guess timing is one of the features of the standardization, 0:20:05.276 --> 0:20:07.516 and we can do research on what you're saying. 0:20:07.636 --> 0:20:12.156 But have you well, we've done research on the timing 0:20:12.236 --> 0:20:14.996 of the questions before they were ever introduced. 0:20:15.676 --> 0:20:18.516 Many how long it takes for people to do this 0:20:18.756 --> 0:20:23.876 number of questions reasonably well to get your optimal score? 0:20:24.076 --> 0:20:27.196 Is it necessarily to try every question? So some students 0:20:28.916 --> 0:20:30.396 to get a better score. 0:20:30.316 --> 0:20:34.716 By spending more time per question and then leaving to 0:20:34.756 --> 0:20:38.796 you skipping a few than by trying every question. Some students' 0:20:38.796 --> 0:20:42.236 best strategy is to try every question, so we advise 0:20:42.356 --> 0:20:45.756 them to experiment on themselves when they're practicing and see 0:20:45.756 --> 0:20:47.556 what that's the best strategy. 0:20:47.596 --> 0:20:51.756 But that's the only reason you need to have those 0:20:51.796 --> 0:20:55.156 strategies is because you have this arbitrary time constraint. 0:20:54.796 --> 0:20:57.276 Right, I just take a suit with the arbitrary. 0:20:57.876 --> 0:21:01.756 Am I being obnoxious? Maybe I am. It's like I've 0:21:01.796 --> 0:21:04.316 gone to push a head coaters and strictgart and I'm 0:21:04.316 --> 0:21:06.916 badgering them about why they aren't building something with sliding 0:21:06.916 --> 0:21:10.676 doors in third row seating. I don't know. Doesn't it 0:21:10.676 --> 0:21:12.836 strike you that they should have at least thought about 0:21:12.836 --> 0:21:13.516 this a bit more? 0:21:13.876 --> 0:21:17.116 I mean you you started by going through a really 0:21:17.156 --> 0:21:20.356 elegant description about how much care you take to make 0:21:20.396 --> 0:21:24.236 sure tests do not have some element of cultural or 0:21:25.116 --> 0:21:29.756 uh you know, group unfairness, which I thought was super interesting. 0:21:30.516 --> 0:21:35.076 But now you just you. But you simultaneously have imposed 0:21:35.076 --> 0:21:38.876 a system which which discriminates against someone who, for example, 0:21:38.916 --> 0:21:41.476 is a slow reader. You're You're, on the one hand, 0:21:41.596 --> 0:21:44.756 beautifully sensitive to the notion that the test might be 0:21:44.756 --> 0:21:49.196 disadvantaging a certain kind of person. But in this, in this, 0:21:49.396 --> 0:21:54.436 in the same breath, you are completely insensitive to the 0:21:54.516 --> 0:21:58.196 kind of person who wants to take their time that 0:21:58.236 --> 0:22:00.196 will may be difficult. I'm just this is genuinely this 0:22:00.356 --> 0:22:00.876 was my question. 0:22:02.036 --> 0:22:02.756 You're so. 0:22:04.276 --> 0:22:06.636 To provide the information the test, does we have to 0:22:06.676 --> 0:22:09.996 do it in the standardized way you're suggests is a 0:22:10.036 --> 0:22:11.676 different approach to these. 0:22:11.476 --> 0:22:16.756 Tests, And we couldn't, of course do that. Willy nilly. 0:22:19.396 --> 0:22:23.996 They'll tinker and rewrite and rethink and restructure the questions, 0:22:24.636 --> 0:22:28.876 but not the format. No, that's willy nilly. The thirty 0:22:28.916 --> 0:22:32.276 five minute time limits on each section are cast in stone. 0:22:33.436 --> 0:22:36.796 Why they cast in stone because the job of the 0:22:36.956 --> 0:22:39.516 l set is to make it easier for law schools 0:22:39.516 --> 0:22:43.476 to decide which students to admit. And what would have 0:22:43.516 --> 0:22:45.836 happened if I had been able to carry over my 0:22:45.916 --> 0:22:49.356 extra time, or if that thirty five minutes was turned 0:22:49.396 --> 0:22:53.316 into forty five minutes, I would have scored higher, so 0:22:53.356 --> 0:22:56.756 would have lots of other tortoises. Give tortoises an extra 0:22:56.836 --> 0:22:59.236 ten minutes and suddenly some of them catch up to 0:22:59.236 --> 0:23:03.436 the hairs. But then what is that done? Now it's 0:23:03.436 --> 0:23:06.076 harder for law schools to decide which students to admit. 0:23:08.756 --> 0:23:11.116 Back I was preparing for the l SAT over at 0:23:11.156 --> 0:23:14.476 the ed tech company Noodle, I asked their experts to 0:23:14.516 --> 0:23:18.276 game this out. One of the Noodle guys, Fritz Stewart, said, 0:23:18.276 --> 0:23:21.076 you could relieve the time pressure for a significant number 0:23:21.116 --> 0:23:23.956 of tortoises if you extended the l SAT to one 0:23:24.036 --> 0:23:26.716 hundred and twenty five percent of its current length. 0:23:27.036 --> 0:23:29.076 If we did win to one hundred and twenty five percent, 0:23:29.436 --> 0:23:30.716 So what specifically we do it. 0:23:30.796 --> 0:23:33.836