WEBVTT - Using AI to Help Doctors Save Lives 0:00:15.356 --> 0:00:23.516 Pushkin. When you walk into a hospital, technology is everywhere. 0:00:23.716 --> 0:00:25.756 In one room, a surgeon is giving a patient a 0:00:25.796 --> 0:00:29.716 bionic knee. In another room, a CT scanner is creating 0:00:29.716 --> 0:00:32.676 this incredible three D picture of the inside of a 0:00:32.716 --> 0:00:37.636 person's body. But in other places the hospital feels less 0:00:37.676 --> 0:00:42.636 high tech. Doctors are still reading patients charts and making 0:00:42.676 --> 0:00:46.956 decisions partly on evidence but largely on instinct. This part 0:00:46.996 --> 0:00:49.116 of the hospital is not so different from what it 0:00:49.196 --> 0:00:52.756 might have looked like fifty years ago, and bringing new 0:00:52.836 --> 0:00:56.476 technology to this part of medicine to care at the 0:00:56.476 --> 0:01:01.196 bedside is a really hard, really interesting problem, because you 0:01:01.316 --> 0:01:03.676 not only have to figure out how to use technology 0:01:03.756 --> 0:01:07.076 to deliver useful information to the doctor at the right time, 0:01:07.676 --> 0:01:10.396 you also have to figure out how to convince the 0:01:10.476 --> 0:01:19.836 doctor that the information is actually worth listening to. I'm 0:01:19.916 --> 0:01:22.396 Jacob Bolgstein and this is What's Your Problem, the show 0:01:22.396 --> 0:01:24.676 where I talk to people who are trying to make 0:01:24.796 --> 0:01:29.276 technological progress. My guest today is Succi Sariya. She's the 0:01:29.356 --> 0:01:32.516 founder and CEO of a company called Baesian Health, and 0:01:32.556 --> 0:01:35.716 she's also a professor at Johns Hopkins, where she runs 0:01:35.756 --> 0:01:40.316 a lab focused on machine learning and healthcare Succi's problem 0:01:40.396 --> 0:01:44.236 is this, how can you use artificial intelligence to detect 0:01:44.476 --> 0:01:48.676 when hospital patients are at risk of potentially deadly complications? 0:01:49.276 --> 0:01:51.796 And then once you've done that, how can you get 0:01:51.836 --> 0:01:55.476 doctors to believe that the AI's warning is worth paying 0:01:55.516 --> 0:01:58.716 attention to. She told me she first got interested in 0:01:58.756 --> 0:02:01.396 healthcare sort of by accident, when she was a grad 0:02:01.396 --> 0:02:04.356 student at Stanford studying AI and robots. 0:02:08.156 --> 0:02:11.196 You know, I crew actually being fascinated by AI. I 0:02:11.236 --> 0:02:13.876 love DAI, and really most of my interest was in 0:02:13.996 --> 0:02:16.436 the algorithm front and like looking at robotics and building 0:02:16.556 --> 0:02:19.436 robots that were really smart, you know. And I got 0:02:19.476 --> 0:02:24.036 acquainted with medicine through a friend colleague who was a 0:02:24.076 --> 0:02:27.996 doctor taking care of babies, And what I learned through 0:02:28.036 --> 0:02:31.076 her was that this is all this data we're starting 0:02:31.076 --> 0:02:35.796 to collect, but literally nobody was doing designing any software 0:02:35.836 --> 0:02:38.596 to make sense of it. So it was just coming 0:02:38.636 --> 0:02:42.956 from a world where you know, I studied all kinds 0:02:42.996 --> 0:02:48.396 of data day in day out, with robots doing fun 0:02:48.516 --> 0:02:50.756 tasks like getting the robot to hold the ball or 0:02:50.796 --> 0:02:54.556 juggle the ball to then realizing, holy crap, there's like 0:02:54.996 --> 0:02:57.196 so many more useful things we could be doing. So 0:02:57.196 --> 0:03:00.636 that was really my first discovery of like how big 0:03:00.636 --> 0:03:03.156 a gap there was between people who thought about AI 0:03:03.276 --> 0:03:06.196 versus people versus the problems that needed to be solved, 0:03:06.196 --> 0:03:08.156 and how little we understood about these problems. 0:03:08.636 --> 0:03:11.716 So so you decide that this is going to be 0:03:11.796 --> 0:03:13.996 your thing, right, this is your life's work now. 0:03:14.156 --> 0:03:17.036 I mean in the beginning, I wasn't convinced. In the beginning, 0:03:17.156 --> 0:03:20.196 it was just about spending a few years helping out 0:03:20.436 --> 0:03:22.556 and making sure we are able to make you know, 0:03:22.996 --> 0:03:25.116 in the beginning, it was about my next three years. 0:03:25.836 --> 0:03:28.556 Like I was afraid of investine. I was afraid of 0:03:28.596 --> 0:03:31.756 the complexity of medicine. Like it wasn't an easy field. 0:03:31.756 --> 0:03:35.036 It's not one where they welcome you, right, just as 0:03:35.076 --> 0:03:37.596 an engineer, you don't come in and like at least 0:03:37.636 --> 0:03:40.036 twelve thirteen years ago, that wasn't the culture. 0:03:39.756 --> 0:03:42.996 That like right, Like like an MD an MD at 0:03:42.996 --> 0:03:46.196 a hospital does not want to hear from some AI researcher. 0:03:46.236 --> 0:03:50.436 They're busy, Oh no, for sure, And they're like, we're busy, 0:03:50.516 --> 0:03:51.516 we have real work to do. 0:03:51.796 --> 0:03:53.316 Yeah, what is this? 0:03:53.756 --> 0:03:56.036 Like this all sounds an esoteric mumbo jumbo. 0:03:56.356 --> 0:03:59.556 Yeah, And so you say, you know, we're collecting all 0:03:59.556 --> 0:04:02.116 this data and healthcare and we're not doing anything with it. 0:04:02.956 --> 0:04:06.956 That is not intuitive, Like, that's not you know, I 0:04:06.956 --> 0:04:08.956 think most people sort of prior and this is that 0:04:09.156 --> 0:04:11.756 an academic hospital, right, Your friend is at Stanford Hospital, 0:04:11.756 --> 0:04:14.636 a very prestigious academic hospitel. I think Stanford Hospital, I 0:04:14.636 --> 0:04:17.236 think data. I think these are people doing research. So 0:04:17.276 --> 0:04:18.996 what do you mean when you say we're collecting all 0:04:19.036 --> 0:04:20.476 this data and not doing anything with it. 0:04:20.676 --> 0:04:23.796 Yeah, So twelve thirteen, fourteen years ago, this field was 0:04:23.916 --> 0:04:27.396 very new and at the time even collecting and storing 0:04:27.436 --> 0:04:30.876 this data, natural question was can be afforded? It costs 0:04:30.916 --> 0:04:33.196 dollars to store this data? Why would we do that? 0:04:33.436 --> 0:04:35.036 And when you say, what what kind of data are 0:04:35.036 --> 0:04:36.996 you talking about here? When you say collect and store 0:04:37.036 --> 0:04:37.876 this data? 0:04:37.956 --> 0:04:41.476 So literally, this was at the time babies entering, you know, 0:04:41.516 --> 0:04:44.436 in the new natle ICU, these are premature babies are 0:04:44.436 --> 0:04:47.716 born in real time. Devices are collecting heart rate and 0:04:47.796 --> 0:04:51.636 vitals and oxygen saturation data and like, and so that 0:04:51.716 --> 0:04:54.596 kind of detailed data, which is much more bulky, was 0:04:54.756 --> 0:04:57.276 historically not stored. Instead, what they would do is they'd 0:04:57.276 --> 0:05:01.756 take like fifteen minute averages and capture that okay, And 0:05:02.196 --> 0:05:04.236 naturally the question came up, do we need to store it? 0:05:04.276 --> 0:05:06.636 This is really expensive data. Let's just throw it away 0:05:07.036 --> 0:05:09.236 after forty eight hours, we don't need it anymore, or 0:05:09.316 --> 0:05:10.796 let's just throw a quick summary of it. 0:05:10.956 --> 0:05:12.996 Huh. So you might do a study, you might track 0:05:13.116 --> 0:05:15.636 certain data points, but the idea that you're going to 0:05:15.756 --> 0:05:18.996 just as a matter of course, be storing all of 0:05:19.036 --> 0:05:22.956 this data that is now being generated and saved because 0:05:23.356 --> 0:05:27.276 electronic medical records are just being adopted. Nobody was doing that. 0:05:27.356 --> 0:05:28.876 Nobody had really thought to do it. It was an 0:05:28.916 --> 0:05:31.796 expensive prospect, It didn't seem like there would be a 0:05:31.796 --> 0:05:33.796 good reason to do it exactly. 0:05:34.196 --> 0:05:39.036 And coming from AI, where we looked at you know, 0:05:39.156 --> 0:05:43.596 fingerprint data on the internet in retail or finance, then 0:05:43.836 --> 0:05:47.116 the you know, we it was so natural to think 0:05:47.156 --> 0:05:50.756 about how this data teaches you things that it felt 0:05:51.036 --> 0:05:54.156 crazy to me that like, we one similarly than all 0:05:54.196 --> 0:05:57.276 sorts of amazing things about these babies or human body 0:05:57.356 --> 0:05:59.596 or how we're involved, or like what are the signs 0:05:59.596 --> 0:06:02.276 and fingerprints of disease? How did they show up? 0:06:02.516 --> 0:06:05.276 When you say fingerprint data, that's a that's a metaphor, right, 0:06:05.476 --> 0:06:08.276 what does fingerprint data mean in the context of sort 0:06:08.316 --> 0:06:10.316 of eCOM online finance. 0:06:11.076 --> 0:06:13.316 Well, like they went to this site and then they 0:06:13.316 --> 0:06:16.316 came to this site, or like they saw and add 0:06:16.436 --> 0:06:19.396 somewhere else about this, and now you know they're searching 0:06:20.276 --> 0:06:22.836 for something, and it shows you intense. 0:06:22.436 --> 0:06:26.316 It's this moment ten years ago when like the when 0:06:26.316 --> 0:06:29.236 people are using data to know like everything about what 0:06:29.316 --> 0:06:32.396 I do when I'm shopping for new shoes. But you 0:06:32.396 --> 0:06:35.236 you're but they're not collecting data on like sick newborn 0:06:35.276 --> 0:06:36.836 babies exactly right. 0:06:37.236 --> 0:06:39.316 Is that mind blowing to you? Because it was a 0:06:40.036 --> 0:06:41.476 crazy mind blowing to me. 0:06:41.916 --> 0:06:44.876 Okay, yes, my mind is blown. So what do you 0:06:44.916 --> 0:06:47.036 do well? 0:06:47.436 --> 0:06:50.316 I mean, it seemed like such a pressing problem. It 0:06:50.356 --> 0:06:53.196 also helped that we were funded as a moonshot project 0:06:53.276 --> 0:06:58.516 by the Google founders, that it was a high profile investment, 0:06:58.596 --> 0:07:01.316 and it sort of naturally led away for United place 0:07:01.396 --> 0:07:05.796 like Stanford curiosity, and we had some amazing collaborators who 0:07:05.796 --> 0:07:09.876 were equally curious, who said, well, let's dive in and 0:07:09.916 --> 0:07:13.876 see what we'll understand. And that was the start of it. 0:07:14.396 --> 0:07:17.316 I literally, you know, got hold of this massive, twelve 0:07:17.396 --> 0:07:21.596 hundred page like this huge, big book to learn about 0:07:21.676 --> 0:07:23.876 babies and what conditions they experience and what does it 0:07:23.916 --> 0:07:25.916 all mean, and then starting to understand how does it 0:07:25.916 --> 0:07:29.516 show up in the data, and you know, spent evenings 0:07:29.556 --> 0:07:32.796 and weekends, and actually I remember sitting in the basement 0:07:32.876 --> 0:07:38.596 of Stanford Hospital at over Christmas trying to work on 0:07:38.676 --> 0:07:40.676 trying to get data out of the health record in 0:07:40.716 --> 0:07:42.636 the first place. And we were trying to experiment with 0:07:42.676 --> 0:07:45.276 all sorts of techniques for pulling the data out, which 0:07:45.556 --> 0:07:47.636 you know now is a whole lot easier than it 0:07:47.716 --> 0:07:49.276 was twelve years ago because. 0:07:49.076 --> 0:07:52.396 It's not built for that, right, It's basically built somewhat 0:07:52.396 --> 0:07:54.556 to track the patient and to a significant degree to 0:07:54.636 --> 0:07:58.276 like build insurance. Right, that's traditionally what electronic medical records 0:07:58.316 --> 0:07:58.676 were for. 0:07:59.036 --> 0:08:00.156 That's exactly right. 0:08:00.156 --> 0:08:02.556 Kind of amazing and kind of weird. I mean, I 0:08:02.556 --> 0:08:05.996 want to talk more about the bigger idea of data 0:08:06.196 --> 0:08:11.476 and healthcare, but just to kind of land this moment 0:08:11.996 --> 0:08:14.876 early in your career at Stanford, like, is there some 0:08:15.036 --> 0:08:17.076 project you do, Like what is the end of your 0:08:17.076 --> 0:08:17.956 work at Stanford. 0:08:18.396 --> 0:08:22.436 So the project was, you know, we're monitoring these premature 0:08:22.476 --> 0:08:26.556 babies right anywhere between twenty four week old babies which 0:08:26.596 --> 0:08:29.396 are very very tiny, like very twenty. 0:08:29.116 --> 0:08:31.876 Four weeks of gestation exactly. 0:08:31.956 --> 0:08:36.556 To like twenty eight thirty thirty two. And the idea was, 0:08:36.596 --> 0:08:40.436 these babies, you know, are like they're at risk for significant, 0:08:40.476 --> 0:08:43.596 like an array of complications. And the idea is, the 0:08:43.716 --> 0:08:47.236 sooner you know, the earlier you can do something about it, 0:08:47.356 --> 0:08:49.876 the greater the chance that you're going to actually resuscitate them. 0:08:50.356 --> 0:08:52.476 So our job was, like, could we look at this 0:08:52.556 --> 0:08:55.596 data from the second they're born and collect this data 0:08:55.636 --> 0:08:59.596 to start analyzing and modeling which babies at risk for 0:08:59.636 --> 0:09:02.116 which of these complications? And if you could, then you 0:09:02.156 --> 0:09:06.236 could start to put more of these preventative prophylactic type 0:09:06.436 --> 0:09:08.916 pathways or approaches in place for caring. 0:09:08.676 --> 0:09:12.556 Basically identify problems more quickly leading to better outcomes. That's 0:09:12.596 --> 0:09:14.836 the basic desire exactly. 0:09:14.836 --> 0:09:17.876 And in the process I discovered, like, you know, a 0:09:17.956 --> 0:09:21.276 long time ago, there was a physician named Virginia Apcar, 0:09:21.796 --> 0:09:24.996 and what she figured out is like, just by measuring 0:09:25.796 --> 0:09:29.316 five different things from when the baby's born, she can 0:09:29.556 --> 0:09:31.596 compute a very simple score that tells you how the 0:09:31.636 --> 0:09:35.836 baby's doing. And so so naturally, the question we asked is, okay, 0:09:35.876 --> 0:09:37.756 so now that we are seeing all these ways in 0:09:37.756 --> 0:09:41.676 which the machine learning and AI is discovering novel signs 0:09:41.676 --> 0:09:45.156 and patterns that are predictive. Could we just simply combine 0:09:45.196 --> 0:09:48.316 this to come up with a simple score. Huh that says, 0:09:48.796 --> 0:09:51.596 you know, can I predict complications? And what we found 0:09:51.676 --> 0:09:54.276 was this new simple score that uses data that no 0:09:54.396 --> 0:09:56.676 special thing you have to do, it's already being collected. 0:09:57.156 --> 0:09:59.076 We just analyze it and we aught to compute the 0:09:59.116 --> 0:10:01.756 score turns out to be much more predictive than the 0:10:01.756 --> 0:10:04.036 APCAR at predicting complications. 0:10:04.996 --> 0:10:07.876 And so so it worked. I mean, did do people 0:10:07.956 --> 0:10:10.236 use it? Is it standard of care? Now? What happened 0:10:10.236 --> 0:10:11.516 with that research? 0:10:12.036 --> 0:10:13.716 So at that point I was like, oh, this is 0:10:13.756 --> 0:10:16.556 so cool. And literally we got all these journalists who 0:10:16.596 --> 0:10:18.236 wanted to write about it, and it was on the 0:10:18.276 --> 0:10:22.316 fundraising you know, it was like Stanford's fundraising highlight for 0:10:22.356 --> 0:10:25.276 like the next five years, et cetera. But what was 0:10:25.316 --> 0:10:27.076 the saddest thing about it is that there was no 0:10:27.196 --> 0:10:31.356 natural mechanism for implementing it in practice, And it had 0:10:31.396 --> 0:10:33.316 to do with so many different pieces to it, Like 0:10:33.676 --> 0:10:36.956 we didn't have the infrastructure, we didn't have the like 0:10:37.236 --> 0:10:39.716 know how of like how do you get physicians to 0:10:39.756 --> 0:10:42.156 trust something like this? How do you build this in 0:10:42.196 --> 0:10:44.756 a way that is trustworthy and reliable? How do you 0:10:44.796 --> 0:10:46.996 do this so that it's not just like a pet 0:10:47.076 --> 0:10:50.036 project in one hospital, but it's like a system that 0:10:50.156 --> 0:10:54.156 is scalable nationally. And you know, what is the incentive structure? 0:10:54.156 --> 0:10:56.036 Who pays for it and why would they pay for it? 0:10:56.076 --> 0:10:59.676 And all of that is literally what sort of got me, 0:10:59.996 --> 0:11:02.316 like got me super interested in the field where I 0:11:02.516 --> 0:11:04.716 started to feel, wow, we're at the start of what 0:11:05.156 --> 0:11:09.396 feels like is a massive movement, has many component to 0:11:09.436 --> 0:11:12.516 be figured out, but we need to figure this out. Interestingly, 0:11:13.356 --> 0:11:16.996 at the time on Sandhill Road, you know why, virtually 0:11:17.036 --> 0:11:18.156 being in Palo Alto. 0:11:18.116 --> 0:11:21.716 Yeah, Santel Road, where all the venture capitalists are exactly. 0:11:21.316 --> 0:11:24.796 People were like, this is fantastic, here's money. Why don't 0:11:24.836 --> 0:11:28.236 you start a company on this topic? And I spent 0:11:28.356 --> 0:11:31.756 six months investigating, you know, talking to lots of peers 0:11:33.356 --> 0:11:38.596 health systems, hospitals and realizing were just too early. There's 0:11:38.636 --> 0:11:40.596 a lot of work that needs to go in place 0:11:40.836 --> 0:11:43.756 for this to become something that will stale nationally. Now 0:11:43.796 --> 0:11:45.276 fast forward ten years. 0:11:45.116 --> 0:11:46.836 Later, I want to fast forward, but give me just 0:11:46.876 --> 0:11:50.276 another moment when you say it's too early, Like in 0:11:50.316 --> 0:11:53.356 what ways was it too early? Like specifically what was 0:11:53.436 --> 0:11:56.236 not ready in the world to start a company at 0:11:56.236 --> 0:11:56.596 that time. 0:11:56.676 --> 0:11:58.636 So the first thing we needed is for hospitals to 0:11:58.636 --> 0:12:00.916 be ready to implement a system like that. For that 0:12:00.956 --> 0:12:04.116 to happen, they needed to have implemented the electronic health record, 0:12:05.036 --> 0:12:07.516 be stable users of the HR so that they'd be 0:12:07.516 --> 0:12:09.756 willing to plug in third party systems on top of it. 0:12:09.996 --> 0:12:12.956 And it's kind of amazing that ten years ago, you know, 0:12:13.316 --> 0:12:19.156 twenty whatever, twenty teens, still hospitals were not sort of 0:12:19.276 --> 0:12:23.316 ubiquitous users of electronic medical records, right, like doctors were 0:12:23.356 --> 0:12:24.556 still writing on paper. 0:12:25.276 --> 0:12:28.196 Honestly, coming from computer science where I did you know, 0:12:28.236 --> 0:12:30.836 where I was involved in other areas of AI and 0:12:30.876 --> 0:12:34.356 computer science, like this was like the biggest like shift 0:12:34.636 --> 0:12:38.156 in mindset I felt every time I came back into 0:12:38.156 --> 0:12:39.956 the healthcare side of the equation, it felt like I 0:12:40.036 --> 0:12:42.836 was going at least twenty thirty years back. 0:12:42.796 --> 0:12:45.036 Right like at a time machine going into the past 0:12:45.076 --> 0:12:48.636 when you walk into the hospital, which is particularly I 0:12:48.676 --> 0:12:53.356 don't know, ironic surprising, given how in some ways healthcare 0:12:53.436 --> 0:12:56.956 feels very cutting edge, right, Like A central interesting thing 0:12:56.996 --> 0:13:00.596 to me about the work that you do is the 0:13:00.596 --> 0:13:02.916 way in which healthcare is. You know, you go get 0:13:02.956 --> 0:13:06.396 a whatever, a CT scan. It's this incredible machine and 0:13:06.436 --> 0:13:10.476 it uploads to a computer and a whatever AI radiologists. 0:13:10.476 --> 0:13:13.076 Can you read the scan blah blah blah. And yet 0:13:14.196 --> 0:13:17.156 on the kind of data side, on the complicated patient 0:13:17.236 --> 0:13:20.636 at the bedside side, it's still very kind of old 0:13:20.676 --> 0:13:22.316 fashioned and almost artisanal. 0:13:23.396 --> 0:13:26.276 I mean, you raised like a fantastic point, which is 0:13:26.556 --> 0:13:31.076 I think when it comes to introducing and designing new medicines, Yeah, 0:13:31.116 --> 0:13:35.036 we've become really really good, But in terms of once 0:13:35.076 --> 0:13:39.476 the medicine is produced, in terms of actually accelerating the adoption, 0:13:39.676 --> 0:13:43.876 optimizing the update, designing who gets it and what does 0:13:43.916 --> 0:13:48.236 and when detecting early who would benefit from it, that's 0:13:48.276 --> 0:13:50.556 what I call the healthcare delivery side of the equation. 0:13:50.836 --> 0:13:54.156 I feel like there's a very very vast gap of 0:13:54.196 --> 0:13:55.716 what needs to happen to get better. 0:13:56.476 --> 0:14:00.396 So, okay, so you do this project. You see that 0:14:00.476 --> 0:14:04.916 it's too early to start a company because the world 0:14:04.956 --> 0:14:08.476 isn't ready yet, because hospitals aren't even widely using electronic 0:14:08.516 --> 0:14:11.156 medical record yet, much less being ready to sort of 0:14:11.396 --> 0:14:14.756 expert the data and listen to the data, et cetera, 0:14:15.156 --> 0:14:19.476 and you take a job as a professor at Johns Hopkins. Right, 0:14:19.516 --> 0:14:20.356 is that the next step? 0:14:20.956 --> 0:14:23.956 That's right? And part of the move to Hopkins was 0:14:24.556 --> 0:14:28.756 realizing there's so much depth and breadth of medicine, not 0:14:28.796 --> 0:14:31.956 just around on the actual devices or the engineering or 0:14:31.956 --> 0:14:34.516 the chemical or the drug development, but also on the 0:14:34.556 --> 0:14:37.476 delivery side, like how what does it take to scale 0:14:37.516 --> 0:14:41.516 ideas nationally? How do you design policy around it? There 0:14:41.596 --> 0:14:46.076 was sort of a whole institute dediclicated to scaling ideas nationally. 0:14:46.516 --> 0:14:49.956 So to me that was extremely exciting to learn about 0:14:50.396 --> 0:14:54.276 what would it take to really build the foundations of 0:14:54.356 --> 0:14:56.756 a field like this. And moving to Baltimore was a 0:14:56.796 --> 0:15:00.196 big move, but I was just excited by the idea 0:15:00.236 --> 0:15:02.476 of learning it all and learning it especially as an 0:15:02.476 --> 0:15:05.636 engineer as ERNII research as an outsider coming into healthcare. 0:15:08.076 --> 0:15:10.636 In a minute, Succi and her colleagues figure out how 0:15:10.676 --> 0:15:13.636 to use AI to detect when certain patients are at 0:15:13.716 --> 0:15:18.316 risk for complications and also how to get doctors to listen. 0:15:28.396 --> 0:15:31.156 So Sucia is at Johns Hopkins in Baltimore and she 0:15:31.236 --> 0:15:35.276 has this big idea using AI to help doctors treat 0:15:35.356 --> 0:15:38.556 hospital patients, but she has to figure out exactly what 0:15:38.716 --> 0:15:39.956 to focus on. 0:15:39.956 --> 0:15:42.036 One of the big areas was this idea of like 0:15:42.156 --> 0:15:48.036 early detection of patients at risk for complications and diagnostic 0:15:48.156 --> 0:15:50.836 errors being the third leading cause of death. Like, huh, 0:15:50.916 --> 0:15:54.996 that's nuts. Like, so today, you know there are critical 0:15:54.996 --> 0:15:57.956 moments that are missed. We get patients the wrong diagnosis 0:15:58.036 --> 0:16:01.036 or that they're developing something subtly and slowly. That's like 0:16:01.036 --> 0:16:04.596 a whole branch of diagnostic errors where you know, complication 0:16:04.876 --> 0:16:07.316 or a condition develops, but they don't get noticed in 0:16:07.356 --> 0:16:12.436 a timely fashion. And so these seemed perfect for AI 0:16:12.556 --> 0:16:14.756 to come in with the kind of data that exists 0:16:15.116 --> 0:16:17.636 to be able to flag patients that are high risk 0:16:18.076 --> 0:16:20.076 and make it easy to provide a second pair of VICE. 0:16:20.076 --> 0:16:24.076 Because it's basically pattern matching, right, I mean, differential diagnosis 0:16:24.156 --> 0:16:28.436 is taking lots of different variables from the patient and 0:16:29.516 --> 0:16:33.236 trying to put those variables together to match the patient 0:16:33.316 --> 0:16:36.876 to you know, thousands of other patients and say, oh, 0:16:36.996 --> 0:16:40.436 all of these, all of these variables, all of these 0:16:40.556 --> 0:16:43.556 health indicators suggest that the patient has disease X. Like 0:16:43.556 --> 0:16:47.036 that's fundamentally what a differential diagnosis is, and like machine 0:16:47.076 --> 0:16:50.196 learning should be very good at that exactly. 0:16:50.316 --> 0:16:54.236 And previously people have attempted a differential diagnosis with very 0:16:54.356 --> 0:16:57.916 coarse symptoms like high level description of like you have 0:16:57.996 --> 0:17:00.916 cop your fever. What was different this time around is 0:17:00.916 --> 0:17:03.356 because of the HR, we had very detailed data. 0:17:03.156 --> 0:17:05.236 The EHR, the electronic health record right. 0:17:05.116 --> 0:17:09.876 Exactly, and so it provided this brand new opportunity to 0:17:10.436 --> 0:17:12.756 do this. And then you know, naturally when you go 0:17:12.796 --> 0:17:16.116 down the list and start looking at problem areas, sepsis 0:17:16.236 --> 0:17:19.756 is a model disease. We chose to demonstrate the idea. 0:17:20.196 --> 0:17:23.036 So let's just talk about sepsis for a minute. What 0:17:23.196 --> 0:17:23.756 is sepsis? 0:17:24.036 --> 0:17:27.396 So let's say a patient gets infected. Your immune system 0:17:27.476 --> 0:17:30.156 is now going to do respond in order to protect 0:17:30.156 --> 0:17:34.876 your body. But in sepsis, it overreacts and starts attacking 0:17:34.916 --> 0:17:40.876 your organ systems, leading to organ failure and your depth. 0:17:41.036 --> 0:17:43.876 And so the idea of its sepsis treatment is very 0:17:43.956 --> 0:17:46.196 much the earlier you can detect it, the better you 0:17:46.236 --> 0:17:48.156 are at like tackling it. 0:17:48.276 --> 0:17:54.036 Right, Okay, so I buy it. It seems seems like 0:17:54.076 --> 0:17:55.916 a big problem and it seems like one that might 0:17:55.956 --> 0:17:59.676 be solved or at least, you know, made less bad 0:17:59.916 --> 0:18:03.436 by with the application of machine learning. So how do 0:18:03.516 --> 0:18:06.316 you how do you actually do it? What do you 0:18:06.316 --> 0:18:08.836 have to do to build a model and see if 0:18:08.876 --> 0:18:10.476 it were and get people to use it. 0:18:10.596 --> 0:18:13.436 Yeah, so this is almost like what you're about to 0:18:13.476 --> 0:18:16.076 describe in two minutes what was almost a five year journey. 0:18:16.476 --> 0:18:18.956 So first, it's collecting a huge amount of data where 0:18:18.996 --> 0:18:22.636 you can identify both patients of subtic versus non septic 0:18:22.636 --> 0:18:24.556 and when they had it, and what other conditions did 0:18:24.556 --> 0:18:26.636 they have, and what else was happening in their life, 0:18:27.236 --> 0:18:29.196 and you know, all the data leading up to that 0:18:29.276 --> 0:18:32.236 episode and what was done after the fact. So you 0:18:32.276 --> 0:18:34.316 get the data. Then the next part is, you know, 0:18:34.356 --> 0:18:36.956 you have to actually understand the biological process or the 0:18:36.956 --> 0:18:39.636 clinical process that's happening and layer that on top of 0:18:39.716 --> 0:18:41.556 the data to make sure you're going from like just 0:18:41.596 --> 0:18:44.476 bits and bytes to data that makes sense, okay, And 0:18:44.796 --> 0:18:48.596 then you implement lots of different learning algorithms to be 0:18:48.676 --> 0:18:51.236 able to experiment, you know, the thing that we first 0:18:51.236 --> 0:18:54.436 did versus the thing we do now. There's like lots 0:18:54.476 --> 0:18:56.716 of generations of improvements in order to get to a 0:18:56.716 --> 0:19:00.756 place where you're going from, like you know, not very 0:19:00.756 --> 0:19:02.676 good signal to very good signal. 0:19:03.156 --> 0:19:06.156 So you're you're building a model through trial and error, 0:19:06.156 --> 0:19:09.636 basically trying to get an AI model that that has 0:19:09.676 --> 0:19:13.756 a high sensitivity and specificity that's good at issuing an 0:19:13.796 --> 0:19:17.396 alert when a patient has sepsis and doesn't issue too 0:19:17.436 --> 0:19:19.476 many alerts when the patient doesn't have sepsis. 0:19:19.116 --> 0:19:21.636 Basically exactly, and also does it in a way that 0:19:21.996 --> 0:19:24.676 you know, when it says somebody has sepsis, it's able 0:19:24.676 --> 0:19:27.596 to explain why. It's able to provide enough information so 0:19:27.636 --> 0:19:30.796 that the clinician can act on it. And it's not 0:19:30.876 --> 0:19:33.556 doing it sorely that there's not enough to work on, 0:19:33.596 --> 0:19:35.916 and it's not doing it so late that it's useless. 0:19:36.236 --> 0:19:40.876 Like often people talk about AI models machine learning models 0:19:40.956 --> 0:19:44.236 as black boxes, right, like very good at pattern matching, 0:19:44.316 --> 0:19:46.716 very good at predicting the next word, but we don't 0:19:46.756 --> 0:19:49.076 know why, And so you're saying, in this instance, you 0:19:49.156 --> 0:19:50.396 sort of need to know why. 0:19:51.796 --> 0:19:54.676 My very key evolution of a scientist working in this 0:19:54.796 --> 0:19:56.636 area was, in the beginning, I saw it all as 0:19:56.716 --> 0:20:00.876 data and math, and then as I started working more 0:20:00.876 --> 0:20:03.356 and more in interfacing and actually deploying systems like this. 0:20:03.476 --> 0:20:06.276 What I started realizing it's actually not math and data. 0:20:06.316 --> 0:20:10.956 It's about trust, huh, Because ultimately to get adoption and 0:20:10.996 --> 0:20:14.036 to get outcomes, I need to get trust from these 0:20:14.236 --> 0:20:19.316 highly trained clinicians who studied this year and year out, 0:20:19.516 --> 0:20:22.916 and they have a process and a system for working, 0:20:22.956 --> 0:20:24.956 and you have to fit within this system. 0:20:25.076 --> 0:20:27.596 And they're very busy, and it's very high stakes, and 0:20:27.676 --> 0:20:31.036 they kind of think they know everything, and it's so 0:20:31.196 --> 0:20:36.156 presumably very hard to get them to trust you in 0:20:36.236 --> 0:20:38.756 making their clinical judgments exactly. 0:20:38.836 --> 0:20:41.876 But moreover, I've also been on the other side of 0:20:41.996 --> 0:20:44.716 like tons of engineers making all sorts of claims about 0:20:44.716 --> 0:20:47.596 their system knows better. But when you actually go and 0:20:47.636 --> 0:20:50.876 make sense of what the evaluations they've done, they literally 0:20:50.956 --> 0:20:54.196 have very little understanding of medicine and the practice of healthcare. 0:20:54.276 --> 0:20:58.436 So like their claims are mostly not good. So a 0:20:58.556 --> 0:21:01.276 huge part of it is like developing respect and humility 0:21:01.716 --> 0:21:04.436 for the system, the complexity, so that when you're bringing 0:21:04.476 --> 0:21:08.036 in this new thing, it really truly fits. It's easy 0:21:08.076 --> 0:21:12.996 to use sense it creates value. Without all that, you're 0:21:13.036 --> 0:21:14.796 not going to get to the benefit. 0:21:16.196 --> 0:21:19.196 So now you say it creates value and suddenly you 0:21:19.276 --> 0:21:23.676 sound like a founder, an entrepreneur, and not like an academic. 0:21:24.276 --> 0:21:28.276 Where where in this arc do you start a company? 0:21:28.396 --> 0:21:30.636 You know, it was somewhere in twenty eighteen. I remember 0:21:31.036 --> 0:21:33.356 twenty eighteen was a transformative video for me for a 0:21:33.396 --> 0:21:37.636 number of reasons. I'll start with the very simple thing 0:21:37.676 --> 0:21:42.076 of like, when we first built this system and deployed it, 0:21:42.356 --> 0:21:45.956 only like two or three clinicians used it, and it 0:21:46.036 --> 0:21:48.156