Speaker 1: Pushkin. Imagine something that is sort of like chat GPT,
Speaker 1: but for the human body. Chat GPT looks at a
Speaker 1: sentence and predicts what words are likely to come next.
Speaker 1: This thing would look at a human body and predict
Speaker 1: what diseases are likely to come next. The body is
Speaker 1: wildly complex and unpredictable. This seems like a very, very
Speaker 1: hard problem, but it is a problem people are working on,
Speaker 1: and at least in some circumstances, they're figuring out how
Speaker 1: to make predictions that are truly useful. I'm Jacob Goldstein,
Speaker 1: and this is What's Your Problem, the show where I
Speaker 1: talk to people who are trying to make technological progress.
Speaker 1: My guest today is Charles Fisher, co founder and CEO
Speaker 1: of Unlearned. Charles' problem is how do you build an
Speaker 1: AI model that can predict human health. Charles and his
Speaker 1: colleagues have built a predictive model of human health that's
Speaker 1: already being used in clinical trials for new drugs and
Speaker 1: new medical devices. But we started out talking about the
Speaker 1: big picture, about the very idea of trying to predict
Speaker 1: what's going to happen to a human body.
Speaker 2: It's funny when I talk about trying to quantify biology
Speaker 2: and make it predictable. I often get hit with this
Speaker 2: critique that biology isn't physics. Biology is complex, biology is
Speaker 2: not physics. We're not going to be able to do that.
Speaker 1: Let's deterministic.
Speaker 2: Right, So for physics, for two thousand years, right, people
Speaker 2: started working on physics in ancient Greece. And for two
Speaker 2: thousand years, physics wasn't physics. Physics was unpredictable. Physics was
Speaker 2: too complex to understand until something was invented. And that
Speaker 2: thing was calculus.
Speaker 1: Until new right.
Speaker 2: Yeah, So once calculus was invented, all of the sudden,
Speaker 2: we had a new language. In this language, this new
Speaker 2: kind of mathematics allowed us to really easily describe lots
Speaker 2: of physical phenomena. And so now physics has become this
Speaker 2: thing that's very predictable and well understood. And that's what
Speaker 2: we've been waiting for in biology. We've been waiting for
Speaker 2: a new tool, a new language, a new mathematics that
Speaker 2: will allow us to understand these complex systems. And that's
Speaker 2: really what I think these new tools are.
Speaker 1: So I think so your hope, your hope is that
Speaker 1: machine learning generative AI will do for medicine biology. What
Speaker 1: Calculus did for physics.
Speaker 2: Exactly. That is big, big, It's exactly what I hope.
Speaker 2: That's exactly what I hope.
Speaker 1: So okay, so this is your hope. You're starting this
Speaker 1: company to test your hypothesis. Uh, what do you do?
Speaker 2: What do you mean? What do I do? What I
Speaker 2: do on day one? Or like, what are we doing? No?
Speaker 1: No, no, We're back to twenty seventeen. You have this
Speaker 1: big up in the sky, two thousand year, thirty thousand
Speaker 1: foot idea. But you got to make a thing that
Speaker 1: somebody is going to pay you for that will hopefully
Speaker 1: use AI in medicine in some way. So what do
Speaker 1: you do?
Speaker 2: So we didn't know what would work, so we focused
Speaker 2: on two different problems at the time. So one problem is,
Speaker 2: let's imagine we're going to have a bunch of data
Speaker 2: from some maybe a big large collection of patients. We're
Speaker 2: gonna have this data all over time, so the symptoms
Speaker 2: that a patient might have every week, four year or
Speaker 2: something like that. And our goal is to be able
Speaker 2: to create a simulator of a patient's future health. So,
Speaker 2: given what I know about a patient in the past,
Speaker 2: can I simulate what will happen to them in the future.
Speaker 1: And presumably that is sort of probabilistic. I mean, what
Speaker 1: we know about health, Like you can say there's an
Speaker 1: x percent chance that in why years this person will
Speaker 1: have a heart attack something like that.
Speaker 2: Exactly. Yeah, we want to yes, because so many things
Speaker 2: are undetermined in that you know, maybe yeah, exactly.
Speaker 1: Right, and it's just the nature of the world, right,
Speaker 1: one hundred percent.
Speaker 2: Yeah.
Speaker 1: So okay, so you have this idea of basically where
Speaker 1: chat GBT, which didn't exist yet, but predicts the next
Speaker 1: word with some probability you want to predict the next
Speaker 1: health outcome.
Speaker 2: For exactly that is the big idea. Yeah, So that
Speaker 2: that was one of them. The other that was not
Speaker 2: the only one that was the one that is what
Speaker 2: we do. The one that we didn't do is we
Speaker 2: were interested as well potentially so that's at a very
Speaker 2: macroscopic scale, that's at the scale of the person, whereas
Speaker 2: the other thing we were interested in was potentially could
Speaker 2: we go at the micro scale and look at what's
Speaker 2: happening inside individual cells. We were interested in this at
Speaker 2: the beginning. Basically, the way we figured this out is
Speaker 2: we signed a few deals with farmer companies to try
Speaker 2: these things, and we found found that the technology worked
Speaker 2: really well in this simulating health outcomes, and it didn't
Speaker 2: work very well when it comes down to simulating what's
Speaker 2: inside the cell. And I think this comes down to data,
Speaker 2: which is that we get a ton of data on
Speaker 2: human health outcomes, like literally every time you go to
Speaker 2: the doctor, there's data there on your health outcomes. But
Speaker 2: the data from the things inside the cell, there is
Speaker 2: a lot of it, but it's much more difficult to
Speaker 2: work with. So I think that was a lot of
Speaker 2: what drove us in this direction is really the focus
Speaker 2: on what we think we have the data to solve
Speaker 2: these kinds of problems.
Speaker 1: So, Okay, you go in the direction of simulating health
Speaker 1: outcomes for patients, and in particular, sort of where you
Speaker 1: get to is working with companies that are running clinical trials.
Speaker 1: And I know eventually you get to a point where
Speaker 1: companies can use your model, use your software to run
Speaker 1: clinical trials with fewer patients. So just tell me about that,
Speaker 1: arc tell me how you get there.
Speaker 2: Clinical trials are, well, they're super tick forever, and they're
Speaker 2: really really expensive. Something might take like five years and
Speaker 2: cost one hundred million dollars to run a clinical trial. Yeah,
Speaker 2: in the way that these are hundreds or thousands of patients, right, oh,
Speaker 2: thousands of patients typically, right, Yeah, And typically half of
Speaker 2: the patients in a clinical trial are receiving a PLACBO.
Speaker 2: So you're going to randomly assign half to receive an
Speaker 2: experimental treatment have to receive a PLACBO. And the reason
Speaker 2: is that every clinical trial is ultimately just doing a comparison.
Speaker 2: You're comparing how a patient responds to the new treatment
Speaker 2: to how they respond if they don't get that treatment.
Speaker 1: And let me just give a shout out to the
Speaker 1: randomized controlled trial as like a really beautiful construct, right,
Speaker 1: not that old? Not that old. I learned that a
Speaker 1: ring for this interview, like less than one hundred years old, amazingly.
Speaker 1: But it's a perfect way to assess not perfect, it's
Speaker 1: a very very good way to assess causality. It's really elegant.
Speaker 2: It is an elegant idea. But if you're a patient,
Speaker 2: why are you participating a clinical trial at all? What's
Speaker 2: the number one reason people participate in clinic trials. They
Speaker 2: participate in clinical trials because they want access to this
Speaker 2: experimental treatment that you can't get any other way. That's
Speaker 2: the number one reason why patients are participating in clinical trials.
Speaker 2: Number one, Now they.
Speaker 1: Don't they don't want to be randomized to the placebo.
Speaker 2: No, no, no, they don't.
Speaker 1: I can certainly understand that it is the case, right
Speaker 1: that most trials fail, meaning the drug is not helping
Speaker 1: you and possibly hurting you, meaning on average, you're better
Speaker 1: off being in the placebo arm Like that is true, right.
Speaker 2: Yea, there's a principle of equipoise. But that's an academic
Speaker 2: Ivory tower principle.
Speaker 1: I mean, it also is true. Just sue, that's fine, that's.
Speaker 2: Fine, but in the end, that's like, in the end,
Speaker 2: patients choose not to participate in clinical trials because they
Speaker 2: don't want to get a placebo. Patients drop out of
Speaker 2: clinical trials when they think they are getting a placbo.
Speaker 2: Those are also true. Number one reason those things happen.
Speaker 2: Are those reasons? Fair?
Speaker 1: Okay?
Speaker 2: Right? So, And in fact, twenty percent of clinical trials
Speaker 2: failed not because the drug didn't work, but because they
Speaker 2: just couldn't find enough people to participate, okay. And what
Speaker 2: we realized though, is that there was a way for
Speaker 2: us not to try to replace the randomized control trial,
Speaker 2: but to make it better, and that what we are
Speaker 2: doing is we could take what we call digital twins
Speaker 2: of the patients, so these are these simulations of their
Speaker 2: of their future outcomes, and that we could incorporate those
Speaker 2: data into our cts directly randomized control trials. We call
Speaker 2: it just kind of like a reimagining of our cts.
Speaker 2: It's it's you're going to have a RCT that is
Speaker 2: more accurate, that is has requires fewer patients, and as
Speaker 2: a result, you get a lot of the benefits of
Speaker 2: faster trials of things that are better for the patients.
Speaker 2: We can talk about that in a minute, but you
Speaker 2: keep all of the same scientific rigger.
Speaker 1: So specifically, okay, that's a good like big picture. Specifically,
Speaker 1: how does it.
Speaker 2: Work right now? We build one model per disease. So,
Speaker 2: for example, we have a model for patients with Alzheimer's disease.
Speaker 2: We have a separate model for patients with als, we
Speaker 2: have a separate model for multiple scleroses, et cetera.
Speaker 1: Let's pick one model and talk about it. What's the
Speaker 1: one that's farthest along, Which is the one that works
Speaker 1: the best?
Speaker 2: Yeah, So our Alzheimer's disease model is that was our
Speaker 2: first one that we've published scientific papers on and things
Speaker 2: like this, so that ones our most well known.
Speaker 1: Okay, so you're setting out to build a model that
Speaker 1: will predict whether what's going to happen, presumably to a
Speaker 1: patient who has the early stages of Alzheimer's disease, How
Speaker 1: will their disease progress? A hard thing to know in
Speaker 1: the real world. How do you build that? What do
Speaker 1: you do?
Speaker 2: So the first thing is that you need data to
Speaker 2: learn from. Yeah, it's kind of obvious. So our first
Speaker 2: step was like, oh, we say, okay, we want to
Speaker 2: have data sets where we get a ton of information
Speaker 2: about each patient. What's that mean? That means that any
Speaker 2: individual time, I want to have a lot of different
Speaker 2: different measurements made on that patient at each time.
Speaker 1: So alsumably you want to have a lot of moments
Speaker 1: when lots of information exactly.
Speaker 2: You also want to have lots of.
Speaker 1: Lots of times over a long period of time, over
Speaker 1: a long period.
Speaker 2: Yeah, and so you know these are going to be
Speaker 2: for Alzheimer's. You're looking at a bunch of things related
Speaker 2: to the patient's cognitive performance on different assessments. Just also
Speaker 2: there's things about just their daily life. How are they
Speaker 2: able to function in their daily life. There's things related
Speaker 2: to their caregivers actually, like how does their caregiver rate
Speaker 2: their behavior? Brain imaging, blood tests, all that kind of information.
Speaker 2: You want to have as much of it about each patient.
Speaker 2: You want to have it as many times as possible. Sure,
Speaker 2: and we'll try to get that for you know, like
Speaker 2: fifty thousand people. And that's the kind of data set
Speaker 2: that we that we're starting with.
Speaker 1: And like, is there one repository that when you get that,
Speaker 1: you're like jackpot or what.
Speaker 2: No, we we have to aggregate data from lots and
Speaker 2: lots of different places to be able to build a
Speaker 2: big enough data set.
Speaker 1: Okay, so now you got the data, what do you
Speaker 1: do next?
Speaker 2: Then we got to train a model to to to
Speaker 2: be able to learn from those data how to simulate things.
Speaker 2: And now actually what we do.
Speaker 1: In particular in this case, how to predict, given some
Speaker 1: set of inputs for a patient, what's going to happen
Speaker 1: next exactly?
Speaker 2: And so this does look you were using that analogy
Speaker 2: of like a language model predicts the next word. So
Speaker 2: given these words I've seen before, predicts the next word.
Speaker 2: And that's that is similar to how our models and
Speaker 2: these diseases work. So we're going to say, given I've
Speaker 2: observed these things in the past about a patient, what
Speaker 2: will happen to them next? That is is very analogous
Speaker 2: to kind of what we're doing.
Speaker 1: It's okay, so you build the model, how does it work?
Speaker 1: How does it work in a clinical trial, specifically so
Speaker 1: that you know the people running the trial can can
Speaker 1: do it with fewer patients.
Speaker 2: Sure. So in a typical case, we're involved at the
Speaker 2: beginning of the clinical trial in the design of the protocol. Okay,
Speaker 2: So there's a question of how many patients should you
Speaker 2: randomize to your control group, how many patients do you
Speaker 2: need overall, and how many should be in the treatment,
Speaker 2: how many should be in.
Speaker 1: The control It's not always fifty to fifty.
Speaker 2: It's not always fifty to fifty in our studies. Our
Speaker 2: typical goal is to try to minimize the number of
Speaker 2: people that you need to put in the control group. Okay,
Speaker 2: And so we're involved in doing helping to do that
Speaker 2: calculation to say, here's how big your trial should be.
Speaker 2: And so then as patients enroll in the study, we
Speaker 2: take data from their first visit before they receive whatever
Speaker 2: new treatment they're going to receive and we take those data,
Speaker 2: we input them into our pre trained model. So I
Speaker 2: like to think about you know, CHATCHBTU give it a
Speaker 2: prompt and it gives us output. Same thing. We take
Speaker 2: the data from the patient, we prompt the model and
Speaker 2: it outputs their predictions for what will happen.
Speaker 1: In the And to be clear, you do that for
Speaker 1: all of the patients in both arms the treatments.
Speaker 2: Yes, yeah, and we don't know, right, it's blinded blind
Speaker 2: it's you, it's blinded to us. We don't know what. Yeah,
Speaker 2: So we do that for one hundred percent of the
Speaker 2: patients and then we give those data to the customer,
Speaker 2: to the farmer company.
Speaker 1: So then what happens next? What happens next?
Speaker 2: We wait around for a while. Yeah. And then when
Speaker 2: the study is actually completed, right, and they they they
Speaker 2: do unblind the data. We have to help to to
Speaker 2: say here's how you now can incorporate these these predicted
Speaker 2: outcomes into the analysis.
Speaker 1: Like so this is this is it. Now We're at
Speaker 1: the moment now when the thing you have built is useful.
Speaker 1: So so now it's it's they have done the study,
Speaker 1: they have the outcomes for the real human beings and
Speaker 1: they have the predicted outcomes from your model. How is
Speaker 1: your system? How's your model useful?
Speaker 2: So the very first thing that we're basically going to
Speaker 2: do is what I'm going to say, We're going to
Speaker 2: recalibrate our model. Recalibrate and you're going to figure out
Speaker 2: a relationship between your predicted outcomes and your observed outcomes
Speaker 2: for the patients who really received the placebo, for.
Speaker 1: The patients in the placebo group, And basically you're going
Speaker 1: to see how you did how do we do.
Speaker 2: Yes, and in particularly going to find out not just
Speaker 2: it's not like a measure of was it good or bad,
Speaker 2: You're going to find out exactly how are they related?
Speaker 2: And then you can take that information in adjust your predictions.
Speaker 2: Okay for everybody. So you can say, let's imagine that
Speaker 2: I find out, well, on average, I'm i underestimating how
Speaker 2: much a patient would progress by one point per year.
Speaker 2: I'm on average underestimating it. Well, then I'll go through
Speaker 2: and I'll take my prediction and I'll be like, well
Speaker 2: add one point, add one point forer you. And then
Speaker 2: now you have said, okay, well, now I've taken the
Speaker 2: model and I've been able to do it in such
Speaker 2: a way where I've fixed these mistakes by looking at
Speaker 2: the actual patients who got place ebo, And now I'm
Speaker 2: going to apply that model to the patient and the
Speaker 2: treatment group, and I'm going to look at Now, I
Speaker 2: just look at that difference between the patients and the
Speaker 2: treatment group and their predictions from the model, and I
Speaker 2: average that and I get an estimate for the treatment effect.
Speaker 2: Now that is described in a two stage procedure. That
Speaker 2: it's not actually a two stage procedure. It's one mathematical
Speaker 2: analysis that you do it. But the thing that's really
Speaker 2: I think quite amazing actually is that this has a
Speaker 2: bunch of mathematical guarantees to it. We can actually prove
Speaker 2: that the estimate that you get for how effective the
Speaker 2: treatment is is still unbiased. So it's not an overestimate,
Speaker 2: it's not under ustan, it's on average correct. Can prove
Speaker 2: that if you compute a P value from the analysis
Speaker 2: like you would typically do, that it has exactly the
Speaker 2: right properties as it does out of a regular RCT.
Speaker 1: P value is roughly the probability that the funding was
Speaker 1: a fluke.
Speaker 2: Ye right, Yeah. If you compute an arabar the arabar
Speaker 2: you get from our analysis the air bar you would
Speaker 2: get from a normal there. They all have exactly identical statistics.
Speaker 1: This is not intuitive, but but you're saying, the mathematical
Speaker 1: fact is that it works. Yes, And just to be clear,
Speaker 1: what this allows you or the people running the trial
Speaker 1: to do is to enroll fewer people in the placebo
Speaker 1: arm not none, but fewer than they otherwise would have
Speaker 1: had to get the same amount of statistical power. Right,
Speaker 1: that is the bottom line thing that you are delivering. Yes,
Speaker 1: that's correct, And it's something like a quarter or a
Speaker 1: third less, is that right? Yeah?
Speaker 2: So it depends on how accurate our models are. The
Speaker 2: more accurate the model is, the fewer patients you need
Speaker 2: in your placebo group. Sure so typically right now, yet
Speaker 2: somewhere between like a quarter, like fifty percent. It depends
Speaker 2: on the specific details.
Speaker 1: So tell me what is the effect of that at
Speaker 1: a macro scale? What does it mean to say a
Speaker 1: drug company can get the same statistical power by enrolling
Speaker 1: twenty five percent fewer people in their study, specifically in
Speaker 1: the placeboar.
Speaker 2: Well, I think that there are two things. First is
Speaker 2: I think people don't always understand how expensive clinical trials
Speaker 2: are you know, companies are paying one hundred sometimes two
Speaker 2: hundred thousand dollars per patient in one of their clinical trials,
Speaker 2: So finding and enrolling and monitoring a patient for all
Speaker 2: that time is very, very expensive. It also just takes
Speaker 2: a long time to find people who are willing to participate.
Speaker 2: And so if you're talking about a large phase three trial,
Speaker 2: reducing the size of the control group by twenty five percent,
Speaker 2: that might mean like one hundred fewer patients that you
Speaker 2: need to actually recruit and enroll in your study, and
Speaker 2: that that could be like a year. But you know,
Speaker 2: so you can save six months to a year off
Speaker 2: of your total clinical trial timeline. That means a lot, right,
Speaker 2: but both for patients. If the drug is actually successful,
Speaker 2: that's a year faster it gets to market. And you know,
Speaker 2: for the farmer company, that's office a big value proposition
Speaker 2: being able to get the drug to market a year faster.
Speaker 1: In a minute, moving from clinical trials to individual patients,
Speaker 1: now back to the show. What is the what's the
Speaker 1: big picture? Where are you trying to get to and
Speaker 1: you know, in the medium termament in the long term, So.
Speaker 2: The ability to understand what a person's health outcome is
Speaker 2: going to be under different scenarios. This is I think
Speaker 2: what's really important. Is it not just hey, given that
Speaker 2: they would get a placebo, what's going to happen to
Speaker 2: the health outcomes? That's nice for clinical trials, but we
Speaker 2: want to know, hey, there's ten different treatment options for
Speaker 2: this patient, and if I were to give them each
Speaker 2: one of these different treatment options, what would their health
Speaker 2: outcomes look like in those different scenarios.
Speaker 1: So there you're also moving out of the clinical trial
Speaker 1: into the realm of like a doctor seeing a patient.
Speaker 1: Let's just be very clear, like that that's a huge leap,
Speaker 1: and like that's what you're talking about.
Speaker 2: I think that there's a really good pathway to being
Speaker 2: able to build these things and make them useful for
Speaker 2: problems that are at the individual patient level.
Speaker 1: And is the narrow way to think about it, Like
Speaker 1: before you get to the magical computer that can predict
Speaker 1: everything for everybody, that you get to a very very
Speaker 1: good model that can predict for individuals in certain circumstances
Speaker 1: a certain set of outcomes. So, for example, you might
Speaker 1: have a very very good Alzheimer's model for certain patients
Speaker 1: at a certain stage of disease. This model is very
Speaker 1: powerful at the level of the individual. Is that the
Speaker 1: way to think about it, Yeah, the way I'll tell you.
Speaker 2: The way I think about it. I think that the
Speaker 2: most important thing that models can do, which actually things
Speaker 2: like a chat ept are not good at, is that
Speaker 2: they can give you really well calibrated estimates of their
Speaker 2: own confidence. That's the most important thing that a model
Speaker 2: can do, because, like we said earlier, health is stochastic.
Speaker 2: There are all kinds of things that happens fundamentally exactly right.
Speaker 2: And so you know, we're going to make a prediction
Speaker 2: about somebody in the future, and sometimes we're going to
Speaker 2: be really confident in that prediction and then it's actionable,
Speaker 2: but sometimes you're not. It's not you're not confident, and
Speaker 2: maybe it's not actionable because you're really unconfident. And the
Speaker 2: most we're never going to get to the point that's
Speaker 2: going to say, hey, you're going to have a heart
Speaker 2: attack on July seventeenth of twenty thirty seven. It's like,
Speaker 2: it's never going to be like that detail. But the
Speaker 2: point question is can you believe the model's estimates of
Speaker 2: its own confidence? And if you can, then you when
Speaker 2: it is confident, you can act on it, and when
Speaker 2: it's not confident, you can do other things. And that's
Speaker 2: the that's so it's actually a really key technical thing,
Speaker 2: and we know what we need to work on.
Speaker 1: If I were going to answer pomorphis it, I'd be like,
Speaker 1: it's like a it's like a humility. It's like an
Speaker 1: epistemic humility, Like it knows what it doesn't know.
Speaker 2: It knows what it doesn't know, and it will tell
Speaker 2: you like I, yeah, here's my prediction. But yeah, exactly
Speaker 2: So if you can get it to that point where
Speaker 2: we were, where it's well calibrated that way, then they
Speaker 2: become really really useful for a whole bunch of things.
Speaker 2: And it's not going to say.
Speaker 1: Become probably useful if they can have a relatively high
Speaker 1: degree of certainty about at least some things, right yea, just.
Speaker 2: Like yeah, it's not very course, yeah, but exactly so.
Speaker 2: I think that that's the most important thing for these
Speaker 2: applications of AI in medicine is to have models that
Speaker 2: are going to be able to do that effectively.
Speaker 1: If everything goes well, what problem will you be trying
Speaker 1: to solve in five years?
Speaker 2: In five years, I hope that we are rolling out
Speaker 2: something that is a model for everything. That's what we
Speaker 2: want to be rolling out, not this one disease at
Speaker 2: a time thing, but one model for all disease. And
Speaker 2: the reason why I really want to do this is
Speaker 2: because if it's one model per disease, I need a
Speaker 2: ton of data on that disease, a ton. So we
Speaker 2: can work on these areas like Alzheimer's where I can
Speaker 2: get data from fifty thousand patients, But how do I
Speaker 2: work on the disease where I have fifty patients fifty
Speaker 2: patients in the world who have this rare disease. Those
Speaker 2: are really really important things. And the only way that
Speaker 2: we're going to be able to do that is to
Speaker 2: unlock a new kind of capability in our models to
Speaker 2: learn from a handful of examples. And so this is
Speaker 2: this is to me, the next frontier for our work
Speaker 2: is figuring out how we can do that and then
Speaker 2: bring that to market, because it opens up the ability
Speaker 2: to work on rare diseases that are really really important
Speaker 2: market very difficult to develop drugs for. And it's and
Speaker 2: again I'm I'm you know, as a scientist, I'm drawn
Speaker 2: to the technical challenges. Those are the things that.
Speaker 1: It seems so hard, right, I mean, it seems like
Speaker 1: this really basic insight about genitive models is that like
Speaker 1: gigantic amounts of data feeding. You know, for a language model,
Speaker 1: you feed it the whole internet is the way to
Speaker 1: get it to understand how language works. And so how
Speaker 1: how can you do something for fifty people?
Speaker 2: Like?
Speaker 1: How how that in five years?
Speaker 2: Yeah, it's really hard. How the analogy is actually perfect? Okay,
Speaker 2: if you want to build what we've learned is that
Speaker 2: if you want to build a really amazing language model
Speaker 2: that's really specific to some domain, so you only want
Speaker 2: a language model that's really good at biophysics, it knows
Speaker 2: biophysics really well. Would you be better off training a
Speaker 2: model trying to find as much biophysics as you can
Speaker 2: and training it on that or just training a model
Speaker 2: on the entire init And what we've learned is much
Speaker 2: better to train a model on the entire init that
Speaker 2: there's a lot of things that transfer from one domain
Speaker 2: to another. And so what we can do now is
Speaker 2: say we train the model on the whole Internet, and
Speaker 2: we have one biophysics paper, and we give it that
Speaker 2: one or two papers on the background of all of
Speaker 2: the knowledge from everywhere else, and that's much better than
Speaker 2: trying to get lots and lots of biophysics papers. So
Speaker 2: the analogy works perfectly in the exact same direction. That's
Speaker 2: the whole point. We want to be able to take
Speaker 2: all of the world's Imagine taking a model that has
Speaker 2: all of the world's health data and putting all of
Speaker 2: that into one So what seen everything and it can
Speaker 2: now draw analogies between because there's a lot of things
Speaker 2: you think about, like Parkinson's and Alzheimer's, they have a
Speaker 2: lot of similarities, Huntington's a lot of similarities. So why
Speaker 2: aren't we drawing kind of information or knowledge from one
Speaker 2: disease area and using it to inform another because they
Speaker 2: are similar. And so I'm allowing a model to have
Speaker 2: access to all of the data and figure out how
Speaker 2: to do it. I think is the right path forward.
Speaker 2: So is that.
Speaker 1: Wildly capital intensive? Like what do you actually do to
Speaker 1: do that? You just get all the health data about
Speaker 1: all the people you can and say to the machine
Speaker 1: figure it out, Like what do you do?
Speaker 2: Yeah? Yes, I mean the first step for us is
Speaker 2: you need to get a lot of data. The biggest
Speaker 2: thing is that we need to figure out a way
Speaker 2: to have the model map all of those data to
Speaker 2: the same representation.
Speaker 1: What does that mean, map all of those data to
Speaker 1: the same representation.
Speaker 2: So let's imagine that there is some unobservable state of
Speaker 2: a person which just describes their health. We can't actually
Speaker 2: observe it directly. It's we don't exactly know what it is,
Speaker 2: but we can make these measurements of it that tell
Speaker 2: us something about that underlying state. So I can measure BMI,
Speaker 2: I can measure heart rate, I can measure all the
Speaker 2: I can measure all of these different things. And what
Speaker 2: we want to be able to do is instead of
Speaker 2: working in the world of measurements, which is where we
Speaker 2: currently work, we want to be able to work at
Speaker 2: that underlying unobservable state because if you can, if you
Speaker 2: can see that, if you could reach through into that
Speaker 2: underlying state, you can answer any question about any.
Speaker 1: Patient's health, like like like a number like this one
Speaker 1: state that is just like one.
Speaker 2: High dimension the high dimension, right right, Well, okay, yeah,
Speaker 2: so I mean yeah, but basically talking about is there
Speaker 2: some vector, some really high dimensional space where we're able
Speaker 2: to take all diseases and look at them how they're
Speaker 2: related to each other in this really high dimensional space.
Speaker 2: That is the way language models work. That's exactly how I.
Speaker 1: Love And that's intense, Like, that's pretty far out right.
Speaker 1: Doesn't that feel far out too?
Speaker 2: I would say, talk like a hippie, But I if
Speaker 2: I describe this to a machine learning researcher, they're like,
Speaker 2: that sounds exactly like what you should do. So it
Speaker 2: doesn't seem far out to me. It seems it seems
Speaker 2: very clear that that's the direction that we should be
Speaker 2: taking things.
Speaker 1: And does five years seem like a like you might
Speaker 1: actually do it in five years.
Speaker 2: Yeah, we were hoping to be able to have a
Speaker 2: version next year. That's a pan neuroscience model, so we're
Speaker 2: starting with all. So we're starting with start with something
Speaker 2: more attractable, build a more tractable thing. So right now
Speaker 2: we're working on a neuroscience model. So we're hoping, I
Speaker 2: mean which to be totally. This might not work. This
Speaker 2: is a research idea, right, so it may work, it
Speaker 2: might not work. But that's you ask where I would
Speaker 2: hope to be. That's where I hope to be is
Speaker 2: that we're able to solve those those problems.
Speaker 1: So we'll be back in a minute. With the Lightning Round,
Speaker 1: including what Charles learned when he worked as an ice
Speaker 1: hockey wrap back to the show. I'm going to finish
Speaker 1: with the Lightning Round. Will just be a few more minutes.
Speaker 2: Okay.
Speaker 1: As the name suggests, I've heard you say that you
Speaker 1: read academic preprints, which is basically studies that are about
Speaker 1: to be published, that you read them every day. What's
Speaker 1: one you read recently that you found particularly interesting?
Speaker 2: Recently? There have been a number of papers that I've
Speaker 2: been reading around different ways of so training, the kind
Speaker 2: of neural networks that we use. All of them use
Speaker 2: a particular algorithm that people call ADAM. It's been used
Speaker 2: for a really long time, like everyone uses it now,
Speaker 2: and it has I don't know, it has some problems.
Speaker 2: There's a paper that was just really recently on a
Speaker 2: new algorithm people call LION. I don't know what it
Speaker 2: stands for. L I O N stands for something. And
Speaker 2: this was a discovered So they used a machine learning
Speaker 2: out a reinforcement learning algorithm to discover a new kind
Speaker 2: of optimizer.
Speaker 1: So if this works, if Lion is better than ADAM,
Speaker 1: will it be like machine learning figuring out a better
Speaker 1: way to build machine learning. Is that what's happening here?
Speaker 2: Yeah, that's what people are working on exactly.
Speaker 1: This is like the takeoff. This is like the moment
Speaker 1: when GPT five builds GBT six or whatever.
Speaker 2: I think the claim is it's like five percent better something.
Speaker 2: It's not. It's not.
Speaker 1: Yes, Lion couldn't find the.
Speaker 2: Time another thing yet Yeah. So yeah, that was a
Speaker 2: paper I read really recently.
Speaker 1: If you couldn't work in AI, what field would you
Speaker 1: work in?
Speaker 2: If I couldn't work in AI. Uh, I guess I
Speaker 2: would probably try to work in energy, maybe tim a
Speaker 2: change something related to that.
Speaker 1: I think seeing bummed at the prospect of not being
Speaker 1: able to work in AI, I appreciate that. I don't
Speaker 1: want to make it.
Speaker 2: I'm very bummed. Yeah, you know, I think it's the
Speaker 2: most exciting thing that's happened on Earth since the Industrial Revolution.
Speaker 2: So it's a new industrial revolution. Yeah.
Speaker 1: Weirdly, you used to work at a virtual reality hardware company.
Speaker 1: I feel like VR is always about to break through,
Speaker 1: you know, like Apple just had this big announcement, had
Speaker 1: a Facebook did a while ago, but yet it never
Speaker 1: quite happens. Why not, Like, why are we not doing
Speaker 1: this interview in the metaverse.
Speaker 2: So I only worked at that company for a few months.
Speaker 2: I spent my whole career working in biophysics. I moved
Speaker 2: to Pfiser. I was working at Pfiser, and then I
Speaker 2: got im just like, I'm gonna try something totally different,
Speaker 2: and I went and tried this work at the VR company.
Speaker 2: I was interested in that because of the underlying technical
Speaker 2: problems research that I had to do, not because I
Speaker 2: was drawn to the product. I have only ever used
Speaker 2: a virtual reality headset twice my entire life. Once was
Speaker 2: in the interview for that job, and once was testing
Speaker 2: something while I was working at that job. I'm not
Speaker 2: interested in it, so you want to know I was
Speaker 2: interested in the engineering. So you want to know why
Speaker 2: I don't think it's taken off. Is because most people
Speaker 2: don't have a compelling reason to use it. Neither do I. Yeah,
Speaker 2: what'd you learn working as an ice hockey referee? Ice
Speaker 2: hockey referee? Oh, that was like my super super young job.
Speaker 2: I would say that I learned it's best not to
Speaker 2: call penalties on little children. That's what I learned. You know,
Speaker 2: people would just like like run into each other and
Speaker 2: they'd fall down. You're like, is that a penalty. Was
Speaker 2: it on purpose? Not on purpose? If you call a penalty,
Speaker 2: the parents are going to be real upset at you.
Speaker 2: So you just just let them play.
Speaker 1: Good early experiments, cost benefit analysis.
Speaker 2: Just let them play.
Speaker 1: Charles Fisher is the co founder and CEO of Umler.
Speaker 1: Today's show was edited by Sarah Nis, produced by Gabriel
Speaker 1: Hunter Chang and Edith Russlo, and engineered by Amanda k Wong.
Speaker 1: I'm Jacob Goldstein. One last note, the show is going
Speaker 1: to be off for the next several weeks and we'll
Speaker 1: be back with new episodes in August. Have a rad summer.