Speaker 1: AI seems like it burst out of the gate a
Speaker 1: few years ago, But is it actually the latest chapter
Speaker 1: in a three hundred year trajectory to turn thought into math?
Speaker 2: Can the mind be captured with equations?
Speaker 1: Why do current AI models need petabytes of data but
Speaker 1: a child can learn from just a few examples. Why
Speaker 1: does AI have jagged intelligence, meaning it looks brilliant in
Speaker 1: one moment and then it does something totally nonsensical. In physics,
Speaker 1: we have various laws, like the law of gravity or
Speaker 1: the laws of motion, And today we're joined by cognitive
Speaker 1: scientist Tom Griffiths from Princeton to talk about whether we
Speaker 1: are moving towards nailing down laws of thought. Welcome to
Speaker 1: Inner Cosmos with me David Eagleman. I'm a neuroscientist and
Speaker 1: author at Stanford, and in these episodes we sail deeply
Speaker 1: into our three pound universe to understand why and how.
Speaker 2: Our lives look the way they do.
Speaker 1: One thing that distinguishes Homo sapiens from all our cousins
Speaker 1: in the animal kingdom is that we watch the world
Speaker 1: around us and we try to abstract patterns from it.
Speaker 1: For example, you might watch the way that a stone
Speaker 1: falls to the ground and maybe you see a tree
Speaker 1: branch fall, and maybe you see a glacier and one
Speaker 1: day a huge wall of ice falls off it, and
Speaker 1: pretty soon you start seeing an underlying similarity to the
Speaker 1: way that things move. And eventually someone very very smart
Speaker 1: comes along, like Isaac Newton and summarizes all this in
Speaker 1: the law of gravity. And then the same smart guy,
Speaker 1: Newton comes up with the three laws of motion. And
Speaker 1: then another smart person is Einstein. He figures out the
Speaker 1: conservation of mass and energy, which seems to be another
Speaker 1: ironclad law, and then we have the laws of thermodynamics
Speaker 1: and electrostatic laws, and all of this speaks to the
Speaker 1: great success that we've had as the species in figuring
Speaker 1: out the lowest level of code that's running in the universe.
Speaker 1: But for most of human history, the concept of a
Speaker 1: thought has felt like the most intimate thing we experience
Speaker 1: and the least tractable thing to study.
Speaker 2: What a thought is and how it occurs.
Speaker 1: That seems to live in a different category of mystery
Speaker 1: from how an object falls. Why, Well, it's because the
Speaker 1: thought pops into your head and somehow it carries memory
Speaker 1: and expectation and language and often a feeling. But it
Speaker 1: feels vaporous and private. It feels like the one thing
Speaker 1: that will forever escape formal description. But what's interesting is
Speaker 1: that for centuries people have tried, there's always been a
Speaker 1: deep human urge.
Speaker 2: To ask whether thought has laws to it?
Speaker 1: In other words, does the mind have principles that you
Speaker 1: can write down? Does reasoning have a grammar to it?
Speaker 1: Can you describe intelligence in a language that's precise enough
Speaker 1: that once you understand the rules, you can begin to
Speaker 1: build with them, like build artificial intelligence. Most of us
Speaker 1: are old enough to remember that this question of AI
Speaker 1: once lived in philosophy seminars and math departments, but now
Speaker 1: it's sitting at the center of our economy.
Speaker 2: Okay, So what is thought?
Speaker 1: Can we capture it in formal systems like laws or equations?
Speaker 1: Do different parts of intelligence come from logic, from learning,
Speaker 1: from uncertainty, from memory, from prior knowledge, from living inside bodies,
Speaker 1: from living inside our cultures?
Speaker 2: From the particular.
Speaker 1: Constraints of being a human animal with a short lifespan
Speaker 1: and limited bandwidth. Our guest today is someone who lives
Speaker 1: right at the intersection of all these questions. Tom Griffiths
Speaker 1: is a professor at Princeton, where He directs the Computational
Speaker 1: Cognitive Science Lab and the Princeton Laboratory for Artificial Intelligence.
Speaker 1: He has spent years asking how minds work through the
Speaker 1: different lenses of math and computation and learning. And he's
Speaker 1: the author of a wonderful new book called The Laws
Speaker 1: of Thought, which traces the long history of thinkers asking
Speaker 1: are their rules to this? Can we understand what human
Speaker 1: thinking is? In his book we get the lengthy arc
Speaker 1: of minds trying to understand mind. This begins millennia ago
Speaker 1: with Aristotle, who wondered whether logic itself could be math matized,
Speaker 1: and Tom follows the trail through the architects of symbolic reasoning,
Speaker 1: through the birth of computation, through the rise of neural networks,
Speaker 1: through the realization that probability theory might serve as a
Speaker 1: language for our beliefs about things. Along the way, in
Speaker 1: his book, a picture emerges that there may not be
Speaker 1: just a single tool for capturing their mind, but instead
Speaker 1: there are different ways of trying to tackle the problem,
Speaker 1: and each one sheds light on a different aspect of cognition.
Speaker 1: So we're going to talk about ourselves human minds, and
Speaker 1: we'll talk about AI what kind of intelligence is this
Speaker 1: and what is missing? Here's my interview with Tom Griffiths.
Speaker 3: As soon as you turn thought into math, it becomes
Speaker 3: something that machines would be able to do. And so
Speaker 3: our modern AI systems are really a consequence of, you know,
Speaker 3: that thought that people were having hundreds of years ago,
Speaker 3: of being able to turn thought into something that can
Speaker 3: be expressed in mathematical terms.
Speaker 1: And so one of the things that I loved about
Speaker 1: your book, by the way, is that you really tell
Speaker 1: stories of all the thinkers.
Speaker 2: You dive into the lives, you tell them with real color.
Speaker 1: If you were going to start with one thinker that
Speaker 1: you think is the most important, who would that be.
Speaker 3: There are a couple of people who have this sort
Speaker 3: of enduring influence throughout the book. One of them is Leibnitz,
Speaker 3: who kind of started this enterprise in some sense. He
Speaker 3: was really trying to take the idea of logic as
Speaker 3: expressed by Aristotle and turn it into math, but ultimately
Speaker 3: failed in doing that. But along the way he also
Speaker 3: discovered the calculus, which turned out to be really important
Speaker 3: when people wanted to make neural networks that could learn
Speaker 3: from data. It turns out that the trick for doing
Speaker 3: that is actually a trick that lad had figured out
Speaker 3: all that time ago. And then another key figure here,
Speaker 3: as might be suggested by the title of the book,
Speaker 3: is George Bull, who was a nineteenth century mathematician. He
Speaker 3: was a school teacher for most of his life and
Speaker 3: did a lot of like serious math on the side
Speaker 3: instead of you know, had a big effect on the
Speaker 3: history of mathematics. But he was really the person who
Speaker 3: then first solved that problem that Leibnitz had posed. And
Speaker 3: in addition to the impact that that work had, he's
Speaker 3: also the great grandfather of Jeff Hinton, who was one
Speaker 3: of the people who played an important role in developing
Speaker 3: these algorithms for learning from your own networks. And so
Speaker 3: you could make an argument that without Boole we would
Speaker 3: be a fair way back from where we are today.
Speaker 2: You know.
Speaker 1: Interestingly, when most people think about Boole, they only know
Speaker 1: about Boolean numbers. They know about zero and one binary numbers,
Speaker 1: and that's essentially the extent of the think. But he
Speaker 1: was quite celebrated in his life right even though he
Speaker 1: was a headmaster and not formally involved as a professor.
Speaker 1: Am I correct about this? He nonetheless was quite recognized
Speaker 1: as a mathematician.
Speaker 3: Yeah, he became a university professor later in his life,
Speaker 3: but spent most of his life as a teacher and
Speaker 3: a head master. But yeah, he won a gold medal
Speaker 3: in mathematics from the Royal Society. Was a very prestigious award,
Speaker 3: and you know, was this amazing person who was having
Speaker 3: these high level correspondences with the leading mathematicians of the
Speaker 3: day while holding down his job running a small school.
Speaker 2: Yeah.
Speaker 1: Now, in the book, you essentially use three different frameworks.
Speaker 1: What phenomenon does each framework explain?
Speaker 2: Unusually Well, the.
Speaker 3: Three frameworks I talk about in the book are what
Speaker 3: I call rules and symbols, which is what we've been
Speaker 3: talking about. This kind of like approach that stems out
Speaker 3: of logic, where the idea is that you're going to
Speaker 3: be able to write down some rules that characterize the
Speaker 3: structure of thought, and by following those rules, you end
Speaker 3: up with interesting consequences. The second approach is networks, features
Speaker 3: and spaces.
Speaker 4: Right.
Speaker 3: This is neural networks, which you can kind of think
Speaker 3: about as a system for doing computation when you start
Speaker 3: representing things as points in a space. Right, So if
Speaker 3: you start to think about you know, every object that
Speaker 3: you could see in the world is not being something
Speaker 3: that's described by rules, but being described by a location
Speaker 3: along some dimensions. You need to have a way of
Speaker 3: talking about how to map between those spaces and your
Speaker 3: all network solve that problem. And then the third is
Speaker 3: probability and statistics. And probability theory is really powerful because
Speaker 3: it is the complement to logic, where logic tells us
Speaker 3: how to go from things that we know to be
Speaker 3: true to other things that we're equally certain or true.
Speaker 3: Probability theory tells us what to do when we're uncertain.
Speaker 3: So if we get some information we want to draw
Speaker 3: a conclusion, but we're not able to draw that conclusion
Speaker 3: with perfect certainty, Probability theory tells us how to do that,
Speaker 3: and it tells us how to combine our sort of
Speaker 3: background beliefs, the other sources of information we have our
Speaker 3: biases in with the data that we see in a
Speaker 3: way that helps us to explain how it's possible to
Speaker 3: learn from small amounts of data. And that's one thing
Speaker 3: which is still something that discriminates human learning from the
Speaker 3: learning that's done by AI systems today.
Speaker 1: Okay, great, so we're going to dive into each of
Speaker 1: these three lenses. But just before we do, do you
Speaker 1: see the AI conversation today over indexing on one of
Speaker 1: these lenses over the others.
Speaker 3: I think there's a lot of emphasis on neural networks,
Speaker 3: which are fundamentally the sort of engineering technology which is
Speaker 3: making possible the creation of our chatbots and the other
Speaker 3: sort of big AI systems that are deployed. I think
Speaker 3: that potentially misses out the importance of these other threads
Speaker 3: right where. One thing that's important to remember is that
Speaker 3: those neural networks are being trained on what is essentially
Speaker 3: a system of rules and symbols. They're being trained on
Speaker 3: human language, which is symbolic and rule like in various ways,
Speaker 3: and they're being trained on code, which is even more
Speaker 3: symbolic and even more rule like, And those things together
Speaker 3: provide some of the substrate for developing the kind of
Speaker 3: intelligence that they demonstrate. And then the way that they're
Speaker 3: trained is by learning to predict the next token, right,
Speaker 3: the next word or part of word, based on what
Speaker 3: they've seen so far. And that way of training them
Speaker 3: is actually using probability theory. So that's a probabilistic problem
Speaker 3: because you're making a guess about what the next thing
Speaker 3: is going to be based on the things that you see,
Speaker 3: and so that's an important ingredient in their success as well,
Speaker 3: is that they're essentially learning to approximate a big probability distribution.
Speaker 1: So let's dive into the first one, rules and symbols.
Speaker 1: So take us back to the original urge. Why did
Speaker 1: early thinkers believe that this could be used to explain thinking.
Speaker 3: I think a lot of the draw of rules and
Speaker 3: symbols was that that really was, in some way what
Speaker 3: mathematics was to people, right, So Leibniz, part of the
Speaker 3: reason why he wasn't able to solve this problem of
Speaker 3: figuring out how to turn thought into math is that
Speaker 3: what he thought math was, or the kind of math
Speaker 3: that he was trying to use to solve that problem,
Speaker 3: was really arithmetic, right, And arithmetic was kind of like
Speaker 3: the model that they had for a mathematical system. So
Speaker 3: you can think about ideas being added together or subtracting
Speaker 3: one idea from another, and really thinking about the operators
Speaker 3: that you're using as being the things that are sort
Speaker 3: of coming from this familiar mathematical language.
Speaker 4: And so I think part of.
Speaker 3: The reason that we end up with that approach is
Speaker 3: because of the kind of math that has been successful
Speaker 3: in other settings, right where we need to do arithmetic
Speaker 3: to you know, that's a good description of certain.
Speaker 4: Kinds of things that human minds do.
Speaker 3: Google had the insight that you needed a different kind
Speaker 3: of algebra in order to describe thought, and then that's
Speaker 3: what leads to modern mathematical logic. But it's still in
Speaker 3: this kind of symbolic language, although Gooole also talked about
Speaker 3: probability theory as being important for capturing languages as well.
Speaker 3: So I think it's really more about what are the
Speaker 3: kinds of mathematical systems that it was sort of straightforward
Speaker 3: to formalize, and that gave us something that we could
Speaker 3: try to map thought onto. And that's what we do
Speaker 3: as scientists is often taking mathematical systems that mathematicians have
Speaker 3: defined for us and then saying, oh, I think this
Speaker 3: mathematical system maps onto the thing that I want to understand,
Speaker 3: and so trying to establish that correspondence and not just
Speaker 3: then allow us to derive its consequences.
Speaker 1: So speaking of rules and symbols, So thinkers like Newl
Speaker 1: and Simon, they popularize this idea of goals and sub goals.
Speaker 1: What did that viewpoint get exactly right about human problem solving.
Speaker 3: So now we're fast forwarding a bit right from we
Speaker 3: have Boule figuring out the structure of logic. That turns
Speaker 3: into you know, lots of people then sort of turn
Speaker 3: that into a sort of mature theory of logic. You
Speaker 3: get aalenteering kind of turning this into a theory of computation,
Speaker 3: thinking about what an abstract mathematician is doing when they're
Speaker 3: doing something like logic, and thinking about how you can
Speaker 3: make a machine do that. And then we have people
Speaker 3: starting to realize that, you know, as digital computers are
Speaker 3: being developed, maybe those provide a good model for how
Speaker 3: thinking works in general, and then trying to use a
Speaker 3: computer as a sort of foundation for you know, thinking
Speaker 3: about things like how people might solve problems. And so
Speaker 3: Alan Ewele and Herbert Simon were influential cognitive scientists who
Speaker 3: did exactly that. They had this idea that maybe there
Speaker 3: is a way that you could make computers smarter by
Speaker 3: using insights from human cognition, but also get a better
Speaker 3: understanding of what humans are doing when they're solving problems
Speaker 3: by using the sort of ideas that come from things
Speaker 3: like computer programming, and so they set up you know this,
Speaker 3: you know, when we're trying to solve a problem or
Speaker 3: prove a mathematical theorem or play a game of chess,
Speaker 3: they set this up as a problem of searching through
Speaker 3: a tree of possibilities, where what you're doing is making choices,
Speaker 3: and then each of those choices gives you a new
Speaker 3: set of choices, and each of those choices gives you
Speaker 3: a new set of choices, and the hard thing is
Speaker 3: finding a path through those choices that leads to the
Speaker 3: point that you want to end up at. And so
Speaker 3: that's something where you can take inspiration from how human
Speaker 3: mathematicians solve problems. You can take inspiration from the kind
Speaker 3: of you know, tricks like working backwards from the end
Speaker 3: towards the start. Right, Those were principles that they were
Speaker 3: able to use to try and explain these aspects of
Speaker 3: human cognition as well as making the machines work better.
Speaker 1: Okay, but then one of the things that happened is
Speaker 1: that at least one of these attempts had ballooned into
Speaker 1: twenty five million rules. And so what does that teach
Speaker 1: us about the shape of human intelligence.
Speaker 4: This rules and symbols enterprise.
Speaker 2: Right.
Speaker 4: The sort of appeal that this had was that maybe
Speaker 4: one day.
Speaker 3: You could just write down all of the rules that
Speaker 3: you need to write down, and then you've characterized how
Speaker 3: intelligence works. Right, So it's just a matter of getting
Speaker 3: enough rules in a way that's very reminiscent today, right
Speaker 3: of you know, the way that our modern AI systems
Speaker 3: are being made is by training them on more and
Speaker 3: more data, right, feeding in more and more language. There
Speaker 3: was a hope that you could just like, yeah, like
Speaker 3: document all of the rules that you need to capture
Speaker 3: the structure of human knowledge. And so that led to
Speaker 3: you know, companies being started to try and engage in
Speaker 3: that enterprise, ultimately I would say, unsuccessfully, but giving us
Speaker 3: some kind of characterization of like particular subsets of human knowledge.
Speaker 3: And so I think the thing that came out of
Speaker 3: that enterprise was revealing that maybe you need something more
Speaker 3: than just rules, right, that maybe thinking about logic as
Speaker 3: a basis for our model of intelligence was missing something.
Speaker 3: It's an approach that worked really well for certain kinds
Speaker 3: of problems like doing arithmetic, playing games or chess, but
Speaker 3: it didn't work very well for other kinds of problems
Speaker 3: like figuring out what you're seeing in the world, or
Speaker 3: actually learning language or these other kinds of things.
Speaker 1: And so this is what leads to your second lens,
Speaker 1: which is neural networks. And you talk about these as
Speaker 1: having you know, a boom and bust history. So, first,
Speaker 1: what happened in the last decade that allowed them to
Speaker 1: turn into the dominant paradigm.
Speaker 3: The big breakthrough in the last decade was really about
Speaker 3: being able to make bigger in neural networks that could
Speaker 3: be trained on more data in a way that could scale, right,
Speaker 3: and so bigger here means what these are. An artificial
Speaker 3: neural network is a set of units that are communicating
Speaker 3: with one another. They're communicating along weighted connections, a sort
Speaker 3: of you know, imagine like how neurons are connected in
Speaker 3: your brain, and those neurons are connected to one another
Speaker 3: and sending each other signals. An artificial neural network is
Speaker 3: basically simulating that kind of structure inside a computer. And
Speaker 3: so for a long time, the sort of the history
Speaker 3: of neural networks has been one of people figuring out
Speaker 3: how to make bigger neural networks work. So the very
Speaker 3: first you know, learning neural networks. They had a learning
Speaker 3: algorithm that worked for one layer of weights, and then
Speaker 3: there was a breakthrough in the nineteen eighties that meant,
Speaker 3: now you had a learning algorithm that could work for
Speaker 3: multiple layers of weights, but it didn't work for very
Speaker 3: deep neural networks with lots of layers of net weights
Speaker 3: because it I can sort of explain the technical reasons
Speaker 3: behind it, but you know, sort of like the basic
Speaker 3: algorithm didn't quite work. And so the big breakthroughs of
Speaker 3: the last you know, ten to fifteen years have been
Speaker 3: about coming up with ways to take those algorithms and
Speaker 3: actually make them work for neural networks that are bigger
Speaker 3: and bigger and deeper and deeper, that are able to
Speaker 3: easily learn more complex functions and can do so from
Speaker 3: massive amounts of data in a way that means that
Speaker 3: they're able to discover sort of complex relationships between things
Speaker 3: that are necessary to produce intelligent behavior.
Speaker 1: And so, what are these neural networks capture about cognition
Speaker 1: that symbols missed, especially in terms of things like similarity
Speaker 1: and fuzziness and graded concepts.
Speaker 3: Fuzziness is a really good way of describing it. It's
Speaker 3: that you know, if you ask somebody, you know, whether
Speaker 3: something is a piece of furniture, they're going to say,
Speaker 3: you know, if you show them a chair, they'll say, yes,
Speaker 3: definitely a piece of furniture. If you show them a rug,
Speaker 3: they'll say, yeah, maybe a piece of furniture. Right, it
Speaker 3: doesn't sort of fit with our you know, week sort
Speaker 3: of have a prototypical idea of what furniture is, which
Speaker 3: contains things like chairs and tables and ottomans and these
Speaker 3: other kinds of things, and then rugs and treadmills, and you.
Speaker 4: Know, like these are things that maybe.
Speaker 3: You're in this category, but maybe an't right. And so
Speaker 3: we need to have a way of thinking about concepts
Speaker 3: that's not just the sort of yes or no, true
Speaker 3: or false one or zero that logic would give us.
Speaker 3: We need to have something which has that fuzziness in it.
Speaker 3: One way of getting fuzziness is by thinking about a
Speaker 3: concept in terms of points in space, right where you
Speaker 3: could think chairs are here in one location, rugs are
Speaker 3: here in another location, and maybe what it is to
Speaker 3: be a piece of furniture is to just be in
Speaker 3: some part of that space, and how close you are
Speaker 3: to that part of the space is like how good
Speaker 3: you are as an example of that kind of furniture.
Speaker 3: And so as soon as you think in those terms,
Speaker 3: you have a new problem, which is with our rules
Speaker 3: and symbols. We knew how to do computation, we knew
Speaker 3: how to describe thinking. Thinking was a matter of applying
Speaker 3: the rules and seat of you know, repeating that process.
Speaker 3: But we don't have a way of doing computation in spaces.
Speaker 3: And that's what youral networks give us. So you can
Speaker 3: kind of think about a space corresponding to the activation
Speaker 3: of the units inside this neural network.
Speaker 4: How much you know, how much input.
Speaker 3: Each neural unit in that neural network is receiving, and
Speaker 3: how much of a response it's making that characterizes some
Speaker 3: kind of space. And then neural network gives us a
Speaker 3: way of mapping from the inputs that it's getting to
Speaker 3: some output.
Speaker 4: So you could put in you know.
Speaker 3: Your picture of a chair, and it maps that to
Speaker 3: some point in space, and then it put sort of
Speaker 3: produces out an output the corresponds to, yes, this is
Speaker 3: a piece of furniture. And because those outputs can now
Speaker 3: be continuous values, you can capture the fuzziness and other
Speaker 3: kinds of things that you want for your concepts.
Speaker 1: And so, in what sense are these modern systems, these
Speaker 1: artificial neural networks learning, and in what sense are they
Speaker 1: doing something that's maybe categorically different from how children learn.
Speaker 3: This is a fundamental question, right, That's the kind of
Speaker 3: thing that we cognitive scientists think about a lot, and
Speaker 3: I think that AI researchers are starting to care about
Speaker 3: a lot too, which is, you know, what are these
Speaker 3: sort of meaningful differences between human minds, human brains and
Speaker 3: what we building in these AI systems or these sort
Speaker 3: of artificial brains. I think one very salient difference is
Speaker 3: the amount of data which is needed for a human
Speaker 3: to learn language compared to the amount of data you
Speaker 3: need to put into on neural network. So if you
Speaker 3: take a system like chat GPT, right, one of these chatbots,
Speaker 3: those systems are trained on the equivalent of something like
Speaker 3: five thousand to fifty thousand years of continuous speech. There's
Speaker 3: sort of massive amounts of data that are going into
Speaker 3: that system. So it's like on the order of a
Speaker 3: thousand or ten thousand times as much data as a
Speaker 3: human child might get in order to learn language. And
Speaker 3: the reason is that those artificial neural networks are really
Speaker 3: kind of like undifferentiated learning machines. You can take that
Speaker 3: same kind of neural network, you can get it to
Speaker 3: learn all sorts of different kinds of things. It works
Speaker 3: really well for learning language, but you can use it
Speaker 3: to learn something about vision or something. You know, you
Speaker 3: can sort of take all sorts of problems and give
Speaker 3: it to them and it can learn how to do that.
Speaker 3: And so as a consequence, they have what we call
Speaker 3: in cognitive science machine learning inductive biases. They're not biased
Speaker 3: to towards any particular solution to the learning problem, and
Speaker 3: human brains have stronger inductive biases for things like learning language. Right,
Speaker 3: we're sort of disposed towards certain kinds of things, which
Speaker 3: are human languages. The things that we call human languages
Speaker 3: are the things that we're disposed to learn. And as
Speaker 3: a consequence, you know, we're able to sort of narrow
Speaker 3: down the space of possibilities in a way that means
Speaker 3: that we're able to learn from less data.
Speaker 1: Okay, this makes a great segue to the third lens.
Speaker 1: So you talked about rules and symbols, and you talked
Speaker 1: about artificial neural networks. The third part of your book
Speaker 1: is about probabilities and statistics. So why did probability become
Speaker 1: an attractive candidate language for thinking about cognition?
Speaker 3: Probability there is a good way of answering certain kinds
Speaker 3: of why questions that we might have, right so, and
Speaker 3: the reason is that it's a way of characterizing how
Speaker 3: a rational agent should make an inference. So all the
Speaker 3: way back in the eighteenth century, British nonconformist minister, the
Speaker 3: Reverend Thomas Bays had this radical idea that you could
Speaker 3: talk about, you know, again, like take a mathematical system
Speaker 3: probability theory, which we would use for describing what happens
Speaker 3: when you roll dice or flip coins right, sort of
Speaker 3: you know, sort of language of gambling and saying, oh,
Speaker 3: in fact, that mathematical system might also be a really
Speaker 3: good system for describing how beliefs work. And so what
Speaker 3: he was interested in was if you think about a
Speaker 3: belief as you know, a degree of belief, right, you
Speaker 3: can say, oh, I think it's going to rain tomorrow,
Speaker 3: and I'll put put that on a scale which goes
Speaker 3: from zero to one, where zero is you know, not
Speaker 3: going to rain, and one is one hundred percent it's
Speaker 3: going to rain.
Speaker 4: Tomorrow.
Speaker 3: Right, That is a belief that you've expressed in the
Speaker 3: form of a probability. And now if you, you know,
Speaker 3: wake up in the morning and look out the window
Speaker 3: and you see gray storm clouds, you've got a new
Speaker 3: piece of data. You need to revise your beliefs, and
Speaker 3: probability theory actually tells you how to do that. It says,
Speaker 3: you know, for each hypothesis, right, so our hypotheses here
Speaker 3: are it's going.
Speaker 4: To rain or it's not going to rain.
Speaker 3: Right, You're going to modify that belief based on how
Speaker 3: likely the data is that you saw if that hypothesis
Speaker 3: were true. So, because gray storm clouds are more likely
Speaker 3: if it's going to rain that day, we should increase
Speaker 3: our belief that it's going to rain. And as a consequence, well,
Speaker 3: we'll end up with a number that's a little bit
Speaker 3: higher than the number we had before.
Speaker 4: And probability theory tells us how to do that.
Speaker 3: There's a principle of probability theory called Bays rule after
Speaker 3: the Reverend Thomas Beys that tells you how to take
Speaker 3: your original beliefs and then turn them into the beliefs
Speaker 3: that you get after seeing data. And that turns out
Speaker 3: to be exactly the tool that you need to answer
Speaker 3: these kinds of questions about how inductive biases work. Right, So,
Speaker 3: how is it that children are able to learn from
Speaker 3: less data than anural networks. Well, it's a consequence of
Speaker 3: you know, these things that we can describe using different
Speaker 3: probabilities being assigned to different hypotheses, whether hypotheses correspond to
Speaker 3: the structure of the languages that are being learned.
Speaker 1: And when people call humans irrational, what changes if we
Speaker 1: look at mistakes as resource limited inferences.
Speaker 3: This is one of the things that I explore in
Speaker 3: my own research is this question of how we should
Speaker 3: actually think about rationality for real agents, right, And this
Speaker 3: is relevant if you're building an AI system or if
Speaker 3: you're just trying to understand human behavior. So I think,
Speaker 3: like I said, probability theory gives us a characterization of
Speaker 3: what you should do as an ideal rational agent, But
Speaker 3: that assumes that you have infinite computational resources. We mirror,
Speaker 3: humans don't have infinite computational resources, nor do our AI systems,
Speaker 3: And so you can ask what should a rational agent
Speaker 3: do if they don't have all of the computational resources
Speaker 3: that you might need and then out of that you
Speaker 3: get the answer is that you know, you should follow
Speaker 3: an algorithm, follow us strategy makes sense given the resources
Speaker 3: that you have. That's what it means to be rational
Speaker 3: in those circumstances where you're sort of doing the best
Speaker 3: job you can of approximating probabilistic inference given those resource constraints.
Speaker 3: And so some of the things that people do when
Speaker 3: people do weird things, and we do lots of weird things,
Speaker 3: and we don't always follow probability theory, things that we
Speaker 3: can understand as us, you know, running into those resource
Speaker 3: limitations and then coming up with, you know, reasonable strategies
Speaker 3: for trying to approximate the right answer.
Speaker 2: So, if we look.
Speaker 1: At probability being the grammar of uncertainty, one thing we
Speaker 1: know is that our prior expectations matter. And one of
Speaker 1: the things I've been obsessed with and doing a lot
Speaker 1: of research on and talking a lot about on the
Speaker 1: podcast is the way that all of us drop into
Speaker 1: the world and our cultures influence us and our language
Speaker 1: and our moment in time and our neighborhood, and this
Speaker 1: leads to people being quite different on the inside. Is
Speaker 1: this something that you think about sometimes about how we
Speaker 1: develop our priors differently based on you know what where
Speaker 1: we grow up.
Speaker 3: Yeah, So priors is that Daysian language, right, for talking
Speaker 3: about the beliefs that you have before you get data
Speaker 3: that you then update into what we call posterior probabilities
Speaker 3: that are informed by those data. And so yeah, I
Speaker 3: spend a lot of my research time thinking about these
Speaker 3: kinds of questions of you know what are these sort
Speaker 3: of prior distributions for humans? How do we acquire good
Speaker 3: prior distributions for solving different kinds of problems. One thing
Speaker 3: there is that calling it a prior makes it sound
Speaker 3: like maybe it's something you're born with, but in fact,
Speaker 3: it just means.
Speaker 4: It's before you get data.
Speaker 3: Right, So when you're seeing that storm cloud in the morning,
Speaker 3: you had a prior probability from last night, and then
Speaker 3: that prior probability was informed by everything else that you know, Right,
Speaker 3: The priors are all of the biases and knowledge that
Speaker 3: we bring to bear when we're trying to make an difference.
Speaker 3: And so yeah, I think I think understanding that is
Speaker 3: a big part of the project of understanding human cognition.
Speaker 2: So let's zoom the camera out.
Speaker 1: We've talked about the three lenses that you describe in
Speaker 1: the book. Now, you also point out that we have
Speaker 1: a lot of constraints like finite lives in limited compute,
Speaker 1: and limited bandwidth, and so how do these constraints sculpt
Speaker 1: human intelligence?
Speaker 3: I think this is really important to just thinking about
Speaker 3: the moment that we're in where there's a lot of
Speaker 3: anxiety around AI, right, And I think if you think
Speaker 3: about intelligence as a kind of one dimensional quantity, you
Speaker 3: can kind of imagine that you know, humans are somewhere,
Speaker 3: our AI systems are somewhere.
Speaker 4: It seems like they're approaching us.
Speaker 3: Maybe they're going to overtake us, and then, oh my god,
Speaker 3: what is going to happen when that happens, Right, We're
Speaker 3: just going to become redundant. There's not going to be
Speaker 3: any jobs. Everything is going to fall apart. And so
Speaker 3: that's a consequence of having a particular conception of what
Speaker 3: intelligence is, which is this kind of one dimensional way
Speaker 3: of thinking about it. And I think there's a different
Speaker 3: way of thinking about it which gives us a little
Speaker 3: more flexibility and maybe a little more hope in the
Speaker 3: way that we think about what's going to happen with AI,
Speaker 3: and that is thinking about intelligence as being an adaptation
Speaker 3: to the kinds of computational problems that a system has
Speaker 3: either of or being trained to solve, right, And so
Speaker 3: for human beings, those computational problems are shaped by the
Speaker 3: constraints that we operate under. And a lot of those
Speaker 3: constraints come from our biology, right that we, as you said,
Speaker 3: have limited lives, have you know, limited compute resources what
Speaker 3: we can carry around inside our heads, have limited bandwidth
Speaker 3: for communication. We have to like make noises at each other,
Speaker 3: or wiggle our fingers or you know, somehow use our
Speaker 3: bodies to transfer data from one human mind to another
Speaker 3: human mind. It's very inefficient. And so those constraints are
Speaker 3: things that mean that human intelligence takes a particular shape,
Speaker 3: which is we're able to learn from limited data because
Speaker 3: we have to because we don't live that long. Right,
Speaker 3: You can't rely on getting five thousand years of language
Speaker 3: data or multiple human lifetimes of you know, chess playing
Speaker 3: or whatever it is. Right, you have to be good
Speaker 3: at using the resources that you have in ways that
Speaker 3: are efficient. And so that's kind of like deciding what
Speaker 3: to think about being able to recognize when a problem
Speaker 3: has a structure that you I've seen before being able
Speaker 3: to you know, sort of like become sort of automatic
Speaker 3: in using certain kinds of patterns of thinking and strategies
Speaker 3: for solving problems, really trying to make it as easy as.
Speaker 4: Possible for us to use the resources that we have.
Speaker 3: And then you need to develop capacities for trying to
Speaker 3: circumvent those bandwidth constraints in order to be able to
Speaker 3: do things that transcend what any individual human can do,
Speaker 3: and that means developing things like language writing societies LLCs,
Speaker 3: you know, all of the sort of libraries, right institutions,
Speaker 3: all of the theory of mind, right for reasoning about
Speaker 3: what someone else might be trying to communicate to you.
Speaker 3: All of this stuff is actually sort of like human
Speaker 3: stuff that's a consequence of these constraints. And so as
Speaker 3: we make AI systems that are smarter, those AI systems
Speaker 3: are in turn being shaped by what they're being trained
Speaker 3: to do and what constraints they operate under. But those
Speaker 3: constraints are different from the ones that humans have. They
Speaker 3: can you know, get more data, they can get access
Speaker 3: to more compute, they don't have bandwidth limitations.
Speaker 4: You can just copy.
Speaker 3: A you know, a state of an AI system across machines.
Speaker 3: You can use the same data to train multiple AI systems,
Speaker 3: and all of those things mean that I think, rather
Speaker 3: than being sort of on one axis where we're sort
Speaker 3: of thinking about better and worse, it's more that there
Speaker 3: are many axes that we can think about intelligent systems
Speaker 3: developing along, and we're just going to end up in
Speaker 3: a state where we have human intelligence and we have AIS,
Speaker 3: and they're going to be meaningfully different from one another,
Speaker 3: rather than things that are sort of directly competing in
Speaker 3: terms of the capacities that they have.
Speaker 2: Yeah, I agree with you on that.
Speaker 1: When you think about the way that humans beat machines
Speaker 1: on data efficiency, what do you think that means is
Speaker 1: missing architecturally from our AI systems.
Speaker 3: I think it's actually it's a great question, and the
Speaker 3: way I would express it is not in terms of architecture.
Speaker 3: So it's actually in terms of a different part of
Speaker 3: a neural network. So when we think about this problem
Speaker 3: of inductive bias, right, which is we're you know, what
Speaker 3: a system is sort of disposed towards learning. As I said,
Speaker 3: our neural networks, the way we normally set them up,
Speaker 3: are pretty weak inductive biases. They can learn all sorts
Speaker 3: of things. The inductive bias that a neural network has
Speaker 3: it is constrained by its architecture, but it's also constrained
Speaker 3: by where it starts out in the space of the
Speaker 3: settings of all of those weights. And normally the default
Speaker 3: is that you set up your neural networks so those
Speaker 3: weights start out really small, close to zero, and then
Speaker 3: they sort of grow away from that as it starts
Speaker 3: to learn how to do things. We've had success in
Speaker 3: taking neural networks that are architecturally identical but setting them
Speaker 3: with different initial weights in order to create an inductive
Speaker 3: bias that enables rapid learning. So we act to use
Speaker 3: a technique called meta learning, which is a method from
Speaker 3: machine learning where you take the same neural network architecture
Speaker 3: and the same initial weights, and you use it to
Speaker 3: learn to solve lots of different problems, like you can
Speaker 3: use it to learn lots of different languages, say, from
Speaker 3: limited data, and then you try and optimize the initial
Speaker 3: weights of the neural network to make it so it
Speaker 3: can learn all of those languages better using the same
Speaker 3: kinds of algorithms we use for training the weights of
Speaker 3: the neural network. When we have these giant data sets.
Speaker 3: You can instead use those algorithms to train the initial
Speaker 3: weights of a neural network for a small data set,
Speaker 3: for lots of small data sets, And when you do that,
Speaker 3: you end up with a neural network that has an
Speaker 3: inductive bias that makes it possible to learn from small
Speaker 3: amounts of data. And so that's the kind of thing
Speaker 3: we've been exploring in my lab is can we find
Speaker 3: a way of taking exactly these same neural network architectures
Speaker 3: and just starting them out in a different place that
Speaker 3: maybe aligns better with the kinds of things that humans do. Okay, well,
Speaker 3: this is a really good segue to what I wanted
Speaker 3: to ask you, which is, if you're looking at rules
Speaker 3: and systems is one sort of math to describe the mind,
Speaker 3: and artificial neural networks is another kind of math, and
Speaker 3: probability is another. What does an optimal hybrid look like?
Speaker 3: Given that no single, no one of these describes everything
Speaker 3: about what's going on with minds, So what does the
Speaker 3: hybrid for an aisystem look like?
Speaker 2: In twenty twenty six, the place.
Speaker 3: Where I end up in the book is saying that
Speaker 3: these different kinds of math really do all fit together
Speaker 3: in an interesting way, and in order to understand that
Speaker 3: we can talk about different levels of analysis when we're
Speaker 3: trying to make sense of an information processing system. So
Speaker 3: the most abstract level, this is an idea that was
Speaker 3: introduced by the computational neuroscientist David Marr. He said, the
Speaker 3: most abstract level is just thinking about the problem that
Speaker 3: the system is solving in its ideal solution, right. And
Speaker 3: I think logic and symbolic systems and probability theory give
Speaker 3: us a good way of characterizing the kinds of problems
Speaker 3: that minds have to solve, right They you know, probabilistic
Speaker 3: inference because we have to make these uncertain inferences. And
Speaker 3: then logic as a way of characterizing the kinds of
Speaker 3: things that are in the world that have this rich
Speaker 3: structure of you know, like a sort of combinatorial structure
Speaker 3: that you get from from having symbols and rules that
Speaker 3: combine together with things like language and dance and all
Speaker 3: of these you know, structured things. Even you know, if
Speaker 3: you look at trees, you can see they have like
Speaker 3: recursive structures that are expressed in them. Right, So these
Speaker 3: kind of occur in nature and are important to be
Speaker 3: able to understand. And then at the level below that
Speaker 3: you have how the system solves those problems, right, like
Speaker 3: what algorithms it might use, what representations it might use,
Speaker 3: And then below that it's you know, how that's actually
Speaker 3: implemented in some kind of physical system, right, and artificial
Speaker 3: neural networks give us a kind of story at those
Speaker 3: levels where we can think about them as being a
Speaker 3: good general purpose system for learning to approximate the things
Speaker 3: that probabilistic in front tells you to do, and learning
Speaker 3: to approximate the structure that's contained within those symbolic systems.
Speaker 3: So I actually think, you know, the kind of story
Speaker 3: that we have right now that's emerged out of these
Speaker 3: advances in AI is actually a pretty good story for
Speaker 3: how we could think about human minds working. The thing
Speaker 3: that's missing, most important thing that's missing is this kind
Speaker 3: of aspect of inductive bias, right where we haven't been
Speaker 3: able to capture what human inductive biases are like in
Speaker 3: machines and so that you have these meaningful differences that come.
Speaker 4: Out of that.
Speaker 3: But it's not a bad place for thinking about how
Speaker 3: these pieces might fit together to give us an explanation
Speaker 3: for how it is that mind's work.
Speaker 1: So along these lines, which AI benchmarks feel to you misleading?
Speaker 2: And how would you make better benchmarks?
Speaker 3: So, in general, I'm not a huge fan of benchmarks,
Speaker 3: because I think benchmarks are useful as an engineering tool,
Speaker 3: but I, as a cognitive scientist, don't just want to
Speaker 3: know how well something is doing something. I want to
Speaker 3: know how it's doing that thing right and how it
Speaker 3: might be sort of messing that up right. So when
Speaker 3: we are designing experiments as cognitive scientists, we don't just say, oh,
Speaker 3: here's one hundred math problems. Go do with one hundred
Speaker 3: math problems and we'll get a score. We say, let's
Speaker 3: choose a set of math problems so that which answers
Speaker 3: people give us tell us about the misconceptions that they
Speaker 3: have in a way that we can then diagnose, oh,
Speaker 3: you know, this is why this person is thinking this
Speaker 3: particular thing. And so I think there's lots of room
Speaker 3: for coming up with better ways of evaluating our AI
Speaker 3: systems that look more like cognitive science experiments. We're really
Speaker 3: targeting understanding what's going on rather than just trying to
Speaker 3: get some brute sort of you know, performance score.
Speaker 1: Okay, good, And you have talked about curiosity as a
Speaker 1: computational problem. So how do you think about what curiosity
Speaker 1: is and how we might measure real curiosity in a machine?
Speaker 3: What problem is curiosity trying to solve? Yeah, this is
Speaker 3: this is a good question. You can you can ask
Speaker 3: this kind of question that we call rational analysis. Right,
Speaker 3: if you have a system that's solving a problem, you
Speaker 3: know what's what's the problem?
Speaker 4: What's the ideal solution?
Speaker 3: Okay, So for curiosity, we've argued and this is work
Speaker 3: with wretched debate.
Speaker 4: Who is that UCLA That.
Speaker 3: One way I think about curiosity is that you're trying
Speaker 3: to find things that are good in increasing your long
Speaker 3: run probability of being able to solve problems in the future. Right,
Speaker 3: So you know, it's sort of like you want data
Speaker 3: which for which the derivative of your total knowledge is
Speaker 3: high relative to that particular data point. And so that
Speaker 3: explanation captures some of the things that happen in human cognition,
Speaker 3: where you know, in some circumstances, we're interested in the
Speaker 3: newest thing, something we've never seen before, Right, But in
Speaker 3: a lot of circumstances, those things aren't the things that
Speaker 3: grab our attention. It's more things that maybe we've seen
Speaker 3: a few times, and you know, we just sort of
Speaker 3: noticed that they're starting to occur. If something happens once,
Speaker 3: you're just say okay, that was weird, and you sort
Speaker 3: of dismiss it. But when something happens a few times
Speaker 3: and it's unfamiliar to you, you say, okay, maybe I
Speaker 3: need to figure that out. Right, And something that happens
Speaker 3: all the time, you're not that curious about. That's just
Speaker 3: the thing that happens all the time. And you can
Speaker 3: explain that by thinking about this sort of derivative, right,
Speaker 3: where if something just happens once, you shouldn't be interested
Speaker 3: in it. Because it just happened once, it's probably never
Speaker 3: going to happen again. If something happens a few times,
Speaker 3: that's a clue that it's probably going to happen again
Speaker 3: in the future, and you've not seen it enough to
Speaker 3: actually know what's going on, And so paying attention to
Speaker 3: that is good in terms of that derivative of your
Speaker 3: future knowledge. And if something happens a lot, then it's
Speaker 3: probably happened enough that you know something about what's going
Speaker 3: on and it's not that interesting, right, And so so
Speaker 3: that sort of sweet spot ends up being around the
Speaker 3: things that are sort of like just happening to your
Speaker 3: enough times that you're starting to realize, oh, this might be.
Speaker 4: A thing that I need to pay attention to.
Speaker 1: If you hadn't been on one capability that's going to
Speaker 1: unlock a broader intelligence, unlock a jump to that.
Speaker 2: What's your candidate?
Speaker 3: I actually think the biggest obstacle at the moment is
Speaker 3: more about generalizability of intelligence rather than any specific capacity, right.
Speaker 3: And so people in the AI world talk about jagged intelligence, right,
Speaker 3: the sort of phenomenon where you have an AI system
Speaker 3: that can do something that's really smart and impress you,
Speaker 3: and then five minutes later does something that's really dumb
Speaker 3: on a problem that's like right next to it, and like,
Speaker 3: if it's able to solve that first problem, it seems
Speaker 3: obvious that it should be able to solve the second problem.
Speaker 4: And you're just like, what happened? You know, why did
Speaker 4: it go wrong there?
Speaker 3: And so that lack of generalizability is also a consequence
Speaker 3: of these kinds of inductive biases, right, So these human
Speaker 3: inductive biases that steer us towards a solution and let
Speaker 3: us learn from limited amounts of data, they constrain the
Speaker 3: kinds of solutions that we find are The kinds of
Speaker 3: solutions that we find are the ones that are sort
Speaker 3: of like generalizable at least to us, right. They are
Speaker 3: things that kind of make sense where if someone's able
Speaker 3: to do one thing, they'll be able to do the
Speaker 3: other thing. And because the AI systems are approaching these
Speaker 3: problems just in a completely different way from a different
Speaker 3: starting point and then getting tons of data that's allowing
Speaker 3: them to sort of approximate what the human solutions are.
Speaker 3: But they're coming at it from another angle. That's the
Speaker 3: thing that makes them jagged. It's not that they don't
Speaker 3: have sort of these same compatible inducted bis is that
Speaker 3: we have that are informed by having evolved in certain
Speaker 3: environments and having had experience of the world, and you know,
Speaker 3: all of these other things that are part of what
Speaker 3: it means to you know, sort of learn anything as
Speaker 3: a human being. And so because they are coming with
Speaker 3: this different set of inductive biases, they're very influenced by
Speaker 3: their training data, they end up doing things that are
Speaker 3: sort of inscrutable to us because they are, you know, yeah,
Speaker 3: like coming at these problems in a way that doesn't
Speaker 3: make sense to us. You know, from the starting point
Speaker 3: that humans come from.
Speaker 1: After writing this book, what do you think we understand
Speaker 1: now about minds that we didn't understand let's say, a
Speaker 1: decade or two ago.
Speaker 3: So it's funny because when I have taught this material
Speaker 3: for you know, twenty twenty years at this point, I
Speaker 3: normally start my cognitive science classes saying, you know, welcome
Speaker 3: to cognitive science. This is going to be different from
Speaker 3: your other science classes. Normally, when you take a science class,
Speaker 3: someone is going to stand up and say, Okay, here's
Speaker 3: all the things that we figured out. Here are the
Speaker 3: answers to the questions. And in cognito science, it's more
Speaker 3: that we figured out how to get better at asking
Speaker 3: the questions.
Speaker 4: We haven't answered them. We don't.
Speaker 3: It's not like you have a consensus across the whole
Speaker 3: field about what those answers look like. And so I
Speaker 3: think that's important that we're still very much And this
Speaker 3: is what got me interested in cognres science in the
Speaker 3: first place. You know, still a field that has deep
Speaker 3: mysteries and lots of opportunities to learn and discover interesting things.
Speaker 3: But I think over the last ten years, like so,
Speaker 3: as I was working on this book, I wrote the
Speaker 3: first chapter, and I had that disclaimer in the first
Speaker 3: chapter and said, okay, look, you know I'm not bromising
Speaker 3: you answers. Well, well, we're going to see if we
Speaker 3: can get a good handle on the questions. But by
Speaker 3: the time I got to the end of the book,
Speaker 3: right after that sort of process of working on it
Speaker 3: for years, I felt like things that actually, you know,
Speaker 3: me going through the process of writing it and exploring
Speaker 3: all these things and thinking about how they fit together,
Speaker 3: but also just where the field was, you know, having
Speaker 3: moved forward, I actually started to feel like, actually, these
Speaker 3: things do fit together in a way where you can
Speaker 3: see the glimpses of what answers are going to look
Speaker 3: like in a way that I think really wasn't there
Speaker 3: ten years ago. And it's that story of Okay, we
Speaker 3: sort of know what the goals are, right, we know
Speaker 3: what the right mathematical systems are for describing what intelligent
Speaker 3: system should be doing. You have these ingredients of symbolic
Speaker 3: systems and probablistic inference, and we've discovered that in fact,
Speaker 3: you can get a remarkable way just using these artificial
Speaker 3: neural networks to learn to approximate those things, and so
Speaker 3: that demonstration I think has shown first of all, that
Speaker 3: language is a extremely good substrate for intelligence right in
Speaker 3: a way that I think people had not anticipated before
Speaker 3: large language models, and that you can make big neural
Speaker 3: networks that can learn to approximate really complex probability distributions.
Speaker 3: And so it gives us some of these ingredients for
Speaker 3: seeing how what originally worth three very different views of
Speaker 3: the mind might start to fit together to make something
Speaker 3: that's a little bit more of a unified hole.
Speaker 1: Excellent, And when you wrote the book what struck You
Speaker 1: is the most beautiful idea in the whole quest, in
Speaker 1: the whole history of this gosh.
Speaker 3: Okay, I mean, I'm a big probability for theory fan,
Speaker 3: so going to you're gonna get me endorsing Bays rule,
Speaker 3: which I really do think is like it's it's when
Speaker 3: you learn it, take it a probability class. It's just
Speaker 3: like it's just a dumb principle of probability theory. But
Speaker 3: when you make this move of saying probability theory isn't
Speaker 3: just about dice and cards, it's about you know, beliefs,
Speaker 3: it suddenly becomes a very deep and insightful sort of principle,
Speaker 3: And in the book, I also show probability theory kind
Speaker 3: of subsumes logic, like everything that's a valid logical inference
Speaker 3: is also a valid inference in probability theory. Probability theory
Speaker 3: just kind of extends the surmountics of logic to these
Speaker 3: cases of uncertainty. So to me, I think that's a
Speaker 3: that's a that's a big one. I kind of like
Speaker 3: that's where I live.
Speaker 1: Yeah, excellent, And and somebody, if we do have a
Speaker 1: mature physics of thought, let's say fifty years from now,
Speaker 1: what is that change from us in terms of education,
Speaker 1: in terms of the way we build machines.
Speaker 3: So I think this is this is exactly where we
Speaker 3: can go, right, which is, once you figure out the
Speaker 3: scientific principles of a domain, you can start to think
Speaker 3: about how to do engineering right. So like you know,
Speaker 3: when you're an engineer and you go to engineering school,
Speaker 3: you take physics, right, and then you learn in your
Speaker 3: physics class what these principles are, and then you take
Speaker 3: your applied engineering classes, which are like taking those physical
Speaker 3: principles and telling you how to build a bridge right
Speaker 3: and explaining that you know not in terms of heuristics
Speaker 3: for what makes a good bridge, but in terms of
Speaker 3: those fundamental physical principles. So I think that's a thing
Speaker 3: that's incredibly exciting here is that as we start to
Speaker 3: converge on what these laws of thought look like, it
Speaker 3: gives us the opportunity to do a much more sort
Speaker 3: of science based form of engineering applied to human cognition,
Speaker 3: thinking about how do we make an optimal you know,
Speaker 3: sort of learning environment, how do we support human decision making.
Speaker 3: That's something that I work on in my lab is like,
Speaker 3: how do we put computation into human environments to overcome
Speaker 3: whatever computational constraints we have as individual decision makers and
Speaker 3: help us make better decisions. And how do we understand,
Speaker 3: you know, the kinds of things that people are doing
Speaker 3: in a way that allows us to then sort of
Speaker 3: like make suggestions about, you know, how they might do
Speaker 3: them better. Right, And so I think there's a there's
Speaker 3: a lot of potential for you know, sort of human
Speaker 3: upside as we start to be able to answer these
Speaker 3: scientific questions.
Speaker 1: That was my interview with Tom Griffith's. To quickly summarize
Speaker 1: his framework, Tom sees three major scientific approaches that all
Speaker 1: try to capture the mind. You've got rules and symbols,
Speaker 1: you've got artificial neural networks, and you've got probability theory.
Speaker 1: These very different approaches, and each which one has delivered
Speaker 1: something a little different. Rules and symbols give us language
Speaker 1: like machinery where pieces can be assembled and reassembled into
Speaker 1: complex ideas. Artificial neural networks they give us graded concepts,
Speaker 1: meaning ideas can be fuzzy, and probability theory gives us
Speaker 1: a language for dealing with uncertainty. Now, what's interesting is
Speaker 1: that human minds seem to traffic in all of these modes.
Speaker 1: We use structured symbols, we also use graded concepts. We
Speaker 1: also revise our belief as new evidence comes in. And
Speaker 1: part of that is that we move through the world
Speaker 1: with prior beliefs shaped by our history, our culture, our language,
Speaker 1: our neighborhood, our moment in time. So none of these
Speaker 1: models by themselves are the final answer. And what this
Speaker 1: means is that, like most scientific stories, this is one
Speaker 1: about humility. Tom's book illustrates how every generation arrives with
Speaker 1: some new formalism, some new piece of math, some new
Speaker 1: model that's powerful enough to illuminate an area of mental
Speaker 1: life and for a moment it feels like, hey, the
Speaker 1: whole mystery is finally collapsing. But then the spotlight widens
Speaker 1: and we see more terrain. So what I love about
Speaker 1: this conversation is that it can leave us with a
Speaker 1: sense of progress and a sense of wonder. At the
Speaker 1: same time, we feel a convergence of different fields, and
Speaker 1: we can also feel how large this subject remains. Cognition
Speaker 1: is still a field in motion, So let's look at
Speaker 1: the big picture. When the field of physics matured, we
Speaker 1: could then build bridges and airplanes and power grids because
Speaker 1: we had firm principles to build on. So once the
Speaker 1: laws of thought come into clearer view, what becomes possible
Speaker 1: for education and for decision making, and for rules that
Speaker 1: help us reason more effectively. So here we are at
Speaker 1: a very cool moment in history where the old dream
Speaker 1: of formalizing thought has escaped to the library and shown
Speaker 1: up in everyone's laptop. The big thinkers of centuries ago
Speaker 1: could sort of squint and see the outline of the project,
Speaker 1: and now we're living much more squarely right in the
Speaker 1: middle of it. If there truly are laws of thought,
Speaker 1: they're going to teach us about our machines, but more importantly,
Speaker 1: they're going to teach us about ourselves, because although it's
Speaker 1: sometimes tempting to view the mind as a ghostly exception
Speaker 1: to the universe, the mind, presumably is part of the universe,
Speaker 1: and it is lawful and wondrous and discoverable, and every
Speaker 1: step towards understanding it enlarges the human story. Go to
Speaker 1: eagleman dot com slash podcast for more information and to
Speaker 1: find further reading. Join the weekly discussions on my substack,
Speaker 1: and check out and subscribe to Inner Cosmos on YouTube
Speaker 1: for videos of each episode and to leave comments until
Speaker 1: next time. I'm David Eagleman, and this is Inner Cosmos.