Speaker 1: Welcome to Stuff to Blow Your Mind from how Stuff
Speaker 1: Works dot com. Hey, welcome to Stuff to Blow your Mind.
Speaker 1: My name is Robert lamp and I'm Joe McCormick. And
Speaker 1: today we're gonna be looking at understanding a little bit
Speaker 1: of the gaps in our knowledge and our metacognition. And
Speaker 1: it's going to be the first part of a two
Speaker 1: part episode on the illusion of explanatory depth. So if
Speaker 1: you like this one, you should also definitely come back
Speaker 1: for the next episode we released next time, which will
Speaker 1: be a follow up to what we're going to talk
Speaker 1: about today, illusion of explanatory depth. So to put that
Speaker 1: in simpler terms, we're talking about a situation where you
Speaker 1: think you know how something works, you think you have
Speaker 1: a working knowledge of the thing, but in reality you don't. Yeah,
Speaker 1: and we've talked about the gaps between the feeling of
Speaker 1: no wing and the actual knowing before this came up
Speaker 1: in the episode we did about the tip of the
Speaker 1: tongue state. You remember that the sense that you know
Speaker 1: something is not necessarily coterminous with actually being able to
Speaker 1: produce that piece of knowledge from memory. There's a gap
Speaker 1: in your mind. So you think you know the name
Speaker 1: of the actor who played Thaiwan Lanister on Game of Thrones.
Speaker 1: Do you Robert Oh, no, I can't. He has that
Speaker 1: weird name that I can never know, Thaiwan. Yeah, wait,
Speaker 1: which one is Taiwan? No, I'm thinking of Jamie. I
Speaker 1: can never remember Jamie's name. What about Thaiwan? Come on,
Speaker 1: he's an alien? Three? Okay, Well you got it. You
Speaker 1: didn't You didn't have that gap. Then some people might
Speaker 1: there was a gap in there somewhere. I did feel
Speaker 1: a gap. If if you're familiar with Game of Thrones
Speaker 1: out there you were, like some of you were probably thinking, oh,
Speaker 1: I know that name, what is it? What is it?
Speaker 1: It was Charles Dance. Okay. The other possible um gap
Speaker 1: there is you hear Taiwan? And then you were is
Speaker 1: a Tyrian? Is it Thaiwan? More names? Well? I picked
Speaker 1: Character actors often fall into this category. The character know
Speaker 1: the actors you've seen in tons of movies throughout the years.
Speaker 1: They become that tip of the tongue name where you
Speaker 1: know the face. You know some movies they've been in.
Speaker 1: You know you know the name, but you don't know
Speaker 1: the name. At the moment. It's this that guy, Oh,
Speaker 1: what's that guy's name? Yeah, And so there's this gap,
Speaker 1: there's this feeling of knowing, and there's the gap between
Speaker 1: the feeling of knowing and the actual knowing itself. But
Speaker 1: the interesting thing is that this gap can be applied
Speaker 1: to other realms of knowledge. It's not just in trying
Speaker 1: to come up with the name for a thing, for example,
Speaker 1: in a quint essentially how stuff works move I think
Speaker 1: we should look at the domain of knowledge that covers
Speaker 1: understanding how things happen, or really in really understanding how
Speaker 1: things work in causal relationships, because of course we live
Speaker 1: in a world of systems. The system is always trying
Speaker 1: to get you down. But but there are causal systems
Speaker 1: all around us, machines, the coffee maker in the office,
Speaker 1: the computer you're working on, and ammals, animals or systems
Speaker 1: of causal relationships. There are natural cycles, like you know,
Speaker 1: the nitrogen cycle or the water cycle. Those are causal systems.
Speaker 1: And then other natural phenomenon uh, tides, rainbows, I don't know,
Speaker 1: pooping a all natural phenomenon. Well, these are all things
Speaker 1: that I mean too. We have to mention, of course,
Speaker 1: the famous quote other C. Clark, right, that any sufficiently
Speaker 1: advanced technology is indistinguishable from magic. But you could pretty
Speaker 1: much say that about like any system, Uh that if
Speaker 1: if you if it's if it's it's it's if it's
Speaker 1: advanced enough and complex enough, and and most systems are U.
Speaker 1: It can it can seem magical in the fact that
Speaker 1: the sun rises in the morning. Um, there's a magic
Speaker 1: to that. We we've we've we've observed it his magic
Speaker 1: and felt it is magic since the time. Out of
Speaker 1: mind we sometimes are even though we have the the
Speaker 1: actual scientific explanation for what's happening, you still also have
Speaker 1: this magical version of the event, uh pared right beside
Speaker 1: it on on the shelf in your mind. Oh totally.
Speaker 1: I mean, we have strong intuitions to give to give
Speaker 1: magical or kind of fuzzy causal relationships. And and it's
Speaker 1: funny because one way of interpreting the idea of magic
Speaker 1: or the supernatural is it's just causal anddeterminacy, right, Like,
Speaker 1: what do you mean when you say something happened by
Speaker 1: magic or something happened with the supernatural cause, it just
Speaker 1: means that the cause. Essentially you're saying, well, the cause
Speaker 1: isn't clear. It's just kind of like getting vague about
Speaker 1: what it means to be a cause I like this.
Speaker 1: You know, my my son who's almost five. He we
Speaker 1: try to explain how things work to him, as one should,
Speaker 1: but he also has this concept of magic. It's very
Speaker 1: loose concept. So the other day he had a new
Speaker 1: helium balloon and it was one of those uh those
Speaker 1: fancy shiny ones that that you get. What's the material?
Speaker 1: My large, my large? So it was a mile balloons,
Speaker 1: so it last, it was lasting longer. He's used to
Speaker 1: getting these cheap balloons and they the helium goes down
Speaker 1: and they're on the floor, but this one was floating
Speaker 1: the next day and he said, hey, that my balloon
Speaker 1: is still floating. Is this is it magic helium? Um?
Speaker 1: Which I think was maybe like his definition of magic
Speaker 1: is more in line with what you just said. There's
Speaker 1: a there's a mystery there. Uh like he knows that
Speaker 1: this helium is not behaving like like like normal helium.
Speaker 1: That's that he's encountered and he has no other explanation
Speaker 1: for it. Yeah. But yeah, so you don't have to
Speaker 1: at that point explain how the magic does what it does.
Speaker 1: If you did, it would sort of stop being magic.
Speaker 1: But Yeah. So so there are these systems all around us.
Speaker 1: We we sort of naturally feel like they're magic, but
Speaker 1: we can come to understand the causal processes that that
Speaker 1: that sustained them and that make them work. But as
Speaker 1: we've said, understanding and the feeling of understanding are actually
Speaker 1: separate things. And whenever you've got two different binary variables
Speaker 1: like this, I think it's interesting to try to make
Speaker 1: that the grid table, you know, where you've got one
Speaker 1: binary area on a column and one binary on a row.
Speaker 1: So you can think of things that we understand and
Speaker 1: that we don't understand, and then you can think of
Speaker 1: things that you feel like you understand or that you
Speaker 1: don't feel like you understand. So there are things that
Speaker 1: we understand and we feel like that we understand them
Speaker 1: like a hammer. Yes, you think you get it, you
Speaker 1: really do get it. Yeah, there's there's some very simple
Speaker 1: physics involved here. There's a there's a there's a definite
Speaker 1: causal um process going on. Yeah. Then there's maybe how
Speaker 1: microprocessor engineering works. That's one where you probably don't understand
Speaker 1: it and you probably feel like you don't understand it. Right.
Speaker 1: This is one of those where you have a problem
Speaker 1: with your computer and you just you tell your tech
Speaker 1: tech guy or gal, you say, it's all magic to me.
Speaker 1: I don't know how this works. Can you help me
Speaker 1: fix this problem? Right? So, those are the ones where
Speaker 1: are understanding and our feelings are basically in agreement. But
Speaker 1: what about the other two boxes? What about things that
Speaker 1: you understand but you don't feel like you understand can
Speaker 1: actually happen sometimes, and I think it's often the starting
Speaker 1: place of a Socratic dialogue or if you ever you know,
Speaker 1: the Socratic teaching method is where instead of telling students
Speaker 1: what to believe, you ask them questions and sort of
Speaker 1: lead them to understand that they already knew the answer,
Speaker 1: but they just didn't know how to articulate it. And
Speaker 1: so in that case, the child already understands, they just
Speaker 1: didn't know how to put the answer with the question
Speaker 1: in context. But then there's the other box, the things
Speaker 1: you feel like you understand but you don't actually understand.
Speaker 1: And the research we're going to talk about today is
Speaker 1: addressing how there is tons of stuff in this box.
Speaker 1: This box box is filled to the brim. Uh toilets
Speaker 1: are probably in this box for you? What do you?
Speaker 1: What do you think? Unless you're a plumber, or you've
Speaker 1: really done some work on your toilet. I bet toilets
Speaker 1: or in this box. Yeah, I mean they're they're fairly complicated,
Speaker 1: a little little mechanisms, despite the fact that they maybe
Speaker 1: haven't buying large advanced as much as they should, because
Speaker 1: it's kind of one of those technologies that we tend
Speaker 1: to think, all right, it's good enough, and we don't
Speaker 1: want to we don't want to put too much extra
Speaker 1: thought into into its design and function. Yeah, here's another one.
Speaker 1: How How what about mirrors? Mirrors is a great one,
Speaker 1: and I love this example. I think I've brought it
Speaker 1: up before, but yeah, I think it's a perfect example
Speaker 1: of an everyday object that we take mostly for granted,
Speaker 1: but it is ultimately this insane, freaky mystery in our lives.
Speaker 1: I mean, really, it's amazing that we don't just run
Speaker 1: around constantly smashing them like maniacs. I think you, like me,
Speaker 1: love a good creepy mirror story, like a haunted mirror.
Speaker 1: What's the Stephen King won the Representage? The Representage fabrical
Speaker 1: short stories one of his best, in my opinion. Uh,
Speaker 1: and there are tons of them, Lovecraft wrote one Clark
Speaker 1: Ashton Smith wrote one, you could probably fill an entire
Speaker 1: book with just creepy mirror stories, and then I would
Speaker 1: buy said book. But wait a minute. Of course, we
Speaker 1: understand how a mirror works. That's easy. It's just uh, well,
Speaker 1: like the light goes in and then it comes back. Right. Well, yeah,
Speaker 1: we think we we have we have it under wraps, right,
Speaker 1: because we encounter them all the time and we have
Speaker 1: this sort of ubiquitous environmental knowledge of them. But when
Speaker 1: we're put to the test the office seems to be
Speaker 1: uh the case. We we don't really understand how they work.
Speaker 1: And and I think this is why we have all
Speaker 1: these fictional tales about weird creepy mirrors, because we need
Speaker 1: that that cultural release valve, that psychic release valve for
Speaker 1: our uneasiness about them. But in terms of just proving
Speaker 1: this out, there was a two thousand five psychological study
Speaker 1: from the University of Liverpool and they looked into this
Speaker 1: and they asked participants in the study to consider a
Speaker 1: draped mirror, so it's, you know, like a haunted mirror
Speaker 1: that's been covered up to keep monsters from coming out
Speaker 1: of it, and they had to predict at which points
Speaker 1: in the room. They would be able to see themselves
Speaker 1: if the mirror in the mirror, if the mirror was uncovered. Okay,
Speaker 1: so if you really had a solid understanding of what
Speaker 1: a mirror, how a mirror works, you should be able
Speaker 1: to predict how you can use it. And they weren't
Speaker 1: able to to do that. They weren't. They weren't able
Speaker 1: to Another thing they couldn't do is they weren't able
Speaker 1: to grasp the fact that your reflection in the mirror
Speaker 1: is always half your size, because the mirror is always
Speaker 1: halfway between the viewer and the viewers reflection. So they'd
Speaker 1: be asked to say, well, they'd be asked how big
Speaker 1: is your your head in that reflection, and they would
Speaker 1: assume that it was the same size as their own head.
Speaker 1: So I would have assumed, yeah, I mean I wouldn't.
Speaker 1: I really had to read the that sentence a couple
Speaker 1: of times. Towie, oh, yeah, there is the mirror is
Speaker 1: halfway between me and the spectral doppelganger with that that
Speaker 1: has his his hair parted on the opposite side. Um
Speaker 1: the But the study basically revealed that we we tend
Speaker 1: to assume the size of the reflection. We tend to
Speaker 1: assume that we know exactly how the angles work for
Speaker 1: the reflection. Uh, we're terrible determining what will be seen
Speaker 1: in a mirror based on the observer's vantage point. And
Speaker 1: a major example of this is the Venus effect that
Speaker 1: we see in so many paintings. Venus. Okay, so you
Speaker 1: have Venus in the painting. Venus is looking at her
Speaker 1: face in a mirror, and we're looking at the painting
Speaker 1: and we see Venus's face. But if she's looking at
Speaker 1: her face in the mirror, how does that work. It's
Speaker 1: It's like, next time you're watching a TV show or
Speaker 1: a movie and there's a scene with a mirror over analysis,
Speaker 1: really think about where's the camera, where's the camera? What
Speaker 1: are they looking at it? It really begins to open
Speaker 1: up your eyes to the fact that it said, Wow,
Speaker 1: I I was completely hoodwinked by this, and maybe I
Speaker 1: don't have the firmest idea of the optical scenario going
Speaker 1: on here. Uh. Slightly related, Also, anytime you're watching a
Speaker 1: movie where there's a mirror on the lid of a
Speaker 1: medicine cabinet and the person opens the medicine cabinet and
Speaker 1: then shuts it be prepared to see another face in
Speaker 1: the mirror behind the person when they shut the lid.
Speaker 1: It happens every time. You know. One more, just very
Speaker 1: quick optical example is just site itself. I think we've
Speaker 1: touched on this that the idea that site is something
Speaker 1: that leaves our eyes. Oh yeah, it's like laser vision. Uh.
Speaker 1: This is one of those things like I talked about earlier,
Speaker 1: where we have this magical unrealistic idea of how it works.
Speaker 1: And even if you have the the realistic idea of
Speaker 1: how it works, the idea of that light is entering
Speaker 1: your eyes, you still you still end up thinking about
Speaker 1: the world in terms of the the fictional scenario. I
Speaker 1: think that's sort of a different gap because I think
Speaker 1: most people do know really they know that the light
Speaker 1: is entering the eyes, that nothing, nothing's going out. But
Speaker 1: you're talking there about the difference between what what we
Speaker 1: know and what we feel, and I think that where
Speaker 1: those two converge, there's room for a lot of confusion.
Speaker 1: I think that's absolutely right. Well, I think so today
Speaker 1: we're going to look at the one big original study
Speaker 1: in the illusion of explanatory depth, and then in the
Speaker 1: next episode we're gonna look at some some takeaways and
Speaker 1: some applications from it. But so I guess we should
Speaker 1: get into the study itself, right, Yeah, do you want
Speaker 1: to take a quick break before we get into it.
Speaker 1: I want to take a quick break, Joe, and then
Speaker 1: when we come back, let's get into this study. All right,
Speaker 1: we're back, all right. So this landmark study is called
Speaker 1: The Misunderstood Limits of Folk Science, an Illusion of Explanatory Depth,
Speaker 1: published in Cognitive Science in two thousand two by Frank
Speaker 1: Kyle and Leonard Rosenblit. And so they start by discussing
Speaker 1: the idea of folk theories. Have you ever heard this
Speaker 1: concept before, Robert folk theories or folk science. Yeah, this
Speaker 1: is just kind of the It was like folk medicine, right,
Speaker 1: It's not necessarily there's not necessarily any science to it.
Speaker 1: It's just kind of the the the general understanding of
Speaker 1: how something works or how it's supposed to work. Yeah,
Speaker 1: it's what we come up with when our methods are
Speaker 1: not Garris. Essentially, it's what we all do sort of
Speaker 1: intuitively all the time. And so they say, you know,
Speaker 1: sort of a theory can be defined as a system
Speaker 1: of ideas that are designed to explain something observed. The
Speaker 1: theory gives an explanation, and theories are a totally common
Speaker 1: feature of science and of everyday life. You know, we
Speaker 1: we use theories all the time. They might not be
Speaker 1: good or correct theories, but we're constantly having theories about
Speaker 1: the explanation of the workings of objects and systems. A
Speaker 1: great example of this is that blue blood in your veins.
Speaker 1: Oh yeah, do you have an explanation for that? Well,
Speaker 1: there's the Well, it's because it's deprived of oxygen, right,
Speaker 1: and that why it turns blue. That's not correct, is it. No,
Speaker 1: it's not correct, but it's It's one of those that
Speaker 1: is often thrown out there sometimes but very intelligent people.
Speaker 1: It's I you know, I don't mean to to mention
Speaker 1: any of these is an example of intellectual failing, but
Speaker 1: they just they pick up esteem. They're passed around, and
Speaker 1: it's easy to go through life thinking that they're true. No,
Speaker 1: and that that can to another thing. We should say.
Speaker 1: This episode is going to be all about our cognitive
Speaker 1: limitations and failures and overconfidence in what we know. But
Speaker 1: this isn't to say that people are stupid or you know,
Speaker 1: we're not accusing the people featured in the studies or
Speaker 1: people in general of being dumb. It's just good to
Speaker 1: reckon with what the mistakes human brains usually make. Our
Speaker 1: human brains make mistakes continually, and uh, I mean, the
Speaker 1: best you can do is be aware of the limitations.
Speaker 1: But one of the things about these folk theories is
Speaker 1: that they often feel like they explain more than they
Speaker 1: actually do. And take the blue blood in the vein
Speaker 1: that that seems intuitive. What if somebody you believe that, Okay,
Speaker 1: I'm looking at my veins in their blue and it's
Speaker 1: because the blood turns blue. What if somebody asked you
Speaker 1: to write down an explanation of how that happens. Then
Speaker 1: you'd start being like, well, wait, so I'm trying to
Speaker 1: write the steps down, so the blood is deprived of
Speaker 1: oxygen and turns blue, how does that happen? Don't You'd
Speaker 1: start encountering gaps in your knowledge. And the authors of
Speaker 1: the study right about this, they say, quote, we frequently
Speaker 1: discovered that the theory that seems crystal clear and complete
Speaker 1: in our head suddenly develops gaping holes and inconsistencies when
Speaker 1: we try to set it down on paper. Uh, intuitively,
Speaker 1: I think that's they're exactly correct about that. I've had
Speaker 1: this experience plenty of times, or not even on paper.
Speaker 1: I bet Robert, I bet you've had this experience too.
Speaker 1: I know I have. Here in the podcast studio. In
Speaker 1: the middle of a podcast, maybe a tangent comes up
Speaker 1: where you briefly want to explain how something works that's
Speaker 1: not central to your research, and you think you do,
Speaker 1: so you just start talking, and you get a sentence
Speaker 1: or two in and you wait. You're like, oh, wait
Speaker 1: a minute. I thought I understood that when I started talking,
Speaker 1: But now that I'm saying the words, I don't actually
Speaker 1: know how this works. And you have to stop and
Speaker 1: figure out, Okay, what am I gonna do now? Yeah? Yeah,
Speaker 1: you have to make that decision. Do I do I
Speaker 1: own up to the fact that I that I really
Speaker 1: don't know what I'm talking about. I'm gonna make everybody
Speaker 1: wait while I read about this for fifteen minutes. Or
Speaker 1: do I just plow ahead and somehow easyl my way
Speaker 1: out of it. Um. I think where I encountered this
Speaker 1: a lot is is in the preparation for a podcast episode.
Speaker 1: My wife will ask me what we're recording on this week,
Speaker 1: and I'll say, oh, recording on such and side, and
Speaker 1: she's like, oh, really, what's what's that about? Give me
Speaker 1: the elevator pitch and and then I'll start to explain it,
Speaker 1: and then I'll realize, oh, you know, it's you know,
Speaker 1: A plus VEH equals C, except I can't adequately describe
Speaker 1: step B in the scenario. It made sense in your
Speaker 1: head until you started trying to use words, and then
Speaker 1: that's where the that's where it became problematic. And what
Speaker 1: it reveals is that, in fact, it didn't actually make
Speaker 1: sense in my head. It just felt like it did.
Speaker 1: And it's useful to it for us because then you know, oh, well,
Speaker 1: that's that's what I don't understand. That's what I need.
Speaker 1: That needs to make sense to me truly, because if
Speaker 1: it doesn't truly make sense to me, it's not going
Speaker 1: to make sense to the listener. Right. Okay, So folk
Speaker 1: theories in contrast us too scientific theories, where you've got
Speaker 1: scientists trying to constantly hunt down the gaping holes and
Speaker 1: inconsistencies in their theories and fix them. With folk theories, uh,
Speaker 1: the explanatory systems are sort of produced in the minds
Speaker 1: of lay people by non rigorous processes. And uh so,
Speaker 1: if you're not a telecommunications engineer, you probably have some
Speaker 1: kind of folk theory about how your cell phone works. Right,
Speaker 1: You've got some basic skeletal idea of well, there's a
Speaker 1: signal in the phone. Maybe you know, it's electromagnetic radiation
Speaker 1: that goes from the phone from the antenna part maybe
Speaker 1: or the antenna is hidden inside now, But it goes
Speaker 1: from part of the phone to a tower. Does it
Speaker 1: go to a satellite, I don't know. If it goes
Speaker 1: to the cloud, that's a big one. And I and
Speaker 1: I've been guilty of this too, not really stopping to realize,
Speaker 1: oh wait, the cloud, Like I know that there's not
Speaker 1: an invisible wonder Woman's airplane type computing system floating in
Speaker 1: the sky. And yet somehow I fall back on that idea,
Speaker 1: just perhaps just out of like I out of a
Speaker 1: lack of desire to understand um the details and our
Speaker 1: telecommunication system. But but yeah, I find myself at least
Speaker 1: putting it up, putting the non realists, the unrealistic version
Speaker 1: up on the shelf with a more realistic expectation of technology. Yeah,
Speaker 1: and yet nevertheless, you sort of think you understand how
Speaker 1: a cell phone works right at a basic level, at
Speaker 1: a basic level, and and then you start, oh no.
Speaker 1: But so the authors of the study, they're they're talking
Speaker 1: about the problems with the way people hold them. So
Speaker 1: they say, quote first, they are novice scientists. People people
Speaker 1: in general are novice scientists. Their knowledge of most phenomena
Speaker 1: is not very deep. We have shallow understandings. But then
Speaker 1: they also say, quote second, their novice epistemologists, meaning people
Speaker 1: who study how knowledge is generated, how we know things, uh, continuing,
Speaker 1: their sense of the properties of knowledge itself, including how
Speaker 1: it is stored, is poor and potentially misleading. So we
Speaker 1: have both an incomplete understanding of how many things work,
Speaker 1: but we also fail to recognize that we have an
Speaker 1: incomplete understanding, uh, exhibited by the fact that when we
Speaker 1: get put on the spot where we're sort of caught
Speaker 1: off guard, we're like, oh, wait, I thought I understood that,
Speaker 1: but now I'm realizing maybe I didn't. UM. So their
Speaker 1: central thesis in this paper is quote we argue here
Speaker 1: that people's limited knowledge and they're misleading. Intuitive epistemology combined
Speaker 1: to create an illusion of explanatory depth or io e ed.
Speaker 1: Most people feel they understand the world in far greater detail, coherence,
Speaker 1: and depth than they actually do. Um Also, they say
Speaker 1: that we're more overconfident about our understanding of some types
Speaker 1: of knowledge than others. Specifically, are our knowledge dealing with
Speaker 1: explanations for how things work that that is that is
Speaker 1: to be singled out. So to test these ideas, the
Speaker 1: authors performed a big series of studies. They're actually twelve
Speaker 1: different studies inside this this massive paper, UH, to measure
Speaker 1: people's level of confidence in their understanding compared with what
Speaker 1: their actual level of understanding is as measured by their
Speaker 1: confidence after they've had some calibration, and then then UH,
Speaker 1: comparing that within various different domains of knowledge, meaning just
Speaker 1: different types of knowing things do you know facts about geography,
Speaker 1: or do you know the narratives of movie plots, or
Speaker 1: do you know how a toilet works. So there have
Speaker 1: been a lot of previous studies about overconfidence, and one
Speaker 1: of the things that's important to establish is that a
Speaker 1: lot of previous research has sort of focused on general knowledge,
Speaker 1: that people might be overconfident about knowledge in general. And
Speaker 1: the authors are not into this idea. That they don't
Speaker 1: like the idea of general knowledge. Instead, they like the
Speaker 1: idea of breaking out knowledge into these different categories, because,
Speaker 1: as they will end up showing in their research, the
Speaker 1: brain estimates its own knowledge in different categories in with
Speaker 1: different levels of accuracy. Yeah. I think we all, most
Speaker 1: healthy individuals realize that they know a lot about some
Speaker 1: things maybe, but certainly little or nothing about other topics. Correct, right,
Speaker 1: and especially uh, not just topics, but different types of
Speaker 1: things to know. Like, you might be way more if
Speaker 1: I ask you, um, Robert, what is the capital of England?
Speaker 1: Before you answer, tell me how confident are you that
Speaker 1: you know the right answer? On a scale of one
Speaker 1: to ten, I would say a ten, Okay, what's the
Speaker 1: capital London? Okay, you're right there, you go. Okay, but
Speaker 1: tell me how confident are you that you can explain
Speaker 1: how a lightsaber works? Well? Uh, not not very because
Speaker 1: that's a it's essentially a magical device. Yeah, I forgot
Speaker 1: and uh and and I also don't I actually I
Speaker 1: think I rewrote the intro page for how lightsabers were
Speaker 1: on how stuff works dot com. Yeah, so I have
Speaker 1: actually worked with the an article, an article that explains
Speaker 1: how it supposedly works, but I don't recall it at all.
Speaker 1: Do you think working on that article would have made
Speaker 1: you more or less confident in your own understanding. I
Speaker 1: think if I had actually worked on the meat of it.
Speaker 1: But that's an article. I think I just brewsed up
Speaker 1: the landing page. I just sexed it up a little.
Speaker 1: Did Tracy Wilson write that one? No, I think it
Speaker 1: was an older piece. Well, anyway, let's go onto the study.
Speaker 1: So study number one out of this. First thing they
Speaker 1: wanted to do was document the illusion of explanatory depth.
Speaker 1: If this thing exists, let's see if we can get
Speaker 1: some evidence that it is there. So they got sixteen
Speaker 1: graduate students from various departments at Yale and these are
Speaker 1: the participants that this was done by professors at Yale University,
Speaker 1: so there's a lot of Yale's in this. And the
Speaker 1: test dealt with their ability to explain how a bunch
Speaker 1: of devices work. So participants were given instructions on how
Speaker 1: to rate their level of explanatory knowledge of a device
Speaker 1: on a scale of one to seven with the help
Speaker 1: of a couple of examples GPS system and a crossbow.
Speaker 1: So with the example of a crossbow, basically a seven
Speaker 1: means you know all the parts and you know how
Speaker 1: all of them work together to make the device work.
Speaker 1: You know all the causal relationships. You could you could
Speaker 1: almost build the thing yourself if you had all the parts.
Speaker 1: Um A one means you you basically don't know anything
Speaker 1: more than what it looks like and what it does.
Speaker 1: You don't know what the parts are, how the parts
Speaker 1: work together. It's almost magic to you. Okay. Then the
Speaker 1: participants were given a list of forty eight objects and
Speaker 1: asked to rate their level of understanding of how the
Speaker 1: object works. So you just go down this list, uh
Speaker 1: you know, uh l C D screen, car, battery, a zipper,
Speaker 1: a spiedometer, piano, key, can opener, hydroelectric turbine, flush toilet,
Speaker 1: cylinder lock, helicopter, quartz watch, sewing machine, And you're supposed
Speaker 1: to give the number on the scale of one to seven.
Speaker 1: Then how well do you understand what all the parts are,
Speaker 1: how they work together? How well do how well do
Speaker 1: you understand how it works? And just to use a
Speaker 1: little terminology because it will recur throughout throughout all the
Speaker 1: different studies here. This first rating is known as T one.
Speaker 1: This thing they give on the first questions their own
Speaker 1: self rating of their explanatory knowledge of each item. Is
Speaker 1: T one. And then in the next phase, the students
Speaker 1: are asked to write a detailed explanation for half of
Speaker 1: the of some of these items in the test category,
Speaker 1: to explain in detail how a sewing machine works. So
Speaker 1: you rated maybe a four on how well you know
Speaker 1: how a sewing machine works? Now we need you to
Speaker 1: explain it step by step in in words, And then
Speaker 1: they wrote that detailed explanation. Then they were asked to
Speaker 1: rate their initial understanding again. So now that you've written
Speaker 1: that explanation, how well did you understand it to begin with? Uh?
Speaker 1: Then they were given and that that rating is T
Speaker 1: two uh. Then they're given a diagnostic question. For example,
Speaker 1: if one of the items they had to explain was
Speaker 1: a cylinder lock, the diagnostic question might be do you
Speaker 1: know how to pick a cylinder lock? And this question
Speaker 1: is designed to force the person to think even more
Speaker 1: about what the parts are and how they work together.
Speaker 1: And then after the diagnostic question, they're asked to rate
Speaker 1: their initial understanding yet again how well did you understand
Speaker 1: it to begin with? And then finally the participants got
Speaker 1: to read a brief explanation written by an expert of
Speaker 1: how these items worked that they explained. And these expert
Speaker 1: explanations came from a cd ROM titled The Way Things
Speaker 1: Worked two point oh. I was hoping they'd use some
Speaker 1: vintage how stuff Works articles. No, no, alas it was
Speaker 1: two thousand two. How staff Works existed then, but we
Speaker 1: were not here anyway. After reading these expert explanations, they
Speaker 1: had to rate again how well they had initially understood
Speaker 1: the device, and then how well they understood it now
Speaker 1: after having read the explanation. So what are the results?
Speaker 1: What does this graph look like? You start with your
Speaker 1: initial guests, and then you get adjusted by having had
Speaker 1: to make an explanation, answer a diagnostic question, and then
Speaker 1: read an expert's explanation. Well, the graph forms a kind
Speaker 1: of U shape or an inverted bell shape, where initially
Speaker 1: the students rate their level of understanding really high or
Speaker 1: relatively high. Not necessarily really high, but it's like, yeah,
Speaker 1: you know, I give it a four. I I understand
Speaker 1: pretty well how a cylinder lock works. Then then they
Speaker 1: have to give the explanation and the ratings drop off significantly.
Speaker 1: Now note that this is not somebody coming in from
Speaker 1: the outside and telling them their explanation is wrong. This
Speaker 1: is their own self evaluation after having had to do
Speaker 1: nothing but just put their own ideas into words. Then
Speaker 1: it continued to drop again after the diagnostic question, and
Speaker 1: then finally shot back up again after reading the experts explanation. No,
Speaker 1: no surprise. If you read somebody telling you how it works,
Speaker 1: now you understand how it works. So it's a perfect
Speaker 1: story arc. It's kind of like a most to kung
Speaker 1: fu movies, right where you have the the the young
Speaker 1: student who is overconfident and then his uh, his, he
Speaker 1: gets his his rear end handed to him by the villain,
Speaker 1: and then he has to learn, he has to accept
Speaker 1: what he doesn't know, and then he has to to
Speaker 1: learn the craft from a from a master, and then
Speaker 1: in the end he can defeat the villain. It's a
Speaker 1: kind of a cry kid situation. Yeah. I think that's
Speaker 1: interesting how how our our narratives play on this this
Speaker 1: fact about us. It almost suggests that somehow we might
Speaker 1: intuitively be somewhat aware of the illusion of explanatory depth.
Speaker 1: But so anyway, looking at this graph, so you know
Speaker 1: that there were drops from T one to T two,
Speaker 1: and then again slightly from T two to T three,
Speaker 1: and then pretty much no drop from T three to
Speaker 1: T four, and then a large increase from T four
Speaker 1: to T five. So one of the things is the
Speaker 1: pattern rules out the idea that confidence is dropping merely
Speaker 1: because of the elapsing of time in the experiment. Right,
Speaker 1: It's not just people are steadily going lower. You know
Speaker 1: that there they eventually stopped lowering their own score, and
Speaker 1: then it comes back up after they read the expert's explanation.
Speaker 1: So so that basically you have to confront what you
Speaker 1: don't know in order to learn, yes, exactly. And the
Speaker 1: interesting thing is that if they're they they sort of
Speaker 1: rate themselves lower, but then they don't keep dropping, You're
Speaker 1: not in free fall. Maybe that suggests that they're adjusting
Speaker 1: more toward real accuracy in their judgment of how much
Speaker 1: they knew. There's also an interesting note that they have
Speaker 1: though this is not quantified data, but this is just
Speaker 1: sort of a subjective report from the debriefing afterwards. You know,
Speaker 1: they talked to the people who were in the experiments,
Speaker 1: and many participants subjectively said they were surprised and felt
Speaker 1: humbled by how much less they knew than they had
Speaker 1: originally assumed. But also, and this is really interesting, even
Speaker 1: with this new humility. Some of the participants showed that
Speaker 1: they were still susceptible to the illusion of explanatory depth,
Speaker 1: because here's what they said. If only I had gotten
Speaker 1: the cylinder lock instead of the flush toilet or whatever,
Speaker 1: then I would have done better overall. So if only
Speaker 1: I'd gotten these other devices instead of the ones I had,
Speaker 1: And the experimenters say, this judgment seems unlikely to be true,
Speaker 1: given that the average level of performance on the two
Speaker 1: different device sets used in the test was pretty much identical. Okay,
Speaker 1: so it to use the kung fu advantage. It's like
Speaker 1: the the young foolish hero enters into combat with the
Speaker 1: villain and is defeated in a sword fight. Uh, and
Speaker 1: then afterwards he's like like, oh my goodness, Yeah, I
Speaker 1: really didn't know how to fight with a so hard.
Speaker 1: After all. If only he had fought me in judo
Speaker 1: right then, then I truly I would have taken him
Speaker 1: out like that. But what if this guy everybody else
Speaker 1: said that about judo, and this guy has defeated everybody
Speaker 1: in Judo also, Yeah, I mean, if if he's wrong
Speaker 1: about this thing, then it could he is it true
Speaker 1: that he's right about everything else? I would doubt it.
Speaker 1: Well it is if he's very special. Maybe he's very special.
Speaker 1: But but yeah, it shows this um you can still
Speaker 1: you still have the blinders on, like you've been humbled
Speaker 1: on this one category, but you're still susceptible to the
Speaker 1: to the illusion of understanding in all other aspects of life. Yeah,
Speaker 1: if only I'd had the toilet, then I would have
Speaker 1: been golden. Okay, anyway, so established here. But this is
Speaker 1: a pretty small sample sixteen grad students, also Yale grad students.
Speaker 1: That's pretty rarefied group to draw from. So we need
Speaker 1: to do some more experiments of the same type to
Speaker 1: try to replicate the results. So they did another one.
Speaker 1: Study number two, they repeat the same experiment, same conditions,
Speaker 1: with a larger, younger sample, a group of thirty three
Speaker 1: Yale undergrads. Undergrads from the same school were picked because,
Speaker 1: in the words of the author's quote, conceivably graduate study
Speaker 1: leads to an intellectual arrogance, and the allude should of
Speaker 1: explanatory competence might be less in undergraduates who are still
Speaker 1: awed by what they do not know. There were there
Speaker 1: were some parts of the study where the writing was
Speaker 1: a little cheeky. I appreciated it. But the thing is
Speaker 1: it replicated basically got very similar results, producing the same
Speaker 1: pattern with respect to the responses over time. Uh. They
Speaker 1: initially rated their own understanding higher than after they had
Speaker 1: to explain it, not got knocked down, uh, and then
Speaker 1: down by the diagnostic question, and then up again at
Speaker 1: the end after they got to read what the expert
Speaker 1: had to say. But one thing that's interesting about the
Speaker 1: undergrads is that the effect was actually just a little
Speaker 1: bit not significantly, not statistically significantly, but a little bit
Speaker 1: stronger with undergrads than with graduate students. So the uh,
Speaker 1: the the graduate student arrogance theory, we can say is
Speaker 1: probably disproved by this. The the undergrads actually did a
Speaker 1: little worse in over over confidence about their understand I
Speaker 1: can certainly remember being a weirdly overconfident undergraduate, for sure,
Speaker 1: I think we all can. Man, wasn't that a great
Speaker 1: time when you knew everything about everything? You know? I
Speaker 1: do remember the kind of the trajectory of sort of
Speaker 1: in particular, I remember going into some religious studies classes
Speaker 1: with certain ideas about the values of certain religions over others,
Speaker 1: and how religion kind of worked, and and uh, and
Speaker 1: it was just completely foolhardy. And then I was opened
Speaker 1: up to do some some generally basic ideas and religious
Speaker 1: studies and you know the importance of world views and
Speaker 1: how the similarity between systems, the history of these different
Speaker 1: religious systems and uh and and I do remember there
Speaker 1: being like this, this resistance to it at first, giving
Speaker 1: in realizing I didn't know anything, and then a real
Speaker 1: excitement they built up from there and really there's you know,
Speaker 1: continued my entire life. And that's something I always trying
Speaker 1: to keep in mind on our show because sometimes we
Speaker 1: do encounter listeners who have uh an adverse reaction to
Speaker 1: studies that we talk about or different different takes on topics,
Speaker 1: and I always remind myself that, well that to put
Speaker 1: it in terms of our study here, that not quite
Speaker 1: free fall, but that descent that occurs doesn't feel good.
Speaker 1: It doesn't feel it can it can feel It's a
Speaker 1: fearful situation at times and humbling. It's humbling, and it's
Speaker 1: in the process of being humbled. Is is not necessarily enjoyable.
Speaker 1: It's like being beaten by the villain and a karate
Speaker 1: movie and the first the first act, but the true
Speaker 1: wise person seeks to be constantly humbled by what they
Speaker 1: don't know. I agree, I am humbled weekend, week out
Speaker 1: by the things I don't know. Oh so, how wise
Speaker 1: are you then, Robert, Well, that that's the thing. I
Speaker 1: admit that I am not the you know, the wisest
Speaker 1: guy in the room, but I am. I'm willing to
Speaker 1: admit that there's a lot there, a lot of things
Speaker 1: I don't know, and I'm continually hungry to to fill
Speaker 1: in those gaps, as we should all be, sir um. Okay,
Speaker 1: So back to the study. So we've looked at one
Speaker 1: sample and then a larger sample of undergrads. Maybe we
Speaker 1: need to look at a different university. Maybe Yale students
Speaker 1: are just generally more arrogant about their own understanding. So
Speaker 1: they figured they should try this at a different university.
Speaker 1: Sixteen students from a regional, less selective state university were
Speaker 1: given the exact same experiment UH and they judged the
Speaker 1: selectivity by comparing the students s A T scores. The
Speaker 1: students at the Southern University had an average of five
Speaker 1: and forty points less on their combined math and verbal
Speaker 1: and the result was The pattern of the pattern of
Speaker 1: results was very similar to the first two studies, a
Speaker 1: steep drop off in confidence after being asked to explain
Speaker 1: what you thought you knew, and then rising confidence and
Speaker 1: new understanding after reading the expert explanation. In fact, though
Speaker 1: the overall pattern was similar between the Yale students and
Speaker 1: the students from this other university, the students at the
Speaker 1: less selective university actually showed a slightly stronger illustion of
Speaker 1: explanatory depth effect uh, mostly due to the fact that
Speaker 1: their initial ratings of knowledge were about a point higher
Speaker 1: than those of the Yale students, and so the results
Speaker 1: results of the first two studies were basically replicated. But
Speaker 1: basically you could you could rule out various different interpretations
Speaker 1: of what this means. Yeah, I mean again, because we're
Speaker 1: all susceptible to this. I think we should trying not
Speaker 1: to put judgment, you know, moral judgments on people's I know,
Speaker 1: I said arrogant a minute ago, because I think the
Speaker 1: authors were being a little bit cheeky when they were
Speaker 1: talking about grad Yale graduate arrogant um. But yeah, it's
Speaker 1: not that you're a bad person if you overestimate how
Speaker 1: well you understand the workings of a toilet we've all
Speaker 1: been there. Yeah, especially if you have attempt to fix one.
Speaker 1: That's generally where the humbling comes. Where if something breaks
Speaker 1: in the house and I think, oh, well, maybe I
Speaker 1: can fix that. Of course I've got an Ikea toolkit,
Speaker 1: let me add it. And then it's you know, hours
Speaker 1: later you realize I'm in over my head. I need
Speaker 1: to actually call an expert. But you don't want to
Speaker 1: admit defeat, right, Okay, So next study studying number four, Well,
Speaker 1: maybe a strange selection of devices is driving the effect. Right,
Speaker 1: they're asking people about certain things cylinder lock, helicopter, toilet,
Speaker 1: all that stuff. What if it's just particularly strong for
Speaker 1: cylinder locks and toilets and helicopters. What if this effect
Speaker 1: wouldn't show up is strongly for other devices. So they
Speaker 1: did the same experiment again, got thirty two undergrads, what
Speaker 1: with many more options for devices. Uh, to explain in
Speaker 1: the experiment and to keep the experiment time under one hour,
Speaker 1: the last couple of ratings T four and T five
Speaker 1: were taken off, so participants only did the first three
Speaker 1: ratings and the results where the different devices didn't change anything,
Speaker 1: the results were the same, So it seems to be
Speaker 1: robust across all different types of machines that you would
Speaker 1: need to explain the workings of. People generally overestimate their
Speaker 1: understanding of them, and then the explanation makes them realize
Speaker 1: that they overestimate aided. So next study, well, what if
Speaker 1: the subjects are just being cautious? This is something I
Speaker 1: thought about when I was sort of running through this
Speaker 1: uh with with Rachel on the way to work today.
Speaker 1: I was like, how well would you think you understand
Speaker 1: how you know A can open our works or something?
Speaker 1: And we discovered that we would tend to just rate
Speaker 1: ourselves very low, maybe because we've been primed with the
Speaker 1: fact that there isn't an illusion of explanatory depth. So
Speaker 1: I'm just gonna I'm gonna start with a two to
Speaker 1: be safe. Yeah, because if you're asking me, hey, do
Speaker 1: you know how do you know how it can opener works?
Speaker 1: I would think you're trying to trick me, right, And
Speaker 1: so maybe maybe the experiment is doing the same thing
Speaker 1: in that once the experience. As the experiment goes on,
Speaker 1: people are just becoming more cautious. They're being put on
Speaker 1: guard and regardless of the actual accuracy of their original
Speaker 1: explanatory depths. Does that make sense, Like they're not adjusting
Speaker 1: toward how well they actually understand things, they're maybe they're
Speaker 1: just adjusting towards call and just lower in general and
Speaker 1: so um. Some students were recruited to subjectively, so they
Speaker 1: basically they did the same study again, the same test,
Speaker 1: you know, hadding people make the assessments. But then they
Speaker 1: also got some people to subjectively rate the explanations written
Speaker 1: by the original people in the study. And some really
Speaker 1: complicated statistical analysis was required on this one, but the
Speaker 1: basic result was that, according to the independent raiders who
Speaker 1: read people's explanations and rated them on the scale, the
Speaker 1: participants initially overestimated their level of understanding, and then their
Speaker 1: confidence ratings became more accurate when they dropped after being
Speaker 1: asked to give the explanation on the and on the
Speaker 1: calibration question. So this seems to rule out the idea
Speaker 1: that people are just becoming more timid or more modest
Speaker 1: or cautious as the ratings go on through the test.
Speaker 1: According to some independent judges who come in and said,
Speaker 1: oh wow, this explanation of a can opener is a
Speaker 1: two um. According to these people, the participants are actually
Speaker 1: becoming more accurate as the test goes on, that they're
Speaker 1: getting closer to how good their understanding was for real.
Speaker 1: Here's another thing related to the priming I was just
Speaker 1: talking about studying number six. Can you destroy the effect
Speaker 1: just by warning people that they're going to have to
Speaker 1: give an explanation for how some of the items work.
Speaker 1: So think about it this way, Robert. You know, I
Speaker 1: ask you, um, on a scale of one to seven,
Speaker 1: how well do you understand how a toilet works? And
Speaker 1: be prepared to explain your answer. Yeah. Another example of
Speaker 1: this would be when someone asks you, hey, have you
Speaker 1: ever seen such and such movie? Your answer might be
Speaker 1: different if you know that the follow up is tell
Speaker 1: me the breakdown of the plot. Yeah, because because that
Speaker 1: I've had that happened before and says, hey, you know
Speaker 1: such and such movie? And sometimes you interpret that as
Speaker 1: do I know of that movie? Did I see the
Speaker 1: trailer for it once? Uh? Did I watch it twenty
Speaker 1: years ago? Yes? Yes? Maybe, But then if you actually
Speaker 1: have to prove that you know this film, that's sometimes
Speaker 1: a different can of worms, right, Yeah, So it could
Speaker 1: put you on guard. And so the question is if
Speaker 1: the illusion of explanatory depth effect is real. What we
Speaker 1: would expect is that maybe maybe warning people this way
Speaker 1: might reduce the effect, elimit a little bit, but it
Speaker 1: shouldn't eliminate it. It shouldn't make it completely go away. Right. Um?
Speaker 1: So with thirty one undergrads again, uh, they did the
Speaker 1: exact same test, except they added a paragraph warning the
Speaker 1: subjects that they were going to have to give a
Speaker 1: written explanation and answer a diagnostic question. So what happened here? Well,
Speaker 1: the results on this one were pretty odd. The same
Speaker 1: pattern presented itself in that the first ratings they gave
Speaker 1: were higher, and then they dropped after being asked to
Speaker 1: write an explanation, and then again after the diagnostic question,
Speaker 1: but the magnitude of the effect was reduced, so the
Speaker 1: drop off was much less. Um, there's still a difference
Speaker 1: between the initial and the later ratings. But the odd
Speaker 1: part is it wasn't because the subjects who were warned
Speaker 1: initially rated their understanding any lower. That's what you would expect, right,
Speaker 1: you'd expect that if you've been warned, your first rating
Speaker 1: would be more cautious, Right Yeah, I mean, if someone
Speaker 1: warns you, whatever you say, someone's going to call you
Speaker 1: on it. So don't. Don't b S is because you
Speaker 1: you will be you'll be corrected, you'll have to have
Speaker 1: to prove your answer. Yeah, but that's not what happened here. Instead,
Speaker 1: they were no less confident in their initial understanding. It
Speaker 1: was because their later self ratings were higher. And that's
Speaker 1: kind of weird, right, So this seems to reveal that
Speaker 1: it's it's not just a it's not it's certainly not
Speaker 1: a conscious matter of I really thinking, Oh, I really
Speaker 1: don't understand how toilets work, but I don't want anybody
Speaker 1: to know, so I'll just tell them I understand. Could
Speaker 1: I mean it? Could? I mean? Maybe that's what's going
Speaker 1: on the author's right quote. One path stability is that
Speaker 1: the new instruction changed the way participants used the rating scales.
Speaker 1: For example, hearing the explicit instructions may have caused participants
Speaker 1: to try to be more consistent with their subsequent ratings
Speaker 1: because they had less justification for being surprised at their
Speaker 1: poor performance. Basically like being it's like you were warned.
Speaker 1: What excuse did you have for overcome for being overconfident
Speaker 1: in how much you knew UM and the fact that
Speaker 1: you were warned? Maybe I don't know, it makes you
Speaker 1: more embarrassed that you were overconfident, and thus you're less
Speaker 1: likely to admit how overconfident you were initially. I don't
Speaker 1: know that that's a that's an odd result here, so
Speaker 1: that's worth keeping in mind. But at this point the
Speaker 1: study considers the initial effect basically satisfactorily satisfactorily replicated for
Speaker 1: how we understand the mechanical workings of devices, and then
Speaker 1: it's gonna move on to other things, other types of
Speaker 1: knowledge and what the researchers called different domains of knowledge.
Speaker 1: Does the same illusion of knowledge hold true for things
Speaker 1: other than next like explanations of causally complex phenomenon like
Speaker 1: how a machine works, how a device works. Does it
Speaker 1: exist for facts? Does it exist for narratives for procedures?
Speaker 1: Can it be lumped in with general over confidence effects?
Speaker 1: Or is the illusion of explanatory depth its own thing?
Speaker 1: So maybe we should take a quick break and then
Speaker 1: when we come back we will get into the rest
Speaker 1: of this study. All right, we're back, So study number seven.
Speaker 1: One of the things is what if people are just
Speaker 1: generally overconfident about what they know, regardless of the type
Speaker 1: of knowledge. What if it's not just explaining things. What
Speaker 1: if everybody's overconfident about all their knowledge. Yeah, I could
Speaker 1: see it being sort of like the scenario in which
Speaker 1: the brain just sort of convinces you that you have
Speaker 1: an answer to a question just so you don't have
Speaker 1: to worry about Because the brain is ultimately an economic system,
Speaker 1: it can't it doesn't need a waste reas sources. So
Speaker 1: it's it's I've read, for instance, the individuals who have
Speaker 1: been quizzed on where they were and what they were
Speaker 1: doing during the September eleventh attacks. People have very specific answers,
Speaker 1: so saying I was wearing a blue shirt, was eating
Speaker 1: hunting nut cheerios, but that in many cases what seems
Speaker 1: to be happening is you're in this state of fight,
Speaker 1: fight or flight. Really uh, there's you're not sure how
Speaker 1: you're gonna survive on some level, and your brain just
Speaker 1: goes ahead and makes up an answer for you. Because
Speaker 1: if to say they don't worry about it was a
Speaker 1: blue shirt, why don't you worry about your shirt that
Speaker 1: there's this awful catastrophic event taking place. Don't worry about
Speaker 1: the cereal bam, I'll just check something off. Don't don't
Speaker 1: even don't even fret. Yeah, I've I've heard about this too,
Speaker 1: like memory confabulation in these like momentary memories, you know,
Speaker 1: the flash bold memories from some big event in your past. Yeah.
Speaker 1: So it's like, if I go into the bathroom and
Speaker 1: on some level one thing, do I know how a
Speaker 1: toilet works? My brain is kind of saying, yeah, you
Speaker 1: know how toilet works, use the restroom, and then you
Speaker 1: flush it obviously. Yeah, don't worry. You've got other things
Speaker 1: to do. Stop worrying about the toilet. Okay, So let's
Speaker 1: test some basic geography here. So specifically, what they did
Speaker 1: was naming the capitals of countries around the world. Experiment
Speaker 1: Ers that came up with a list of forty eight countries,
Speaker 1: and they split it roughly in thirds between countries where
Speaker 1: it's easy to know the capital, or where at least
Speaker 1: where you would expect American students to easily guess the capital.
Speaker 1: How about England. We hit that one already, you know,
Speaker 1: man genius here. Uh, then the ones where they were
Speaker 1: moderate likely moderately likely to know the capital, and then
Speaker 1: the ones where they were very unlikely to know. So
Speaker 1: split into thirds. Uh, Robert, what's the capital of Brazil. Oh,
Speaker 1: this one, it's like Brazilia, but my Portuguese is not good.
Speaker 1: You are correct. I was hoping i'd trick you into
Speaker 1: saying Rio de jann Arrow. Well, this is You would
Speaker 1: have caught me with various states for sure, because yeah,
Speaker 1: you think of the what's the most thing this city
Speaker 1: from that country or US state, and then you assume
Speaker 1: that's the capital. If I recall correctly, the Brazilia Rio
Speaker 1: de Janeiro confusion is actually a major plot point in
Speaker 1: one of the I Know what you did last summer sequels,
Speaker 1: which I am here publicly admitting that I've seen. But
Speaker 1: then also, here's a hard one. What's the capital of Tajikstan. Yeah,
Speaker 1: that one. That one is one that I I probably
Speaker 1: should have a leg up on that one because I
Speaker 1: took because I remember taking a course in college about
Speaker 1: former Soviet states in that region. Yeah, I'm drawn up
Speaker 1: complete blank on that one. I think it used to
Speaker 1: be called Stalinabad, but now it's a ducham Bay Okay.
Speaker 1: But yeah, anyway, so participants fifty two college undergraduates, um
Speaker 1: and they were first shown a list of all the countries.
Speaker 1: So here all the countries you're gonna have to know
Speaker 1: the capitals of go down and rate them on the
Speaker 1: seventh same seven point scale. Rate your confidence in how
Speaker 1: well do you know the capital of all these countries? Reason?
Speaker 1: And then they're asked to actually list the names of
Speaker 1: capitals for half the countries and then asked to re
Speaker 1: rate their knowledge. So essentially it's the same thing, except
Speaker 1: instead of giving an explanation of how something works, you're
Speaker 1: just listing the capital. Then they're told the real names
Speaker 1: of the capitals and asked to re rate their knowledge.
Speaker 1: So what are the results here, Well compared to a
Speaker 1: combined group from studies one through four, and the authors
Speaker 1: justify combining them into one group in their discussion. Uh,
Speaker 1: the students who were tested on the facts showed a
Speaker 1: different pattern in the same direction, but with less magnitude.
Speaker 1: So confidence dropped off significantly between the first and second rating,
Speaker 1: So after people had to answer the questions, they were
Speaker 1: less confident, But UH, it stayed almost the same for
Speaker 1: the final rating. And though the drop off from T
Speaker 1: one to T two is statistically significant, UH, it's significantly
Speaker 1: smaller than the drop off in explanation. So essentially, with facts,
Speaker 1: we're seeing a pattern that's going in the same direction,
Speaker 1: but it's just much smaller. The line graph shows a
Speaker 1: slight decrease, but it's closer to being flat than the
Speaker 1: graph line for the device explanations. So we've got some
Speaker 1: overconfidence with capitals, and I think we've all got to
Speaker 1: be that way given our schooling. Right, How how many
Speaker 1: capitals did you have to learn in school? Why why
Speaker 1: do they do capitals? You know? I remember I remember
Speaker 1: exercises where we had to of course remember the states
Speaker 1: and their capitals, but I also remember just a lot
Speaker 1: of geography quizzes that had no substance to them, like
Speaker 1: you'd have to you'd have to memorize all the nations
Speaker 1: of Africa, and yes, some of the some of the
Speaker 1: nations you were learning about, you know, every everyone's learning
Speaker 1: about Egypt, everyone's learning something about South Africa. But then
Speaker 1: there are all these other African states. You're not even
Speaker 1: asked to know anything about them, just except for their name,
Speaker 1: and so they're useless facts because there's no substance behind it. Yeah,
Speaker 1: I feel like it would be much more useful if
Speaker 1: if you're doing that instead of learning capitals, to learn
Speaker 1: like primary language is main ethnic groups and main exports,
Speaker 1: but I guess there's only so much time in a day.
Speaker 1: Then again, I guess I won't complain when I I
Speaker 1: don't know. It's good being able to produce a capital.
Speaker 1: You still you still get that third grade rush. Oh yeah,
Speaker 1: I did it, Brazilia. Okay, So next next test study
Speaker 1: number eight. Uh, let's look let's look at a different
Speaker 1: domain of knowledge. So we've looked at we've looked at
Speaker 1: explanations for causal phenomenon with devices, and we've looked at facts.
Speaker 1: What about procedures? So this, uh, this type of knowledge
Speaker 1: in a way is very similar to explaining how devices work.
Speaker 1: It involves explaining how you do something. How do you
Speaker 1: tie a tie? How do you bake chocolate chip cookies
Speaker 1: from scratch? How do you drive from New York to
Speaker 1: New Haven? This makes me think of all the wonderful
Speaker 1: wiki how um explanations out there, often with pictures that
Speaker 1: explain and yet don't explain the thing you looked that.
Speaker 1: Those things are my favorite looking up obscure e how
Speaker 1: articles used to be one of my favorite games on
Speaker 1: the Internet, the best one I ever came across. I
Speaker 1: swear this is true. It was an eHow article because
Speaker 1: I think it doesn't exist anymore. But it was called
Speaker 1: how to pray for Money? Oh really, yeah, for money?
Speaker 1: So it's so it would be like it had instruction
Speaker 1: meal pray asked for money? Well, I think it was.
Speaker 1: Actually it was actually kind of complex because it was
Speaker 1: like recognized that money might not be the most important thing.
Speaker 1: Um that was like step number five. I guess, okay, well,
Speaker 1: these are the kind of things that I guess occur
Speaker 1: when when writing assignments are going out, Um, you know
Speaker 1: lickety split based purely on you know, search engine terms,
Speaker 1: right right, Okay, So this part, so they're going to
Speaker 1: run the same test they've done before, all the same
Speaker 1: rating steps, everything's the same, except instead of explanations, it's
Speaker 1: going to be how do you do this? Here's a
Speaker 1: procedure right down the steps and and what order they
Speaker 1: come in and how they work together. Results are very interesting. Uh,
Speaker 1: this pattern was completely unlike anything we've seen before. So
Speaker 1: instead of the ratings dropping off between T one and
Speaker 1: T two, your first guests and then your adjustment after
Speaker 1: you have to write something, write the answer out, the
Speaker 1: ratings actually showed a slight but statistically non significant increase
Speaker 1: from one to two and so after you have to
Speaker 1: give an account of how to bake chocolate chip cookies,
Speaker 1: you're actually more confident in your knowledge than you were
Speaker 1: before you wrote down the steps. Um, and I thought
Speaker 1: that was interesting, but it also sort of matches. I
Speaker 1: can see how that would be true. Like you think
Speaker 1: you probably know how to do something, then you write
Speaker 1: down the steps to do it, and looking at them
Speaker 1: there you're like, oh, yeah, I was right, I knew,
Speaker 1: And so you're a little more confident. Yeah, especially if
Speaker 1: it's something you've chocolate chip cookies as an example, Like
Speaker 1: so often you're going off a recipe, or at least
Speaker 1: I I'm not being a real baker. So I'm gonna
Speaker 1: look up the recipe and then I'm gonna follow the
Speaker 1: recipe with no intent of memorizing it. And then afterwards
Speaker 1: I may be able to recall those steps and list
Speaker 1: them out and say, all right, that looks accurate. But
Speaker 1: then if I actually take that list and compared to
Speaker 1: the recipe, I'm bound to have left out some like
Speaker 1: key steps well like baking them, like something I don't know,
Speaker 1: like licking the raw egg laden spoon. All on that
Speaker 1: part I got down. Oh yeah, So so the authors
Speaker 1: also report again this is some non quantified information, but
Speaker 1: just some post test debriefing. They report that the students
Speaker 1: didn't show any of the now characteristic surprise and all
Speaker 1: the other stuff, you know. After the test they'd be like, Wow,
Speaker 1: I can't believe how much I didn't know. Instead, they
Speaker 1: seemed perfectly aware of how much or how little they
Speaker 1: knew about how to do things. On the other hand,
Speaker 1: I guess, yeah, I would say, this isn't really all
Speaker 1: that surprising. Our mental process of remembering how to do
Speaker 1: something is very different from our mental process of remembering
Speaker 1: how something x sternal to the self works because when
Speaker 1: you're remembering how to do something, it's usually a first
Speaker 1: person memory. You picture yourself being the thing doing the thing. Yeah,
Speaker 1: often it's like an unlanguaged experience. I've had this this
Speaker 1: experience with Legos recently because because I'm building Legos with
Speaker 1: my son. Haven't built built anything out of Legos since
Speaker 1: I was a kid, and I'm realizing that I'm I'm
Speaker 1: sure there are industry terms for all the different blocks
Speaker 1: and the sizes of blocks, but I do not know
Speaker 1: what those terms are, so I'm and the the of course,
Speaker 1: the instructions are wordless, so I don't have any length
Speaker 1: or I have very little language to describe the steps
Speaker 1: that are taking place, but I can, you know, I
Speaker 1: can picture myself doing it. Yeah, And there there are
Speaker 1: also plenty of things where there is like something you
Speaker 1: know how to do through muscle memory that would be
Speaker 1: difficult to put into words, like could you explain the
Speaker 1: steps of how to ride a bicycle? Right? Or a
Speaker 1: big one is is tying a long neck? Hi? Yeah,
Speaker 1: but like I can I can tie one on myself,
Speaker 1: I cannot tie one on another person all the time
Speaker 1: with people where if they're going to tie a tie
Speaker 1: for someone, they have to wear it themselves. Uh. Well,
Speaker 1: at least that is interesting and I think that's true
Speaker 1: in my experience. But it does not seem so much
Speaker 1: born out in these results. It seems like people are
Speaker 1: or maybe actually it's not that people were perfect at
Speaker 1: being able to explain how to do procedures. They were
Speaker 1: just very accurate in predicting how well they would be
Speaker 1: able to explain them, because you are you. It's based
Speaker 1: on an actual memory of doing the thing or trying
Speaker 1: to do the thing. Yeah. Um so yeah, So next
Speaker 1: next study, what about a different type of knowledge. We've
Speaker 1: looked at facts, We've looked at procedures. How about narratives.
Speaker 1: One of my favorite things to recall is what happened
Speaker 1: in that part of Big Trouble and Little China after
Speaker 1: the monster for first Pokes's head. I don't anyway, So yeah,
Speaker 1: recalling a narrative if if the plot of a narrative
Speaker 1: is basically realist in terms of genre, you're not talking
Speaker 1: about el topo or something. There is a causal logic
Speaker 1: to the events that take place in it, right, in
Speaker 1: the sense that a narrative, like the plot of a
Speaker 1: book or a movie, is a kind of machine. It's
Speaker 1: a structure built out of causal relationships that can be labeled, explained,
Speaker 1: and summarized. So so let's let's look at the machine
Speaker 1: of movie plots. Thirty nine students were given a list
Speaker 1: of twenty popular movies UH, selected to be things college
Speaker 1: students were likely to have seen. I think Forest Gump
Speaker 1: was one of them. UH. And they're asked which of
Speaker 1: the movies they had seen, and then asked to rate
Speaker 1: their understanding of the plots of five of the movies
Speaker 1: they'd seen, and then, after their t one first ratings,
Speaker 1: they had to describe each of those five plots and
Speaker 1: then rewrite their original understanding, So the same procedure we've
Speaker 1: seen the whole time, except instead of explaining devices or procedures,
Speaker 1: it's plots of movies. And then finally they read reviews
Speaker 1: from a professional movie website, not review summaries of plots
Speaker 1: from a professional movie review website, and then compared those
Speaker 1: to what they had and rated again, and the results
Speaker 1: were that the pattern was closer to the one for
Speaker 1: procedures than the one for devices. I thought this was
Speaker 1: interesting because in a narrative, you were recalling a narrative
Speaker 1: that's not something you had to do with your body.
Speaker 1: It's so that it's taking that element out, but it's
Speaker 1: closer to the pattern for procedures. There's no significant drop
Speaker 1: off from T one to T two. People were pretty
Speaker 1: accurate at predicting how well they knew narratives. That's interesting
Speaker 1: because just thinking back on movies I've seen, like you mentioned,
Speaker 1: Big Trouble a Little China, and I instantly started trying
Speaker 1: to in my mind sort of piece out of timeline
Speaker 1: of that movie. And it's a movie I've seen a lot,
Speaker 1: and and and I have a lot of Love for
Speaker 1: But there I think there's some definite holes in my
Speaker 1: attempt to restructure. You know, at what point they go
Speaker 1: to the to the import export business there, and then
Speaker 1: they come out, and then they go back in And
Speaker 1: when did this encounter fall in line? You know, they
Speaker 1: ran a different study to test different devices just to
Speaker 1: make sure the device list they had wasn't peculiar. I
Speaker 1: wonder if they should have run another test with different movies,
Speaker 1: Like what if the movies they had were unusually perspicuous
Speaker 1: and clear in terms of plot relationship? Yeah, like say
Speaker 1: a romantic comedy, say like the movie Amala. Despite the
Speaker 1: fact that I've seen Big Trouble A Little China far
Speaker 1: more than I've seen Amlay, I'm far more confident in
Speaker 1: my ability to to just rattle off the plot points
Speaker 1: and the basic movement of the narrative for Amlay because
Speaker 1: it one thing follows from another, Right, it follows a
Speaker 1: basic Uh, there's a basic blueprint for that sort of film.
Speaker 1: And I'm not saying Big Trump A Little China doesn't
Speaker 1: follow a very basic blueprint as well. Now, I think
Speaker 1: some kind of random things happened in it, you wouldn't
Speaker 1: necessarily infer from one scene, what's going to happen in
Speaker 1: the Uh? Yeah? Anyway, So next study, let's look at
Speaker 1: one more different type of knowledge. So we've looked at
Speaker 1: explanations for machines, facts about geography, procedures on how to
Speaker 1: do things, and narratives from movie plots. How about explaining
Speaker 1: natural phenomenon. Natural phenomena are complex causal systems. In a way,
Speaker 1: they're very much like devices or like machines, except they're
Speaker 1: you know, they're not made by humans. But in other senses,
Speaker 1: they are like that. They have causal relationships, different components
Speaker 1: that work together, and they in the end they make
Speaker 1: something happen. Uh. So participants were thirty one Yale undergrads
Speaker 1: and study was identical to the ones before, except instead
Speaker 1: of explaining how device works, you explain how tides occur,
Speaker 1: how why comets have tales, how earthquakes occur, how rainbows
Speaker 1: are formed, things like that. So, just like in the
Speaker 1: first four studies, they gave the initial confidence rating, you know, rainbows, Oh,
Speaker 1: I'm six on rainbows. Then they had to explain how
Speaker 1: they're formed, YadA YadA, and the results were it's a jackpot.
Speaker 1: The results distribution from the explanations of natural phenomen we're
Speaker 1: very similar to those four devices. They were closest to devices. So,
Speaker 1: whether it's tides or whether it's toilets, we think we
Speaker 1: understand how things work, but when we try to explain it,
Speaker 1: we realize there are lots of gaps in our understanding.
Speaker 1: So to summarize the results across all these studies we've
Speaker 1: we've seen that people are significantly overconfident in their understanding
Speaker 1: of how devices work and how natural phenomena occur, um
Speaker 1: that asking for an explanation makes this overconfidence apparent and
Speaker 1: reduces it. People are somewhat overconfident, but less so about
Speaker 1: their knowledge of facts like capital cities, and people are
Speaker 1: fairly accurate at judging their own knowledge of procedures, how
Speaker 1: to do stuff, and narratives what happened in a movie? Now,
Speaker 1: the big question is this is the thing we haven't
Speaker 1: gotten to yet. Why? Why? So? What's causing these differences
Speaker 1: in metacognition across different domains of knowledge? Why are we
Speaker 1: more overconfident about some types of knowledge than other is?
Speaker 1: What is it that would make us more confident about
Speaker 1: knowing how a toilet works than about knowing the plot
Speaker 1: of Forest Gump. Well, my my initial answer would be
Speaker 1: that we assume a certain simplicity of its design, Like
Speaker 1: without even really reminiscing too much on Forest Gump, I'm
Speaker 1: given the fact that it was a big blockbuster summer movie,
Speaker 1: I'm assuming it has a pretty simplistic structure. And as
Speaker 1: for the toilet, I mean it has such a it
Speaker 1: has such a a low standing in the household when
Speaker 1: it's functioning that you you just assume it couldn't be
Speaker 1: that complicated. Why would why would it take high technology
Speaker 1: to simply dispose of human waste? Yeah, I guess I
Speaker 1: can see that. I mean, so one of the answers
Speaker 1: that was given in the piloting they did for this,
Speaker 1: you know, when they were trying to think, what, what
Speaker 1: are some good hypotheses that could explain this this difference
Speaker 1: is complexity of the device, and so the hypothese sees
Speaker 1: that they ended up wanting to test where how about confusion?
Speaker 1: They called it confusion of environmental support with internal representation,
Speaker 1: And what that means is confusion of the fact that
Speaker 1: you can see the parts and how they work with
Speaker 1: the idea that you can represent the parts and how
Speaker 1: they work in your mind. Um. And so what this
Speaker 1: predicts is devices like bicycles and can openers and stuff
Speaker 1: that are very clear and perspicuous are the things that
Speaker 1: were most likely to overestimate our knowledge of the workings
Speaker 1: of because when we can just look at them and
Speaker 1: see all the parts, there's no there's no sensation that
Speaker 1: the workings are being hidden from us. But in your mind,
Speaker 1: try to draw a picture of a bicycle right now,
Speaker 1: I think actually, even if you've seen lots of bicycles,
Speaker 1: chances are you just mentally illustrated a bicycle that could
Speaker 1: not work. I don't know, I feel I feel a
Speaker 1: sense of perhaps false confidence here. Well maybe I mean,
Speaker 1: maybe I'm just because I assembled one on Christmas. Well,
Speaker 1: if you've if you've actually assembled a bicycle, then you
Speaker 1: might be in a different category. But I'm willing to
Speaker 1: accept that I'm completely foolhardy on this. The bicycle might
Speaker 1: be if you've actually worked on them with your hands,
Speaker 1: it might be more in the procedures category. But with
Speaker 1: an ikea tool kits so place. Well, I mean a
Speaker 1: lot of people we would try to draw try to
Speaker 1: draw a bicycle, and then they'd have like a you know,
Speaker 1: like a single bar running from the spokes of both wheels,
Speaker 1: and that would make it impossible to steer the bicycle
Speaker 1: and stuff like that, or they'd have, um, you know,
Speaker 1: the chain running to the front wheel and the back
Speaker 1: wheel or something like that. Well, it also makes me
Speaker 1: think that one of the scenarios here is that you
Speaker 1: feel that one should be able to understand a bicycle. Yes,
Speaker 1: I'm reminded that's the ease of representation. Yeah, Like I'm
Speaker 1: reminded of a bid ends in in the Art of
Speaker 1: Motorcycle Maintenance where uh, it's one of the parts where
Speaker 1: he's talking about motorcyle Coleman. It's that he says that
Speaker 1: the motorcycle is a perfect vehicle, a perfect perfect just
Speaker 1: a perfect system to have control of because it is
Speaker 1: a it is a complex system, but it's not so
Speaker 1: complex that a single individual can't master it and care
Speaker 1: for it. And whereas if you get into progressively more
Speaker 1: complicated mechanical systems or just systems in general, then it
Speaker 1: becomes increasingly difficult for one person to be able to
Speaker 1: have a grasp of it. Can you be the master
Speaker 1: of your prius? I don't know. I mean, I'm sure
Speaker 1: there are people that can. I I could not be
Speaker 1: the master of a motorcycle. I'm and I'm willing to
Speaker 1: to admit that I probably can't even be the master
Speaker 1: of my son's bicycle at this point. Well, I bet
Speaker 1: you're more master than me, now that you've actually used
Speaker 1: your hands on it. I I know now. I was primed,
Speaker 1: actually had to think about what's in a bicycle and
Speaker 1: look it up. But I'm quite confident that if I
Speaker 1: had just been asked to draw a bicycle and the
Speaker 1: different parts and what they do, I would not have
Speaker 1: been able to do it correctually if I hadn't thought
Speaker 1: about it ahead of time. Okay, another hypothesis. What about
Speaker 1: confusing higher and lower levels of an analysis. Basically, this
Speaker 1: just means, uh, if you've got an idea of the
Speaker 1: causal relationships at a high level, you know the big
Speaker 1: parts of a machine and basically what the machine does,
Speaker 1: you assume you have an understanding for the things at
Speaker 1: the lower level, even if you don't. So you think
Speaker 1: about car breakes. Car brakes slow the spinning of the
Speaker 1: wheels by applying friction. I understand how car brakes work,
Speaker 1: but there there are tons of things involved in the
Speaker 1: brakes that you've got some kind of hydraulics probably, or
Speaker 1: you know, fluid or some kind of How how is
Speaker 1: the pressure applied from the brake pedal to the brakes?
Speaker 1: What are all the different little gears and connections and parts.
Speaker 1: There's tons of stuff there that you're not even thinking about.
Speaker 1: But at the high level you basically know what it does,
Speaker 1: and so that makes you assume that you know how
Speaker 1: it works. It's a confusion of the what with the how. Yeah,
Speaker 1: Like examples that come to mind, like a chainsaw. You
Speaker 1: know how the cutting occurs, but do you really know
Speaker 1: how the all the intricacies of the saw itself hydraulic
Speaker 1: press like the one of the end of terminator. Oh yeah,
Speaker 1: you know, but it's pretty simple concept. The two pieces
Speaker 1: come together and flatten the terminator. But there's a lot
Speaker 1: more involved there with the hydraulic system and everything else.
Speaker 1: I like, I don't I don't even have a firm
Speaker 1: enough understanding of that of how how the intense pressure
Speaker 1: is applied via hydraulics. Yeah, yeah, um that that that's
Speaker 1: a good one. How about another explanation. What if it's
Speaker 1: the problem that explanations of of how things work, explanations
Speaker 1: unlike facts and stuff like that have indeterminate end states,
Speaker 1: and that if I ask you the capital of a country,
Speaker 1: how confident are you that you know the capital of
Speaker 1: a country? Whether or not you're right about the answer,
Speaker 1: you know what the answer will look like. It will
Speaker 1: be you know a short word, and you you think
Speaker 1: you probably know what that word is, um with an explanation.
Speaker 1: It's just it's very open ended. You know, you don't
Speaker 1: exactly know what the answer should look like, exactly how
Speaker 1: detailed is it supposed to be? UM? What you know?
Speaker 1: What are all the things that would be in it?
Speaker 1: It's it's it's more amorphous in terms of structure, even
Speaker 1: if you haven't colored inside the lines yet. And then
Speaker 1: the final hypothesis is what about rarity of production? We just, Robert,
Speaker 1: here's one where you and I might be different than
Speaker 1: a lot of people. Not to say we're better, We're
Speaker 1: probably worse. But most people don't have to give explanations
Speaker 1: of how things work very often. But we do often
Speaker 1: have to give recountings of facts, narratives and uh and
Speaker 1: about procedures. Right, these are things that are common for
Speaker 1: everybody to explain, but it's not all that common for
Speaker 1: people to explain how things work, and this may make
Speaker 1: us overestimate our performance at it. Yeah, I think that's reasonable.
Speaker 1: I mean we we are in kind of a privileged
Speaker 1: situation where we are constantly having to confront the things
Speaker 1: we don't know and and research them and form form
Speaker 1: and understanding of ourselves and then share that understanding with
Speaker 1: listeners or readers or viewers what have you. Yeah, we
Speaker 1: we we were practiced enough to know how little we know. Hopefully, No,
Speaker 1: we probably don't know how little we know. We foolishly
Speaker 1: think we know how little we know. But yeah, we
Speaker 1: have an illusion of depth of understanding our own ignorance. Uh,
Speaker 1: hopefully we have a lag up on the on the situation.
Speaker 1: That's the main hope. Maybe. Well we try, we try,
Speaker 1: probably fail, but we try, okay, well to examine how
Speaker 1: these figure. And there are a couple more studies there
Speaker 1: two more in this uh, in this research. So one
Speaker 1: of them, study number eleven is what if the difference
Speaker 1: in the different knowledge types is just that some knowledge
Speaker 1: types are more socially desirable than others. I thought about that.
Speaker 1: That's kind of interesting. What if we're more likely to
Speaker 1: overestimate our knowledge in say, uh devices, because it's much
Speaker 1: cooler to know how a toilet works, uh, and thus
Speaker 1: much more socially desirable, And thus we're sort of internally
Speaker 1: bluffing on the most socially important categories that could be possible.
Speaker 1: So twenty four Yale undergrads participated in this. They rate
Speaker 1: it on a seven point scale how embarrassed they would
Speaker 1: be if they have to admit if they had to
Speaker 1: admit they were ignorant about certain things. And the things
Speaker 1: on the list were pulled from a combined master list
Speaker 1: of the contents of previous experiments. So you're asked, like
Speaker 1: four each item, please rate how embarrassed do you think
Speaker 1: you would be if someone asked you to explain the
Speaker 1: item and it turned out that you did not have
Speaker 1: a good understanding or knowledge of that item. So apply
Speaker 1: what I just said to a flesh toilet, the plot
Speaker 1: of Forrest Gump, how to tie a bow tie, the
Speaker 1: capital of England, how rainbows are formed, And the results
Speaker 1: are that people were the least embarrassed to be ignorant
Speaker 1: about how devices worked. They were moderately embarrassed to be
Speaker 1: ignorant about facts, procedures, and natural phenomena, And then this
Speaker 1: was crazy. They were the most embarrassed to be ignorant
Speaker 1: about narratives. Interesting because it seems like you would have
Speaker 1: that that you have the most plausible deniability there I
Speaker 1: haven't seen it in a while, or I haven't seen it,
Speaker 1: or I didn't like it all that. I guess for
Speaker 1: college students, having seen certain movies carries a lot of
Speaker 1: social cache and don't know it's it's you know anyway.
Speaker 1: So this response pattern does not show a correlation between
Speaker 1: overconfidence and a knowledge domain and the social desirability of
Speaker 1: the knowledge domain. People are not bluffing themselves on the
Speaker 1: important stuff, or they would be convinced they know way
Speaker 1: more about what happens in movies than they actually do
Speaker 1: last study. In this research, so what exactly is correlated
Speaker 1: with overconfidence? Having established that people are the most overconfident
Speaker 1: about their understandings of devices and natural phenomenon, ruling out
Speaker 1: the idea that this is because those domains of knowledge
Speaker 1: are socially accepted or desirable. Uh, the experimenters, they were
Speaker 1: trying to measure what are the other factors that are
Speaker 1: correlated with overconfidence and understanding? So they returned to our
Speaker 1: old friends, the devices, the cylinder lock, the flush toilet
Speaker 1: the Grand list from studies one through four. Now, this
Speaker 1: tested a lot of different correlates, like familiarity with the item,
Speaker 1: the ratio of visible versus hidden parts, the number of
Speaker 1: mechanical versus electrical parts, the total number of parts, and
Speaker 1: the number of parts for which one knows the names. Uh.
Speaker 1: There was a lot of complicated analysis on this one
Speaker 1: as well, but in the end the researchers ruled that
Speaker 1: the visible or two hidden parts ratio explained the most
Speaker 1: of the variation in over confidence. In other words, a
Speaker 1: device with visible trans parent mechanisms, in their words, seems
Speaker 1: to be the most likely to trick you into thinking
Speaker 1: you understand how it works, when in fact you would
Speaker 1: discover yourself unable to explain it. So like we were
Speaker 1: talking about the can open or the bicycle, things that
Speaker 1: seem very clear and easy to to look at and
Speaker 1: think you understand are the most likely to make people
Speaker 1: over confident in their understanding um. They also believe that
Speaker 1: the results indicate that the quote levels of analysis confusion
Speaker 1: and the label mechanism confusion may contribute to feelings of knowing,
Speaker 1: so that means confusing the higher level with the lower
Speaker 1: level you know knowing confusing what it does with how
Speaker 1: it does it at the granular level, and then also
Speaker 1: knowing the names for parts of a thing might make
Speaker 1: you overconfident in thinking that you know how the thing works.
Speaker 1: A little knowledge is a dangerous thing, it certainly is.
Speaker 1: And I can definitely think of this in like biology.
Speaker 1: Remember in high school when you learn the names of
Speaker 1: all the parts of the cell. Maybe not high school,
Speaker 1: I don't know what, but some s class you have
Speaker 1: in school. If you learn all the parts of the
Speaker 1: cell in human maybe there's an art project involved. And
Speaker 1: then you think you know how the cell works. I
Speaker 1: don't know. You don't know how the cell works? Are
Speaker 1: you a fool? Yeah? I mean the same can be
Speaker 1: said of the human body, right, I mean you you
Speaker 1: you learn all these different anatomical parts, the different organs.
Speaker 1: But to say you know what a liver is is
Speaker 1: the different thing than saying you know how the liver
Speaker 1: works exactly right. So they say in their final discussion, Uh,
Speaker 1: they're they're thinking about, you know, the explanations for what
Speaker 1: causes the illusion of explanatory depth, and and they're sort
Speaker 1: of focusing a lot on this idea of the the
Speaker 1: environmental support being able to look at an object and
Speaker 1: see the parts and confusing that for an understanding. And
Speaker 1: I thought this was a good, good passage, They say, quote,
Speaker 1: it would be easy to assume that you can derive
Speaker 1: the same kind of representational support from the mental movie
Speaker 1: that you could from observing a real phenomenon. So that's
Speaker 1: like when you play a movie of a thing in
Speaker 1: your head. Uh. They that you could confuse that with
Speaker 1: the same level of information that you get from looking
Speaker 1: at the object working the right quote. Of course, the
Speaker 1: mental movie is more like Hollywood than it is like
Speaker 1: real life. It fails to respect reality constraints. When we
Speaker 1: try to lean on the seductively glossy surface, we find
Speaker 1: that the facades of our mental films are hollow cardboard.
Speaker 1: That discovery, the revelation of the shallowness of our mental
Speaker 1: representations for perceptually salient processes, maybe what causes the surprise
Speaker 1: and our participants. And that seems very plausible to me.
Speaker 1: Like you, you try to put together a mental movie
Speaker 1: of how they can opener works, and you're playing the
Speaker 1: cartoon in your mind, and because you can do that,
Speaker 1: you're like, oh, okay, I know how it works. Like
Speaker 1: I just made the parts work in my mind, so
Speaker 1: I know what all the parts are and what they do.
Speaker 1: And it's something. Uh, it's something about this trick where
Speaker 1: our imagination is less vivid than we think it is.
Speaker 1: Like I'm picturing it in my head, I can see
Speaker 1: it in my head, but then you try to explain
Speaker 1: it and you realize there are blind spots in your
Speaker 1: own imagination that you do not realize are there. Yeah,
Speaker 1: our minds kind of tricks, isn't thinking We've filled in
Speaker 1: all those little gaps. Um, Like I was having similar
Speaker 1: situation just with Big Trouble a Little China. I feel
Speaker 1: like my memory of it, when I summon it is
Speaker 1: more of it, just a flash of images and uh
Speaker 1: and and probably leaning heavy on just the film score,
Speaker 1: just all these different ideas, scenes and sounds from the
Speaker 1: film that I have encapsulated as my memory of the film.
Speaker 1: I think that's true for a lot of movies with
Speaker 1: me yea. And yet for some reason, people are generally
Speaker 1: better at predicting how well they'll be able to describe
Speaker 1: a narrative, so that that's one of the outliers for me.
Speaker 1: I'm wondering what that really means. Well, I mean you
Speaker 1: could could certainly take that apart and say, well, it's
Speaker 1: a lot of it has to do with the way
Speaker 1: that we make sense of our lives being narratives when
Speaker 1: they're really not looking for the story shape in everything
Speaker 1: from you know, your personal life to current events. It's
Speaker 1: we're continually bashing our head up against the reality that
Speaker 1: things do not play out with the economy or the
Speaker 1: form of a traditional narrative. Well, unless you have anything else, Robert,
Speaker 1: I think we should wrap up this first part, and
Speaker 1: then when we return next time, we can look at
Speaker 1: some of the applications of the fact that we have
Speaker 1: an illusion of understanding, an illusion of explanatory depth about
Speaker 1: the world around us. How can this knowledge be brought
Speaker 1: to bear in various domains of life. Yeah, we'll consider
Speaker 1: the children, will consider politics. Um, we might even consider
Speaker 1: zombies a little bit. We'll see. Uh So. One of
Speaker 1: the one thing though, I do want to to keep
Speaker 1: in mind about this is that you shouldn't just take
Speaker 1: this as pessimistic, right, uh Like, oh, we we don't
Speaker 1: actually understand things as well as we do. How horrible
Speaker 1: you could be pessimistic. You could say, why do we
Speaker 1: know so much less about? How things work, then we
Speaker 1: feel like we do. Or you could look at this
Speaker 1: in a very optimistic way, and then and instead ask
Speaker 1: the question, how are we so good at surviving in
Speaker 1: and traveling through and manipulating the world when our models
Speaker 1: for understanding causal relationships are so skeletal and bare bones, Like,
Speaker 1: why are we so good at life compared to how
Speaker 1: absolutely uh uh sparse are our mental imagery that animates
Speaker 1: our understanding of the workings of things? Is? Yeah, And
Speaker 1: I think at two other positive spins. Hey, if I
Speaker 1: forget details of the plot and the narrative structure of
Speaker 1: Big Trouble and Little China, that means the next time
Speaker 1: I see it, a lot of stuff's gonna be new again.
Speaker 1: Oh and then, and we talked about our own privileged
Speaker 1: place of continually exploring new topics and and confronting what
Speaker 1: we don't know and learning more about the world around us.
Speaker 1: We should also point out, at risk of sounding like
Speaker 1: I'm pandering uh, that our audience probably is in much
Speaker 1: the same boat. The mere fact that you listen to
Speaker 1: stuff to blow your mind, um, that you engage in
Speaker 1: uh educational infocational uh podcast it just means. It means
Speaker 1: that you to realize. Hey, Like, for instance, we had
Speaker 1: to recently had an episode on butter. Some people might say,
Speaker 1: I know how better works. I'm not gonna listen to that.
Speaker 1: But people who did listen to it, they realized, well,
Speaker 1: I think I know how better works. But if they
Speaker 1: did an episode on it, then I guess there's more
Speaker 1: to the scenario than I than I than I give
Speaker 1: it credit. Oh that, I guess there's more. Moment. I
Speaker 1: feel like it's very central to what we do. Yeah. Uh,
Speaker 1: but don't let it go to your head. Robert, you
Speaker 1: and I and you out there listening. We're no better.
Speaker 1: We're no better. We just we just strive to understand
Speaker 1: the depths of our ignorance, all right, And if you
Speaker 1: want to strive to explore the depths of your ignorance,
Speaker 1: head on over to stuff to Blow your Mind dot com.
Speaker 1: That's where we find all the podcast episodes, videos, blog post,
Speaker 1: you name. It leaks out to social media accounts. We're
Speaker 1: on Facebook, Twitter, Tumbler, et cetera. About the Mothership is
Speaker 1: Stuff to Blow your Mind dot com. And of course,
Speaker 1: as always, if you want to email us directly to
Speaker 1: get in touch about this episode or any other you
Speaker 1: can hit us up at blow the Mind at how
Speaker 1: stuff works dot com for more on this and thousands
Speaker 1: of other topics. Is it how stuff works dot com.