Speaker 1: Welcome to the Psychology Podcast, where we give you insights
Speaker 1: into the mind, brain, behavior and creativity. I'm doctor Scott
Speaker 1: Barry Kaufman, and in each episode I have a conversation
Speaker 1: with a guest who will stimulate your mind and give
Speaker 1: you a greater understanding of yourself, others, and the world
Speaker 1: to live in. Hopefully we'll also provide a glimpse into
Speaker 1: human possibility. Thanks for listening and enjoy the podcast. Today
Speaker 1: we have Brian Nosek on the podcast. Brian is co
Speaker 1: founder and executive director of the Center for Open Science
Speaker 1: that operates the Open Science Framework. The Center for Open
Speaker 1: Science is enabling open and reproducible research practices worldwide. Brian
Speaker 1: is also a professor in the Department of Psychology at
Speaker 1: the University of Virginia. He received his PhD from Yale
Speaker 1: University in two thousand and two. He co founded Project
Speaker 1: and Plicit, a multi university collaboration for research education investigating
Speaker 1: implicit cognition, thoughts and feelings that occur outside of awareness
Speaker 1: or control. Brian investigates the gap between values and practices,
Speaker 1: such as when behavior is influenced by factors other than
Speaker 1: one's intentions and goals. Research Applications of this interest include
Speaker 1: implicit bias, decision making, attitudes, ideology, morality, innovation, and barriers
Speaker 1: to change. Noseek applies this interest to improve the alignment
Speaker 1: between personal and organizational values and practices. In twenty fifteen,
Speaker 1: he was named one of Nature's Ten and to the
Speaker 1: Chronicle for Higher Education Influence List. Brian's so glad to
Speaker 1: chat with you today. Yeah, thanks for having me on
Speaker 1: quite a lengthy bio. There we can keep going. I've
Speaker 1: got all kinds of history with my family. Maybe we
Speaker 1: should take out of it. Well, we get to that
Speaker 1: if you want me to psychoanalyze you. But that you
Speaker 1: stay busy, don't you? I mean you don't like do
Speaker 1: you ever have existential crisses? I feel like you don't
Speaker 1: have enough to think about yourself. I certainly have a
Speaker 1: full life, and I enjoy time to time looking up
Speaker 1: and trying to think about what it is we're doing
Speaker 1: and why we're trying to do it. But you're right
Speaker 1: that that doesn't happen as frequently as I would like.
Speaker 1: What spending time to yourself, it doesn't happen as frequently.
Speaker 1: Yeah that's right, Yeah, Yeah, gotcha, gotcha. Well that's that's
Speaker 1: a good thing for the world. I don't know if
Speaker 1: it's good for you. Probably could use some more downtime,
Speaker 1: but I, you know, I want to just like I
Speaker 1: do like to do with my guests is go back.
Speaker 1: Let's go back. So you got your PhD at Yale
Speaker 1: in two thousand and two. I got my PhD at
Speaker 1: Yale in two thousand and nine. So who did you
Speaker 1: study with? Their? Lazarine Banaji was my primary advisor, right,
Speaker 1: So when she was at Yale and then's right, she
Speaker 1: went to Harvard and you did you move to Harvard
Speaker 1: as well? Yeah? So in my last year of grad school,
Speaker 1: she relocated to Harvard, and it so happened that my
Speaker 1: spouse got her clinical internship in Boston at that same time.
Speaker 1: So I in fact moved before Mazarine did and help
Speaker 1: set up the lab that she was going to be
Speaker 1: moving too. And so I spent my last year sort
Speaker 1: of on a postop at Harvard instead of but it
Speaker 1: was really just my last year of grad school. Gotcha,
Speaker 1: Are you like officially a co creator of the implicit
Speaker 1: association test? Yeah? Tony Greenwald is the inventor, Okay, And
Speaker 1: then he and Mazarine had just started working on it
Speaker 1: with some of their collaboration, and then I got very
Speaker 1: involved in it right away and then built the website
Speaker 1: that became Project Implicit, and so the three of us
Speaker 1: are co founders of Project Implicit as a nonprofit organization,
Speaker 1: and then I ran that for the first eleven years
Speaker 1: of my faculty career. Oh wonderful. Yeah. I remember about
Speaker 1: seven years ago or so, I was teaching cognitive psychology
Speaker 1: at NYU, and I showed my students every semester of
Speaker 1: video of the IT and you're you're the one that
Speaker 1: talks in this video that they show a younger looking
Speaker 1: version of you. No offense. What are you talking about?
Speaker 1: No offense age? I hate to break it to you.
Speaker 1: Oh yeah, yeah, I don't look in mirrors just to
Speaker 1: be but yeah, I mean that's an irrelevant detail, but
Speaker 1: it's kind of surreal to and I'll be talking to you,
Speaker 1: interacting with the real you. I showed that video to
Speaker 1: my students every semester, and at that time I talked
Speaker 1: about the IT and about how, you know, there's still
Speaker 1: a lot of research that needs to be conducted on
Speaker 1: it in the future. I wrote an article for Psychology
Speaker 1: Today around that time with some research kind of starting
Speaker 1: to cast some doubt on whether or not it's really
Speaker 1: how strong the correlation is with explicit racism, for instance.
Speaker 1: And I remember telling my students about, you know, we
Speaker 1: should really look at this with an open mind. Okay,
Speaker 1: let me ask this question first. Were you like ever
Speaker 1: a zalot of the I? Are you like a reformer
Speaker 1: now or is it more of like you always liked
Speaker 1: the it, but you had some of your own criticisms
Speaker 1: from the beginning, and and then you spent many years
Speaker 1: refining and adding nuance to How would you describe the
Speaker 1: difference between where your state of mind was at back
Speaker 1: then when you were creating this thing and talking about
Speaker 1: it in public. Yeah, and your state of mind now?
Speaker 1: How discrepant are those two state of minds? Yeah? Well,
Speaker 1: I hope I've learned a lot from all of the
Speaker 1: research that we and others have done on it over
Speaker 1: the years. But in the big picture, I have the
Speaker 1: same view that I've had since we did the first
Speaker 1: set of studies to just sort of get a handle
Speaker 1: on what is this tool at all, and that is
Speaker 1: that it's a great tool for research purposes. We've been
Speaker 1: able to learn a lot about the tool itself, and
Speaker 1: about human behavior and interaction with the tool, and a
Speaker 1: lot about the psychology of things that are occur with
Speaker 1: less control and less awareness than just asking people how
Speaker 1: they feel about topics. So that it has been and
Speaker 1: continues to be a very productive research area for trying
Speaker 1: to understand better how humans work. And then the main
Speaker 1: concern that we had it onset and this is actually
Speaker 1: a lot of the discussion of even creating the website
Speaker 1: is the same anticipated some of the concerns and overuses
Speaker 1: that has happened with the itIt in the present, and
Speaker 1: that is the natural I don't know if natural is
Speaker 1: the right word, but the common desire that people have
Speaker 1: for simple solutions and thinking. Well, a measure is a
Speaker 1: direct indicator of something that we care about, and it
Speaker 1: shouldn't have any error in measurement, and it should be
Speaker 1: applicable to lots and lots and lots of situations. And
Speaker 1: there's lots of potential misuse of the itIt despite it
Speaker 1: being a very productive research tool and education tool. I
Speaker 1: like the experience of doing it and delivering it in
Speaker 1: an audience and the discussion that that provokes about what
Speaker 1: is it that it means and what does it mean
Speaker 1: about me? What does it mean about the world. Those
Speaker 1: are really productive intellectual discussions and debates. But the risk
Speaker 1: parts are the over application of the IAT two selection processes.
Speaker 1: We should use this for deciding who gets a job
Speaker 1: or not. We should use this to decide who's on
Speaker 1: a jury or not. Those are the kinds of app
Speaker 1: real world applications of IT as a measure that go
Speaker 1: far beyond its validity. And so this isn't exactly answering
Speaker 1: your question, because we even at the very beginning when
Speaker 1: we launched the website, we said it should not be
Speaker 1: used for these purposes, and I still believe that to
Speaker 1: be true. But what I have changed over time is
Speaker 1: the refinement of where it is we understand the evidence
Speaker 1: base against some of the major questions. And what's amazing
Speaker 1: about it is there's been so much research and we
Speaker 1: still don't have a great handle on really big questions
Speaker 1: relating to the itIt and measures like it. So that
Speaker 1: just is part of illustrating this field of how hard
Speaker 1: it is to actually make progress on studying human behavior.
Speaker 1: Oh yeah, and we'll get to your more recent open
Speaker 1: science initiative and how that relates to that. But let's
Speaker 1: stay in this. Let's stay in two thousand and two
Speaker 1: to two thousand and twelve for a second. You know, sure,
Speaker 1: this time period of your life. Now, let's talk shop
Speaker 1: a second. So my dissertation at Yale a little bit
Speaker 1: a couple of years after yours, was looking at the
Speaker 1: question is there such a thing as individual differences and
Speaker 1: implicit cognition? And the idea was to like ask this
Speaker 1: question because, you know, from a trade perspective, I felt
Speaker 1: like that was a huge gap in the literature. Was
Speaker 1: there's so much research on the reliability and validity of
Speaker 1: IQ tests, for instance, But I wanted to ask that question,
Speaker 1: like if we adapt some of these implicit cognition measures
Speaker 1: from the social psychological and experimental literature for an individual
Speaker 1: differences paradigm, you know, are there reliable and stable differences?
Speaker 1: And I have a whole appendix of failed experiments. By
Speaker 1: the way, you should tell me how to publish that
Speaker 1: some but we'll get to that in a second. But
Speaker 1: I have a hope. But so much of my dissertation
Speaker 1: where I guess you know, I'm putting failed in quotes
Speaker 1: because you know, I mean, what do you mean that
Speaker 1: was so useful? What you got. Wasn't that useful? I
Speaker 1: thought that was useful information that like, wow, the majority
Speaker 1: of these and puts it well. So I also worked
Speaker 1: and puss it learning, so not just the kind of
Speaker 1: the if I'm talking about. I looked at like, you know,
Speaker 1: pus a learning test, artificial grammar learning and SEI re
Speaker 1: reaction and all this stuff, and I found that virtually
Speaker 1: all of them it was almost impossible to capture reliable
Speaker 1: individual differences that like cohered over time. And but I
Speaker 1: did find like one that did and I published that
Speaker 1: it was the Sill reaction time task. But anyway, before
Speaker 1: we completely lose my audience, which is a general audience,
Speaker 1: I just want to say that I'm trying to link
Speaker 1: this because I feel like for me, one of the
Speaker 1: things that I am most wary about with the IAT
Speaker 1: is like and then this might be more of a
Speaker 1: feature than a bug. But it may be capturing you know,
Speaker 1: at this given moment at time when the person is
Speaker 1: taking that test, it's capturing a a lot of what
Speaker 1: the societal norms and influences are on that person's associations,
Speaker 1: but not capturing so much an intrinsic sort of stable
Speaker 1: individual differences variable so I just wanted to throw that
Speaker 1: out there and see what your current thoughts on that are. Yeah, well,
Speaker 1: it's clear it is not trait like in the same
Speaker 1: way that a measure like the Big five for personality
Speaker 1: is trait like. It does show stability over time, but
Speaker 1: much more weakly than that. So across a variety of
Speaker 1: different topics you might see a test retest correlation for
Speaker 1: the IT trying to measure the same construct of a
Speaker 1: round point five. The curiosity for this is, well, I
Speaker 1: guess it's a few curiosities. One is, does that mean
Speaker 1: that we have it measures some degree of trade variance?
Speaker 1: It seems to do so, because there is some stability
Speaker 1: over time. Then what's the rest? Is the rest error?
Speaker 1: Or is it state variance in some way? Right, variation
Speaker 1: that is meaningful variation that is sensitive to the context
Speaker 1: of measurement. And surely it's some of both, but we
Speaker 1: don't know how much, and there isn't yet real good
Speaker 1: insight on where the prediction components of the IIT are
Speaker 1: in how it anticipates behavior. Right, if we could separate
Speaker 1: in a real reliable way the trait part, the state part,
Speaker 1: and the error part, then we should be able to
Speaker 1: uniquely predict different kinds of things between the strait, the
Speaker 1: trait and the state and trait components. Another twist which
Speaker 1: is very interesting that's totally understudied in my view, is
Speaker 1: that these the variation that agree to which it's state
Speaker 1: versus trait like seems to vary by the topic that
Speaker 1: you're investigating. So when you do a Democrat Republican iat
Speaker 1: where you see to what extent do people favor one
Speaker 1: or the other in US respondents, the correlation with self
Speaker 1: report is very strong, and the stability over time is
Speaker 1: stronger than when you measure black, white, or some of
Speaker 1: the other types of topics. So there is also something
Speaker 1: about the attitude construct itself that you're assessing that's not
Speaker 1: as much measurement based, but is interacting with the measure
Speaker 1: that may anticipate the extent to which is trade or
Speaker 1: state like. So these are all interesting things that if
Speaker 1: I had time to study them, that was there would
Speaker 1: be the problems that I would continue to be studying,
Speaker 1: But I've had to leave that aside. Oh, there's a
Speaker 1: million million interesting questions. Thank you for bringing that up.
Speaker 1: Relating to the point I was trying to make about
Speaker 1: cultural influences on IT. I mean, have you done in
Speaker 1: and out like it seems to me like it's a
Speaker 1: testable hypothesis to be like, okay, for instance, in more
Speaker 1: gender equality societies, Let's say you, like, you know, you
Speaker 1: look at average global IT differences around the world. Do
Speaker 1: you find has that study been done like to show that,
Speaker 1: like you know, in more gender equality societies there's less
Speaker 1: bias in certain ways and others. I feel like that's
Speaker 1: an interest and open question. Yeah. Yeah, there are a
Speaker 1: number now because we've made the project implicit data publicly accessible,
Speaker 1: there are a number of investigations looking at regional differences.
Speaker 1: One that we published in two thousand and nine, I
Speaker 1: think it was was an examination of the strength of
Speaker 1: implicit gender stereotypes about science versus humanities, right, the extent
Speaker 1: to which I associate male with science or male with math.
Speaker 1: And we looked at that by nation and by nation
Speaker 1: that based on who came to the website. The strength
Speaker 1: of that stereotype is related to the sex gap in
Speaker 1: performance among eighth graders in that country, So places where
Speaker 1: the stereotype is stronger, the sex gap of men doing
Speaker 1: better than women in science and math standardized tests or
Speaker 1: boys versus girls in this case is larger, and we
Speaker 1: can't say anything about the causal directions there, but at
Speaker 1: least it shows that there is a covariation with things
Speaker 1: that are happening at a cultural scale, differences in engagement
Speaker 1: performance in these domains, and these implicit stereotypes as they're
Speaker 1: measured indirectly. That is really really cool. It was in PNAS,
Speaker 1: I think in two thousand and nine. So well, that's
Speaker 1: really interesting. So, yeah, there are a lot of unanswered questions.
Speaker 1: In what ways do you think that the it has
Speaker 1: been misapplied? I mean, you hint it at one, which
Speaker 1: is personnel selection. Yeah, so that seems like in what
Speaker 1: other ways can you just riff a little bit about,
Speaker 1: you know, I mean that's really important when you think
Speaker 1: of the audience you have right now. You have a
Speaker 1: very large general audience, So this is an opportunity to really,
Speaker 1: you know, explain a little bit about, you know, ways
Speaker 1: in which you think that society maybe has taken this
Speaker 1: a little too far beyond the science. Yeah. So the
Speaker 1: other main thing that I spend a lot of time
Speaker 1: trying to correct the record as it were in the
Speaker 1: public discussion is on what is the likely impact of
Speaker 1: implicit bias training. So this isn't the IAT per se,
Speaker 1: but certainly the IAT research has led to this sort
Speaker 1: of brad embrace of wanting to educate and train people
Speaker 1: about implicit bias. It's great for education. I'm all for education,
Speaker 1: but then with this idea that if you go through
Speaker 1: an implicit bias training in your organization and learn about
Speaker 1: these then you will no longer be a biased person,
Speaker 1: and the evidence there is not good for that being
Speaker 1: an effective method for actually changing behavior. I give an
Speaker 1: education like this, and I think it's useful to educate
Speaker 1: organizations and individuals about implicit bias. But what I think
Speaker 1: the limits are, and this is based on some evidence
Speaker 1: that we've gathered, is that the training is really just education.
Speaker 1: It's like learning about a topic and knowing what the
Speaker 1: state of the science is. And at most it will
Speaker 1: provide a basis for increasing motivation to do something about
Speaker 1: addressing unwanted biases within an organizational setting, but the training
Speaker 1: itself is not giving skills that will reduce it. This
Speaker 1: is sort of a very hopeful but not very behaviorally
Speaker 1: informed approach to trying to address biases, thinking that just
Speaker 1: teaching someone about the existence of bias will be sufficient
Speaker 1: to get rid of it, and that I think has
Speaker 1: been count in some cases been counterproductive to actually addressing
Speaker 1: some of the real disparities that are occurring in society,
Speaker 1: because it's an oversimplified view of what really is a
Speaker 1: complex and structural problem of how it is that organizations define,
Speaker 1: how hiring gets done, create decision processes, promotion processes, succession planning,
Speaker 1: all of the different aspects of effective organizational management. Really
Speaker 1: the solutions are in how those decision processes are structured,
Speaker 1: rather than in trying to get people to have better intentions. Yeah,
Speaker 1: it seems like that's very consistent with a recent study
Speaker 1: at Warden by Rebra Belle and Adam Grant and others.
Speaker 1: I don't know if you saw that study that cannot
Speaker 1: recently try to I didn't assess the effectiveness of well
Speaker 1: diversity training. Maybe it's a little bit okay, maybe it's
Speaker 1: a little bit different, but it's probably the broad broader brush.
Speaker 1: But but surely a diverse implicit bias is featuring in
Speaker 1: a lot of diversity training these days. Yeah, and you
Speaker 1: touched on a really interesting point about like, well, what
Speaker 1: is the best Like how would you measure the outcome
Speaker 1: of this like two day or week training thing. I mean,
Speaker 1: it seems like that would not be a very good
Speaker 1: thing to like then go back to the I T
Speaker 1: and see a difference between it pre and I T posts.
Speaker 1: That doesn't seem like the best sort of outcome you know,
Speaker 1: you would want to I mean, it's like, yeah, yeah,
Speaker 1: you could start just focusing on let's change the I
Speaker 1: T and have that be the end of it. But yeah,
Speaker 1: of course it's that doesn't actually shift behavior. Then what
Speaker 1: was the point? Yeah, what was the point? Yeah? Yeah,
Speaker 1: So there's there's so many dealing with such a systemic problem.
Speaker 1: It is dealing with a system of things, you know,
Speaker 1: that's like of interacting parts, and it does seem too
Speaker 1: simplistic to kind of just treat one part of the
Speaker 1: system as the cure. All so great, I'm glad you
Speaker 1: made that point. What about the association here between this
Speaker 1: kind of research and the literature on microaggressions for instance?
Speaker 1: Tell me if this is too far different, But it
Speaker 1: inttally feels to me like it's related to some more
Speaker 1: genus of things where you have this assumption that some
Speaker 1: sort of like implicit or unconscious motivations are going to
Speaker 1: creep out, I guess into external behavior of manifestation of behavior. Yeah, yeah,
Speaker 1: I think they are conceptually related. I haven't seen a
Speaker 1: lot of good research showing a functional relationship. But the
Speaker 1: notion that things leak out and in ways that are
Speaker 1: unintended or automatic or hard to detect is thematic in
Speaker 1: the in the microaggressions discussion, and certainly that is consistent
Speaker 1: in a very global sense with this notion that we
Speaker 1: have thoughts and feelings that are either outside of our
Speaker 1: awareness or outside of our control that may be different
Speaker 1: than our conscious values what we're trying to do in
Speaker 1: the moment, and we can be genuinely saying I'm not
Speaker 1: trying to be biased, I am trying to be fair.
Speaker 1: I'm trying to engage this person as a person, and
Speaker 1: nevertheless be influenced by otnizer influencing my behavior, and the
Speaker 1: detectability of those may differ by the perceiver and by
Speaker 1: the actor. And that's I think a key part in
Speaker 1: that microaggressions literature. Well, what is the latest state of
Speaker 1: the science on not just the micro aggressions such, all
Speaker 1: of these genus of things. Because you're in the forefront
Speaker 1: of this, you know, to what extent do you think,
Speaker 1: like we are really making advances in showing that there
Speaker 1: are these impuls influences on our explicitly behavior that operate
Speaker 1: outside of our value system. Where are we at right now? Yeah,
Speaker 1: it's a good question. I can't really characterize the microaggressions
Speaker 1: literature per se. I don't follow that as a distinct literature,
Speaker 1: but i'm a general point. I think it's the big
Speaker 1: picture story is pretty clear with evidence, which is we
Speaker 1: do things automaticity. We do things that are counter to
Speaker 1: our interests all the time, and sometimes we recognize that
Speaker 1: we're doing it, sometimes we don't, but a lot of
Speaker 1: times it's not controllable. So but that's a very big picture,
Speaker 1: very global, very non specific point. By the way, so
Speaker 1: that wasn't true. If that wasn't true, the mindfulness field
Speaker 1: would not be people wouldn't be making so much money.
Speaker 1: Oh so you're saying that you can we make money
Speaker 1: if things are true, Well, we've got some other conversations
Speaker 1: to have here, But it wouldn't have been so hot
Speaker 1: and popular people need it people. Yeah, so yeah, it's
Speaker 1: in Yeah, many parts of that are sort of obviously true.
Speaker 1: You're right in the sense that, yeah, we recognize we're
Speaker 1: doing things counter to our interests all the time. Right,
Speaker 1: we want to lose weight and we can't. We want
Speaker 1: to exercise more and we can't. We recognize that we're
Speaker 1: not interact with our loved ones and the way that
Speaker 1: we want to, and yet we still do it. So
Speaker 1: there's it's easy to recognize that there are factors that
Speaker 1: are outside of our own control, that are inside our
Speaker 1: own minds, that are influenced in our behavior. And that's
Speaker 1: a very important insight where the science does not have
Speaker 1: a great handle yet is clarifying when that happens and
Speaker 1: precisely how that happens, and then what to do about
Speaker 1: it when it happens. And there's some places where it's
Speaker 1: a lot better. For example, the training of specific phobias
Speaker 1: right cognitive behavioral therapies for specifics is really effective. Even
Speaker 1: though that's a problem that people can't easily overcome just
Speaker 1: by sheer will, there are treatment approaches that address it
Speaker 1: in a very effective and efficient way, but at scale
Speaker 1: across the range of human behaviors that we have, we
Speaker 1: still have a lot to learn about how to address that.
Speaker 1: That's for sure. So give me, like something that that
Speaker 1: you discover psychologists have discovered is not obvious, Like give me,
Speaker 1: like you know what I mean, It's like it's like,
Speaker 1: you know, it's like, why are we doing all this research,
Speaker 1: you know, to just show that the unconscious has influences
Speaker 1: on behavior? Like give me. Let's talk about some specific studies.
Speaker 1: I'll pick out one because I was going down your
Speaker 1: Google scholar things yesterday of new studies preprints that have
Speaker 1: come out. Here's an interesting one, how ideology impairs sound reasoning.
Speaker 1: So and this is research that it was led by Anugampa. Yeah,
Speaker 1: and colleagues that you're on this list. So this seems
Speaker 1: very relevant to our current political landscape. So do you
Speaker 1: think a lot of people do you think the research
Speaker 1: is suggesting that our ideology like creeps into what you know,
Speaker 1: we think is this pure reasoning process? Yeah, yeah, exactly.
Speaker 1: So it's easy for us to RECOGNI eyes that when
Speaker 1: if I am a conservative and you hear and you're liberal,
Speaker 1: and you hear me make an argument about some issue
Speaker 1: that has a conservative liberal difference, it's easy to understand
Speaker 1: that you would disagree with my claims and perhaps even
Speaker 1: the assumptions underlying my claims, right, I just don't agree
Speaker 1: that you know, this is the way that society should
Speaker 1: work or the economy actually works. That's not surprising. What
Speaker 1: we'd found in this research is that that disagreement, your
Speaker 1: dislike of my conservative position on something even makes it
Speaker 1: harder for you to recognize the logical reasoning in it,
Speaker 1: whether the reasoning. You'll be better at detecting logical errors
Speaker 1: in or liberal, and you'll be worse at recognizing sound
Speaker 1: reasoning regardless of whether you believe the content or not. Right.
Speaker 1: So there's a difference between soundness the premises, the conclusions
Speaker 1: follow from the premises, then from whether the premisses and
Speaker 1: the conclusions are themselves true. And ideology is so strong
Speaker 1: that it impacts even our ability to see whether the
Speaker 1: arguments have logical coherence to them when we disagree with
Speaker 1: the claims in them. So that just to me shows
Speaker 1: how deep and difficult it is to actually have productive
Speaker 1: debates across different when there are ideological differences. Yeah, it's
Speaker 1: a great example. It seems to me like it's becoming
Speaker 1: increasingly increasingly more difficult in this society or this political
Speaker 1: landscape to be ideology free, even if you want to
Speaker 1: be Yeah, do you know what I mean. It's like
Speaker 1: almost like you're penalize these days for trying your best
Speaker 1: to be logical. Yeah, and see things from other perspectives.
Speaker 1: It's like, yeah, pick a stand man. Yeah, right. So
Speaker 1: we even have a paper from a few years ago,
Speaker 1: Carl Hawkins was the lead author that we called Motivated Independence,
Speaker 1: and what we did was we identified people who said
Speaker 1: that they were independent right, they do not subscribe to
Speaker 1: a political left or political right, and we gave them
Speaker 1: an IAT and the IAT measured their implicit preference for
Speaker 1: Democrats versus Republicans. And even among people who declare that
Speaker 1: they are independent, there is wide variation and whether they
Speaker 1: were pro Democrat or pro Republicans. Some were close to
Speaker 1: the middle, some were very pro left, some were very
Speaker 1: pro right. But that's not where it stopped. What we
Speaker 1: then did was we presented them with a paper where
Speaker 1: two different policies were proposed, and we randomized whether policy
Speaker 1: AA was reposed by the Republican or policy B was
Speaker 1: proposed by the Republican, and then the Democrat proposed the
Speaker 1: other one. And then we just asked them which policy
Speaker 1: to prefer AARB and it turns out that among these independents,
Speaker 1: their implicit preference for Democrats as republican predicted whether they
Speaker 1: which policy they supported. If they were implicitly pro left,
Speaker 1: they tended to select the policy that whichever one it
Speaker 1: was that the Democrats supported. So these ideologies creep in everywhere,
Speaker 1: and even when we're trying to be unbiased, as it were,
Speaker 1: or independent, if we have any of that in our minds,
Speaker 1: it may yet shape how it is we make our decisions. Wow,
Speaker 1: that is a truly cool study, by the way, really
Speaker 1: neat good example of how this is relevant to a
Speaker 1: current landscape. But the idea of I'm really trying to
Speaker 1: wrap my head around this idea of an unconscious association
Speaker 1: being pro like you said, pro republican, pro Democrat, as
Speaker 1: if like our unconscious has its own value system that
Speaker 1: may be separate from her explicit value. So there are
Speaker 1: there implicit values and explicit values. Has the field distinguished
Speaker 1: between those? That's a good question, and that that may
Speaker 1: imply more richness than what is justified and how it
Speaker 1: is these are represented in the mind. When I say
Speaker 1: pro Democrat or Republican, all I mean is that what
Speaker 1: we do with these measures is a relative assessment how
Speaker 1: much more easily do you associate goodness with Democrats versus Republicans?
Speaker 1: And if it's easier for you to put good with
Speaker 1: Democrat than good with Republican, then you're pro Democrat. So
Speaker 1: it may be as very base level as a simple
Speaker 1: associative or affective relationship, or it could be that a
Speaker 1: lot of the stuff is more rich like you're describing,
Speaker 1: and actually has some representation about value systems and things
Speaker 1: that we normally would associate with much more deliberate thinking. Yeah,
Speaker 1: because I mean you could imagine someone living in a
Speaker 1: society where because of the people around them, let's say
Speaker 1: you live in a side of everyone's racist around you,
Speaker 1: you're explicitly racist explicitly and you're not explicitly racist, and
Speaker 1: that is very much against your value system, and it's
Speaker 1: picking up and you take this test, it just shows
Speaker 1: you're easier to categorize you know, African Americans as bad,
Speaker 1: I guess, or whites as good. Yeah, I mean you
Speaker 1: wouldn't want to say that like that means that person
Speaker 1: is pro white, you know what I mean? Like using
Speaker 1: the same nowgy, do you say pro Republican or pro Democrat?
Speaker 1: I mean, would you say that some one is just
Speaker 1: because they're easier to categorize something. Yeah, So that you're
Speaker 1: raising a good question, which is what do we call
Speaker 1: these things? Yeah, when there are associations in the mind,
Speaker 1: and usually when we talk about things like beliefs or
Speaker 1: attitudes or related concepts, we're thinking in terms of endorsement. Right,
Speaker 1: this is what that person says they believe, and so
Speaker 1: they do believe it. And when we're talking about implicit measures,
Speaker 1: we're not talking about endorsement. People don't have an opportunity
Speaker 1: to say I agree or disagree with this, and they
Speaker 1: may genuinely disagree when confronted with those associations. Right, So
Speaker 1: I have these implicit biases in my own mind. I
Speaker 1: would reject them, the ones that are different than my
Speaker 1: conscious values. I would reject them out of hand, and
Speaker 1: yet they are still in my mind. We have embraced
Speaker 1: the notion of calling the association between a racial categories
Speaker 1: or political categories and evowness or badness. We've called those
Speaker 1: implicit attitudes deliberately to try to make the argument that
Speaker 1: an attitude isn't just what we consciously believe, that the
Speaker 1: same sorts of behavioral consequences could occur with that kind
Speaker 1: of association of a group with goodness or badness, even
Speaker 1: if you don't endorse it, just because it exists in
Speaker 1: the mind and that is different than their conscious beliefs.
Speaker 1: But it is still attitudinal in the sense of its consequences.
Speaker 1: And so this is a real interesting part of how
Speaker 1: we debate and think about terminology and then what its
Speaker 1: implications are for how you understand both the concepts and
Speaker 1: their implications. Oh, for sure, I'm not fully understanding why
Speaker 1: attitudes is the right word. They're like, even that seems
Speaker 1: like it's a low subjective call. Why not just call
Speaker 1: it impussit habits or impussit patterns, like pus associations is
Speaker 1: calling it what it is versus attitudes. It seems like
Speaker 1: you're imparting some sort of subjective label on what it is. Yeah, well,
Speaker 1: I think ultimately every label is a subjective label in
Speaker 1: that sense. So the question is what is the most
Speaker 1: psychologically appropriate for what the terms mean? And there's a
Speaker 1: great Mazarine Bnage and Tony Greenwald, my advisors and collaborators
Speaker 1: on this work, published a paper in nineteen ninety five
Speaker 1: where they made the case for implicit social cognitions being
Speaker 1: attitudes and stereotypes with the qualifier implicit to make very clear,
Speaker 1: not endorsed. What they did, and their development of that argument,
Speaker 1: in part was go reference the define of attitudes that
Speaker 1: we have been working with as a field for many
Speaker 1: years and sort of unpack them and said, look at
Speaker 1: these definitions. None of them imply what a lot of
Speaker 1: our measurement approaches have done to try to assess attitudes.
Speaker 1: None of them imply, for example, that the person believes it,
Speaker 1: or endorses it, or says it. They all talk about
Speaker 1: attitudes in a much more functional way or whatever each
Speaker 1: of the definitions were. So these are consequential, but I
Speaker 1: think they all are also important for all of our
Speaker 1: psychological theories and terminologies is that we imbue them with meaning,
Speaker 1: and it is important to represent as clearly as possible
Speaker 1: what that meaning is and the attention of that meaning
Speaker 1: is because words have ambiguity and people use them in
Speaker 1: different ways. Yeah. Absolutely, and it'll be easy for me
Speaker 1: to say, okay, let's move on now to open science.
Speaker 1: But I think this is a you know, this is
Speaker 1: something it says, A gret opportunity to talk to you
Speaker 1: about this, because it's just been. It's been on my
Speaker 1: mind a while, Like you know, like the book blind Spot.
Speaker 1: It's called the book is called blind Spot, subcol and
Speaker 1: Hidden Biases of Good People, And that was written by
Speaker 1: both of your advisors, right the Naji and Greenwald. Now
Speaker 1: when you read that, obviously coming from a place of
Speaker 1: immense respect and love for them and their research, I mean,
Speaker 1: what do you think they were exaggerated things at all
Speaker 1: in that book? I don't know if I would say
Speaker 1: that or not. I'd have to go and look at
Speaker 1: specific claims to decide whether I think the calibration of
Speaker 1: the on the evidence is off the mark. In general,
Speaker 1: my recollection of read It's been a while now was
Speaker 1: that I thought it was a good popular summary of
Speaker 1: the state of the research literature at that point in time.
Speaker 1: And I don't think a whole lot has changed in
Speaker 1: the research literature in terms of the broad brush conclusions. Okay, yeah,
Speaker 1: but this is an you know, like I said before,
Speaker 1: there's lots that were learning and there have been lots
Speaker 1: of things that each of us has assumed along the
Speaker 1: way of Oh, I bet this is going to happen
Speaker 1: where we're all confronted with evidence of Oh no, that's
Speaker 1: not actually how it works. Lazarine has this great chapter
Speaker 1: from like mid two thousands. I'm forgetting what when exactly
Speaker 1: was published, but it's a chapter that's all about how
Speaker 1: her grad students showed she was wrong on different things,
Speaker 1: and one of the examples is on the malleability of
Speaker 1: implicit evaluations. She went into the work saying, no, no, no,
Speaker 1: these things are fixed. They're not going to change at all.
Speaker 1: They're not going to change easily. This is not going
Speaker 1: to happen. And then she had a series of grad
Speaker 1: students that said, I don't know about that, and did
Speaker 1: some studies to show, like we were just talking about
Speaker 1: that they're not so trait based, that they are state influenced,
Speaker 1: and she was blown away by that, and oh, geez, okay,
Speaker 1: I guess they that isn't how it is. And of
Speaker 1: course even that the pendulum has swung back some. So
Speaker 1: we've had more recent studies that we have found that, yeah,
Speaker 1: they may be variable by the context, but they're actually
Speaker 1: really hard to get able change from any of the
Speaker 1: interventions that we've tried. So even just one area of
Speaker 1: evidence about implicit biases, we sort of think, okay, they're
Speaker 1: they're not just stable, they're variable. Oh, in fact, wait
Speaker 1: a second, they're not totally variable. They're they're varying, but
Speaker 1: they're not changing. Waita wait, okay, so when I get changing,
Speaker 1: and what does that mean if they're not changing and
Speaker 1: they're variable. So that as we dig into the problem,
Speaker 1: it just gets more and more complicated, which you know,
Speaker 1: that's great in many ways. It keeps us very busy
Speaker 1: and learning and everything else, but it just shows, like
Speaker 1: we were talking before about how hard it is to
Speaker 1: make real progress on real behaviors like what people do. Yeah,
Speaker 1: for sure, absolutely. You know, I'm really trying to think
Speaker 1: this through in terms of, like, at the unconscious level
Speaker 1: of associations, won't there always be bias in one direction?
Speaker 1: Like what would a quality of implicit associations look like?
Speaker 1: You know, let's say we reverse it and we say,
Speaker 1: you know, white's bad, Black's good. Is that the society
Speaker 1: we want to live in? Either, you know, I'm just trying.
Speaker 1: I'm trying to think what is a quality? You're never
Speaker 1: going to have an implus association test where it's like
Speaker 1: everything is good and nothing is bad, or I'm just
Speaker 1: thinking like our im plus it's always going to be
Speaker 1: biased in some direction. Bias is not always and bias
Speaker 1: is not always bad, right, Yeah, I think that's a
Speaker 1: critical point, right, is that we've societally we've sort of
Speaker 1: pejorative judgment about if you're biased, that means something's wrong
Speaker 1: with you, right, And that's a really unfortunate because a
Speaker 1: lot of these biases come from very ordinary operations of
Speaker 1: the mind, for one, and are in fact things that
Speaker 1: we endorse. Right, the same mental systems that lead me
Speaker 1: to be wanting to call my doctor rather than my
Speaker 1: mechanic when my back hurts are the same things that
Speaker 1: lead me to have biases of kinds that I would
Speaker 1: disagree with. And that is, you hear things in the world,
Speaker 1: you see things related to other things, and your brain
Speaker 1: stores that information and sometimes it's information that you would
Speaker 1: agree with and you want to use. And it's still
Speaker 1: a bias in the very functional sense of it leads
Speaker 1: you to do one thing over another thing. But it's
Speaker 1: not an unwanted bias. It's not even as a bias.
Speaker 1: I'd be perfectly happy to say, yes, of course I
Speaker 1: prefer my doctor to my mechanic one of my back
Speaker 1: hurts because I have a bias A belief that's grounded
Speaker 1: in whatever is grounded in that I think that person
Speaker 1: is going to do a better job than that person.
Speaker 1: The challenge, of course, is that we don't always get
Speaker 1: purely accurate information exposed to us, and we don't necessarily
Speaker 1: agree with the information that we do get exposed to,
Speaker 1: but nevertheless it gets in our heads and it has
Speaker 1: potential to influence our actions. Yeah. Absolutely, I've seen some
Speaker 1: instances where some people will say things like, oh, came out,
Speaker 1: I'm biased against males who I'm safe, you know, and
Speaker 1: then you know and everyone Wilaugh said that. But then
Speaker 1: you say you're biased against females, and everyone come like
Speaker 1: the Uni gay mobbed on Twitter. So it's like, I
Speaker 1: guess the point is, like, I'm a humanist, you know,
Speaker 1: a humanistic psychologist. I want to think through what would
Speaker 1: mean to live in a world where you know, we
Speaker 1: didn't say it was great to be biased against anyone
Speaker 1: except Nazis. Well, it all can be biased against Nazis.
Speaker 1: But do you see where I'm going with this? Do
Speaker 1: you see I'm saying yeah, totally yeah, And I think
Speaker 1: so one sort of thing where I think I resonate
Speaker 1: with what you're saying, is that the goal of the project,
Speaker 1: if there is a social agenda, a linked to the
Speaker 1: scientific agenda, just to understand how these things work is
Speaker 1: not very usefully thought of as let's get rid of bias,
Speaker 1: because that's sort of like saying, let's get rid of perception, right,
Speaker 1: you know, it's just it's the mind does that, and
Speaker 1: it's going to do that. If there is a social
Speaker 1: justice project, it's let's make sure that our values are
Speaker 1: the things that are driving our behaviors to the extent
Speaker 1: that we can right, and when they're misaligned, that's an
Speaker 1: opportunity to figure out how is it that we can
Speaker 1: improve our decision making process. The identification of those potential
Speaker 1: biases some intervention, but it really should be about matching
Speaker 1: our values with our behavior rather than saying, well, we
Speaker 1: just have to be bias free in some I don't
Speaker 1: even know what it would mean as you were saying,
Speaker 1: I have to be bias free and self functional. Great, perfect,
Speaker 1: let's end it. Let's end that this topic, that this
Speaker 1: conversation on that note, So let's now move into the
Speaker 1: reproducibility project. Sure, now, why did you start that? What
Speaker 1: year did you start that? Why did you start that
Speaker 1: what was going on in the culture in psychology at
Speaker 1: that time, that was the impetus for this. So we
Speaker 1: launched the Reproducibility Project in twenty eleven, and it came
Speaker 1: up in the context of a discussion that was had
Speaker 1: been going at a low hum in psychology and other
Speaker 1: fields for decades, which is, we're not so sure that
Speaker 1: the research literature is as credible as we assume it is.
Speaker 1: Published findings we presume are based on some degree of
Speaker 1: likelihood of being true or credible evidence. But there are
Speaker 1: things that happen in research practice that may undermine that
Speaker 1: credibility right like, for example, positive results finding a relationship
Speaker 1: between something or finding that intervention works are more likely
Speaker 1: to be reported than not finding anything, finding a negative relationship,
Speaker 1: finding this thing doesn't work, and so that might be
Speaker 1: biasing the literature. People might be making decisions about how
Speaker 1: they analyze their data and only reporting the ones that
Speaker 1: make the findings look as publishable or as credible as possible,
Speaker 1: undermining the actual credibility the evidence. So there's been this
Speaker 1: discussion for decades about these things seem to be happening
Speaker 1: might be undermining the credibility of the literature. And we're
Speaker 1: kind of worried about it, but there hadn't been much change.
Speaker 1: But in twenty eleven a few different things happened that,
Speaker 1: at least within psychology, seem to help to foment a
Speaker 1: much broader cultural discussion. One was a major fraud scandal
Speaker 1: Diedrich Stoppel in oh Yea, the Netherlands, you know, created
Speaker 1: out of fifty papers ended up something like that getting
Speaker 1: retracted of made up data that people hadn't seen. They've
Speaker 1: seen the papers, but no one recognized for many years
Speaker 1: that these papers were based on no evidence at all.
Speaker 1: People hadn't tried to replicate them to in order to
Speaker 1: evaluate whether those are reproducible evidence. So there was a
Speaker 1: worry that how is it that we never even recognized
Speaker 1: as a field about this fraudulent evidence getting into the literature.
Speaker 1: What's wrong with our practices? So the other major event
Speaker 1: in twenty eleven was that Daryl Bem who was a
Speaker 1: prominent social psychologist and has done lots of excellent work
Speaker 1: for many years, published a paper in the most prestigious
Speaker 1: social psychology journal showing evidence for ESP and people were shocked,
Speaker 1: how is it that this journal, it is a premiere
Speaker 1: peer reviewed journal would publish evidence for something that we're
Speaker 1: pretty sure isn't true. And the answer was that the
Speaker 1: paper was beautiful. It was beautiful in the sense that
Speaker 1: it followed all of the rules for what one does
Speaker 1: to get evidence and to publish that evidence in psychology.
Speaker 1: And so the editor said, well, look, it's just like
Speaker 1: every other paper that we would end up accepting because
Speaker 1: it did all of the things we expect to do.
Speaker 1: It just comes to a conclusion that we don't believe.
Speaker 1: And so the debate that followed that was this followed
Speaker 1: all the rules. No one's suggesting he did something wrong
Speaker 1: or different than everyone else does. If he followed all
Speaker 1: the rules, then either we now need to believe VSP
Speaker 1: or we need to question the rules. How is it
Speaker 1: that we end up reporting, doing and reporting on the
Speaker 1: evidence for our research findings. So that was sort of
Speaker 1: the global context was the sort of stunning events along
Speaker 1: with this long history of concern about the credibility of
Speaker 1: the literature. So we started the Reproducibility Project with a
Speaker 1: goal of saying, boy, everybody's worrying about the credibility of
Speaker 1: research findings. The normal way that researchers evaluate the credibility
Speaker 1: of findings is to replicate them. Right, you see a finding,
Speaker 1: you say that's interesting, I'm not so sure, or oh
Speaker 1: I want to use that and extend it in some way.
Speaker 1: I'll run a replication see if I can use the
Speaker 1: same methodology and then get the similar result. But people
Speaker 1: don't tend to do that because it's not rewarded very much.
Speaker 1: So we said, let's organize a project with a bunch
Speaker 1: of us, and we'll try to replicate a sample of
Speaker 1: findings and see if we can reproduce the results that
Speaker 1: are in the literature. And so we so yeah, right, yeah,
Speaker 1: So we just started it as a you know, sort
Speaker 1: of informally and made an announcement on some discussion group
Speaker 1: online social media somewhere, and hundreds of people joined the project.
Speaker 1: We ended up the final paper, which was published in
Speaker 1: twenty fifteen, had two hundred and seventy co authors and
Speaker 1: another eighty something people helped in some way just short
Speaker 1: of authorship. So it was this massive crowdsourced effort where
Speaker 1: one hundred replications were done from a sample of paper
Speaker 1: leading journals in psychology. And you know, the short summary
Speaker 1: is we were able to successfully replicate less than half
Speaker 1: about forty percent of the findings that we tried to reproduce,
Speaker 1: and so that just spawned or at least helped to
Speaker 1: fullment this what was now a very robust discussion about
Speaker 1: what are the challenges for reproducibility in psychology and beyond psychology.
Speaker 1: This is now an issue that is prominent across all
Speaker 1: research domains. I would say less so my field, which
Speaker 1: is personality psychology. Oh you would you all right, but
Speaker 1: those are kind of fighting words to say that to
Speaker 1: Brian notes. But there was a study that was done
Speaker 1: recently to see whether or not the same sort of
Speaker 1: wreckability crisis as operating in personal psychology. It was like,
Speaker 1: on the much higher than fifty percent of the things
Speaker 1: are replicating in that field. Yeah, although it was a
Speaker 1: very specific subset of findings. So I while I do
Speaker 1: believe that the and we even have evidence in our
Speaker 1: own studies that some of the personality findings that the
Speaker 1: rate might be a bit higher than others, it is
Speaker 1: not a challenge that is that it has been avoided
Speaker 1: across disciplines writ large, including personality research. For sure. I
Speaker 1: don't want to be hubristic. I don't be hubristic, but
Speaker 1: I'm stating a datum. You know, they did this, but
Speaker 1: maybe we'll see if that's study replicats that's like a
Speaker 1: meta replica, meta replication, Yeah for sure, Yeah, does it
Speaker 1: replicate that a lot of the things do replicate? Will
Speaker 1: yours replicate? Well, your science paper replicate will come out
Speaker 1: that it actually is actually anyway higher than forty percent. Yeah, well,
Speaker 1: in fact, we are doing a replication of that subset
Speaker 1: of that study, so that you know, we had those
Speaker 1: one hundred and one of the criticisms that was let it.
Speaker 1: There's lots of people raised criticisms, but one of them
Speaker 1: that yeah, I think that's a real important limitation is
Speaker 1: that maybe the replication team screwed up. Right. There's a
Speaker 1: very plausible reason. It's like, well, geez, who said that
Speaker 1: we did a good job of it? Right, Maybe the
Speaker 1: reason we failed to replicate is because we think at
Speaker 1: doing it. And there are ten of the papers, ten
Speaker 1: of the replications in the reproducibility project where the original
Speaker 1: authors had raised some concerns prior to running the study,
Speaker 1: so they said, you know, they're presumably pretty expert, right,
Speaker 1: they did the original research, and they had identified something
Speaker 1: that they said, I'm not so sure about this, ranging
Speaker 1: from moderate concern to major potentially, but the replication team said, well,
Speaker 1: we think we can still do a fair replication despite
Speaker 1: that concern. Nine of those ten failed to replicate in
Speaker 1: the reproducibility project. So those ten are a perfect pescothesis,
Speaker 1: which is, if we actually can meet those concerns that
Speaker 1: they raised, revise the studies to try to do it
Speaker 1: in the way that the original author said, you need
Speaker 1: to do it this way in order to get the effect,
Speaker 1: then we should be able to find evidence of this
Speaker 1: impact of expertise in improving replicability. So we're running an
Speaker 1: experiment right now where actually the data collection is done.
Speaker 1: We're getting close to the analysis phase where those ten
Speaker 1: studies have now been replicated multiple times with the process
Speaker 1: that we did in the reproducibility project that they had
Speaker 1: raised concerns about, and a revised one that meets the
Speaker 1: concerns that were raised to try to maximize the quality
Speaker 1: of replication. And we'll see is this plausible reason for
Speaker 1: some failures to replicate an actual reason in these cases? Oh,
Speaker 1: that's really great. That's really great, Brian. What are some
Speaker 1: like old chestnuts that have not replicated, like you know
Speaker 1: some things that are taught as staples and introductory psychology textbooks,
Speaker 1: you know what are just name a couple. Yeah, well
Speaker 1: this is there are areas that have had particular attention
Speaker 1: for being challenging to replicate, and of course that doesn't
Speaker 1: mean that they're wrong, but it does mean that we
Speaker 1: don't yet understand what's needed to make these findings reproducible,
Speaker 1: if they are reproducible at all. And the two of
Speaker 1: the biggest ones in the public discussion are about ego depletion,
Speaker 1: the idea that our ego is like a muscle, right,
Speaker 1: you use it a lot and you get tired, and
Speaker 1: so once you've been working hard to regulate yourself in
Speaker 1: some way, then you may be more likely after that
Speaker 1: to be blurting out things that you don't want to
Speaker 1: do or doing things that are less effective in self
Speaker 1: regulation and management. Right right, Yeah, well that's something else
Speaker 1: that you, as a personality psychologist, should be able to
Speaker 1: tell us about. So that's one where there's lots and
Speaker 1: lots of research about it and lots of exciting findings,
Speaker 1: but attempts to replicate some of those central findings have
Speaker 1: not been successful, or at least not very successful, and
Speaker 1: so now there's a lot of debate of what in
Speaker 1: that literature is credible and what isn't. And that's not result.
Speaker 1: And then the other that's gotten a ton of attention
Speaker 1: is very generally called social priming, which isn't a super
Speaker 1: informative label, but it's the idea that very subtle cues
Speaker 1: may influence our behavior in surprising ways. So the classic
Speaker 1: demonstration is by having people get exposed incidentally to words
Speaker 1: relating to oldness Florida, Caine, other things. Then they later
Speaker 1: will do things that make it look like they are older,
Speaker 1: like walk slower down a hallway when they leave the experiment,
Speaker 1: compared to if they weren't primed that these words meaning oldness.
Speaker 1: And so there's a lot of interesting demonstrations of these
Speaker 1: ways that subtle primes may influence our behavior. And that's
Speaker 1: been a very hot both heat and light area for
Speaker 1: a debate about replicability, with some high profile failures to replicate.
Speaker 1: So there's the current state of the literature right now
Speaker 1: is not one where there are clear answers. Oh here, okay,
Speaker 1: here are all the findings in our literature. These are
Speaker 1: the ones that are replicable, and these are the ones
Speaker 1: that aren't. Mostly it is a morass of debates of oh,
Speaker 1: this one's been challenging to replicate, and people say, well,
Speaker 1: I think it's because you did this, or I think
Speaker 1: it's because of this reason, or I think it works
Speaker 1: here and not there. And to me, that's as long
Speaker 1: as it stays focused on the evidence, that's a very
Speaker 1: productive kind of discussion to be having. We work on
Speaker 1: complicated stuff, and so some attention to trying to figure
Speaker 1: out the conditions under which those core findings actually are
Speaker 1: observable is very effective and useful for improving our theories,
Speaker 1: for understanding how it is those things work. Absolutely. What
Speaker 1: about the fundamental attribution not bias? Is that still fundamental? Well,
Speaker 1: the fundamental portion of that has actually evolved a lot
Speaker 1: over time. We change the label. Yeah, so the term
Speaker 1: that is common now for a portion of it is
Speaker 1: called the correspondence bias. So we don't need to get
Speaker 1: into the details for why and the distinction between that
Speaker 1: and the fundelent attribution error. And there's been interesting debates
Speaker 1: on whether it's an error or not or a bias. Right, Yeah,
Speaker 1: you know exactly, So this comf full circle right right,
Speaker 1: And there is a comfull circle that is unproductive, which
Speaker 1: is you just keep spinning in the same place. And
Speaker 1: there is a cyclical nature of scientific advancement that's very productive.
Speaker 1: Revisions that we have about any area of research to
Speaker 1: sort of check in and say, hang on a second.
Speaker 1: You know, we thought about this this way. Let's look
Speaker 1: at it from this new perspective now that we have
Speaker 1: new evidence and see if we can revive an old
Speaker 1: claim or see if we can refine the way that
Speaker 1: we understand it. So, by and large, I'm find what
Speaker 1: has been happening within our literature to be very exciting
Speaker 1: recently because I think we are questioning assumptions in a
Speaker 1: quite productive way. Yeah, a lot of people are excited
Speaker 1: about it. There are a couple of people are not
Speaker 1: excited about it, and those are those whose life's work
Speaker 1: is shown to not replicate. So I mean hopefully they
Speaker 1: I mean, I mean I say that like in a
Speaker 1: cheeky way, but having some capassion as well. For the scientists. Yeah, yeah,
Speaker 1: for sure, scientists buy and large a human and we
Speaker 1: get attached to our findings. Yeah. Most Yeah, we won't
Speaker 1: talk about the words that aren't yeah all the but
Speaker 1: a clue is they all have a last thing that
Speaker 1: starts with Z. But the challenge, of course, is that
Speaker 1: we treat those findings like possessions, right, they become the
Speaker 1: basis of our identity, as in our profession. And so
Speaker 1: it's very understandable if we're thinking of these things as possessions,
Speaker 1: that we would get defensive about them, and that they
Speaker 1: would have implications for our reputation when they get challenged.
Speaker 1: And some challenges are fair and some are not fair,
Speaker 1: and so there is a lot of person in these
Speaker 1: debates rather than them just being purely scientific. And if
Speaker 1: we don't recognize that in how dos we conduct the debates,
Speaker 1: then there'll be more unfortunate interpersonal consequences than there need
Speaker 1: to be. There will inevitably be consequences for people's reputations,
Speaker 1: because science is a reputation based discipline, and and some
Speaker 1: work is more robust than other work. But you're right,
Speaker 1: the compassion part of that is a key part of
Speaker 1: how one walks into this kind of work. Yeah, you know,
Speaker 1: And it's so easy just to say, oh, the problem is, well,
Speaker 1: just don't hitch your identity to it. I actually I
Speaker 1: think that there's a deep issue. There's a deep existential
Speaker 1: issue here. If you spend thirty years of your career,
Speaker 1: it's making sacrifices in your life. We have a short
Speaker 1: life span, and I didn't marry this person, or I
Speaker 1: didn't take this job, I didn't have this child. Yeah,
Speaker 1: because I did this work. I guess. I just I
Speaker 1: really like feel I almost feel like it's a little
Speaker 1: bit too dismissive to say, well, we just have to
Speaker 1: be these spocks and rise above and pitching identity to
Speaker 1: our science. No, we have made these sacrifices in our
Speaker 1: lives what we're going to commit our lives to studying.
Speaker 1: So perhaps in the way we have these debates, we
Speaker 1: could come from a place of like, well, you know, look, hey,
Speaker 1: this work that you spent through year studying added a
Speaker 1: lot of value because it showed us better boundary conditions
Speaker 1: or better ways forward for methodologies. Without you having done
Speaker 1: that work, we couldn't have used that as a stepping stone.
Speaker 1: But instead I see a lot of these discussions devolving
Speaker 1: in a different direction of like it doesn't lead from
Speaker 1: that place. It almost ignores like the fundamental sacredness of
Speaker 1: a human existence. There's scientists who would just make fun
Speaker 1: of things, you know, and mock them, and that's got
Speaker 1: to hurt if you've like you really have contributed, even
Speaker 1: if it didn't all replicate, you still like your life
Speaker 1: had value, do you know what I mean? So it
Speaker 1: gets deep, But yeah, no, I think you're right that
Speaker 1: this is much deeper than a very simple come on,
Speaker 1: don't be so attached to your findings, so that, if anything,
Speaker 1: it's psychologically naive to think that we could do that. Right,
Speaker 1: if anything, right, come on, it'sycholog as. We know that's
Speaker 1: not going to happen. So let's at least recognize that
Speaker 1: we are going to feel that and whatever in whatever
Speaker 1: position we are in any given debate, we should be
Speaker 1: able to at least empathize with the other positions of
Speaker 1: if my work was being attacked or challenged or whatever
Speaker 1: productive or unproductive term you want to use, how would
Speaker 1: I respond. And there's there's a lot that we can
Speaker 1: understand of how we are tied to our findings and
Speaker 1: our claims and the work that we've done in the past,
Speaker 1: and simultaneously we do need to go through that, right
Speaker 1: and that's the real challenge is that it's you know,
Speaker 1: the flip of what you're describing is also equally problematic,
Speaker 1: which is to say, oh, we don't want to challenge
Speaker 1: people because it make them feel bad. No, no no, no no,
Speaker 1: this is we really have to Science is about skepticism,
Speaker 1: and really it is our commitment to what it is
Speaker 1: we are doing in the first place, to make it
Speaker 1: so that our claims are as robust and evidence based
Speaker 1: as possible. And sometimes that's going to mean, in fact,
Speaker 1: most of the time that's going to mean wrong about
Speaker 1: pretty serious claims that we've made, and so at base,
Speaker 1: I think cultivating an intellectual humility is a real key
Speaker 1: part for everybody in the game, right, whether you're criticizing
Speaker 1: or whether you are defending a particular position. The intellectual
Speaker 1: humility of uncertainty, of knowing that I might be wrong
Speaker 1: but I still think that this is a debate worth
Speaker 1: having is hard to do, easy to say, but super important.
Speaker 1: Oh I love that. I love that point you just made.
Speaker 1: So it seems to me like there's a shift going
Speaker 1: on that is healthy and sort of the spirit upon
Speaker 1: which we're doing psychological science. I think for too long
Speaker 1: in the field of psychological science, people's the spirit upon
Speaker 1: which you did it is sort of like, oh, so
Speaker 1: I could own this theory, you know, like it's Zimbardo's model,
Speaker 1: do you know what I mean? Trying to pick out
Speaker 1: there for a second, but you know me subconsciously it works.
Speaker 1: There you go, there you go. But like I think
Speaker 1: there's a lesson we can learn from this that I'm
Speaker 1: personally trying to learn too. And it's like it's like science.
Speaker 1: It's fun though, Like it's fun if you change your
Speaker 1: mindset about this to like it's all in. We're all
Speaker 1: in explorers, we're exploring uncharted territory. The uncharted territory can change,
Speaker 1: you know, like the problem is like clearly when you
Speaker 1: you touch your ego to a result or a finding.
Speaker 1: But psychological science, the field doesn't have to be that way,
Speaker 1: like we can. It just seems like it's more fun
Speaker 1: when like it's like, oh, let's do this study and
Speaker 1: you know, who knows what's good? What well? It is
Speaker 1: good sometimes the pre register predictions. Now you can't be
Speaker 1: totally exploratory, but at least like being open to like
Speaker 1: you know, what's going to happen, Like we're in this
Speaker 1: sort of sailboat or something all together, but all together though,
Speaker 1: you know, so yeah, it seems like there's yeah, so
Speaker 1: does this make sense. There's sort of like a shift
Speaker 1: in the kind of that the healthier spirit of the field. Yeah,
Speaker 1: I think there is. I think that positive shift is underway,
Speaker 1: and and the way that we've phrased it in some
Speaker 1: papers is really cultivating. Instead of the desire to be right,
Speaker 1: it's the desire to get it right right, and just
Speaker 1: that just that shift would open us up more to
Speaker 1: the challenge being an opportunity. Right. There's in the in
Speaker 1: open source software development, when another software developer points out
Speaker 1: a bug in your code and says, here's a fix
Speaker 1: for your bug. People don't say, you jerk, why you're
Speaker 1: pointing out this bug in my code? Right. The reaction
Speaker 1: is so thank you. Now my code works better. That's fantastic,
Speaker 1: and we could do more of that. But an important
Speaker 1: part of what makes that work, I think, is that
Speaker 1: the solution is often part of that discussion. Right. It
Speaker 1: isn't just you are wrong and you're wrong, it's you're wrong,
Speaker 1: and here's some evidence that we can use to get
Speaker 1: less wrong, and that can make it much more productive
Speaker 1: in terms of a scientific dialogue. Absolutely, so, open science
Speaker 1: is this broader framework we' you're giving you explained to why,
Speaker 1: how about the world's authority? And open science explains to
Speaker 1: me what open science is supposed to be explained to you.
Speaker 1: So open science is the idea that one of the
Speaker 1: ways that we can be more effective in accelerating discovery,
Speaker 1: finding solutions, developing knowledge is by showing the basis of
Speaker 1: evidence for the claims that we make. So it's not
Speaker 1: just enough for me to tell you here's my paper
Speaker 1: and this is what I found, but rather I should
Speaker 1: show you here's the process that I went through. Here's
Speaker 1: the initial plans that I had, and what I was
Speaker 1: going to do with the study, and how I was
Speaker 1: going to analyze the data, so you can compare against
Speaker 1: what I ended up saying. Here are the data, and
Speaker 1: here are the materials and methods that I used, so
Speaker 1: that if you want to interrogate the data with alternative specifications,
Speaker 1: you could do that, or you could take the materials
Speaker 1: and you could run a replication more easily yourself. And also,
Speaker 1: open science means that anybody that has the wherewithal the interests,
Speaker 1: the skills, the time, the resources to contribute to the
Speaker 1: scientific process should have avenues to contribute either as a
Speaker 1: producer or a consumer of that research. So it has
Speaker 1: a strong inclusivity aspect that we have thought of the
Speaker 1: scientific process as isolated to elite groups right in the
Speaker 1: elite places as doing the science. But we can distribute
Speaker 1: that a bit better. There are ways to get more
Speaker 1: people involved in more avenues to get involved in scientific research.
Speaker 1: So both of these parts of open science I think
Speaker 1: are critical for making the science itself more robust and
Speaker 1: for making it easier to leverage the wide range of
Speaker 1: skills and interests and talents that people have for contributing. Well,
Speaker 1: thank you so much for advancing the field so much
Speaker 1: in that direction. What is this expression I saw that
Speaker 1: came across pro science pound bro science? What is bro science? Yeah,
Speaker 1: bropen science is a term that's come up in social media.
Speaker 1: That's referencing how open science movement, just like any movement,
Speaker 1: has the foregoing in directions that reinforce some of the
Speaker 1: status or inequality hierarchies and activities that can occur in
Speaker 1: everyday life. And I think every community and every movements
Speaker 1: those risks is ongoing and present risk right, and social
Speaker 1: media in particular can foment the worst in social communication
Speaker 1: because it can amplify the more extreme voices, the more
Speaker 1: hostile voices, and you can lose sort of the view
Speaker 1: of that most people are just trying to be decent,
Speaker 1: trying to do a good job, and just trying to
Speaker 1: get the work done and talk about it. When when
Speaker 1: the real hostility becomes so amplified. So I think that's
Speaker 1: partly the origin of that term, and an ongoing part
Speaker 1: of the discussion is, Yeah, we have values for how
Speaker 1: we're trying to improve science, we also need to be
Speaker 1: attending to the values of who should be able to
Speaker 1: be contributing and how they're contributing to the science. Beautiful. Yeah,
Speaker 1: I like this twitter yours. You wrote Twitter is the
Speaker 1: perfect communication medium if the goal is to escalate conflict
Speaker 1: as rapidly as possible. Yeah, I thought that was great.
Speaker 1: That was great, right, Yeah, well it's true, right, because
Speaker 1: what can you do? That's why create and now suddenly
Speaker 1: just worse and worse. So again, I want to thank
Speaker 1: you for coming on the show and for talking to
Speaker 1: me about all your work, and I hope that we
Speaker 1: can work together to balance getting it right with being
Speaker 1: kind to others. So thank you so much for being
Speaker 1: on the podcast today, Brian. Yeah, my pleasure. Thanks for
Speaker 1: doing it. Thanks for listening to the Psychology Podcast. I
Speaker 1: hope you enjoyed this episode. If you'd like to react
Speaker 1: in some way to something you heard, I encourage you
Speaker 1: to join in the discussion at the Psychology podcast dot com.
Speaker 1: That's the Psychology podcast dot com. Also, please add a
Speaker 1: rating and review of the Psychology Podcast on iTunes. Thanks
Speaker 1: for being such a great supporter of the podcast, and
Speaker 1: tune in next time for more on the mind, brain, behavior,
Speaker 1: and creativity.