Speaker 1: Hey everyone, it's Robert and Joe here. Today we've got
Speaker 1: something a little bit different to share with you. It
Speaker 1: is a new edition of the Smart Talks podcast series,
Speaker 1: which is produced in partnership with IBM. This season of
Speaker 1: Smart Talks with IBM is all about new creators, the developers,
Speaker 1: data scientists, c t o s, and other visionaries creatively
Speaker 1: applying technology and business to drive change. They use their
Speaker 1: knowledge and creativity to develop better ways of working, no
Speaker 1: matter the industry. Join hosts from your favorite Pushkin Industries
Speaker 1: podcast as they use their expertise to deepen these conversations.
Speaker 1: Malcolm Gladwell will guide you through this season as your
Speaker 1: host to provide his thoughts and analysis along the way.
Speaker 1: Look out for new episodes of Smart Talks with IBM
Speaker 1: every month on the I Heart Radio app, Apple Podcasts,
Speaker 1: or wherever you get your podcasts. And learn more at
Speaker 1: IBM dot com slash smart Talks. Hello, Hello, Welcome to
Speaker 1: Smart Talks with IBM, a podcast from Pushkin Industries, I
Speaker 1: Heart Radio and IBM. I'm Malcolm Babbo. This season, we're
Speaker 1: talking to new creators, the developers, data scientists, c t
Speaker 1: o s, and other visionaries who are creatively applying technology
Speaker 1: and business to drive change. Channeling their knowledge and expertise,
Speaker 1: they're developing more creative and effective solutions, no matter the industry.
Speaker 1: Our guest today is Padre Bonadius, trust in AI Practice
Speaker 1: leader within IBM Consulting. Advocating for artificial intelligence built and
Speaker 1: deployed responsibly is no longer just a compliance issue, but
Speaker 1: a business imperative. Part of Phaedre's job is to help
Speaker 1: companies identify potential risks and pitfalls way before any code
Speaker 1: is written. In today's show, you'll hear how Phaedre's team
Speaker 1: and IBM is a pro genus challenge holistically and creatively.
Speaker 1: Pedra spoke with Dr Lois Santos, host of the Pushkin
Speaker 1: podcast The Happiness Lab. Laurie is a professor of psychology
Speaker 1: at Yale University and an expert on human cognition and
Speaker 1: the cognitive biases that impede better choices. Now let's get
Speaker 1: to the interview, Pedro. I'm so excited that we get
Speaker 1: a chance to chat today. You know, just to start off,
Speaker 1: I'm wondering how did you get started in this role
Speaker 1: at IBM, Like, what's the story to how you got
Speaker 1: where you are? Today. Oh goodness. My background is actually
Speaker 1: from the world of video games for entertainment, so AI
Speaker 1: has always been very interesting to me, especially when you
Speaker 1: intersect AI and play. But several years ago I began
Speaker 1: to get very frustrated by what I was reading in
Speaker 1: the news with respect to malintent through the use of AI.
Speaker 1: And the more that I learned and the more that
Speaker 1: I studied about this space of AI and ethics, the
Speaker 1: more I recognized that even organizations that have the very
Speaker 1: very best of intentions could inadvertently cause potential harm. And
Speaker 1: so that's super cool. I love that your interest in
Speaker 1: more responsible AI came from the gaming world. You have
Speaker 1: to talk a little bit about your history with gaming
Speaker 1: and that how that informed your interest and trustworthy AI. Well,
Speaker 1: it wasn't as much necessarily the ethical components of AI
Speaker 1: when I was working in games. It was more things like,
Speaker 1: look at what non player characters can do, you know,
Speaker 1: I mean, if you've got an AI acting as a
Speaker 1: character within the game, and how is it that you
Speaker 1: can use AI in order to make a game a
Speaker 1: more interesting experience. Actually ended up joining IBM to be
Speaker 1: our first global lead for something called serious games, which
Speaker 1: is when you use video games to do something other
Speaker 1: than just entertaining. And so the idea of integrating real
Speaker 1: data and real processes within sophisticated games powered by AI
Speaker 1: to solve complex problems. It wasn't until, as I mentioned,
Speaker 1: like later, when we started to hear all of us
Speaker 1: more and more news about just problems what could happen
Speaker 1: with respect to rendering or putting out models that are
Speaker 1: inaccurate or unfair. I know one of your inspirations for
Speaker 1: hearing other interviews that you've done is sci Fi. I'm
Speaker 1: also a sci Fi nerd, and I know sci Fi
Speaker 1: has talked a lot about, you know, the trustworthiness issues
Speaker 1: that come up when we're dealing with AI and so on,
Speaker 1: and so talk a little bit about how you bring
Speaker 1: that to your work in developing AI. That's a little
Speaker 1: bit more ethical. A lovely question. So, my my parents
Speaker 1: were major techno files. They both were immigrants to the
Speaker 1: United States, came here to study engineering, and they met
Speaker 1: UH in college. Growing up, my sister and I we
Speaker 1: had Star Trek playing every night. My parents were both
Speaker 1: big fans of Gene Roddenberry's vision of how technology could
Speaker 1: really be used to help better humankind, and that was
Speaker 1: the ethos that, of course we grew up in. The
Speaker 1: wonderful thing about science fiction isn't that it predicts cars,
Speaker 1: for example, but that it predicts traffic jams. You know.
Speaker 1: And I think there's just so much we can learn
Speaker 1: from science fiction, or in fact, like I said, play
Speaker 1: as a mechanism to be able to teach science fiction
Speaker 1: predicting traffic jams. I love it. But when we think
Speaker 1: about AI and science fiction, we need to be careful.
Speaker 1: We need to remember that AI is not something that's
Speaker 1: going to enter our lives at some point in the
Speaker 1: distant future. AI is something that's all around us today.
Speaker 1: If you have a virtual assistant in your house, that's AI,
Speaker 1: your phone app that predicts traffic AI. When a streaming
Speaker 1: service recommends a movie, you've guessed it AI, Paeder says.
Speaker 1: AI maybe behind the scenes determining the interest rate on
Speaker 1: your loan, or even whether or not you're the right
Speaker 1: candidate for that job you applied for. AI is both
Speaker 1: ubiquitous and invisible, which is why it is so crucial
Speaker 1: the companies learn how to build trustworthy AI. How do
Speaker 1: we do that? When thinking about what does it take
Speaker 1: to earn trust in something like an AI there are
Speaker 1: fundamentally human centric questions to be asked, right, like, what
Speaker 1: is the intent of this particular AI model? How accurate
Speaker 1: is that model? How fair is it? Is it explainable
Speaker 1: if it makes a decision that could directly affect my livelihood?
Speaker 1: Can I inquire what data did you use about me
Speaker 1: to make this decision? Is it protecting my data? Is
Speaker 1: it robust? Is it protected against people who could trick
Speaker 1: it to disadvantage me over others? I mean, there's so
Speaker 1: many questions to be asked. Earning trust in something like
Speaker 1: AI is fundamentally not a technological challenge but a socio
Speaker 1: technological challenge. It can't just be solved with a tool alone.
Speaker 1: What are the kinds of risks that companies have to
Speaker 1: think through? Is they're developing these technologies to make sure
Speaker 1: they're as trustworthy as possible. Well, you know, they may
Speaker 1: be putting a lot of money into investing in AI
Speaker 1: that gets stuck in proof of concept planned like it's
Speaker 1: get stuck in pilot. We we've done some research where
Speaker 1: we have found about eight percent of investments in AI
Speaker 1: get stuck, and sometimes it's because the investment isn't tied
Speaker 1: directly to a business strategy, or more often than not,
Speaker 1: people simply don't trust the results of the AI model.
Speaker 1: As a company, who is of course thinking about this
Speaker 1: so deeply. What do businesses need to consider when they're
Speaker 1: trying to figure out, you know, how to solve this
Speaker 1: big puzzle of AI ethics. It has to be approached holistically,
Speaker 1: So you've got to be thinking about, for example, what
Speaker 1: culture is required within your organization in order to really
Speaker 1: be able to responsibly create AI, what processes are in
Speaker 1: place to make sure that you're being compliant and that
Speaker 1: your your practitioners know what to do. And then of
Speaker 1: course AI engineering frameworks and tooling that can assist you
Speaker 1: on this journey. There is so much fundamentally to do.
Speaker 1: We found that actually those that were leading responsible AI
Speaker 1: trust where the AI initiatives within their organization has switched
Speaker 1: in the last three years. It used to be technical leaders,
Speaker 1: for example, chief data officer or someone who is a
Speaker 1: PhD in machine learning, and now it's switched to be
Speaker 1: a the percent of those leaders are now non technical
Speaker 1: business leaders, maybe you know, chief compliance officer, chief diversity
Speaker 1: inclusivity officers, chief legal officer. So we're seeing a shift,
Speaker 1: and I believe firmly. It's a recognition from organizations that
Speaker 1: are seeing that in order to really pull this off well,
Speaker 1: there has to be an investment than a focus in culture,
Speaker 1: in people and getting people to understand why they should
Speaker 1: care about this space. And so I see two challenges
Speaker 1: with doing that right. One is, you know a lot
Speaker 1: of these technology companies are really built to be tech companies,
Speaker 1: not necessarily you know, social tech companies or having this
Speaker 1: sort of training and ethics and beyond. Another issue seems
Speaker 1: to be that you're really proposing a switch that's truly holistic, right,
Speaker 1: that's like rethinking the way the company thinks about its
Speaker 1: bottom line. And so as you think about working through
Speaker 1: these kinds of challenges at IBM, how you tackled this,
Speaker 1: like how have you brought new talent in? How have
Speaker 1: you thought really carefully about this big holistic switch that
Speaker 1: needs to come to make AI more trustworthy. Data is
Speaker 1: an artifact of the human experience. And if you start
Speaker 1: with that as your definition and then think about well
Speaker 1: data is curated by data side this all data is
Speaker 1: biased and so if you're not recognizing bias with eyes
Speaker 1: fully open, then ultimately you're calcifying systemic bias into systems
Speaker 1: like AI. So some of the things that we've done
Speaker 1: at IBM again recognizing this important need for culture is big, big,
Speaker 1: big focus on diversity, not only looking at teams of
Speaker 1: data scientists and saying how many women are on this team,
Speaker 1: how many minorities are on this team, but also insisting
Speaker 1: on recognizing that we need to bring in people with
Speaker 1: different world views too, For example, what's your definition of fairness?
Speaker 1: Is your definition equality is an equity? Also bringing people
Speaker 1: with a wider variety of skill sets and roles, including
Speaker 1: our social scientists, anthropologists, sociologists, psychologists like yourself, right, behavioral scientists, designers.
Speaker 1: I mean we have one of the leading AI design
Speaker 1: practices in the world. I mean the effort, the investments
Speaker 1: we've been making in design thinking as a mechanism to
Speaker 1: create frameworks for systemic empathy well before any code is written,
Speaker 1: so people can think through, how would you design in
Speaker 1: order to mitigate for any potential harm given not only
Speaker 1: the values of your organization, but what are the rights
Speaker 1: of individuals asking oneself? These kinds of questions reinforces than
Speaker 1: the idea. The ethics doesn't come at the end like
Speaker 1: it's some kind of quality assurance, like check I passed
Speaker 1: the audit, I've got to go, you know. But instead, really,
Speaker 1: you know, as soon as you're thinking about using an
Speaker 1: AI for a particular use case, thinking about you know,
Speaker 1: what is the intent of this model, what's the relationship
Speaker 1: we ultimately want to have with AI? And again, these
Speaker 1: are non technology questions. This is where social scientists. Having
Speaker 1: a social scientist on your team helping think through these
Speaker 1: kinds of questions is is critical. Let's pause here for
Speaker 1: a second, because this is a really profound idea. Building
Speaker 1: responsible AI does not mean that you create a system
Speaker 1: then check in at the end and say is this okay?
Speaker 1: Is this ethical? If you don't ask those questions until
Speaker 1: the end of the process, you've already failed. You have
Speaker 1: to think about ethics from the jump from the makeup
Speaker 1: of the team to the data you're using to train
Speaker 1: the model to the most basic question of all, is
Speaker 1: this even the right use case artificial intelligence? The big
Speaker 1: lesson from IBM is this responsible AI is something you
Speaker 1: build at every step of the process. So this season
Speaker 1: of smart Talk is all focused on creativity and business.
Speaker 1: My guess is that thinking about trustworthy AI involves a
Speaker 1: lot of creativity. But talk to me about some of
Speaker 1: the spots where you see this work as being most creative.
Speaker 1: Oh goodness, I would say incorporating design design thinking in
Speaker 1: particular as well as straight up design in order to
Speaker 1: craft AI responsibly. You've used this word design thinking, and
Speaker 1: so I'm wondering exactly what you mean here? How do
Speaker 1: you define this idea of design thinking. Design thinking is
Speaker 1: a practice that we established here at IBM many years ago.
Speaker 1: In essence, what it is, it's a way of working
Speaker 1: with groups of people to co create a vision for something,
Speaker 1: for a product or a sir risk or an outcome.
Speaker 1: And typically it starts with things like, for example, empathy maps,
Speaker 1: like if you're thinking about an end user, thinking through
Speaker 1: what is this person thinking, seeing, hearing, feeling, like what
Speaker 1: are they experiencing in order to ultimately craft and experience
Speaker 1: for them that is targeted specifically for them. So we
Speaker 1: use it in a really wide variety of different ways
Speaker 1: with respect to trustworthy AI, even rendering an AI model
Speaker 1: explainable to a subject. And I'll give you an example.
Speaker 1: So we've got this wonderful program with an IBM caller,
Speaker 1: our Academy of Technology, and we take on initiatives that
Speaker 1: steer the company in innovative new directions. So we had
Speaker 1: an initiative where it was titled What the Titanic taught
Speaker 1: Us About Explainable AI, and the project was imagining if
Speaker 1: there was an AI mo utle that could predict the
Speaker 1: likelihood of a passenger getting a life raft on the Titanic.
Speaker 1: And we broke up into two work streams. One was
Speaker 1: the workstream full of the data scientists who were using
Speaker 1: all the different explainers to come up with the predictions
Speaker 1: and they would crank out the numbers. And the other
Speaker 1: team here's where the social scientists lived and the designers
Speaker 1: were right where we were thinking through how do we
Speaker 1: empower people? Well, how do we explain this algorithm and
Speaker 1: this predictor and the accuracy behind this prediction in such
Speaker 1: a way as to ultimately empower an end users? They
Speaker 1: could decide I'm not getting on that boat, or I
Speaker 1: want to get a second opinion please, or I went
Speaker 1: to contest the outputs of this model because I upgraded
Speaker 1: to first class just yesterday. See what I'm saying. And
Speaker 1: that takes a lot of creativity. How do you design
Speaker 1: and experience for someone in order to ultimately empower them.
Speaker 1: So design design design is critically critically important. And why
Speaker 1: I mentioned you know, we we've got to open up
Speaker 1: the aperture with respect to who we invite to the table,
Speaker 1: and these kinds of conversations. Taking the time to really
Speaker 1: understand other people's perspectives is so important when you're doing
Speaker 1: anything creative, and it is fundamental to the way the
Speaker 1: new creators work. The core question you should always be
Speaker 1: asking is where will the user be meeting this product?
Speaker 1: As Peder said, what will they be thinking, seeing, hearing, feeling.
Speaker 1: If you can answer those questions the way IBM does
Speaker 1: in its design thinking practice, you will be in great
Speaker 1: shape to create almost anything. Really, let's hear how it
Speaker 1: works in practice. And so we've been mostly talking kind
Speaker 1: of at the metal level about, you know, how to
Speaker 1: think about AI ethics generally. But of course the way
Speaker 1: this probably occurs in the trenches as a client approach
Speaker 1: as IBM, and they want to help with a specific
Speaker 1: problem in AI. And so I'm wondering, from a client
Speaker 1: based perspective, where do you start having some of these
Speaker 1: tough conversations. It has varied, to tell you the truth,
Speaker 1: we had one client that approached us to expand the
Speaker 1: use of an AI model to infer skill sets of
Speaker 1: their employees, but not just to infer their technical skills
Speaker 1: but also their soft foundational skills, meaning, let me use
Speaker 1: an AI to determine what kind of communicator you might
Speaker 1: be Laurie right. Others might come to us with, Okay,
Speaker 1: we recognize we need help setting an AI ethics board.
Speaker 1: Is this something you can assist us with? Or we
Speaker 1: have these values, we need to establish AI ethics principles
Speaker 1: and processes to help us ensure that we're compliant given
Speaker 1: regulations coming down the pike. Or we've had clients come
Speaker 1: to us saying, please train our people how to assess
Speaker 1: for unexpected patterns in an AI model, but then also
Speaker 1: how to holistically mitigate to prevent any potential harm. And
Speaker 1: those have been phenomenal engagements. They're huge learning moments. And
Speaker 1: so it seems like the real additional value that IBM
Speaker 1: is bringing through this process isn't necessarily just providing an
Speaker 1: AI algorithm or consulting on sam AI algorithm. It seems
Speaker 1: like the real value added is explaining how this design
Speaker 1: thinking works. You're almost like this therapist or like a
Speaker 1: really good bartender who talks to people, who talks whole
Speaker 1: companies through some of their problems to try to figure
Speaker 1: out where they're going astray before they start implementing these things.
Speaker 1: Can I put Chief Bartender Office on my metaphor, I'll
Speaker 1: tell you some of our our most valuable people on
Speaker 1: the team for that engagement. We had an industrial organization psychologist,
Speaker 1: we had an anthropologist. That's why I'm saying it's important
Speaker 1: we bring in the social scientists because you're exactly right,
Speaker 1: it's more than just scrutinizing the algorithm in its state.
Speaker 1: You have to be thinking about how is it being
Speaker 1: used holistically? And so if I was a business that
Speaker 1: was trying to think about how a company like IBM
Speaker 1: could come in and help out with more trustworthy AI,
Speaker 1: what would this process really look like. Well, what we're
Speaker 1: finding more often than not is that there'll be smaller
Speaker 1: teams within broader organizations that either have the responsibility of
Speaker 1: compliance and see the writing on the wall, or they've
Speaker 1: been the ones investing in AI and are trying to
Speaker 1: figure out how to get the rest of the organization
Speaker 1: on board with respect to things like setting up an
Speaker 1: ethics board or establishing principles or things like that. So
Speaker 1: some things that we've done help companies do this is
Speaker 1: we kick off engagements with what we called our our
Speaker 1: AI for leaders workshops. On the one hand, it's teaching
Speaker 1: why you should care, but on the other hand, it's
Speaker 1: meant to get people so excited across the organization that
Speaker 1: they want to raise their hand and say, I want
Speaker 1: to represent this part, like, for example, I want to
Speaker 1: be part of the ethics board as it is being
Speaker 1: stood up. The heart parts, not the tech. The hard
Speaker 1: part is human behavior. And I know I'm preaching to
Speaker 1: the choir given your background, it's so nice as a
Speaker 1: psychologist to hear this. I'm like snapping my fingers like
Speaker 1: peach exactly. The hard part is human behavior. So it's
Speaker 1: been like drinking from a fire hose. I mean in
Speaker 1: terms of the kinds of things that that we've all
Speaker 1: been learning, and there's still so much to learn. It
Speaker 1: really bugs me that those who are lucky enough to
Speaker 1: be able to take classes and things like data ethics
Speaker 1: or AI ethics self categorized as coders machine learning dissert
Speaker 1: data scientists. If we're living in a world where AI
Speaker 1: is fundamentally being used to make decisions that could directly
Speaker 1: affect our livelihoods. We need to know more, We need
Speaker 1: to have more literacy, and also make sure that there
Speaker 1: is a consistent message of accessibility such that we are
Speaker 1: saying you don't just have to be interested in coding,
Speaker 1: like you're interested in social justice or psychology or anthropologies.
Speaker 1: There's a seat at the table for you here because
Speaker 1: we desperately need you. We desperately need that kind of
Speaker 1: skill set. Just getting people to think about how do
Speaker 1: you design something given an empathy lens to protect people?
Speaker 1: I mean that, I think is such a crucial skill
Speaker 1: to learn. You know, one thing I love about your
Speaker 1: approaches that when you're talking to clients, you're almost doing
Speaker 1: what I'm doing is a professor, where you're kind of
Speaker 1: instructing students, getting them to think in different ways. But
Speaker 1: I know from my field that I wind up learning
Speaker 1: as much from students as I think sometimes they learned
Speaker 1: from me. And so I'm wondering what what you've learned
Speaker 1: in the process of helping so many businesses approach AI
Speaker 1: a little bit more ethically, Like, have there been insights
Speaker 1: that you've gotten through your interaction with clients and the
Speaker 1: challenges they've been facing. I'm learning with every single interaction.
Speaker 1: For example, in my mind, given the experiences that IBM
Speaker 1: has had with respect to setting up our principles are
Speaker 1: pillars ARII, ethics board. There's a process to follow, right
Speaker 1: if you're thinking about it like a book, these are
Speaker 1: the chapters in order to to optimize the approach, let's say,
Speaker 1: but sometimes we work with clients that say, I'm going
Speaker 1: to install this tool and I want to jump to
Speaker 1: chapter seven, and it's like, okay, you know, how how
Speaker 1: do we help navigate clients that want to skip over
Speaker 1: steps that we think are important. Another on is again
Speaker 1: the social scientists and bringing them into really push hard
Speaker 1: on what is the right context of where this data
Speaker 1: tell me the origin story? Again like really pushing us
Speaker 1: to think hard and with their perspective, you don't know,
Speaker 1: just constant, constant learning. Which is why one of the
Speaker 1: things we did at IBM is we've established something called
Speaker 1: our Center of Excellence where we said, you know what
Speaker 1: IBM or we don't care what your background is, we
Speaker 1: don't care who you are. If you're interested in this space,
Speaker 1: you can become a member. The Center of Excellence is
Speaker 1: a way in which we have not only projects people
Speaker 1: can join in order to get real life experience, but
Speaker 1: then also share back. Here's what we learned. We did
Speaker 1: this with this particular I had. Here was our epiphany,
Speaker 1: because if we're not sharing back and we're not constantly educating,
Speaker 1: then we're missing the opportunity to establish the right culture.
Speaker 1: Establishing the right culture to share what we're learning is
Speaker 1: so important. And so I wanted to end. But going
Speaker 1: back to where we started, you with your technofile family
Speaker 1: watching Star Trek. I think if we were to fast
Speaker 1: forward a couple of decades, we probably couldn't have imagined
Speaker 1: that we'd be in the place with AI generally where
Speaker 1: we are now, and especially as we think through more
Speaker 1: trustworthy AI. And so you know, with such change happening
Speaker 1: right now, with the fact that it's a fire hose
Speaker 1: that's gonna just get even more powerful over time, what
Speaker 1: do you think is next in this world of thinking
Speaker 1: through more trustworthy AI. I would say next is far
Speaker 1: more education, far more understanding, and we're starting to see
Speaker 1: that shift far more CEO saying, yeah, ethics has to
Speaker 1: be corridor our business. There's that, but there's a shift
Speaker 1: barely half of the CEO is in we're saying that
Speaker 1: a ethics was key or important to their business, and
Speaker 1: now you're saying the great majority so education, education, education,
Speaker 1: And again I would underscore making it far more accessible
Speaker 1: to far more people, which means it's not just our
Speaker 1: classes and higher ed institutions, it's our conferences, it's anytime
Speaker 1: we write white papers, anytime we publish articles, anytime we
Speaker 1: do podcasts like this. Right, the way we talk about
Speaker 1: this space has to be far more accessible and open
Speaker 1: and inviting two people with different roles, different skill sets,
Speaker 1: different worldviews, because else again we're just codifying our own bias. Well, Fature,
Speaker 1: I want to express my gratitude today for making AI
Speaker 1: a little bit more accessible to everyone. This has been
Speaker 1: such a delightful conversation. Thank you so much for joining
Speaker 1: me for it. The pleasure was mine. Looie, thank you
Speaker 1: for being the consummate host. I want to close by
Speaker 1: going back to that moment when Lorie suggested that Phedra
Speaker 1: was actually IBM's Chief Bartender Officer, not just because that's
Speaker 1: the best C suite title ever, but because it gets
Speaker 1: at what I think is the biggest, most important idea
Speaker 1: in today's episode, pedro Boiled it down into a single
Speaker 1: line when she said, the hard part is not the tech,
Speaker 1: the hard part is human behavior. Why is building AI
Speaker 1: so complicated? Because people are complicated. IBM believes that building
Speaker 1: trust into AI from the start can lead to better outcomes,
Speaker 1: and that to build trustworthy AI, you don't just need
Speaker 1: to think like a computer scientist. You need to think
Speaker 1: like a psychologist, like an anthropologist, You need to understand people.
Speaker 1: Smart Talks of IBM is produced by Molly Sosha, Alexandra Garratton,
Speaker 1: Royston Reserve, and Edith Russolo with Jacob Goldstein. We're edited
Speaker 1: by Jen Guerra. Our engineers are Jason gam Brell, Sarah
Speaker 1: Brugre and Ben Holliday. Theme song by Grandmascope. Special thanks
Speaker 1: to Carli Migliore, Andy Kelly, Kathy Callaghan and the eight
Speaker 1: Bar and IBM teams, as well as the Pushkin marketing team.
Speaker 1: Smart Talks with IBM is a production of Pushkin Industries
Speaker 1: and I Heart Media. To find more Pushkin podcasts, listen
Speaker 1: on the I Heart Radio app, Apple Podcasts, or wherever
Speaker 1: you listen to podcasts. I'm Malcolm Gladwell. This is a
Speaker 1: paid advertisement from IBM.