Speaker 1: Welcome to tech Stuff, a production from I Heart Radio.
Speaker 1: This season of Smart Talks with IBM is all about
Speaker 1: new creators, the developers, data scientists, c t o s
Speaker 1: and other visionaries creatively applying technology in business to drive change.
Speaker 1: They use their knowledge and creativity to develop better ways
Speaker 1: of working, no matter the industry. Join hosts from your
Speaker 1: favorite pushkin industries podcasts as they use their expertise to
Speaker 1: deepen these conversations, and of course Malcolm Gladwell will guide
Speaker 1: you through the season as your host and provide his
Speaker 1: thoughts and analysis along the way. Look out for new
Speaker 1: episodes of Smart Talks with IBM on the I Heart
Speaker 1: Radio app, Apple Podcasts, or wherever you get your podcasts,
Speaker 1: and learn more at IBM dot com slash smart talks. Hello, Hello,
Speaker 1: Welcome to Smart Talks with IBM, a podcast from Industries
Speaker 1: Ighart Radio and IBM. I'm Malcolm Glabbo. This season, we're
Speaker 1: talking to new creators, the developers, data scientists, c t
Speaker 1: o s and other visionaries who are creatively applying technology
Speaker 1: and business to drive change. Channeling their knowledge and expertise,
Speaker 1: they're developing more creative and effective solutions, no matter the industry.
Speaker 1: Our guest today is Padre Bonadius, trust in AI practice
Speaker 1: leader within IBM Consulting. Advocating for artificial intelligence built and
Speaker 1: deployed responsibly is no longer just a compliance issue, but
Speaker 1: a business imperative. Part of Phedre's job is to help
Speaker 1: companies identify potential risks and pitfalls way before any code
Speaker 1: is written. In today's show, you'll hear how Phaeder's team
Speaker 1: and IBM is approaching this challenge holistically and creatively. Phedre
Speaker 1: spoke with Dr Loris Santos, host of the Pushkin podcast
Speaker 1: The Happiness Lab. Laurie is a professor of psychology at
Speaker 1: Yale University and an expert on human cognition and the
Speaker 1: cognitive biases that impede better choices. Now let's get to
Speaker 1: the interview, Padre. I'm so excited that we get a
Speaker 1: chance to chat today. You know, just to start off,
Speaker 1: I'm wondering how did you get started in this role
Speaker 1: at IBM, Like, what's the story to how you got
Speaker 1: where you are today? Oh goodness. My background is actually
Speaker 1: from the world of video games for entertainment, so AI
Speaker 1: has always been very interesting to me, especially when you
Speaker 1: intersect AI and play but several years ago, I began
Speaker 1: to get very frustrated by what I was reading in
Speaker 1: the news with respect to malintent through the use of AI.
Speaker 1: And the more that I learned and the more that
Speaker 1: I studied about this space of AI and ethics, the
Speaker 1: more I recognize that even organizations that have the very
Speaker 1: very best of intentions could inadvertently cause potential harm. And
Speaker 1: so that's super cool. I love that your interest in
Speaker 1: more responsible AI came from the gaming world. You have
Speaker 1: to talk a little bit about your history with gaming
Speaker 1: and that how that informed your interest in trustworthy AI. Well,
Speaker 1: it wasn't as much necessarily the ethical components of AI
Speaker 1: when I was working in games. It was more things like,
Speaker 1: look at what nonplayer characters can do, you know, I mean,
Speaker 1: if you've got an AI acting as a character within
Speaker 1: the game, and how is it that you can use
Speaker 1: AI in order to make a game a more interesting experience. Actually,
Speaker 1: I ended up joining IBM to be our first global
Speaker 1: lead for something called serious games, which is when you
Speaker 1: use video games to do something other than just entertaining,
Speaker 1: and of the idea of integrating real data and real
Speaker 1: processes within sophisticated games powered by AI to solve complex problems.
Speaker 1: It wasn't until, as I mentioned, like later, when we
Speaker 1: started to hear all of us more and more news
Speaker 1: about just problems. What could happen with respect to rendering
Speaker 1: or putting out models that are inaccurate or unfair. I
Speaker 1: know one of your inspirations for hearing other interviews that
Speaker 1: you've done is sci Fi. I'm also a sci Fi nerd,
Speaker 1: and I know sci Fi has talked a lot about,
Speaker 1: you know, the trustworthiness issues that come up when we're
Speaker 1: dealing with AI and so on, and so talk a
Speaker 1: little bit about how you bring that to your work
Speaker 1: in developing AI. That's a little bit more ethical. A
Speaker 1: lovely question. So, my my parents were major techno files.
Speaker 1: They both were immigrants to the United States, came here
Speaker 1: to study engineering, and they met uh in college. Growing up,
Speaker 1: my sister and I, we had Star Trek playing every night. UH.
Speaker 1: My parents were both big fans of Gene Roddenberry's vision
Speaker 1: of how technology could really be used to help better humankind,
Speaker 1: and that was the ethos that, of course we grew
Speaker 1: up in. The wonderful thing about science fiction isn't that
Speaker 1: it predicts cars, for example, but that it predicts traffic jams,
Speaker 1: you know, and I think there's just so much we
Speaker 1: can learn from science fiction or in fact, like I said,
Speaker 1: play as a mechanism to be able to teach science
Speaker 1: fiction predicting traffic jams. I love it. But when we
Speaker 1: think about AI and science fiction, we need to be careful.
Speaker 1: We need to remember that AI is not something that's
Speaker 1: going to enter our lives at some point in the
Speaker 1: distant future. AI is something that's all around us today.
Speaker 1: If you have a virtual assistant in your house, that's AI,
Speaker 1: your phone app that predicts traffic AI. What a streaming
Speaker 1: service recommends a movie? You've guessed it AI, Paeder says.
Speaker 1: AI maybe behind the scenes determining the interest rate on
Speaker 1: your loan, or even whether or not you're the right
Speaker 1: candidate for that job you applied for. AI is both
Speaker 1: ubiquitous and invisible, which is why it is so crucial
Speaker 1: the companies learn how to build trustworthy AI. How do
Speaker 1: we do that? When thinking about what does it take
Speaker 1: to earn trust in something like an AI, there are
Speaker 1: fundamentally human centric questions to be asked, right like what
Speaker 1: is the intent of this particular AI model? How accurate
Speaker 1: is that model? How fair is it Is it explainable
Speaker 1: if it makes a decision that could directly affect my livelihood?
Speaker 1: Can I inquire what data did you use about me
Speaker 1: to make this decision? Is it protecting my data? Is
Speaker 1: it robust? Is it protected against people who could trick
Speaker 1: it to disadvantage me over others? I mean, there's so
Speaker 1: many questions to be asked. Earning trust in something like
Speaker 1: AI is fundamentally not a technological challenge but a socio
Speaker 1: technological challenge. It can't just be solved with a tool alone.
Speaker 1: What are the kinds of risks that companies have to
Speaker 1: think through? Is they're developing these technologies to make sure
Speaker 1: they're as trustworthy as possible. Well, you know, they may
Speaker 1: be putting a lot of money into investing in AI
Speaker 1: that gets stuck in proof of concept planned like get's
Speaker 1: get stuck in pilot. We we've done some research where
Speaker 1: we have found about eight of investments in AI get stuck.
Speaker 1: And sometimes it's because the investment isn't tied directly to
Speaker 1: a business strategy, or more often than not, people simply
Speaker 1: don't trust the results of the AI model. As a company,
Speaker 1: who is of course thinking about this so deeply? What
Speaker 1: a businesses need to consider when they're trying to figure out,
Speaker 1: you know, how to solve this big puzzle of AI ethics.
Speaker 1: It has to be approached holistically, So you've got to
Speaker 1: be thinking about, for example, what culture is required within
Speaker 1: your organization in order to really be able to responsibly
Speaker 1: create AI, what processes are in place to make sure
Speaker 1: that you're being compliant and that your your practitioners know
Speaker 1: what to do. And then of course AI engineering frameworks
Speaker 1: and tooling that can assist you on this journey. There
Speaker 1: is so much fundamentally to do. We found that actually
Speaker 1: those that were leading responsible AI trust where the AI
Speaker 1: initiatives within their organization has switched in the last three years.
Speaker 1: It used to be technical leaders, for example, chief data
Speaker 1: officer or someone who is a PhD in machine learning,
Speaker 1: and now it's switched to be a d percent of
Speaker 1: those leaders are now non technical business leaders maybe you know,
Speaker 1: chief compliance officer, diversely inclusivity officers, chief legal officer. So
Speaker 1: we're seeing a shift, and I believe firmly it's a
Speaker 1: recognition from organizations that are seeing that in order to
Speaker 1: really pull this off well, there has to be an
Speaker 1: investment than a focus in culture in people and getting
Speaker 1: people to understand why they should care about this space.
Speaker 1: And so I see two challenges with doing that right.
Speaker 1: One is, you know a lot of these technology companies
Speaker 1: are really built to be tech companies, not necessarily you know,
Speaker 1: social tech companies or having this sort of training and
Speaker 1: ethics and beyond. Another issue seems to be that you're
Speaker 1: really proposing a switch that's truly holistic, right, that's like
Speaker 1: rethinking the way the company thinks about its bottom line.
Speaker 1: And so as you think about working through these kinds
Speaker 1: of challenges at IBM, how have you tackled this, like
Speaker 1: how have you brought new talent in? How have you
Speaker 1: thought really carefully about this big holistic switch that needs
Speaker 1: to come to make AIM more trustworth be Data is
Speaker 1: an artifact of the human experience. And if you start
Speaker 1: with that as your definition and then think about, well
Speaker 1: data is curated by data sideists, all data is biased.
Speaker 1: And so if you're not recognizing bias with eyes fully open,
Speaker 1: then ultimately you're calcifying systemic bias into systems like AI.
Speaker 1: So some of the things that we've done at IBM,
Speaker 1: again recognizing this important need for culture is big, big,
Speaker 1: big focus on diversity, not only looking at teams of
Speaker 1: data scientists and saying how many women are on this team,
Speaker 1: how many minorities are on this team, but also insisting
Speaker 1: on recognizing that we need to bring in people with
Speaker 1: different world views too, for example, what's your definition of fairness?
Speaker 1: Is your definition equality or is it equity? Also bringing
Speaker 1: people with a wider variety of skill sets and roles,
Speaker 1: including our social scientists, anthropologists, sociologist, psychologists like yourself, right,
Speaker 1: behavioral scientists, designers. I mean we have one of the
Speaker 1: leading AI design practices in the world. I mean the effort,
Speaker 1: the investments we've been making in design thinking as a
Speaker 1: mechanism to create frameworks for systemic empathy well before any
Speaker 1: code is written, so people can think through how would
Speaker 1: you design in order to mitigate for any potential harm
Speaker 1: given not only the values of your organization, but what
Speaker 1: are the rights of individuals asking oneself? These kinds of
Speaker 1: questions reinforces then the idea the ethics doesn't come at
Speaker 1: the end, like it's some kind of quality assurance, like
Speaker 1: check I passed the audit, I've got to go, you know,
Speaker 1: But instead, really, you know, as soon as you're thinking
Speaker 1: about using an AI for a particular use case thinking
Speaker 1: about you know, what is the intent of this model,
Speaker 1: what's the relationship we ultimately want to have with AI?
Speaker 1: And again, these are non technology questions. This is where
Speaker 1: social scientists. Having a social scientist on your team helping
Speaker 1: think through these kinds of questions is is critical. Let's
Speaker 1: pause here for a second, because this is a really
Speaker 1: profound idea. Building responsible AI does not mean that you
Speaker 1: create a system then check in at the end and
Speaker 1: say is this okay? Is this ethical? If you don't
Speaker 1: ask those questions until the end of the process, you've
Speaker 1: already failed. You have to think about ethics from the
Speaker 1: jump from the makeup of the team to the data
Speaker 1: you're using to train the model to the most basic
Speaker 1: question of all, is this even the right use case
Speaker 1: for artificial intelligence? The big lesson from IBM is this
Speaker 1: responsible AI is something you build at every step of
Speaker 1: the process. So this season of smart Talk is all
Speaker 1: focused on creativity and business. My guess is that thinking
Speaker 1: about trustworthy AI involves a lot of creativity. But talk
Speaker 1: to me about some of the spots where you see
Speaker 1: this work as being most creative. Oh goodness, I would
Speaker 1: say incorporating design design thinking in particular as well as
Speaker 1: straight up design in order to craft AI responsibly. You've
Speaker 1: used this word design thinking, and so I'm wondering exactly
Speaker 1: what you mean here. How do you define this idea
Speaker 1: of design thinking. Design thinking is a practice that we
Speaker 1: established here at IBM many years ago. In essence, what
Speaker 1: it is, it's a way of working with groups of
Speaker 1: people to co create a vision for something, for a
Speaker 1: product or a service or an outcome. And typically it
Speaker 1: starts with things like, for example, empathy maps. If you're
Speaker 1: thinking about an end user, thinking through what is this
Speaker 1: person thinking, seeing, hearing, feeling like, what are the experiencing
Speaker 1: in order to ultimately craft and experience for them that
Speaker 1: is targeted specifically for them. So we use it in
Speaker 1: a really wide variety of different ways with respect to
Speaker 1: trustworthy AI, even rendering an AI model explainable to a subject.
Speaker 1: And I'll give you an example. So we've got this
Speaker 1: wonderful program with an IBM caller, our Academy of Technology,
Speaker 1: and we take on initiatives that steer the company in
Speaker 1: innovative new directions. So we had an initiative where it
Speaker 1: was titled what the Titanic taught Us About explainable AI,
Speaker 1: and the project was imagining if there was an AI
Speaker 1: model that could predict the likelihood of a passenger getting
Speaker 1: a life raft on the Titanic. And we broke up
Speaker 1: into two work streams. One was the workstream full of
Speaker 1: the data scientists who were using all the different explainers
Speaker 1: to come up with the predictions and they would crank
Speaker 1: out the numbers. And the other team here's where the
Speaker 1: social scientists lived and the designers were right where we
Speaker 1: were thinking through how do we empower people? How do
Speaker 1: we explain this algorithm and this predictor and the accuracy
Speaker 1: behind this prediction in such a way as to ultimately
Speaker 1: empower and end users. They could decide I'm not getting
Speaker 1: on that boat, or I want to get a second
Speaker 1: opinion please, or I want to contest the outputs of
Speaker 1: this model because I upgraded to first class just yesterday.
Speaker 1: See what I'm saying. And that takes a lot of creativity.
Speaker 1: How do you design an experience for someone in order
Speaker 1: to ultimately empower them? So design designed as is critically
Speaker 1: critically important. And why I mentioned you know, we we've
Speaker 1: got to open up the aperture with respect to who
Speaker 1: we invite to the table and these kinds of conversations.
Speaker 1: Taking the time to really understand other people's perspectives is
Speaker 1: so important when you're doing anything creative, and it is
Speaker 1: fundamental to the way the new creators work. The core
Speaker 1: question you should always be asking is where will the
Speaker 1: user be meeting this product? As Peder said, what will
Speaker 1: they be thinking, seeing, hearing, feeling. If you can answer
Speaker 1: those questions the way IBM does in its design thinking practice,
Speaker 1: you will be in great shape to create almost anything. Really,
Speaker 1: let's hear how it works in practice. And so we've
Speaker 1: been mostly talking kind of at the metal level about
Speaker 1: you know, how to think about AI ethics generally, but
Speaker 1: of course the way this probably occurs in the trenches
Speaker 1: as a client approach as IBM and they want help
Speaker 1: with the specific problem in AI. And so I'm wondering,
Speaker 1: from a client based perspective, where do you start having
Speaker 1: some of these tough conversations. It has varied, to tell
Speaker 1: you the truth, We had one client that approached us
Speaker 1: to expand the use of an AI model to infer
Speaker 1: skill sets of their employees, but not just to infer
Speaker 1: their technical skills, but also their soft foundational skills, meaning
Speaker 1: let me use an AI to determine what kind of
Speaker 1: communicator you might be A Laurie right. Others might come
Speaker 1: to us with, Okay, we recognize we need help setting
Speaker 1: an AI ethics board. Is this something you can assist
Speaker 1: us with? Or we have these values, we need to
Speaker 1: establish AI ethics principles and processes to help us ensure
Speaker 1: that we're compliant given regulations coming down the pike. Or
Speaker 1: we've had clients come to us saying, please train our
Speaker 1: people how to assess for unexpected patterns in an aim
Speaker 1: at all, but then also how to holistically mitigate to
Speaker 1: prevent any potential harm. And those have been phenomenal engagements.
Speaker 1: They're huge learning moments. And so it seems like the
Speaker 1: real additional value that IBM is bringing through this process
Speaker 1: isn't necessarily just providing an AI algorithm or consulting on
Speaker 1: sam AI algorithm. It seems like the real value added
Speaker 1: is explaining how this design thinking works. You're almost like
Speaker 1: this therapist or like a really good bartender who talks
Speaker 1: to people, who talks whole companies through some of their
Speaker 1: problems to try to figure out where they're going astray
Speaker 1: before they start implementing. These things can I put Chief
Speaker 1: Bartender Office on my I like the metaphor. I'll tell
Speaker 1: you some of our our most valuable people on the
Speaker 1: team for that engagement. We had an industrial organizational psychologist,
Speaker 1: we had an anthropologist. That's why I'm saying it's important
Speaker 1: that we bring in the social scientists because you're exactly right,
Speaker 1: it's more than just scrutinizing the algorithm in its state.
Speaker 1: You have to be thinking about how is it being
Speaker 1: used holistically? And so if I was a business that
Speaker 1: was trying to think about how a company like IBM
Speaker 1: could come in and help out with more trustworthy AI,
Speaker 1: what would this process really look like. Well, what we're
Speaker 1: finding more often than not is that there'll be smaller
Speaker 1: teams within broader organizations that either have the responsibility of
Speaker 1: compliance and see the writing on the wall, or they've
Speaker 1: been the ones investing in AI and are trying to
Speaker 1: figure out how to get the rest of the organization
Speaker 1: on board with respect to things like setting up an
Speaker 1: ethics board or establishing principles or things like that. So
Speaker 1: some things that we've done to help companies do this
Speaker 1: is we kick off engagements with what we called our
Speaker 1: our AI for leaders workshops On the one hand, it's
Speaker 1: teaching why you should care, But on the other hand,
Speaker 1: it's meant to get people so excited across the organization
Speaker 1: that they want to raise their hand and say, I
Speaker 1: want to represent this part, like, for example, I want
Speaker 1: to be part of the ethics board as it is
Speaker 1: being stood up. The heart parts, not the tech. The
Speaker 1: hard part is human behavior. And I know I'm preaching
Speaker 1: to the choir given your background, it's so nice as
Speaker 1: a psychologist to hear this. I'm like snapping my fingers,
Speaker 1: like preach exactly. The hard part is human behavior. So
Speaker 1: it's been like drinking from a fire hose. I mean
Speaker 1: in terms of the kinds of things that that we've
Speaker 1: all been learning, and there's still so much to learn.
Speaker 1: It really bugs me that those who are lucky enough
Speaker 1: to be able to take classes and things like data
Speaker 1: ethics or AI ethics self categorized as coders, machine learning scientists,
Speaker 1: or data scientists. If we're living in a world where
Speaker 1: AI is fundamentally being used to make decisions that could
Speaker 1: directly affect our livelihoods, we need to know more. We
Speaker 1: need to have more literacy and also make sure that
Speaker 1: there is a consistent message of accessibility such that we
Speaker 1: are saying you don't just have to be interested in coding,
Speaker 1: like you're interested in social justice or psychology or anthropologies.
Speaker 1: There's a seat at the table for you here because
Speaker 1: we desperately need you. We desperately need that kind of
Speaker 1: skill set. Just getting people to think about how do
Speaker 1: you design something given an empathy lens to protect people?
Speaker 1: I mean that, I think is such a crucial skill
Speaker 1: to learn. You know, one thing I love about your
Speaker 1: approaches that when you're talking to clients, you're almost doing
Speaker 1: what I'm doing is a professor, where you're kind of
Speaker 1: instructing students, getting them to think in different ways. But
Speaker 1: I know from my field that I wind up learning
Speaker 1: as much from students as I think sometimes they learned
Speaker 1: from me. And so I'm wondering what what you've learned
Speaker 1: in the process of helping so many businesses approach AI
Speaker 1: a little bit more ethically, Like, have there been insights
Speaker 1: that you've gotten through your interaction with clients and the
Speaker 1: challenges they've been facing. I'm learning with every single interaction.
Speaker 1: For example, in my mind, given the experiences that IBM
Speaker 1: has had with respect to setting up our principles are
Speaker 1: pillars Arii Ethics Board. There's a process to follow, right
Speaker 1: if you're thinking about it like a book, these are
Speaker 1: the chapters in order to to optimize the approach. Let's say,
Speaker 1: but sometimes we work with clients that say I'm gonna
Speaker 1: install this tool and I want to jump to chapter seven,
Speaker 1: and it's like, okay, you know, how how do we
Speaker 1: help navigate clients that want to skip over steps that
Speaker 1: we think are important. Another one is again the social
Speaker 1: scientists and bringing them into really push hard on what
Speaker 1: is the right context where this data tell me the
Speaker 1: origin story? Again like really pushing us to think hard
Speaker 1: and with their perspective, you don't know, just constant, constant learning,
Speaker 1: which is why one of the things we did at
Speaker 1: IBM is we've established something called our Center of Excellence,
Speaker 1: where we said, you know what IBM or we don't
Speaker 1: care what your background is, we don't care who you are.
Speaker 1: If you're interested in this space, you can become a member.
Speaker 1: The Center of Excellence is a way in which we
Speaker 1: have not only projects people can join in order to
Speaker 1: get real life experience, but then also share back here's
Speaker 1: what we learned. We did this with this particular and
Speaker 1: here was our epiphany, because if we're not sharing back
Speaker 1: and we're not constantly educating, then we're missing the opportunity
Speaker 1: to establish the right culture. Establishing the right culture to
Speaker 1: share what we're learning is so important. So I wanted
Speaker 1: to But going back to where we started, you with
Speaker 1: your technofile family watching Star Trek, I think if we
Speaker 1: were to fast forward a couple of decades, we probably
Speaker 1: couldn't have imagined that we'd be in the place with
Speaker 1: AI generally where we are now, and especially as we
Speaker 1: think through more trustworthy AI. And so you know, with
Speaker 1: such change happening right now, with the fact that it's
Speaker 1: a fire hose that's gonna just get even more powerful
Speaker 1: over time, what do you think is next in this
Speaker 1: world of thinking through more trustworthy AI. I would say
Speaker 1: next is far more education, far more understanding. And we're
Speaker 1: starting to see that shift, far more CEO saying yeah,
Speaker 1: ethics has to be corrid or business. There's that, but
Speaker 1: there's a shift. Barely half of the CEO is we're
Speaker 1: saying that a ethics was key or important to their business.
Speaker 1: And now you're saying the great majority so education, education, education,
Speaker 1: And again I would underscore making it far more accessible
Speaker 1: to far more people, which means it's not just our
Speaker 1: classes in higher ed institutions, it's our conferences, it's anytime
Speaker 1: we write white papers, anytime we publish articles, anytime we
Speaker 1: do podcasts like this. Right the way we talk about
Speaker 1: this space has to be far more accessible and open
Speaker 1: and inviting two people with different roles, different skill sets,
Speaker 1: different worldviews, because else again, we're just codifying our own bias. Well, feature,
Speaker 1: I want to express my gratitude today for making AI
Speaker 1: a little bit more accessible to everyone. This has been
Speaker 1: such a delightful conversation. Thank you so much for joining
Speaker 1: me for it. The pleasure was mine. Looie, thank you
Speaker 1: for being the consummate host. Thank you. I want to
Speaker 1: close by going back to that moment when Lorie suggested
Speaker 1: that Phedra was actually IBM's Chief Bartender Officer, not just
Speaker 1: because that's the best C suite title ever, but because
Speaker 1: gets at what I think is the biggest, most important idea.
Speaker 1: In today's episode, Pedro boiled it down into a single
Speaker 1: line when she said, the hard part is not the tech.
Speaker 1: The hard part is human behavior. Why is building AI
Speaker 1: so complicated? Because people are complicated. IBM believes that building
Speaker 1: trust into AI from the start can lead to better outcomes,
Speaker 1: and that to build trustworthy AI, you don't just need
Speaker 1: to think like a computer scientist. You need to think
Speaker 1: like a psychologist, like an anthropologist. You need to understand people.
Speaker 1: Smart Talks with IBM is produced by Molly Sosha, Alexandra Garraton,
Speaker 1: Royston Reserve and Edith Russolo with Jacob Goldstein. We're edited
Speaker 1: by Jan Guerra. Our engineers are Jason Gambrel, Sarah Brugere
Speaker 1: and Ben Holiday theme song by Grandmascope. Special thanks to
Speaker 1: Carlie Migliori, Andy Kelly, Kathy Callaghan and the eight Bar
Speaker 1: and IBM teams, as well as the Pushkin marketing team.
Speaker 1: Smart Talks with IBM is a production of Pushkin Industries
Speaker 1: and i Heart Media. To find more Pushkin podcasts, listen
Speaker 1: on the i Heart Radio app, Apple Podcasts, or wherever
Speaker 1: you listen to podcasts. I'm Malcolm Gladwell. This is a
Speaker 1: paid advertisement from IBM.