WEBVTT - Smart Talks with IBM: Building Trustworthy AI: A Holistic Approach 0:00:00.040 --> 0:00:05.000 Hey everyone, it's Robert and Joe here. Today we've got 0:00:05.040 --> 0:00:06.960 something a little bit different to share with you. It 0:00:07.080 --> 0:00:10.680 is a new edition of the Smart Talks podcast series, 0:00:10.720 --> 0:00:14.319 which is produced in partnership with IBM. This season of 0:00:14.360 --> 0:00:18.640 Smart Talks with IBM is all about new creators, the developers, 0:00:19.040 --> 0:00:22.600 data scientists, c t o s, and other visionaries creatively 0:00:22.640 --> 0:00:27.120 applying technology and business to drive change. They use their 0:00:27.160 --> 0:00:30.640 knowledge and creativity to develop better ways of working, no 0:00:30.720 --> 0:00:34.840 matter the industry. Join hosts from your favorite Pushkin Industries 0:00:34.920 --> 0:00:38.560 podcast as they use their expertise to deepen these conversations. 0:00:39.040 --> 0:00:41.800 Malcolm Gladwell will guide you through this season as your 0:00:41.840 --> 0:00:45.000 host to provide his thoughts and analysis along the way. 0:00:45.320 --> 0:00:48.600 Look out for new episodes of Smart Talks with IBM 0:00:48.680 --> 0:00:52.040 every month on the I Heart Radio app, Apple Podcasts, 0:00:52.159 --> 0:00:55.480 or wherever you get your podcasts. And learn more at 0:00:55.520 --> 0:01:04.480 IBM dot com slash smart Talks. Hello, Hello, Welcome to 0:01:04.560 --> 0:01:08.400 Smart Talks with IBM, a podcast from Pushkin Industries, I 0:01:08.600 --> 0:01:13.040 Heart Radio and IBM. I'm Malcolm Babbo. This season, we're 0:01:13.040 --> 0:01:17.440 talking to new creators, the developers, data scientists, c t 0:01:17.600 --> 0:01:21.360 o s, and other visionaries who are creatively applying technology 0:01:21.400 --> 0:01:25.559 and business to drive change. Channeling their knowledge and expertise, 0:01:25.760 --> 0:01:30.400 they're developing more creative and effective solutions, no matter the industry. 0:01:31.319 --> 0:01:35.800 Our guest today is Padre Bonadius, trust in AI Practice 0:01:35.880 --> 0:01:41.480 leader within IBM Consulting. Advocating for artificial intelligence built and 0:01:41.560 --> 0:01:46.280 deployed responsibly is no longer just a compliance issue, but 0:01:46.440 --> 0:01:50.440 a business imperative. Part of Phaedre's job is to help 0:01:50.480 --> 0:01:55.280 companies identify potential risks and pitfalls way before any code 0:01:55.320 --> 0:01:58.760 is written. In today's show, you'll hear how Phaedre's team 0:01:58.760 --> 0:02:02.840 and IBM is a pro genus challenge holistically and creatively. 0:02:03.560 --> 0:02:06.920 Pedra spoke with Dr Lois Santos, host of the Pushkin 0:02:07.000 --> 0:02:11.160 podcast The Happiness Lab. Laurie is a professor of psychology 0:02:11.200 --> 0:02:14.760 at Yale University and an expert on human cognition and 0:02:14.800 --> 0:02:21.000 the cognitive biases that impede better choices. Now let's get 0:02:21.040 --> 0:02:29.919 to the interview, Pedro. I'm so excited that we get 0:02:29.919 --> 0:02:32.799 a chance to chat today. You know, just to start off, 0:02:32.840 --> 0:02:35.560 I'm wondering how did you get started in this role 0:02:35.560 --> 0:02:37.760 at IBM, Like, what's the story to how you got 0:02:37.760 --> 0:02:41.160 where you are? Today. Oh goodness. My background is actually 0:02:41.760 --> 0:02:45.239 from the world of video games for entertainment, so AI 0:02:45.400 --> 0:02:48.280 has always been very interesting to me, especially when you 0:02:48.280 --> 0:02:52.400 intersect AI and play. But several years ago I began 0:02:52.440 --> 0:02:57.480 to get very frustrated by what I was reading in 0:02:57.520 --> 0:03:03.799 the news with respect to malintent through the use of AI. 0:03:04.000 --> 0:03:06.960 And the more that I learned and the more that 0:03:07.040 --> 0:03:11.160 I studied about this space of AI and ethics, the 0:03:11.200 --> 0:03:15.800 more I recognized that even organizations that have the very 0:03:16.160 --> 0:03:23.160 very best of intentions could inadvertently cause potential harm. And 0:03:23.280 --> 0:03:25.520 so that's super cool. I love that your interest in 0:03:25.720 --> 0:03:29.280 more responsible AI came from the gaming world. You have 0:03:29.360 --> 0:03:31.480 to talk a little bit about your history with gaming 0:03:31.480 --> 0:03:34.880 and that how that informed your interest and trustworthy AI. Well, 0:03:35.240 --> 0:03:40.400 it wasn't as much necessarily the ethical components of AI 0:03:40.440 --> 0:03:43.680 when I was working in games. It was more things like, 0:03:44.600 --> 0:03:48.160 look at what non player characters can do, you know, 0:03:48.400 --> 0:03:50.320 I mean, if you've got an AI acting as a 0:03:50.400 --> 0:03:53.240 character within the game, and how is it that you 0:03:53.280 --> 0:03:55.640 can use AI in order to make a game a 0:03:55.680 --> 0:04:00.160 more interesting experience. Actually ended up joining IBM to be 0:04:00.200 --> 0:04:03.280 our first global lead for something called serious games, which 0:04:03.280 --> 0:04:05.119 is when you use video games to do something other 0:04:05.200 --> 0:04:08.560 than just entertaining. And so the idea of integrating real 0:04:08.640 --> 0:04:12.440 data and real processes within sophisticated games powered by AI 0:04:12.600 --> 0:04:16.960 to solve complex problems. It wasn't until, as I mentioned, 0:04:17.080 --> 0:04:20.240 like later, when we started to hear all of us 0:04:20.279 --> 0:04:25.440 more and more news about just problems what could happen 0:04:25.480 --> 0:04:28.080 with respect to rendering or putting out models that are 0:04:28.080 --> 0:04:31.680 inaccurate or unfair. I know one of your inspirations for 0:04:31.720 --> 0:04:34.160 hearing other interviews that you've done is sci Fi. I'm 0:04:34.200 --> 0:04:36.320 also a sci Fi nerd, and I know sci Fi 0:04:36.400 --> 0:04:40.160 has talked a lot about, you know, the trustworthiness issues 0:04:40.200 --> 0:04:42.360 that come up when we're dealing with AI and so on, 0:04:42.680 --> 0:04:44.520 and so talk a little bit about how you bring 0:04:44.560 --> 0:04:46.960 that to your work in developing AI. That's a little 0:04:46.960 --> 0:04:51.000 bit more ethical. A lovely question. So, my my parents 0:04:51.160 --> 0:04:54.640 were major techno files. They both were immigrants to the 0:04:54.760 --> 0:04:57.839 United States, came here to study engineering, and they met 0:04:58.600 --> 0:05:01.680 UH in college. Growing up, my sister and I we 0:05:01.760 --> 0:05:08.400 had Star Trek playing every night. My parents were both 0:05:08.560 --> 0:05:12.960 big fans of Gene Roddenberry's vision of how technology could 0:05:12.960 --> 0:05:17.440 really be used to help better humankind, and that was 0:05:17.600 --> 0:05:21.279 the ethos that, of course we grew up in. The 0:05:21.360 --> 0:05:25.960 wonderful thing about science fiction isn't that it predicts cars, 0:05:26.120 --> 0:05:29.840 for example, but that it predicts traffic jams. You know. 0:05:30.480 --> 0:05:32.880 And I think there's just so much we can learn 0:05:33.640 --> 0:05:36.840 from science fiction, or in fact, like I said, play 0:05:36.880 --> 0:05:40.240 as a mechanism to be able to teach science fiction 0:05:40.360 --> 0:05:44.760 predicting traffic jams. I love it. But when we think 0:05:44.760 --> 0:05:48.040 about AI and science fiction, we need to be careful. 0:05:48.680 --> 0:05:51.719 We need to remember that AI is not something that's 0:05:51.720 --> 0:05:53.760 going to enter our lives at some point in the 0:05:53.880 --> 0:05:59.200 distant future. AI is something that's all around us today. 0:05:59.480 --> 0:06:03.200 If you have a virtual assistant in your house, that's AI, 0:06:03.480 --> 0:06:07.480 your phone app that predicts traffic AI. When a streaming 0:06:07.480 --> 0:06:12.240 service recommends a movie, you've guessed it AI, Paeder says. 0:06:12.320 --> 0:06:16.400 AI maybe behind the scenes determining the interest rate on 0:06:16.480 --> 0:06:19.400 your loan, or even whether or not you're the right 0:06:19.440 --> 0:06:23.119 candidate for that job you applied for. AI is both 0:06:23.320 --> 0:06:27.520 ubiquitous and invisible, which is why it is so crucial 0:06:27.720 --> 0:06:32.080 the companies learn how to build trustworthy AI. How do 0:06:32.160 --> 0:06:35.320 we do that? When thinking about what does it take 0:06:35.440 --> 0:06:39.839 to earn trust in something like an AI there are 0:06:39.920 --> 0:06:44.120 fundamentally human centric questions to be asked, right, like, what 0:06:44.279 --> 0:06:47.720 is the intent of this particular AI model? How accurate 0:06:47.880 --> 0:06:52.000 is that model? How fair is it? Is it explainable 0:06:52.080 --> 0:06:55.400 if it makes a decision that could directly affect my livelihood? 0:06:56.120 --> 0:06:59.000 Can I inquire what data did you use about me 0:06:59.279 --> 0:07:03.400 to make this decision? Is it protecting my data? Is 0:07:03.440 --> 0:07:07.760 it robust? Is it protected against people who could trick 0:07:07.839 --> 0:07:11.200 it to disadvantage me over others? I mean, there's so 0:07:11.280 --> 0:07:15.680 many questions to be asked. Earning trust in something like 0:07:15.800 --> 0:07:21.240 AI is fundamentally not a technological challenge but a socio 0:07:21.240 --> 0:07:26.560 technological challenge. It can't just be solved with a tool alone. 0:07:27.880 --> 0:07:29.880 What are the kinds of risks that companies have to 0:07:29.920 --> 0:07:32.720 think through? Is they're developing these technologies to make sure 0:07:32.720 --> 0:07:35.520 they're as trustworthy as possible. Well, you know, they may 0:07:35.560 --> 0:07:39.280 be putting a lot of money into investing in AI 0:07:39.720 --> 0:07:42.680 that gets stuck in proof of concept planned like it's 0:07:42.720 --> 0:07:45.000 get stuck in pilot. We we've done some research where 0:07:45.000 --> 0:07:48.440 we have found about eight percent of investments in AI 0:07:48.600 --> 0:07:53.400 get stuck, and sometimes it's because the investment isn't tied 0:07:53.480 --> 0:07:56.200 directly to a business strategy, or more often than not, 0:07:56.360 --> 0:07:59.560 people simply don't trust the results of the AI model. 0:08:00.640 --> 0:08:02.680 As a company, who is of course thinking about this 0:08:02.760 --> 0:08:05.600 so deeply. What do businesses need to consider when they're 0:08:05.640 --> 0:08:07.880 trying to figure out, you know, how to solve this 0:08:07.960 --> 0:08:11.560 big puzzle of AI ethics. It has to be approached holistically, 0:08:11.840 --> 0:08:15.240 So you've got to be thinking about, for example, what 0:08:15.440 --> 0:08:19.840 culture is required within your organization in order to really 0:08:19.880 --> 0:08:23.600 be able to responsibly create AI, what processes are in 0:08:23.640 --> 0:08:26.240 place to make sure that you're being compliant and that 0:08:26.280 --> 0:08:30.480 your your practitioners know what to do. And then of 0:08:30.520 --> 0:08:34.640 course AI engineering frameworks and tooling that can assist you 0:08:34.679 --> 0:08:38.680 on this journey. There is so much fundamentally to do. 0:08:39.440 --> 0:08:44.200 We found that actually those that were leading responsible AI 0:08:44.360 --> 0:08:47.960 trust where the AI initiatives within their organization has switched 0:08:48.000 --> 0:08:51.560 in the last three years. It used to be technical leaders, 0:08:52.120 --> 0:08:55.880 for example, chief data officer or someone who is a 0:08:55.920 --> 0:08:59.720 PhD in machine learning, and now it's switched to be 0:08:59.840 --> 0:09:03.280 a the percent of those leaders are now non technical 0:09:03.800 --> 0:09:07.920 business leaders, maybe you know, chief compliance officer, chief diversity 0:09:07.960 --> 0:09:12.320 inclusivity officers, chief legal officer. So we're seeing a shift, 0:09:12.400 --> 0:09:17.760 and I believe firmly. It's a recognition from organizations that 0:09:17.800 --> 0:09:21.360 are seeing that in order to really pull this off well, 0:09:21.400 --> 0:09:25.560 there has to be an investment than a focus in culture, 0:09:26.240 --> 0:09:30.080 in people and getting people to understand why they should 0:09:30.080 --> 0:09:34.760 care about this space. And so I see two challenges 0:09:34.840 --> 0:09:37.160 with doing that right. One is, you know a lot 0:09:37.200 --> 0:09:40.640 of these technology companies are really built to be tech companies, 0:09:40.640 --> 0:09:44.319 not necessarily you know, social tech companies or having this 0:09:44.440 --> 0:09:47.800 sort of training and ethics and beyond. Another issue seems 0:09:47.840 --> 0:09:51.520 to be that you're really proposing a switch that's truly holistic, right, 0:09:51.600 --> 0:09:54.760 that's like rethinking the way the company thinks about its 0:09:54.800 --> 0:09:57.800 bottom line. And so as you think about working through 0:09:57.840 --> 0:10:00.600 these kinds of challenges at IBM, how you tackled this, 0:10:00.679 --> 0:10:02.720 like how have you brought new talent in? How have 0:10:02.760 --> 0:10:05.520 you thought really carefully about this big holistic switch that 0:10:05.559 --> 0:10:08.520 needs to come to make AI more trustworthy. Data is 0:10:08.559 --> 0:10:12.080 an artifact of the human experience. And if you start 0:10:12.120 --> 0:10:15.480 with that as your definition and then think about well 0:10:16.559 --> 0:10:19.840 data is curated by data side this all data is 0:10:19.920 --> 0:10:25.600 biased and so if you're not recognizing bias with eyes 0:10:25.720 --> 0:10:31.760 fully open, then ultimately you're calcifying systemic bias into systems 0:10:31.800 --> 0:10:35.000 like AI. So some of the things that we've done 0:10:35.280 --> 0:10:40.080 at IBM again recognizing this important need for culture is big, big, 0:10:40.120 --> 0:10:44.520 big focus on diversity, not only looking at teams of 0:10:44.600 --> 0:10:47.520 data scientists and saying how many women are on this team, 0:10:47.640 --> 0:10:52.439 how many minorities are on this team, but also insisting 0:10:52.559 --> 0:10:56.040 on recognizing that we need to bring in people with 0:10:56.120 --> 0:11:00.600 different world views too, For example, what's your definition of fairness? 0:11:01.400 --> 0:11:05.240 Is your definition equality is an equity? Also bringing people 0:11:05.280 --> 0:11:09.760 with a wider variety of skill sets and roles, including 0:11:09.760 --> 0:11:18.640 our social scientists, anthropologists, sociologists, psychologists like yourself, right, behavioral scientists, designers. 0:11:18.679 --> 0:11:23.160 I mean we have one of the leading AI design 0:11:23.440 --> 0:11:27.040 practices in the world. I mean the effort, the investments 0:11:27.040 --> 0:11:31.640 we've been making in design thinking as a mechanism to 0:11:31.679 --> 0:11:36.240 create frameworks for systemic empathy well before any code is written, 0:11:36.840 --> 0:11:41.040 so people can think through, how would you design in 0:11:41.120 --> 0:11:45.000 order to mitigate for any potential harm given not only 0:11:45.040 --> 0:11:47.520 the values of your organization, but what are the rights 0:11:47.520 --> 0:11:52.760 of individuals asking oneself? These kinds of questions reinforces than 0:11:52.960 --> 0:11:56.760 the idea. The ethics doesn't come at the end like 0:11:56.880 --> 0:12:00.360 it's some kind of quality assurance, like check I passed 0:12:00.360 --> 0:12:03.559 the audit, I've got to go, you know. But instead, really, 0:12:03.600 --> 0:12:05.959 you know, as soon as you're thinking about using an 0:12:05.960 --> 0:12:09.760 AI for a particular use case, thinking about you know, 0:12:09.880 --> 0:12:13.640 what is the intent of this model, what's the relationship 0:12:13.679 --> 0:12:17.400 we ultimately want to have with AI? And again, these 0:12:17.440 --> 0:12:22.160 are non technology questions. This is where social scientists. Having 0:12:22.160 --> 0:12:26.400 a social scientist on your team helping think through these 0:12:26.480 --> 0:12:30.480 kinds of questions is is critical. Let's pause here for 0:12:30.520 --> 0:12:34.319 a second, because this is a really profound idea. Building 0:12:34.480 --> 0:12:38.199 responsible AI does not mean that you create a system 0:12:38.440 --> 0:12:41.679 then check in at the end and say is this okay? 0:12:41.720 --> 0:12:45.319 Is this ethical? If you don't ask those questions until 0:12:45.360 --> 0:12:48.920 the end of the process, you've already failed. You have 0:12:49.000 --> 0:12:52.559 to think about ethics from the jump from the makeup 0:12:52.640 --> 0:12:54.800 of the team to the data you're using to train 0:12:54.880 --> 0:12:57.880 the model to the most basic question of all, is 0:12:57.920 --> 0:13:02.240 this even the right use case artificial intelligence? The big 0:13:02.320 --> 0:13:07.360 lesson from IBM is this responsible AI is something you 0:13:07.400 --> 0:13:12.200 build at every step of the process. So this season 0:13:12.200 --> 0:13:15.200 of smart Talk is all focused on creativity and business. 0:13:15.720 --> 0:13:18.360 My guess is that thinking about trustworthy AI involves a 0:13:18.400 --> 0:13:20.800 lot of creativity. But talk to me about some of 0:13:20.800 --> 0:13:23.320 the spots where you see this work as being most creative. 0:13:24.160 --> 0:13:28.760 Oh goodness, I would say incorporating design design thinking in 0:13:28.840 --> 0:13:33.320 particular as well as straight up design in order to 0:13:33.720 --> 0:13:38.240 craft AI responsibly. You've used this word design thinking, and 0:13:38.280 --> 0:13:40.320 so I'm wondering exactly what you mean here? How do 0:13:40.360 --> 0:13:43.760 you define this idea of design thinking. Design thinking is 0:13:43.800 --> 0:13:47.520 a practice that we established here at IBM many years ago. 0:13:47.800 --> 0:13:51.559 In essence, what it is, it's a way of working 0:13:51.920 --> 0:13:58.360 with groups of people to co create a vision for something, 0:13:58.400 --> 0:14:01.280 for a product or a sir risk or an outcome. 0:14:02.160 --> 0:14:07.200 And typically it starts with things like, for example, empathy maps, 0:14:07.200 --> 0:14:10.440 like if you're thinking about an end user, thinking through 0:14:10.559 --> 0:14:14.600 what is this person thinking, seeing, hearing, feeling, like what 0:14:14.679 --> 0:14:19.400 are they experiencing in order to ultimately craft and experience 0:14:19.440 --> 0:14:23.520 for them that is targeted specifically for them. So we 0:14:23.640 --> 0:14:27.200 use it in a really wide variety of different ways 0:14:27.720 --> 0:14:32.040 with respect to trustworthy AI, even rendering an AI model 0:14:32.440 --> 0:14:35.320 explainable to a subject. And I'll give you an example. 0:14:35.840 --> 0:14:39.720 So we've got this wonderful program with an IBM caller, 0:14:39.760 --> 0:14:42.760 our Academy of Technology, and we take on initiatives that 0:14:42.920 --> 0:14:47.480 steer the company in innovative new directions. So we had 0:14:47.520 --> 0:14:51.480 an initiative where it was titled What the Titanic taught 0:14:51.520 --> 0:14:58.920 Us About Explainable AI, and the project was imagining if 0:14:58.960 --> 0:15:02.320 there was an AI mo utle that could predict the 0:15:02.440 --> 0:15:06.479 likelihood of a passenger getting a life raft on the Titanic. 0:15:07.000 --> 0:15:09.800 And we broke up into two work streams. One was 0:15:09.840 --> 0:15:12.680 the workstream full of the data scientists who were using 0:15:12.720 --> 0:15:15.320 all the different explainers to come up with the predictions 0:15:15.320 --> 0:15:17.920 and they would crank out the numbers. And the other 0:15:18.080 --> 0:15:22.160 team here's where the social scientists lived and the designers 0:15:22.240 --> 0:15:25.400 were right where we were thinking through how do we 0:15:25.440 --> 0:15:32.280 empower people? Well, how do we explain this algorithm and 0:15:32.480 --> 0:15:36.440 this predictor and the accuracy behind this prediction in such 0:15:36.480 --> 0:15:39.240 a way as to ultimately empower an end users? They 0:15:39.280 --> 0:15:43.480 could decide I'm not getting on that boat, or I 0:15:43.560 --> 0:15:47.680 want to get a second opinion please, or I went 0:15:47.800 --> 0:15:52.480 to contest the outputs of this model because I upgraded 0:15:52.880 --> 0:15:56.400 to first class just yesterday. See what I'm saying. And 0:15:56.520 --> 0:16:00.680 that takes a lot of creativity. How do you design 0:16:00.760 --> 0:16:04.760 and experience for someone in order to ultimately empower them. 0:16:05.480 --> 0:16:10.400 So design design design is critically critically important. And why 0:16:10.440 --> 0:16:12.480 I mentioned you know, we we've got to open up 0:16:12.480 --> 0:16:15.040 the aperture with respect to who we invite to the table, 0:16:15.080 --> 0:16:18.720 and these kinds of conversations. Taking the time to really 0:16:18.800 --> 0:16:23.160 understand other people's perspectives is so important when you're doing 0:16:23.200 --> 0:16:27.040 anything creative, and it is fundamental to the way the 0:16:27.120 --> 0:16:31.080 new creators work. The core question you should always be 0:16:31.160 --> 0:16:34.560 asking is where will the user be meeting this product? 0:16:35.440 --> 0:16:39.640 As Peder said, what will they be thinking, seeing, hearing, feeling. 0:16:40.360 --> 0:16:43.160 If you can answer those questions the way IBM does 0:16:43.280 --> 0:16:46.600 in its design thinking practice, you will be in great 0:16:46.640 --> 0:16:50.680 shape to create almost anything. Really, let's hear how it 0:16:50.720 --> 0:16:54.520 works in practice. And so we've been mostly talking kind 0:16:54.520 --> 0:16:56.360 of at the metal level about, you know, how to 0:16:56.400 --> 0:16:59.800 think about AI ethics generally. But of course the way 0:16:59.800 --> 0:17:02.800 this probably occurs in the trenches as a client approach 0:17:02.840 --> 0:17:05.080 as IBM, and they want to help with a specific 0:17:05.119 --> 0:17:07.879 problem in AI. And so I'm wondering, from a client 0:17:07.920 --> 0:17:10.560 based perspective, where do you start having some of these 0:17:10.560 --> 0:17:14.840 tough conversations. It has varied, to tell you the truth, 0:17:15.160 --> 0:17:20.000 we had one client that approached us to expand the 0:17:20.080 --> 0:17:24.439 use of an AI model to infer skill sets of 0:17:24.480 --> 0:17:28.160 their employees, but not just to infer their technical skills 0:17:28.200 --> 0:17:32.720 but also their soft foundational skills, meaning, let me use 0:17:32.760 --> 0:17:35.520 an AI to determine what kind of communicator you might 0:17:35.520 --> 0:17:41.280 be Laurie right. Others might come to us with, Okay, 0:17:41.400 --> 0:17:44.400 we recognize we need help setting an AI ethics board. 0:17:44.640 --> 0:17:47.919 Is this something you can assist us with? Or we 0:17:48.000 --> 0:17:52.560 have these values, we need to establish AI ethics principles 0:17:52.720 --> 0:17:56.639 and processes to help us ensure that we're compliant given 0:17:56.680 --> 0:18:00.600 regulations coming down the pike. Or we've had clients come 0:18:00.640 --> 0:18:03.119 to us saying, please train our people how to assess 0:18:03.880 --> 0:18:08.399 for unexpected patterns in an AI model, but then also 0:18:09.160 --> 0:18:15.240 how to holistically mitigate to prevent any potential harm. And 0:18:15.520 --> 0:18:21.560 those have been phenomenal engagements. They're huge learning moments. And 0:18:21.600 --> 0:18:24.320 so it seems like the real additional value that IBM 0:18:24.400 --> 0:18:27.480 is bringing through this process isn't necessarily just providing an 0:18:27.480 --> 0:18:30.600 AI algorithm or consulting on sam AI algorithm. It seems 0:18:30.640 --> 0:18:34.439 like the real value added is explaining how this design 0:18:34.480 --> 0:18:37.679 thinking works. You're almost like this therapist or like a 0:18:37.720 --> 0:18:40.400 really good bartender who talks to people, who talks whole 0:18:40.440 --> 0:18:43.040 companies through some of their problems to try to figure 0:18:43.080 --> 0:18:46.080 out where they're going astray before they start implementing these things. 0:18:46.960 --> 0:18:52.720 Can I put Chief Bartender Office on my metaphor, I'll 0:18:52.720 --> 0:18:56.239 tell you some of our our most valuable people on 0:18:56.280 --> 0:19:01.080 the team for that engagement. We had an industrial organization psychologist, 0:19:01.480 --> 0:19:06.000 we had an anthropologist. That's why I'm saying it's important 0:19:06.080 --> 0:19:09.280 we bring in the social scientists because you're exactly right, 0:19:09.960 --> 0:19:15.480 it's more than just scrutinizing the algorithm in its state. 0:19:15.720 --> 0:19:18.040 You have to be thinking about how is it being 0:19:18.160 --> 0:19:21.399 used holistically? And so if I was a business that 0:19:21.520 --> 0:19:24.080 was trying to think about how a company like IBM 0:19:24.080 --> 0:19:26.880 could come in and help out with more trustworthy AI, 0:19:27.040 --> 0:19:30.320 what would this process really look like. Well, what we're 0:19:30.359 --> 0:19:33.840 finding more often than not is that there'll be smaller 0:19:33.920 --> 0:19:38.960 teams within broader organizations that either have the responsibility of 0:19:39.119 --> 0:19:43.200 compliance and see the writing on the wall, or they've 0:19:43.200 --> 0:19:46.919 been the ones investing in AI and are trying to 0:19:46.960 --> 0:19:50.040 figure out how to get the rest of the organization 0:19:50.359 --> 0:19:53.600 on board with respect to things like setting up an 0:19:53.600 --> 0:19:57.600 ethics board or establishing principles or things like that. So 0:19:58.440 --> 0:20:01.679 some things that we've done help companies do this is 0:20:01.720 --> 0:20:05.840 we kick off engagements with what we called our our 0:20:05.880 --> 0:20:10.520 AI for leaders workshops. On the one hand, it's teaching 0:20:10.640 --> 0:20:13.600 why you should care, but on the other hand, it's 0:20:13.640 --> 0:20:16.439 meant to get people so excited across the organization that 0:20:16.480 --> 0:20:18.280 they want to raise their hand and say, I want 0:20:18.280 --> 0:20:20.920 to represent this part, like, for example, I want to 0:20:20.960 --> 0:20:22.960 be part of the ethics board as it is being 0:20:23.000 --> 0:20:26.520 stood up. The heart parts, not the tech. The hard 0:20:26.600 --> 0:20:28.560 part is human behavior. And I know I'm preaching to 0:20:28.600 --> 0:20:31.520 the choir given your background, it's so nice as a 0:20:31.560 --> 0:20:34.240 psychologist to hear this. I'm like snapping my fingers like 0:20:34.320 --> 0:20:38.640 peach exactly. The hard part is human behavior. So it's 0:20:38.680 --> 0:20:42.199 been like drinking from a fire hose. I mean in 0:20:42.280 --> 0:20:45.200 terms of the kinds of things that that we've all 0:20:45.280 --> 0:20:48.480 been learning, and there's still so much to learn. It 0:20:49.240 --> 0:20:53.439 really bugs me that those who are lucky enough to 0:20:53.480 --> 0:20:56.159 be able to take classes and things like data ethics 0:20:56.280 --> 0:21:00.439 or AI ethics self categorized as coders machine learning dissert 0:21:00.520 --> 0:21:03.920 data scientists. If we're living in a world where AI 0:21:04.119 --> 0:21:07.359 is fundamentally being used to make decisions that could directly 0:21:07.400 --> 0:21:11.360 affect our livelihoods. We need to know more, We need 0:21:11.440 --> 0:21:16.040 to have more literacy, and also make sure that there 0:21:16.160 --> 0:21:21.359 is a consistent message of accessibility such that we are 0:21:21.400 --> 0:21:24.400 saying you don't just have to be interested in coding, 0:21:24.680 --> 0:21:28.240 like you're interested in social justice or psychology or anthropologies. 0:21:28.720 --> 0:21:31.480 There's a seat at the table for you here because 0:21:31.520 --> 0:21:35.000 we desperately need you. We desperately need that kind of 0:21:35.040 --> 0:21:39.840 skill set. Just getting people to think about how do 0:21:39.880 --> 0:21:44.879 you design something given an empathy lens to protect people? 0:21:44.920 --> 0:21:47.200 I mean that, I think is such a crucial skill 0:21:47.240 --> 0:21:50.000 to learn. You know, one thing I love about your 0:21:50.040 --> 0:21:52.560 approaches that when you're talking to clients, you're almost doing 0:21:52.600 --> 0:21:54.840 what I'm doing is a professor, where you're kind of 0:21:54.920 --> 0:21:57.919 instructing students, getting them to think in different ways. But 0:21:58.000 --> 0:22:00.000 I know from my field that I wind up learning 0:22:00.320 --> 0:22:02.920 as much from students as I think sometimes they learned 0:22:03.000 --> 0:22:05.920 from me. And so I'm wondering what what you've learned 0:22:05.920 --> 0:22:08.879 in the process of helping so many businesses approach AI 0:22:08.960 --> 0:22:11.399 a little bit more ethically, Like, have there been insights 0:22:11.400 --> 0:22:13.760 that you've gotten through your interaction with clients and the 0:22:13.840 --> 0:22:20.080 challenges they've been facing. I'm learning with every single interaction. 0:22:20.320 --> 0:22:26.840 For example, in my mind, given the experiences that IBM 0:22:26.960 --> 0:22:31.000 has had with respect to setting up our principles are 0:22:31.160 --> 0:22:36.240 pillars ARII, ethics board. There's a process to follow, right 0:22:36.480 --> 0:22:38.080 if you're thinking about it like a book, these are 0:22:38.080 --> 0:22:43.320 the chapters in order to to optimize the approach, let's say, 0:22:43.359 --> 0:22:45.879 but sometimes we work with clients that say, I'm going 0:22:45.920 --> 0:22:48.119 to install this tool and I want to jump to 0:22:48.200 --> 0:22:52.320 chapter seven, and it's like, okay, you know, how how 0:22:52.400 --> 0:22:55.760 do we help navigate clients that want to skip over 0:22:56.960 --> 0:23:00.679 steps that we think are important. Another on is again 0:23:01.240 --> 0:23:05.879 the social scientists and bringing them into really push hard 0:23:06.040 --> 0:23:08.919 on what is the right context of where this data 0:23:09.000 --> 0:23:12.680 tell me the origin story? Again like really pushing us 0:23:12.680 --> 0:23:17.520 to think hard and with their perspective, you don't know, 0:23:17.640 --> 0:23:21.359 just constant, constant learning. Which is why one of the 0:23:21.400 --> 0:23:24.400 things we did at IBM is we've established something called 0:23:24.440 --> 0:23:27.200 our Center of Excellence where we said, you know what 0:23:27.280 --> 0:23:30.000 IBM or we don't care what your background is, we 0:23:30.040 --> 0:23:32.640 don't care who you are. If you're interested in this space, 0:23:33.119 --> 0:23:36.359 you can become a member. The Center of Excellence is 0:23:36.400 --> 0:23:40.119 a way in which we have not only projects people 0:23:40.119 --> 0:23:42.679 can join in order to get real life experience, but 0:23:42.720 --> 0:23:45.880 then also share back. Here's what we learned. We did 0:23:45.920 --> 0:23:48.879 this with this particular I had. Here was our epiphany, 0:23:48.920 --> 0:23:53.400 because if we're not sharing back and we're not constantly educating, 0:23:54.200 --> 0:23:58.440 then we're missing the opportunity to establish the right culture. 0:24:00.200 --> 0:24:04.560 Establishing the right culture to share what we're learning is 0:24:04.600 --> 0:24:08.080 so important. And so I wanted to end. But going 0:24:08.119 --> 0:24:10.800 back to where we started, you with your technofile family 0:24:10.960 --> 0:24:13.320 watching Star Trek. I think if we were to fast 0:24:13.359 --> 0:24:16.200 forward a couple of decades, we probably couldn't have imagined 0:24:16.240 --> 0:24:18.960 that we'd be in the place with AI generally where 0:24:19.000 --> 0:24:20.920 we are now, and especially as we think through more 0:24:20.960 --> 0:24:25.320 trustworthy AI. And so you know, with such change happening 0:24:25.480 --> 0:24:27.639 right now, with the fact that it's a fire hose 0:24:27.720 --> 0:24:30.760 that's gonna just get even more powerful over time, what 0:24:30.800 --> 0:24:32.679 do you think is next in this world of thinking 0:24:32.680 --> 0:24:37.040 through more trustworthy AI. I would say next is far 0:24:37.200 --> 0:24:41.800 more education, far more understanding, and we're starting to see 0:24:41.840 --> 0:24:45.879 that shift far more CEO saying, yeah, ethics has to 0:24:45.920 --> 0:24:48.399 be corridor our business. There's that, but there's a shift 0:24:48.680 --> 0:24:52.800 barely half of the CEO is in we're saying that 0:24:53.280 --> 0:24:56.879 a ethics was key or important to their business, and 0:24:56.920 --> 0:25:02.680 now you're saying the great majority so education, education, education, 0:25:02.880 --> 0:25:06.360 And again I would underscore making it far more accessible 0:25:06.400 --> 0:25:10.600 to far more people, which means it's not just our 0:25:10.720 --> 0:25:16.000 classes and higher ed institutions, it's our conferences, it's anytime 0:25:16.040 --> 0:25:19.960 we write white papers, anytime we publish articles, anytime we 0:25:20.040 --> 0:25:24.199 do podcasts like this. Right, the way we talk about 0:25:24.240 --> 0:25:27.320 this space has to be far more accessible and open 0:25:27.400 --> 0:25:31.840 and inviting two people with different roles, different skill sets, 0:25:31.880 --> 0:25:37.520 different worldviews, because else again we're just codifying our own bias. Well, Fature, 0:25:37.560 --> 0:25:40.280 I want to express my gratitude today for making AI 0:25:40.320 --> 0:25:43.120