WEBVTT - Smart Talks with IBM - Responsible AI: Why Businesses Need Reliable AI Governance

0:00:04.559 --> 0:00:12.760
<v Speaker 1>Welcome to Tech Stuff, a production from iHeartRadio. Today, we

0:00:12.800 --> 0:00:15.760
<v Speaker 1>are witnessed to one of those rare moments in history,

0:00:16.120 --> 0:00:19.360
<v Speaker 1>the rise of an innovative technology with the potential to

0:00:19.480 --> 0:00:24.200
<v Speaker 1>radically transform business in society forever. That technology, of course,

0:00:24.720 --> 0:00:28.240
<v Speaker 1>is artificial intelligence, and it's the central focus for this

0:00:28.440 --> 0:00:32.440
<v Speaker 1>new season of Smart Talks with IBM. Join hosts from

0:00:32.479 --> 0:00:36.199
<v Speaker 1>your favorite Pushkin podcasts as they talk with industry experts

0:00:36.200 --> 0:00:39.800
<v Speaker 1>and leaders to explore how businesses can integrate AI into

0:00:39.840 --> 0:00:43.160
<v Speaker 1>their workflows and help drive real change in this new

0:00:43.280 --> 0:00:46.960
<v Speaker 1>era of AI, and of course, host Malcolm Gladwell will

0:00:46.960 --> 0:00:49.280
<v Speaker 1>be there to guide you through the season and throw

0:00:49.360 --> 0:00:52.279
<v Speaker 1>in his two cents as well. Look out for new

0:00:52.280 --> 0:00:55.160
<v Speaker 1>episodes of Smart Talks with IBM every other week on

0:00:55.240 --> 0:00:59.440
<v Speaker 1>the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts,

0:00:59.640 --> 0:01:03.880
<v Speaker 1>and learn more at IBM dot com slash smart talks.

0:01:06.400 --> 0:01:10.280
<v Speaker 2>Hello, Hello, Welcome to Smart Talks with IBM, a podcast

0:01:10.280 --> 0:01:16.880
<v Speaker 2>from Pushkin Industri's iHeartRadio and IBM. I'm Malcolm Gladwell. This season,

0:01:16.920 --> 0:01:21.280
<v Speaker 2>we're continuing our conversation with new creators visionaries who are

0:01:21.319 --> 0:01:24.840
<v Speaker 2>creatively applying technology in business to drive change, but with

0:01:24.920 --> 0:01:29.840
<v Speaker 2>a focus on the transformative power of artificial intelligence and

0:01:29.920 --> 0:01:33.040
<v Speaker 2>what it means to leverage AI as a game changing

0:01:33.160 --> 0:01:38.160
<v Speaker 2>multiplier for your business. Our guest today is Christina Montgomery,

0:01:38.560 --> 0:01:42.640
<v Speaker 2>IBM's Chief Privacy and Trust Officer. She's also chair of

0:01:42.680 --> 0:01:48.120
<v Speaker 2>IBM's AI Ethics Board. In addition to overseeing IBM's privacy policy,

0:01:48.480 --> 0:01:53.480
<v Speaker 2>a core part of Christina's job involves AI governance, making

0:01:53.520 --> 0:01:57.320
<v Speaker 2>sure the way AI is used complies with the international

0:01:57.640 --> 0:02:03.480
<v Speaker 2>legal regulations customized for each industry. In today's episode, Christina

0:02:03.520 --> 0:02:08.440
<v Speaker 2>will explain why businesses need foundational principles when it comes

0:02:08.520 --> 0:02:13.240
<v Speaker 2>to using technology, why AI regulation should focus on specific

0:02:13.440 --> 0:02:17.720
<v Speaker 2>use cases over the technology itself, and share a little

0:02:17.720 --> 0:02:23.360
<v Speaker 2>bit about her landmark congressional testimony. Last May, Christina spoke

0:02:23.440 --> 0:02:27.080
<v Speaker 2>with doctor Lori Santos, host of the Pushkin podcast The

0:02:27.120 --> 0:02:32.600
<v Speaker 2>Happiness Lab, a cognitive scientist and psychology professor at Yale University.

0:02:32.919 --> 0:02:38.280
<v Speaker 2>Laurie is an expert on human happiness and cognition. Okay,

0:02:38.800 --> 0:02:40.079
<v Speaker 2>let's get to the interview.

0:02:42.639 --> 0:02:44.720
<v Speaker 3>So Christina, I'm so excited to talk to you today.

0:02:44.880 --> 0:02:47.080
<v Speaker 3>So let's start by talking a little bit about your

0:02:47.160 --> 0:02:49.840
<v Speaker 3>role at IBM. What does a Chief Privacy and trust

0:02:49.880 --> 0:02:50.920
<v Speaker 3>officer actually do.

0:02:51.440 --> 0:02:55.200
<v Speaker 4>It's a really dynamic profession and it's not a new profession,

0:02:55.760 --> 0:02:58.640
<v Speaker 4>but the role has really changed. I mean, my role

0:02:58.760 --> 0:03:02.480
<v Speaker 4>today is broader and just helping to ensure compliance with

0:03:02.600 --> 0:03:07.240
<v Speaker 4>data protection laws globally. I'm also responsible for AI governance

0:03:07.280 --> 0:03:10.239
<v Speaker 4>I co chair or AI Ethics Board here at IBM,

0:03:10.560 --> 0:03:13.600
<v Speaker 4>and for data clearance and data governance as well for

0:03:13.680 --> 0:03:17.240
<v Speaker 4>the company. So I have both a compliance aspect to

0:03:17.280 --> 0:03:19.960
<v Speaker 4>my role, really important on a global basis, but also

0:03:20.520 --> 0:03:25.440
<v Speaker 4>help the business to competitively differentiate because really trust is

0:03:25.440 --> 0:03:29.160
<v Speaker 4>a strategic advantage for IBM and a competitive differentiator as

0:03:29.160 --> 0:03:33.080
<v Speaker 4>a company that's been responsibly managing the most sensitive data

0:03:33.120 --> 0:03:35.720
<v Speaker 4>for our clients for more than a century now and

0:03:35.880 --> 0:03:38.600
<v Speaker 4>helping to usher new technologies into the world with trust

0:03:38.600 --> 0:03:42.000
<v Speaker 4>and transparency. And so that's also a key aspect of

0:03:42.040 --> 0:03:42.480
<v Speaker 4>my role.

0:03:43.240 --> 0:03:45.440
<v Speaker 3>And so you joined us here on smart Talks back

0:03:45.440 --> 0:03:47.680
<v Speaker 3>in twenty twenty one and you chatted with us about

0:03:47.680 --> 0:03:51.360
<v Speaker 3>IBM's approach of building trust and transparency with AI, and

0:03:51.520 --> 0:03:53.520
<v Speaker 3>that was only two years ago, but it almost feels

0:03:53.560 --> 0:03:55.880
<v Speaker 3>like in eternity has happened in the field of AI

0:03:56.040 --> 0:03:59.040
<v Speaker 3>since then, and so I'm curious how much has changed

0:03:59.080 --> 0:04:01.560
<v Speaker 3>since you were here last time. Were the things you

0:04:01.600 --> 0:04:04.440
<v Speaker 3>told us before you are they still true? How are

0:04:04.480 --> 0:04:05.120
<v Speaker 3>things changing?

0:04:05.520 --> 0:04:09.000
<v Speaker 4>You're absolutely right. It feels like the world has changed

0:04:09.080 --> 0:04:12.760
<v Speaker 4>really in the last two years. But the same fundamental

0:04:12.800 --> 0:04:17.720
<v Speaker 4>principles and the same overall governance apply to IBM's program

0:04:18.279 --> 0:04:22.640
<v Speaker 4>for Data Protection and Responsible AI that we talked about

0:04:22.680 --> 0:04:25.360
<v Speaker 4>two years ago, and not much has changed there from

0:04:25.400 --> 0:04:28.560
<v Speaker 4>our perspective. And the good thing is we've put these

0:04:28.600 --> 0:04:33.039
<v Speaker 4>practices and this governance approach into place, and we have

0:04:33.279 --> 0:04:36.920
<v Speaker 4>an established way of looking at these emerging technologies. As

0:04:36.960 --> 0:04:40.360
<v Speaker 4>the technology evolves, the tech is more powerful, for sure,

0:04:40.520 --> 0:04:44.560
<v Speaker 4>foundation models are vastly larger and more capable, and our

0:04:44.640 --> 0:04:48.080
<v Speaker 4>creating in some respects new issues. But that just makes

0:04:48.080 --> 0:04:49.880
<v Speaker 4>it all the more urgent to do what we've been

0:04:49.920 --> 0:04:53.520
<v Speaker 4>doing and to put trust and transparency into place across

0:04:53.560 --> 0:04:56.200
<v Speaker 4>the business to be accountable to those principles.

0:04:57.400 --> 0:04:59.560
<v Speaker 3>And so our conversation today is really centered around this

0:04:59.680 --> 0:05:02.960
<v Speaker 3>need for new AI regulation and part of that regulation

0:05:03.080 --> 0:05:05.960
<v Speaker 3>involves the mitigation of bias. And this is something I

0:05:05.960 --> 0:05:08.320
<v Speaker 3>think about a ton as a psychologist, right, you know,

0:05:08.360 --> 0:05:11.120
<v Speaker 3>I know, like my students and everyone who's interacting with

0:05:11.200 --> 0:05:14.279
<v Speaker 3>AI is assuming that the kind of knowledge that they're

0:05:14.320 --> 0:05:17.320
<v Speaker 3>getting from this kind of learning is accurate, right, But

0:05:17.400 --> 0:05:19.440
<v Speaker 3>of course AI is only as good as the knowledge

0:05:19.440 --> 0:05:21.720
<v Speaker 3>that's going in. And so talk to me a little

0:05:21.760 --> 0:05:25.000
<v Speaker 3>bit about like why bias occurs in AI and the

0:05:25.080 --> 0:05:26.960
<v Speaker 3>level of the problem that we're really dealing with.

0:05:27.800 --> 0:05:30.840
<v Speaker 4>Yeah, well, obviously AI is based on data, right, It's

0:05:31.200 --> 0:05:36.200
<v Speaker 4>trained with data, and that data could be biased in

0:05:36.200 --> 0:05:38.680
<v Speaker 4>and of itself, and that's where issues could come up.

0:05:38.720 --> 0:05:40.600
<v Speaker 4>They come up in the data, they could also come

0:05:40.680 --> 0:05:44.200
<v Speaker 4>up in the output of the models themselves. So it's

0:05:44.320 --> 0:05:49.080
<v Speaker 4>really important that you build bias consideration and bias testing

0:05:49.200 --> 0:05:52.600
<v Speaker 4>into your product development cycle. And so what we've been

0:05:52.640 --> 0:05:55.200
<v Speaker 4>thinking about here at IBM and doing we had some

0:05:55.279 --> 0:05:58.239
<v Speaker 4>of our research teams delivered some of the very first

0:05:58.240 --> 0:06:01.599
<v Speaker 4>toolkits to help detect bias years ago now right, and

0:06:01.640 --> 0:06:05.280
<v Speaker 4>deployed them to open source. And we have put into

0:06:05.320 --> 0:06:08.640
<v Speaker 4>place for our developers here at IBM and Ethics by

0:06:08.680 --> 0:06:11.880
<v Speaker 4>Design playbook that's sort of a step by step approach

0:06:12.640 --> 0:06:19.160
<v Speaker 4>which also addresses very fully biased considerations, and we provide

0:06:19.200 --> 0:06:22.400
<v Speaker 4>not only like here's a point when you should test

0:06:22.440 --> 0:06:24.840
<v Speaker 4>for it, and you consider it in the data you

0:06:24.880 --> 0:06:26.599
<v Speaker 4>have to measure it both at the data and the

0:06:26.640 --> 0:06:30.160
<v Speaker 4>model level or the outcome level, and we provide guidance

0:06:30.160 --> 0:06:32.839
<v Speaker 4>with respect to what tools can best be used to

0:06:32.880 --> 0:06:36.360
<v Speaker 4>accomplish that. So it's a really important issue. It's one

0:06:36.400 --> 0:06:39.360
<v Speaker 4>you can't just talk about. You have to provide essentially

0:06:39.480 --> 0:06:42.719
<v Speaker 4>the technology and the capabilities and the guidance to enable

0:06:42.760 --> 0:06:43.920
<v Speaker 4>people to test for it.

0:06:44.640 --> 0:06:47.320
<v Speaker 3>Recently, you had this wonderful opportunity to head to Congress

0:06:47.360 --> 0:06:50.400
<v Speaker 3>to talk about AI, and in your testimony before Congress,

0:06:50.440 --> 0:06:53.440
<v Speaker 3>you mentioned that it's often said that innovation moves too

0:06:53.480 --> 0:06:56.080
<v Speaker 3>fast for government to keep up, and this is something

0:06:56.080 --> 0:06:58.240
<v Speaker 3>that I also worry about as a psychologist. Right our

0:06:58.279 --> 0:07:01.160
<v Speaker 3>policy makers really understanding the issues that they're dealing with,

0:07:01.640 --> 0:07:03.880
<v Speaker 3>and so I'm curious how you're approaching this challenge of

0:07:03.920 --> 0:07:06.360
<v Speaker 3>adapting AI policies to keep up with the sort of

0:07:06.440 --> 0:07:08.960
<v Speaker 3>rapid pace of all the advancements we're seeing in the

0:07:08.960 --> 0:07:10.320
<v Speaker 3>AI technology itself.

0:07:11.280 --> 0:07:14.520
<v Speaker 4>I think it's really critically important that you have foundational

0:07:14.560 --> 0:07:20.000
<v Speaker 4>principles that applied to not only how you use technology,

0:07:20.040 --> 0:07:21.840
<v Speaker 4>but whether you're going to use it in the first place,

0:07:21.840 --> 0:07:23.520
<v Speaker 4>and where you're going to use and apply it across

0:07:23.600 --> 0:07:27.280
<v Speaker 4>your company, and then your program from a governance perspective,

0:07:27.360 --> 0:07:29.880
<v Speaker 4>has to be agile. It has to be able to

0:07:29.960 --> 0:07:35.680
<v Speaker 4>address emerging capabilities, new training methods, etc. And part of

0:07:35.720 --> 0:07:40.240
<v Speaker 4>that involves helping to educate and instill and empower a

0:07:40.280 --> 0:07:44.080
<v Speaker 4>trustworthy culture at a company so you can spot those

0:07:44.120 --> 0:07:46.360
<v Speaker 4>issues so you can ask the right questions at the

0:07:46.440 --> 0:07:49.640
<v Speaker 4>right time if you try. We talked about during the

0:07:49.680 --> 0:07:53.920
<v Speaker 4>Senate hearing, and IBM's been talking for years about regulating

0:07:54.200 --> 0:07:57.360
<v Speaker 4>the use, not the technology itself, because if you try

0:07:57.360 --> 0:08:01.200
<v Speaker 4>to regulate technology, you're very quickly going to find out

0:08:01.600 --> 0:08:04.560
<v Speaker 4>regulation will absolutely never keep up with that.

0:08:05.040 --> 0:08:07.280
<v Speaker 3>And so in your testimony to Congress, you also talked

0:08:07.280 --> 0:08:10.480
<v Speaker 3>about this idea of a precision regulation approach for AI.

0:08:11.120 --> 0:08:13.320
<v Speaker 3>Tell me more about this. What is a precision regulation

0:08:13.400 --> 0:08:15.400
<v Speaker 3>approach and why could that be so important.

0:08:15.680 --> 0:08:18.640
<v Speaker 4>It's funny because I was able to share with Congress

0:08:19.280 --> 0:08:23.080
<v Speaker 4>our precision regulation point of view in twenty twenty three,

0:08:23.440 --> 0:08:26.240
<v Speaker 4>but that precision regulation point of view was published by

0:08:26.360 --> 0:08:31.040
<v Speaker 4>IBM in twenty twenty So we have not changed our

0:08:31.160 --> 0:08:36.079
<v Speaker 4>position that you should apply the tightest controls, the strictest

0:08:36.120 --> 0:08:40.360
<v Speaker 4>regulatory requirements to the technology where the end use and

0:08:40.480 --> 0:08:44.400
<v Speaker 4>risk of societal harm is the greatest. So that's essentially

0:08:44.440 --> 0:08:47.440
<v Speaker 4>what it is. There's lots of AI technology that's used

0:08:47.440 --> 0:08:51.080
<v Speaker 4>today that doesn't touch people, that's very low risk in nature.

0:08:51.600 --> 0:08:55.040
<v Speaker 4>And even when you think about AI that delivers a

0:08:55.160 --> 0:09:00.520
<v Speaker 4>movie recommendation versus AI that is used to diagnose cancer, right,

0:09:00.640 --> 0:09:04.120
<v Speaker 4>there's very different implications associated with those two uses of

0:09:04.160 --> 0:09:09.000
<v Speaker 4>the technology. And so essentially what precision regulation is is

0:09:09.000 --> 0:09:12.400
<v Speaker 4>apply different rules to different risks, right, more stringent regulation

0:09:12.679 --> 0:09:15.800
<v Speaker 4>to the use cases with the greatest risk. And then

0:09:15.960 --> 0:09:20.120
<v Speaker 4>also we build that out calling for things like transparency

0:09:20.720 --> 0:09:24.280
<v Speaker 4>you see it today with content right, misinformation and the like.

0:09:24.880 --> 0:09:28.760
<v Speaker 4>We believe that consumers should always know when they're interacting

0:09:28.840 --> 0:09:31.800
<v Speaker 4>with an AI system, So be transparent, don't hydra AI

0:09:32.880 --> 0:09:36.480
<v Speaker 4>clearly define the risks. So as a country, we need

0:09:36.520 --> 0:09:39.600
<v Speaker 4>to have some clear guidance right in globally as well

0:09:39.920 --> 0:09:43.520
<v Speaker 4>in terms of which uses of AI or higher risk

0:09:43.600 --> 0:09:47.920
<v Speaker 4>where will apply higher and stricter regulation and have sort

0:09:47.960 --> 0:09:50.880
<v Speaker 4>of a common understanding of what those high risk uses

0:09:50.920 --> 0:09:54.480
<v Speaker 4>are and then demonstrate the impact in the cases of

0:09:54.559 --> 0:09:59.280
<v Speaker 4>those higher risk uses. So companies who are using AI

0:09:59.440 --> 0:10:03.839
<v Speaker 4>in space where they can impact people's legal rights, for example,

0:10:04.440 --> 0:10:08.680
<v Speaker 4>should have to conduct an impact assessment that demonstrates that

0:10:08.760 --> 0:10:11.920
<v Speaker 4>the technology isn't biased. So we've been pretty clear about

0:10:12.360 --> 0:10:15.720
<v Speaker 4>apply that the most stringent regulation to the highest risk

0:10:15.800 --> 0:10:16.480
<v Speaker 4>uses of AI.

0:10:18.200 --> 0:10:20.479
<v Speaker 3>And so so far we've been talking about your congressional

0:10:20.480 --> 0:10:23.320
<v Speaker 3>testimony in terms of, you know, the specific content that

0:10:23.360 --> 0:10:25.840
<v Speaker 3>you talked about, But I'm just curious on a personal level,

0:10:26.160 --> 0:10:28.280
<v Speaker 3>you know what was that like right Like, right now,

0:10:28.280 --> 0:10:30.559
<v Speaker 3>it feels like at a policy level, like there's a

0:10:30.640 --> 0:10:33.000
<v Speaker 3>kind of fever pitch going on with AI right now.

0:10:33.240 --> 0:10:34.679
<v Speaker 3>You know, what did that feel like to kind of

0:10:34.760 --> 0:10:37.200
<v Speaker 3>really have the opportunity to talk to policy makers and

0:10:37.240 --> 0:10:40.240
<v Speaker 3>sort of influence what they're thinking about AI technologies like

0:10:40.320 --> 0:10:41.280
<v Speaker 3>in the coming century.

0:10:41.320 --> 0:10:44.240
<v Speaker 4>Perhaps I was really an honor to be able to

0:10:44.320 --> 0:10:47.280
<v Speaker 4>do that and to be one of the first set

0:10:47.320 --> 0:10:51.400
<v Speaker 4>of invitees to the first hearing. And what I learned

0:10:51.440 --> 0:10:54.600
<v Speaker 4>from it essentially is, you know, really two things. The

0:10:54.640 --> 0:10:58.480
<v Speaker 4>first is really the value of authenticity. So both as

0:10:58.679 --> 0:11:02.640
<v Speaker 4>an individual and as a company, I was able to

0:11:02.679 --> 0:11:05.240
<v Speaker 4>talk about what I do. You know, I need a

0:11:05.240 --> 0:11:08.680
<v Speaker 4>lot of advanced prep right I talked about what my

0:11:08.800 --> 0:11:12.240
<v Speaker 4>job is, what IBM has been putting in place for

0:11:12.360 --> 0:11:16.600
<v Speaker 4>years now, so this isn't about creating something. This was

0:11:16.800 --> 0:11:19.240
<v Speaker 4>just about showing up and being authentic. And we were

0:11:19.240 --> 0:11:22.360
<v Speaker 4>invited for a reason. We were invited because we were

0:11:22.400 --> 0:11:26.120
<v Speaker 4>one of the earliest companies in the AI technology space.

0:11:26.760 --> 0:11:31.920
<v Speaker 4>We're the oldest technology company and we are trusted and

0:11:32.240 --> 0:11:34.640
<v Speaker 4>that's an honor. And then the second thing I came

0:11:34.679 --> 0:11:37.400
<v Speaker 4>away with was really how important this issue is to society.

0:11:37.440 --> 0:11:41.920
<v Speaker 4>I don't think I appreciated it as much until following

0:11:42.040 --> 0:11:46.360
<v Speaker 4>that experience. I had outreached from colleagues I hadn't worked

0:11:46.400 --> 0:11:49.280
<v Speaker 4>with for years. I had an outreach from family members

0:11:49.280 --> 0:11:51.960
<v Speaker 4>who heard me on the radio. You know, my mother

0:11:52.120 --> 0:11:55.000
<v Speaker 4>and my mother in law, and my nieces and nephews

0:11:55.040 --> 0:11:57.439
<v Speaker 4>and my friends of my kids were all like, oh,

0:11:57.440 --> 0:11:59.360
<v Speaker 4>I get it. I get what you do. Now, Wow,

0:11:59.440 --> 0:12:02.840
<v Speaker 4>that's pretty you know. So that was really probably the

0:12:02.960 --> 0:12:05.520
<v Speaker 4>best and most impactful takeaway that I had.

0:12:06.000 --> 0:12:09.640
<v Speaker 2>The mass adoption of generative AI, happening at breakneck speed,

0:12:10.000 --> 0:12:14.000
<v Speaker 2>has spurred societies and governments around the world to get

0:12:14.120 --> 0:12:20.400
<v Speaker 2>serious about regulating AI for businesses. Compliance is complex enough already,

0:12:20.600 --> 0:12:23.880
<v Speaker 2>but throw an ever involving technology like AI into the mix,

0:12:24.320 --> 0:12:30.600
<v Speaker 2>and compliance itself becomes an exercise in adaptability. As regulators

0:12:30.600 --> 0:12:35.160
<v Speaker 2>seek greater accountability in how AI is used, businesses need

0:12:35.240 --> 0:12:40.160
<v Speaker 2>help creating governance processes comprehensive enough to comply with the law,

0:12:40.400 --> 0:12:43.480
<v Speaker 2>but agile enough to keep up with the rapid rate

0:12:43.520 --> 0:12:48.880
<v Speaker 2>of change in AI development. Regulatory scrutiny isn't the only

0:12:48.960 --> 0:12:54.120
<v Speaker 2>consideration either responsible AI governance. Of business's ability to prove

0:12:54.160 --> 0:12:59.000
<v Speaker 2>its AI models are transparent and explainable is also key

0:12:59.040 --> 0:13:04.080
<v Speaker 2>to building trust with customers, regardless of industry. In the

0:13:04.120 --> 0:13:08.319
<v Speaker 2>next part of their conversation, Laurie asked Christina what businesses

0:13:08.360 --> 0:13:12.760
<v Speaker 2>should consider when approaching AI governance. Let's listen.

0:13:13.600 --> 0:13:16.040
<v Speaker 3>So it's a particular role that businesses are playing in

0:13:16.120 --> 0:13:18.920
<v Speaker 3>AI governance, Like why is it so critical for businesses

0:13:18.960 --> 0:13:19.840
<v Speaker 3>to be part of this?

0:13:20.480 --> 0:13:24.959
<v Speaker 4>So, I think it's really critically important that businesses understand

0:13:25.480 --> 0:13:28.400
<v Speaker 4>the impacts that technology can have, both in making them

0:13:28.400 --> 0:13:32.240
<v Speaker 4>better businesses, but the impacts that those technologies can have

0:13:32.720 --> 0:13:37.839
<v Speaker 4>on the consumers that they are supporting. You know, businesses

0:13:37.960 --> 0:13:42.560
<v Speaker 4>need to be deploying AI technology that is in alignment

0:13:42.720 --> 0:13:44.560
<v Speaker 4>with the goals that they set for it and that

0:13:44.720 --> 0:13:47.000
<v Speaker 4>can be trusted, I think for us and for our

0:13:47.040 --> 0:13:50.839
<v Speaker 4>clients A lot of this comes back to trust in tech.

0:13:51.280 --> 0:13:56.960
<v Speaker 4>If you deploy something that doesn't work, that hallucinates, that discriminates,

0:13:57.440 --> 0:14:01.600
<v Speaker 4>that isn't transparent, where decisions can't be explained, then you

0:14:01.720 --> 0:14:05.360
<v Speaker 4>are going to very rapidly erode the trust at best

0:14:05.520 --> 0:14:08.839
<v Speaker 4>right of your clients and at worst for yourself. You're

0:14:08.880 --> 0:14:11.560
<v Speaker 4>going to create legal and regulatory issues for yourself as well.

0:14:11.600 --> 0:14:15.520
<v Speaker 4>So trusted technology is really important, and I think there's

0:14:15.559 --> 0:14:17.760
<v Speaker 4>a lot of pressure on businesses today to move very

0:14:17.840 --> 0:14:20.520
<v Speaker 4>rapidly and adopt technology. But if you do it without

0:14:20.600 --> 0:14:24.080
<v Speaker 4>having a program of governance in place, you're really risking

0:14:24.280 --> 0:14:25.240
<v Speaker 4>eroding that trust.

0:14:25.360 --> 0:14:27.560
<v Speaker 3>So this is really where I think a strong AI

0:14:27.640 --> 0:14:31.200
<v Speaker 3>governance comes in. Talk about from your perspective, how this

0:14:31.280 --> 0:14:35.160
<v Speaker 3>really contributes to maintaining the trust that customers and stakeholders

0:14:35.200 --> 0:14:36.360
<v Speaker 3>have in these technologies.

0:14:37.080 --> 0:14:39.640
<v Speaker 4>Yeah. Absolutely. I mean you need to have a governance

0:14:39.680 --> 0:14:43.680
<v Speaker 4>program because you need to understand that the technology, particularly

0:14:43.720 --> 0:14:48.280
<v Speaker 4>in the AI space, that you are deploying, is explainable.

0:14:48.320 --> 0:14:52.680
<v Speaker 4>You need to understand why it's making decisions and recommendations

0:14:52.720 --> 0:14:54.200
<v Speaker 4>that it's making, and you need to be able to

0:14:54.200 --> 0:14:56.600
<v Speaker 4>explain that to your consumers. I mean, you can't do

0:14:56.680 --> 0:14:58.760
<v Speaker 4>that if you don't know where your data is coming from,

0:14:58.760 --> 0:15:01.240
<v Speaker 4>what data are you using to train those models, if

0:15:01.240 --> 0:15:05.040
<v Speaker 4>you don't have a program that manages the alignment of

0:15:05.080 --> 0:15:08.640
<v Speaker 4>your AI models over time to make sure as AI

0:15:08.880 --> 0:15:13.360
<v Speaker 4>learns and evolves over uses, which is in large part

0:15:13.880 --> 0:15:17.680
<v Speaker 4>what makes it so beneficial that it stays in alignment

0:15:17.720 --> 0:15:21.200
<v Speaker 4>with the objectives that you set for the technology over time.

0:15:21.840 --> 0:15:25.680
<v Speaker 4>So you can't do that without a robust governance process

0:15:25.680 --> 0:15:29.160
<v Speaker 4>in place. So we work with clients to share our

0:15:29.200 --> 0:15:31.680
<v Speaker 4>own story here at IBM in terms of how we

0:15:31.720 --> 0:15:35.320
<v Speaker 4>put that in place, but also in our consulting practice

0:15:36.080 --> 0:15:40.600
<v Speaker 4>to help clients work with these new generative capabilities and

0:15:40.600 --> 0:15:43.560
<v Speaker 4>foundation models and the like in order to put them

0:15:43.560 --> 0:15:45.600
<v Speaker 4>to work for their business in a way that's going

0:15:45.640 --> 0:15:48.280
<v Speaker 4>to be impactful to that business but at the same

0:15:48.360 --> 0:15:49.440
<v Speaker 4>time be trusted.

0:15:49.840 --> 0:15:51.720
<v Speaker 3>So now I wanted to turn a little bit towards

0:15:51.720 --> 0:15:56.200
<v Speaker 3>watsonex governance, and so IBM recently announced their AI platform, watsonex,

0:15:56.440 --> 0:15:59.600
<v Speaker 3>which will include a governance component. Could you tell us

0:15:59.640 --> 0:16:01.960
<v Speaker 3>a little more about watsonex dot governance.

0:16:02.320 --> 0:16:04.560
<v Speaker 4>Yeah, I mean before I do that, I'll just back

0:16:04.640 --> 0:16:08.240
<v Speaker 4>up and talk about the full platform and then lean

0:16:08.280 --> 0:16:11.160
<v Speaker 4>into Watson X because I think it's important to understand

0:16:11.440 --> 0:16:17.360
<v Speaker 4>the delivery of a full suite of capabilities, to get data,

0:16:17.440 --> 0:16:20.360
<v Speaker 4>to train models, and then to govern them over their

0:16:20.400 --> 0:16:25.920
<v Speaker 4>life cycle. All of these things are really important from

0:16:25.960 --> 0:16:28.800
<v Speaker 4>the onset you need to make sure that you have,

0:16:29.440 --> 0:16:33.680
<v Speaker 4>you know, for our watsonex dot AI for example, that's

0:16:33.720 --> 0:16:38.280
<v Speaker 4>the studio to train new foundation models and generative AI

0:16:38.360 --> 0:16:44.000
<v Speaker 4>and machine learning capabilities, and we are populating that studio

0:16:44.360 --> 0:16:49.640
<v Speaker 4>with some IBM trained foundation models which we're curating and

0:16:49.720 --> 0:16:53.480
<v Speaker 4>tailoring more specifically for enterprises. So that's really important. It

0:16:53.520 --> 0:16:55.880
<v Speaker 4>comes back to the point I made earlier about business

0:16:55.880 --> 0:17:00.200
<v Speaker 4>trust and the need you know to have enterprise is

0:17:00.280 --> 0:17:05.600
<v Speaker 4>ready technologies in the AI space. And then the watsonex

0:17:05.600 --> 0:17:08.800
<v Speaker 4>dot data is a fit for purpose data store or

0:17:08.840 --> 0:17:12.560
<v Speaker 4>a data lake, and then watsonex dot gov. So that's

0:17:12.960 --> 0:17:17.600
<v Speaker 4>a particular component of the platform that my team and

0:17:17.680 --> 0:17:21.080
<v Speaker 4>the AI Ethics Board has really worked closely with the

0:17:21.119 --> 0:17:24.680
<v Speaker 4>product team on developing, and we're using it internally here

0:17:24.680 --> 0:17:27.880
<v Speaker 4>in the Chief Privacy Office as well to help us

0:17:28.320 --> 0:17:33.240
<v Speaker 4>govern our own uses of AI technology and our compliance

0:17:33.280 --> 0:17:39.359
<v Speaker 4>program here. And it essentially helps to notify you if

0:17:39.400 --> 0:17:42.520
<v Speaker 4>a model becomes biased or gets out of alignment as

0:17:42.560 --> 0:17:45.440
<v Speaker 4>you're using it over time. So companies are going to

0:17:45.520 --> 0:17:48.040
<v Speaker 4>need these capabilities. I mean they need them today to

0:17:48.119 --> 0:17:53.080
<v Speaker 4>deliver technologies with trust. They'll need them tomorrow to comply

0:17:53.200 --> 0:17:55.280
<v Speaker 4>with regulation, which is on the horizon.

0:17:55.520 --> 0:17:58.399
<v Speaker 3>And I think compliance becomes even more complex when you

0:17:58.440 --> 0:18:02.560
<v Speaker 3>consider international data protection laws and regulations. Honestly, I don't

0:18:02.600 --> 0:18:05.160
<v Speaker 3>know how anyone on any company legal team is keeping

0:18:05.240 --> 0:18:07.480
<v Speaker 3>up with this these days. But my question for you

0:18:07.600 --> 0:18:11.200
<v Speaker 3>is really how can businesses develop a strategy to maintain

0:18:11.240 --> 0:18:14.040
<v Speaker 3>compliance and to deal with it in this ever changing landscape.

0:18:14.119 --> 0:18:18.040
<v Speaker 4>It's increasingly more challenging. In fact, I saw statistic just

0:18:18.119 --> 0:18:22.720
<v Speaker 4>this morning that the regulatory obligations on companies have increased

0:18:22.720 --> 0:18:26.600
<v Speaker 4>something like seven hundred times in the last twenty years.

0:18:26.400 --> 0:18:31.000
<v Speaker 4>So it really is a huge focus area for companies.

0:18:31.160 --> 0:18:34.040
<v Speaker 4>You have to have a process in place in order

0:18:34.080 --> 0:18:36.600
<v Speaker 4>to do that, and it's not easy, particularly for a

0:18:36.680 --> 0:18:40.560
<v Speaker 4>company like IBM that it has a presence in over

0:18:40.560 --> 0:18:43.600
<v Speaker 4>one hundred and seventy countries around the world. There is

0:18:43.640 --> 0:18:48.520
<v Speaker 4>more than one hundred and fifty comprehensive privacy regulations. There

0:18:48.560 --> 0:18:53.080
<v Speaker 4>are regulations of non personal data, there are AI regulations emerging,

0:18:54.040 --> 0:18:58.119
<v Speaker 4>so you really need an operational approach to it in

0:18:58.240 --> 0:19:00.280
<v Speaker 4>order to stay compliant. But one of the things we

0:19:00.320 --> 0:19:02.480
<v Speaker 4>do as we set a baseline, and a lot of

0:19:02.480 --> 0:19:05.919
<v Speaker 4>companies do this as well, So we define a privacy baseline,

0:19:05.960 --> 0:19:10.760
<v Speaker 4>we define an AI baseline, and we ensure then as

0:19:10.800 --> 0:19:13.200
<v Speaker 4>a result of that that there are very few deviances

0:19:13.280 --> 0:19:16.359
<v Speaker 4>because it incorporates in that baseline. So that's one of

0:19:16.400 --> 0:19:18.560
<v Speaker 4>the ways we do it. Other companies, I think are

0:19:18.640 --> 0:19:23.840
<v Speaker 4>similarly situated in terms of doing that. But again, it

0:19:24.000 --> 0:19:26.679
<v Speaker 4>is a real challenge for global companies. It's one of

0:19:26.680 --> 0:19:30.640
<v Speaker 4>the reasons why we advocate for as much alignment as

0:19:30.720 --> 0:19:36.080
<v Speaker 4>possible on the international realm as well as nationally here

0:19:36.119 --> 0:19:39.920
<v Speaker 4>in the US, as much alignment as possible to make

0:19:40.119 --> 0:19:45.080
<v Speaker 4>compliance easier for easier, and not just because companies want

0:19:45.119 --> 0:19:48.280
<v Speaker 4>an easy way to comply, but the harder it is,

0:19:48.520 --> 0:19:52.399
<v Speaker 4>the less likely there will be compliance. And it's not

0:19:52.520 --> 0:19:58.600
<v Speaker 4>the objective of anybody, governments, companies, consumers to have to

0:19:58.760 --> 0:20:02.280
<v Speaker 4>set legal obligation that companies simply can't meet.

0:20:02.520 --> 0:20:04.560
<v Speaker 3>So what advice would you give to other companies who

0:20:04.560 --> 0:20:07.640
<v Speaker 3>are looking to rethink or strengthen their approach to AI. Governance.

0:20:07.720 --> 0:20:11.040
<v Speaker 4>Think you need to start with, as we did, foundational principles,

0:20:11.680 --> 0:20:15.119
<v Speaker 4>and you need to start making decisions about what technology

0:20:15.119 --> 0:20:17.480
<v Speaker 4>you're going to deploy and what technology you're not, What

0:20:17.480 --> 0:20:18.600
<v Speaker 4>are you going to use it for, and what aren't

0:20:18.600 --> 0:20:20.000
<v Speaker 4>you going to use it for? And then when you

0:20:20.040 --> 0:20:24.400
<v Speaker 4>do use it, align to those principles. That's really important.

0:20:24.480 --> 0:20:29.000
<v Speaker 4>Formalize a program, have someone within the organization, whether it's

0:20:29.040 --> 0:20:33.320
<v Speaker 4>the chief privacy officer, whether it's some other role, a

0:20:33.400 --> 0:20:37.840
<v Speaker 4>chief AI ethics officer, but have an accountable individual and

0:20:37.920 --> 0:20:42.080
<v Speaker 4>accountable organization. Do a maturity assessment, figure out where you

0:20:42.160 --> 0:20:44.919
<v Speaker 4>are and where you need to be, and really start

0:20:46.000 --> 0:20:50.480
<v Speaker 4>putting it into place today. Don't wait for regulation to

0:20:50.560 --> 0:20:53.200
<v Speaker 4>apply directly to your business, because it'll be too late.

0:20:54.440 --> 0:20:57.840
<v Speaker 3>So Smart Talks features new creators, these visionaries like yourself

0:20:57.840 --> 0:21:01.320
<v Speaker 3>who are creatively applying technology in business to drive change.

0:21:01.640 --> 0:21:04.320
<v Speaker 3>I'm curious if you see yourself as creative.

0:21:04.480 --> 0:21:07.800
<v Speaker 4>You know, I definitely do. I mean you need to

0:21:07.880 --> 0:21:12.520
<v Speaker 4>be creative when you're working in an industry that evolves

0:21:12.560 --> 0:21:17.040
<v Speaker 4>so very quickly. So you know, I started with IBM

0:21:17.320 --> 0:21:20.440
<v Speaker 4>when we were primarily a hardware company, right, and we've

0:21:20.520 --> 0:21:23.960
<v Speaker 4>changed our business so significantly over the years and the

0:21:24.040 --> 0:21:28.480
<v Speaker 4>issues that are raised with respect to each new technology,

0:21:28.520 --> 0:21:32.160
<v Speaker 4>whether it be cloud, whether it be AI now where

0:21:32.160 --> 0:21:33.760
<v Speaker 4>we're seeing a ton of issues, or you look at

0:21:33.760 --> 0:21:37.919
<v Speaker 4>emergent issues in the space of things like neurotechnologies and

0:21:38.000 --> 0:21:44.600
<v Speaker 4>quantum computers. You have to be strategic and you have

0:21:44.720 --> 0:21:47.960
<v Speaker 4>to be creative and thinking about how you can adapt

0:21:48.600 --> 0:21:53.840
<v Speaker 4>agilely quickly a company to an environment that is changing

0:21:53.880 --> 0:21:56.040
<v Speaker 4>so quickly and with.

0:21:56.080 --> 0:21:59.000
<v Speaker 3>This transformation happening at such a rapid pace. Do you

0:21:59.040 --> 0:22:01.320
<v Speaker 3>think creativity is a role in how you think about

0:22:01.359 --> 0:22:05.520
<v Speaker 3>and implement specifically a trustworthy AI strategy.

0:22:06.600 --> 0:22:10.640
<v Speaker 4>Yeah, I absolutely think it does, because again it comes

0:22:10.680 --> 0:22:13.840
<v Speaker 4>back to these capabilities, and there are ways, I guess

0:22:14.040 --> 0:22:17.800
<v Speaker 4>how you define creativity could be different, right, but I'm

0:22:17.840 --> 0:22:21.040
<v Speaker 4>thinking of creativity in the sense of sort of agility

0:22:21.080 --> 0:22:25.160
<v Speaker 4>and strategic vision and creative problem solving. I think that's

0:22:25.440 --> 0:22:28.560
<v Speaker 4>really important in the world that we're in right now,

0:22:28.600 --> 0:22:33.159
<v Speaker 4>being able to creatively problem solve with new issues that

0:22:33.240 --> 0:22:36.320
<v Speaker 4>are rising sort of every day.

0:22:36.960 --> 0:22:38.520
<v Speaker 3>And so, how do you see the role of chief

0:22:38.560 --> 0:22:42.200
<v Speaker 3>privacy officer evolving in the future as AI technology continues

0:22:42.240 --> 0:22:45.120
<v Speaker 3>to advance, Like what stuff should CPOs take to stay

0:22:45.119 --> 0:22:47.040
<v Speaker 3>ahead of all these changes that are come in their way.

0:22:48.320 --> 0:22:52.240
<v Speaker 4>So the role is evolving in most companies, I would

0:22:52.280 --> 0:22:57.240
<v Speaker 4>say pretty rapidly. Many companies are looking to chief privacy

0:22:57.280 --> 0:23:00.800
<v Speaker 4>officers who are ready understand the data that's being used

0:23:00.800 --> 0:23:04.320
<v Speaker 4>in the organization and have programs to ensure compliance with

0:23:04.440 --> 0:23:08.240
<v Speaker 4>laws that require you to manage that data in accordance

0:23:08.240 --> 0:23:10.880
<v Speaker 4>with data protection laws and the like. It's a natural

0:23:10.920 --> 0:23:16.920
<v Speaker 4>place and position for AI responsibility, and so I think

0:23:16.920 --> 0:23:19.600
<v Speaker 4>what's happening to a lot of chief privacy officers is

0:23:19.640 --> 0:23:23.119
<v Speaker 4>they're being asked to take on this AI governance responsibility

0:23:23.160 --> 0:23:26.680
<v Speaker 4>for companies and if not take it on, at least

0:23:26.720 --> 0:23:30.119
<v Speaker 4>play a very key role working with other parts of

0:23:30.160 --> 0:23:33.800
<v Speaker 4>the business in AI governance. So that really is changing.

0:23:34.040 --> 0:23:38.040
<v Speaker 4>And if chief privacy officers are in companies who maybe

0:23:38.119 --> 0:23:41.879
<v Speaker 4>haven't started thinking about AI yet, they should, so I

0:23:41.920 --> 0:23:45.760
<v Speaker 4>would encourage them to look at different resources that are

0:23:45.800 --> 0:23:50.159
<v Speaker 4>available already in AI governance space. For example, the International

0:23:50.200 --> 0:23:54.640
<v Speaker 4>Association of Privacy Professionals, which is the seventy five thousand

0:23:54.680 --> 0:23:59.760
<v Speaker 4>member professional body for the profession of Chief Privacy officers,

0:23:59.840 --> 0:24:04.520
<v Speaker 4>just recently launched an AI Governance Initiative and an AI

0:24:04.560 --> 0:24:08.520
<v Speaker 4>Governance Certification program. I sit on their advisory board. But

0:24:08.640 --> 0:24:11.320
<v Speaker 4>that's just emblematic of the fact that the field is

0:24:11.400 --> 0:24:12.840
<v Speaker 4>changing so rapidly.

0:24:14.359 --> 0:24:16.920
<v Speaker 3>And so, you know, speaking of rapid change, when you're

0:24:17.200 --> 0:24:19.600
<v Speaker 3>back here on smart Talks in twenty twenty one, you

0:24:19.640 --> 0:24:21.960
<v Speaker 3>said that the future of AI will be more transparent

0:24:22.000 --> 0:24:24.200
<v Speaker 3>and more trustworthy. You know, what do you see the

0:24:24.240 --> 0:24:26.280
<v Speaker 3>next five to ten years holding. You know, when you're

0:24:26.320 --> 0:24:29.240
<v Speaker 3>back on smart Talks in you know, twenty twenty six,

0:24:29.320 --> 0:24:31.000
<v Speaker 3>you know twenty thirty, You know what are we going

0:24:31.080 --> 0:24:33.359
<v Speaker 3>to be talking about when it comes to AI technology

0:24:33.359 --> 0:24:34.000
<v Speaker 3>and governance.

0:24:34.560 --> 0:24:36.719
<v Speaker 4>So I try to be an optimist, right And I

0:24:36.760 --> 0:24:40.600
<v Speaker 4>said that two years ago, and I think we're seeing

0:24:40.640 --> 0:24:45.040
<v Speaker 4>it now come into fruition, and there will be requirements,

0:24:45.840 --> 0:24:48.639
<v Speaker 4>whether they're coming from the US, whether they're coming from Europe,

0:24:48.720 --> 0:24:52.320
<v Speaker 4>whether they're just coming from voluntary adoption by clients of

0:24:52.400 --> 0:24:57.280
<v Speaker 4>things like the NISS Risk Management Framework, really important voluntary frameworks.

0:24:57.880 --> 0:25:01.760
<v Speaker 4>You're going to have to adopt transparent, explainable practices in

0:25:01.800 --> 0:25:04.240
<v Speaker 4>your uses of AI. So I do see that happening.

0:25:04.320 --> 0:25:06.760
<v Speaker 4>And in the next five to ten years, boy, I

0:25:06.800 --> 0:25:12.760
<v Speaker 4>think we'll see more research into trust in techniques because

0:25:12.760 --> 0:25:16.720
<v Speaker 4>we don't really know, for example, how to water mark.

0:25:17.480 --> 0:25:21.000
<v Speaker 4>We're calling for things like watermarking. There'll be more research

0:25:21.040 --> 0:25:26.879
<v Speaker 4>into how to do that. I think you'll see regulation

0:25:27.000 --> 0:25:29.479
<v Speaker 4>that's specifically going to require those types of things. So

0:25:29.600 --> 0:25:31.600
<v Speaker 4>I think again, I think the regulation is going to

0:25:31.680 --> 0:25:34.920
<v Speaker 4>drive research. It's going to drive research into these areas

0:25:34.960 --> 0:25:39.600
<v Speaker 4>that will help ensure that we can deliver new capabilities,

0:25:39.640 --> 0:25:43.000
<v Speaker 4>generated capabilities and the like with trust and explainability.

0:25:43.480 --> 0:25:45.520
<v Speaker 3>Thank you so much Christina for joining me on smart

0:25:45.520 --> 0:25:47.240
<v Speaker 3>Talks to talk about AI and governance.

0:25:47.920 --> 0:25:49.880
<v Speaker 4>Well, thank you very much for having me.

0:25:51.440 --> 0:25:56.159
<v Speaker 2>To unlock the transformative growth possible with artificial intelligence. Businesses

0:25:56.240 --> 0:25:59.359
<v Speaker 2>need to know what they wish to grow into first.

0:26:00.320 --> 0:26:03.159
<v Speaker 2>Like Christina said, the best way forward in the AI

0:26:03.280 --> 0:26:07.320
<v Speaker 2>future is for businesses to figure out their own foundational

0:26:07.359 --> 0:26:11.960
<v Speaker 2>principles around using the technology, drawing on those principles to

0:26:12.040 --> 0:26:15.840
<v Speaker 2>apply AI in a way that's ethically consistent with their

0:26:15.880 --> 0:26:19.560
<v Speaker 2>mission and complies with the legal frameworks built to hold

0:26:19.560 --> 0:26:24.560
<v Speaker 2>the technology accountable. As AI adoption grows more and more widespread,

0:26:24.800 --> 0:26:29.119
<v Speaker 2>so too will the expectation from consumers and regulators that

0:26:29.200 --> 0:26:34.639
<v Speaker 2>businesses use it responsibly. Investing independable AI governance is a

0:26:34.680 --> 0:26:38.639
<v Speaker 2>way for businesses to lay the foundations for technology that

0:26:38.680 --> 0:26:42.560
<v Speaker 2>their customers can trust while rising to the challenge of

0:26:42.680 --> 0:26:48.440
<v Speaker 2>increasing regulatory complexity. Though the emergence of AI does complicate

0:26:48.600 --> 0:26:53.160
<v Speaker 2>an already tough compliance landscape, businesses now face a creative

0:26:53.200 --> 0:26:57.840
<v Speaker 2>opportunity to set a precedent for what accountability in AI

0:26:58.000 --> 0:27:00.800
<v Speaker 2>looks like and rethink what it means means to deploy

0:27:01.200 --> 0:27:07.320
<v Speaker 2>trustworthy artificial intelligence. I'm Malcolm Gladwell. This is a paid

0:27:07.440 --> 0:27:11.800
<v Speaker 2>advertisement from IBM. Smart Talks with IBM will be taking

0:27:11.840 --> 0:27:15.080
<v Speaker 2>a short hiatus, but look for new episodes in the

0:27:15.119 --> 0:27:19.280
<v Speaker 2>coming weeks. Smart Talks with IBM is produced by Matt Romano,

0:27:19.840 --> 0:27:24.760
<v Speaker 2>David jaw Nische Venkat and Royston Deserve with Jacob Goldstein.

0:27:25.040 --> 0:27:29.080
<v Speaker 2>We're edited by Lydia jen Kott. Our engineer is Jason Gambrel.

0:27:29.400 --> 0:27:34.320
<v Speaker 2>Theme song by Gramoscope. Special thanks to Carli Migliori, Andy Kelly,

0:27:34.720 --> 0:27:38.960
<v Speaker 2>Kathy Callahan and the eight Bar and IBM teams, as

0:27:39.000 --> 0:27:43.120
<v Speaker 2>well as the Pushkin marketing team. Smart Talks with IBM

0:27:43.440 --> 0:27:47.720
<v Speaker 2>is a production of Pushkin Industries and Ruby Studio at iHeartMedia.

0:27:48.359 --> 0:27:52.480
<v Speaker 2>To find more Pushkin podcasts, listen on the iHeartRadio, app,

0:27:52.760 --> 0:28:00.359
<v Speaker 2>Apple podcasts, or wherever you listen to podcasts. H