1
00:00:04,559 --> 00:00:12,760
Speaker 1: Welcome to Tech Stuff, a production from iHeartRadio. Today, we

2
00:00:12,800 --> 00:00:15,760
Speaker 1: are witnessed to one of those rare moments in history,

3
00:00:16,120 --> 00:00:19,360
Speaker 1: the rise of an innovative technology with the potential to

4
00:00:19,480 --> 00:00:24,200
Speaker 1: radically transform business in society forever. That technology, of course,

5
00:00:24,720 --> 00:00:28,240
Speaker 1: is artificial intelligence, and it's the central focus for this

6
00:00:28,440 --> 00:00:32,440
Speaker 1: new season of Smart Talks with IBM. Join hosts from

7
00:00:32,479 --> 00:00:36,199
Speaker 1: your favorite Pushkin podcasts as they talk with industry experts

8
00:00:36,200 --> 00:00:39,800
Speaker 1: and leaders to explore how businesses can integrate AI into

9
00:00:39,840 --> 00:00:43,160
Speaker 1: their workflows and help drive real change in this new

10
00:00:43,280 --> 00:00:46,960
Speaker 1: era of AI, and of course, host Malcolm Gladwell will

11
00:00:46,960 --> 00:00:49,280
Speaker 1: be there to guide you through the season and throw

12
00:00:49,360 --> 00:00:52,279
Speaker 1: in his two cents as well. Look out for new

13
00:00:52,280 --> 00:00:55,160
Speaker 1: episodes of Smart Talks with IBM every other week on

14
00:00:55,240 --> 00:00:59,440
Speaker 1: the iHeartRadio app, Apple Podcasts, or wherever you get your podcasts,

15
00:00:59,640 --> 00:01:03,880
Speaker 1: and learn more at IBM dot com slash smart talks.

16
00:01:06,400 --> 00:01:10,280
Speaker 2: Hello, Hello, Welcome to Smart Talks with IBM, a podcast

17
00:01:10,280 --> 00:01:16,880
Speaker 2: from Pushkin Industri's iHeartRadio and IBM. I'm Malcolm Gladwell. This season,

18
00:01:16,920 --> 00:01:21,280
Speaker 2: we're continuing our conversation with new creators visionaries who are

19
00:01:21,319 --> 00:01:24,840
Speaker 2: creatively applying technology in business to drive change, but with

20
00:01:24,920 --> 00:01:29,840
Speaker 2: a focus on the transformative power of artificial intelligence and

21
00:01:29,920 --> 00:01:33,040
Speaker 2: what it means to leverage AI as a game changing

22
00:01:33,160 --> 00:01:38,160
Speaker 2: multiplier for your business. Our guest today is Christina Montgomery,

23
00:01:38,560 --> 00:01:42,640
Speaker 2: IBM's Chief Privacy and Trust Officer. She's also chair of

24
00:01:42,680 --> 00:01:48,120
Speaker 2: IBM's AI Ethics Board. In addition to overseeing IBM's privacy policy,

25
00:01:48,480 --> 00:01:53,480
Speaker 2: a core part of Christina's job involves AI governance, making

26
00:01:53,520 --> 00:01:57,320
Speaker 2: sure the way AI is used complies with the international

27
00:01:57,640 --> 00:02:03,480
Speaker 2: legal regulations customized for each industry. In today's episode, Christina

28
00:02:03,520 --> 00:02:08,440
Speaker 2: will explain why businesses need foundational principles when it comes

29
00:02:08,520 --> 00:02:13,240
Speaker 2: to using technology, why AI regulation should focus on specific

30
00:02:13,440 --> 00:02:17,720
Speaker 2: use cases over the technology itself, and share a little

31
00:02:17,720 --> 00:02:23,360
Speaker 2: bit about her landmark congressional testimony. Last May, Christina spoke

32
00:02:23,440 --> 00:02:27,080
Speaker 2: with doctor Lori Santos, host of the Pushkin podcast The

33
00:02:27,120 --> 00:02:32,600
Speaker 2: Happiness Lab, a cognitive scientist and psychology professor at Yale University.

34
00:02:32,919 --> 00:02:38,280
Speaker 2: Laurie is an expert on human happiness and cognition. Okay,

35
00:02:38,800 --> 00:02:40,079
Speaker 2: let's get to the interview.

36
00:02:42,639 --> 00:02:44,720
Speaker 3: So Christina, I'm so excited to talk to you today.

37
00:02:44,880 --> 00:02:47,080
Speaker 3: So let's start by talking a little bit about your

38
00:02:47,160 --> 00:02:49,840
Speaker 3: role at IBM. What does a Chief Privacy and trust

39
00:02:49,880 --> 00:02:50,920
Speaker 3: officer actually do.

40
00:02:51,440 --> 00:02:55,200
Speaker 4: It's a really dynamic profession and it's not a new profession,

41
00:02:55,760 --> 00:02:58,640
Speaker 4: but the role has really changed. I mean, my role

42
00:02:58,760 --> 00:03:02,480
Speaker 4: today is broader and just helping to ensure compliance with

43
00:03:02,600 --> 00:03:07,240
Speaker 4: data protection laws globally. I'm also responsible for AI governance

44
00:03:07,280 --> 00:03:10,239
Speaker 4: I co chair or AI Ethics Board here at IBM,

45
00:03:10,560 --> 00:03:13,600
Speaker 4: and for data clearance and data governance as well for

46
00:03:13,680 --> 00:03:17,240
Speaker 4: the company. So I have both a compliance aspect to

47
00:03:17,280 --> 00:03:19,960
Speaker 4: my role, really important on a global basis, but also

48
00:03:20,520 --> 00:03:25,440
Speaker 4: help the business to competitively differentiate because really trust is

49
00:03:25,440 --> 00:03:29,160
Speaker 4: a strategic advantage for IBM and a competitive differentiator as

50
00:03:29,160 --> 00:03:33,080
Speaker 4: a company that's been responsibly managing the most sensitive data

51
00:03:33,120 --> 00:03:35,720
Speaker 4: for our clients for more than a century now and

52
00:03:35,880 --> 00:03:38,600
Speaker 4: helping to usher new technologies into the world with trust

53
00:03:38,600 --> 00:03:42,000
Speaker 4: and transparency. And so that's also a key aspect of

54
00:03:42,040 --> 00:03:42,480
Speaker 4: my role.

55
00:03:43,240 --> 00:03:45,440
Speaker 3: And so you joined us here on smart Talks back

56
00:03:45,440 --> 00:03:47,680
Speaker 3: in twenty twenty one and you chatted with us about

57
00:03:47,680 --> 00:03:51,360
Speaker 3: IBM's approach of building trust and transparency with AI, and

58
00:03:51,520 --> 00:03:53,520
Speaker 3: that was only two years ago, but it almost feels

59
00:03:53,560 --> 00:03:55,880
Speaker 3: like in eternity has happened in the field of AI

60
00:03:56,040 --> 00:03:59,040
Speaker 3: since then, and so I'm curious how much has changed

61
00:03:59,080 --> 00:04:01,560
Speaker 3: since you were here last time. Were the things you

62
00:04:01,600 --> 00:04:04,440
Speaker 3: told us before you are they still true? How are

63
00:04:04,480 --> 00:04:05,120
Speaker 3: things changing?

64
00:04:05,520 --> 00:04:09,000
Speaker 4: You're absolutely right. It feels like the world has changed

65
00:04:09,080 --> 00:04:12,760
Speaker 4: really in the last two years. But the same fundamental

66
00:04:12,800 --> 00:04:17,720
Speaker 4: principles and the same overall governance apply to IBM's program

67
00:04:18,279 --> 00:04:22,640
Speaker 4: for Data Protection and Responsible AI that we talked about

68
00:04:22,680 --> 00:04:25,360
Speaker 4: two years ago, and not much has changed there from

69
00:04:25,400 --> 00:04:28,560
Speaker 4: our perspective. And the good thing is we've put these

70
00:04:28,600 --> 00:04:33,039
Speaker 4: practices and this governance approach into place, and we have

71
00:04:33,279 --> 00:04:36,920
Speaker 4: an established way of looking at these emerging technologies. As

72
00:04:36,960 --> 00:04:40,360
Speaker 4: the technology evolves, the tech is more powerful, for sure,

73
00:04:40,520 --> 00:04:44,560
Speaker 4: foundation models are vastly larger and more capable, and our

74
00:04:44,640 --> 00:04:48,080
Speaker 4: creating in some respects new issues. But that just makes

75
00:04:48,080 --> 00:04:49,880
Speaker 4: it all the more urgent to do what we've been

76
00:04:49,920 --> 00:04:53,520
Speaker 4: doing and to put trust and transparency into place across

77
00:04:53,560 --> 00:04:56,200
Speaker 4: the business to be accountable to those principles.

78
00:04:57,400 --> 00:04:59,560
Speaker 3: And so our conversation today is really centered around this

79
00:04:59,680 --> 00:05:02,960
Speaker 3: need for new AI regulation and part of that regulation

80
00:05:03,080 --> 00:05:05,960
Speaker 3: involves the mitigation of bias. And this is something I

81
00:05:05,960 --> 00:05:08,320
Speaker 3: think about a ton as a psychologist, right, you know,

82
00:05:08,360 --> 00:05:11,120
Speaker 3: I know, like my students and everyone who's interacting with

83
00:05:11,200 --> 00:05:14,279
Speaker 3: AI is assuming that the kind of knowledge that they're

84
00:05:14,320 --> 00:05:17,320
Speaker 3: getting from this kind of learning is accurate, right, But

85
00:05:17,400 --> 00:05:19,440
Speaker 3: of course AI is only as good as the knowledge

86
00:05:19,440 --> 00:05:21,720
Speaker 3: that's going in. And so talk to me a little

87
00:05:21,760 --> 00:05:25,000
Speaker 3: bit about like why bias occurs in AI and the

88
00:05:25,080 --> 00:05:26,960
Speaker 3: level of the problem that we're really dealing with.

89
00:05:27,800 --> 00:05:30,840
Speaker 4: Yeah, well, obviously AI is based on data, right, It's

90
00:05:31,200 --> 00:05:36,200
Speaker 4: trained with data, and that data could be biased in

91
00:05:36,200 --> 00:05:38,680
Speaker 4: and of itself, and that's where issues could come up.

92
00:05:38,720 --> 00:05:40,600
Speaker 4: They come up in the data, they could also come

93
00:05:40,680 --> 00:05:44,200
Speaker 4: up in the output of the models themselves. So it's

94
00:05:44,320 --> 00:05:49,080
Speaker 4: really important that you build bias consideration and bias testing

95
00:05:49,200 --> 00:05:52,600
Speaker 4: into your product development cycle. And so what we've been

96
00:05:52,640 --> 00:05:55,200
Speaker 4: thinking about here at IBM and doing we had some

97
00:05:55,279 --> 00:05:58,239
Speaker 4: of our research teams delivered some of the very first

98
00:05:58,240 --> 00:06:01,599
Speaker 4: toolkits to help detect bias years ago now right, and

99
00:06:01,640 --> 00:06:05,280
Speaker 4: deployed them to open source. And we have put into

100
00:06:05,320 --> 00:06:08,640
Speaker 4: place for our developers here at IBM and Ethics by

101
00:06:08,680 --> 00:06:11,880
Speaker 4: Design playbook that's sort of a step by step approach

102
00:06:12,640 --> 00:06:19,160
Speaker 4: which also addresses very fully biased considerations, and we provide

103
00:06:19,200 --> 00:06:22,400
Speaker 4: not only like here's a point when you should test

104
00:06:22,440 --> 00:06:24,840
Speaker 4: for it, and you consider it in the data you

105
00:06:24,880 --> 00:06:26,599
Speaker 4: have to measure it both at the data and the

106
00:06:26,640 --> 00:06:30,160
Speaker 4: model level or the outcome level, and we provide guidance

107
00:06:30,160 --> 00:06:32,839
Speaker 4: with respect to what tools can best be used to

108
00:06:32,880 --> 00:06:36,360
Speaker 4: accomplish that. So it's a really important issue. It's one

109
00:06:36,400 --> 00:06:39,360
Speaker 4: you can't just talk about. You have to provide essentially

110
00:06:39,480 --> 00:06:42,719
Speaker 4: the technology and the capabilities and the guidance to enable

111
00:06:42,760 --> 00:06:43,920
Speaker 4: people to test for it.

112
00:06:44,640 --> 00:06:47,320
Speaker 3: Recently, you had this wonderful opportunity to head to Congress

113
00:06:47,360 --> 00:06:50,400
Speaker 3: to talk about AI, and in your testimony before Congress,

114
00:06:50,440 --> 00:06:53,440
Speaker 3: you mentioned that it's often said that innovation moves too

115
00:06:53,480 --> 00:06:56,080
Speaker 3: fast for government to keep up, and this is something

116
00:06:56,080 --> 00:06:58,240
Speaker 3: that I also worry about as a psychologist. Right our

117
00:06:58,279 --> 00:07:01,160
Speaker 3: policy makers really understanding the issues that they're dealing with,

118
00:07:01,640 --> 00:07:03,880
Speaker 3: and so I'm curious how you're approaching this challenge of

119
00:07:03,920 --> 00:07:06,360
Speaker 3: adapting AI policies to keep up with the sort of

120
00:07:06,440 --> 00:07:08,960
Speaker 3: rapid pace of all the advancements we're seeing in the

121
00:07:08,960 --> 00:07:10,320
Speaker 3: AI technology itself.

122
00:07:11,280 --> 00:07:14,520
Speaker 4: I think it's really critically important that you have foundational

123
00:07:14,560 --> 00:07:20,000
Speaker 4: principles that applied to not only how you use technology,

124
00:07:20,040 --> 00:07:21,840
Speaker 4: but whether you're going to use it in the first place,

125
00:07:21,840 --> 00:07:23,520
Speaker 4: and where you're going to use and apply it across

126
00:07:23,600 --> 00:07:27,280
Speaker 4: your company, and then your program from a governance perspective,

127
00:07:27,360 --> 00:07:29,880
Speaker 4: has to be agile. It has to be able to

128
00:07:29,960 --> 00:07:35,680
Speaker 4: address emerging capabilities, new training methods, etc. And part of

129
00:07:35,720 --> 00:07:40,240
Speaker 4: that involves helping to educate and instill and empower a

130
00:07:40,280 --> 00:07:44,080
Speaker 4: trustworthy culture at a company so you can spot those

131
00:07:44,120 --> 00:07:46,360
Speaker 4: issues so you can ask the right questions at the

132
00:07:46,440 --> 00:07:49,640
Speaker 4: right time if you try. We talked about during the

133
00:07:49,680 --> 00:07:53,920
Speaker 4: Senate hearing, and IBM's been talking for years about regulating

134
00:07:54,200 --> 00:07:57,360
Speaker 4: the use, not the technology itself, because if you try

135
00:07:57,360 --> 00:08:01,200
Speaker 4: to regulate technology, you're very quickly going to find out

136
00:08:01,600 --> 00:08:04,560
Speaker 4: regulation will absolutely never keep up with that.

137
00:08:05,040 --> 00:08:07,280
Speaker 3: And so in your testimony to Congress, you also talked

138
00:08:07,280 --> 00:08:10,480
Speaker 3: about this idea of a precision regulation approach for AI.

139
00:08:11,120 --> 00:08:13,320
Speaker 3: Tell me more about this. What is a precision regulation

140
00:08:13,400 --> 00:08:15,400
Speaker 3: approach and why could that be so important.

141
00:08:15,680 --> 00:08:18,640
Speaker 4: It's funny because I was able to share with Congress

142
00:08:19,280 --> 00:08:23,080
Speaker 4: our precision regulation point of view in twenty twenty three,

143
00:08:23,440 --> 00:08:26,240
Speaker 4: but that precision regulation point of view was published by

144
00:08:26,360 --> 00:08:31,040
Speaker 4: IBM in twenty twenty So we have not changed our

145
00:08:31,160 --> 00:08:36,079
Speaker 4: position that you should apply the tightest controls, the strictest

146
00:08:36,120 --> 00:08:40,360
Speaker 4: regulatory requirements to the technology where the end use and

147
00:08:40,480 --> 00:08:44,400
Speaker 4: risk of societal harm is the greatest. So that's essentially

148
00:08:44,440 --> 00:08:47,440
Speaker 4: what it is. There's lots of AI technology that's used

149
00:08:47,440 --> 00:08:51,080
Speaker 4: today that doesn't touch people, that's very low risk in nature.

150
00:08:51,600 --> 00:08:55,040
Speaker 4: And even when you think about AI that delivers a

151
00:08:55,160 --> 00:09:00,520
Speaker 4: movie recommendation versus AI that is used to diagnose cancer, right,

152
00:09:00,640 --> 00:09:04,120
Speaker 4: there's very different implications associated with those two uses of

153
00:09:04,160 --> 00:09:09,000
Speaker 4: the technology. And so essentially what precision regulation is is

154
00:09:09,000 --> 00:09:12,400
Speaker 4: apply different rules to different risks, right, more stringent regulation

155
00:09:12,679 --> 00:09:15,800
Speaker 4: to the use cases with the greatest risk. And then

156
00:09:15,960 --> 00:09:20,120
Speaker 4: also we build that out calling for things like transparency

157
00:09:20,720 --> 00:09:24,280
Speaker 4: you see it today with content right, misinformation and the like.

158
00:09:24,880 --> 00:09:28,760
Speaker 4: We believe that consumers should always know when they're interacting

159
00:09:28,840 --> 00:09:31,800
Speaker 4: with an AI system, So be transparent, don't hydra AI

160
00:09:32,880 --> 00:09:36,480
Speaker 4: clearly define the risks. So as a country, we need

161
00:09:36,520 --> 00:09:39,600
Speaker 4: to have some clear guidance right in globally as well

162
00:09:39,920 --> 00:09:43,520
Speaker 4: in terms of which uses of AI or higher risk

163
00:09:43,600 --> 00:09:47,920
Speaker 4: where will apply higher and stricter regulation and have sort

164
00:09:47,960 --> 00:09:50,880
Speaker 4: of a common understanding of what those high risk uses

165
00:09:50,920 --> 00:09:54,480
Speaker 4: are and then demonstrate the impact in the cases of

166
00:09:54,559 --> 00:09:59,280
Speaker 4: those higher risk uses. So companies who are using AI

167
00:09:59,440 --> 00:10:03,839
Speaker 4: in space where they can impact people's legal rights, for example,

168
00:10:04,440 --> 00:10:08,680
Speaker 4: should have to conduct an impact assessment that demonstrates that

169
00:10:08,760 --> 00:10:11,920
Speaker 4: the technology isn't biased. So we've been pretty clear about

170
00:10:12,360 --> 00:10:15,720
Speaker 4: apply that the most stringent regulation to the highest risk

171
00:10:15,800 --> 00:10:16,480
Speaker 4: uses of AI.

172
00:10:18,200 --> 00:10:20,479
Speaker 3: And so so far we've been talking about your congressional

173
00:10:20,480 --> 00:10:23,320
Speaker 3: testimony in terms of, you know, the specific content that

174
00:10:23,360 --> 00:10:25,840
Speaker 3: you talked about, But I'm just curious on a personal level,

175
00:10:26,160 --> 00:10:28,280
Speaker 3: you know what was that like right Like, right now,

176
00:10:28,280 --> 00:10:30,559
Speaker 3: it feels like at a policy level, like there's a

177
00:10:30,640 --> 00:10:33,000
Speaker 3: kind of fever pitch going on with AI right now.

178
00:10:33,240 --> 00:10:34,679
Speaker 3: You know, what did that feel like to kind of

179
00:10:34,760 --> 00:10:37,200
Speaker 3: really have the opportunity to talk to policy makers and

180
00:10:37,240 --> 00:10:40,240
Speaker 3: sort of influence what they're thinking about AI technologies like

181
00:10:40,320 --> 00:10:41,280
Speaker 3: in the coming century.

182
00:10:41,320 --> 00:10:44,240
Speaker 4: Perhaps I was really an honor to be able to

183
00:10:44,320 --> 00:10:47,280
Speaker 4: do that and to be one of the first set

184
00:10:47,320 --> 00:10:51,400
Speaker 4: of invitees to the first hearing. And what I learned

185
00:10:51,440 --> 00:10:54,600
Speaker 4: from it essentially is, you know, really two things. The

186
00:10:54,640 --> 00:10:58,480
Speaker 4: first is really the value of authenticity. So both as

187
00:10:58,679 --> 00:11:02,640
Speaker 4: an individual and as a company, I was able to

188
00:11:02,679 --> 00:11:05,240
Speaker 4: talk about what I do. You know, I need a

189
00:11:05,240 --> 00:11:08,680
Speaker 4: lot of advanced prep right I talked about what my

190
00:11:08,800 --> 00:11:12,240
Speaker 4: job is, what IBM has been putting in place for

191
00:11:12,360 --> 00:11:16,600
Speaker 4: years now, so this isn't about creating something. This was

192
00:11:16,800 --> 00:11:19,240
Speaker 4: just about showing up and being authentic. And we were

193
00:11:19,240 --> 00:11:22,360
Speaker 4: invited for a reason. We were invited because we were

194
00:11:22,400 --> 00:11:26,120
Speaker 4: one of the earliest companies in the AI technology space.

195
00:11:26,760 --> 00:11:31,920
Speaker 4: We're the oldest technology company and we are trusted and

196
00:11:32,240 --> 00:11:34,640
Speaker 4: that's an honor. And then the second thing I came

197
00:11:34,679 --> 00:11:37,400
Speaker 4: away with was really how important this issue is to society.

198
00:11:37,440 --> 00:11:41,920
Speaker 4: I don't think I appreciated it as much until following

199
00:11:42,040 --> 00:11:46,360
Speaker 4: that experience. I had outreached from colleagues I hadn't worked

200
00:11:46,400 --> 00:11:49,280
Speaker 4: with for years. I had an outreach from family members

201
00:11:49,280 --> 00:11:51,960
Speaker 4: who heard me on the radio. You know, my mother

202
00:11:52,120 --> 00:11:55,000
Speaker 4: and my mother in law, and my nieces and nephews

203
00:11:55,040 --> 00:11:57,439
Speaker 4: and my friends of my kids were all like, oh,

204
00:11:57,440 --> 00:11:59,360
Speaker 4: I get it. I get what you do. Now, Wow,

205
00:11:59,440 --> 00:12:02,840
Speaker 4: that's pretty you know. So that was really probably the

206
00:12:02,960 --> 00:12:05,520
Speaker 4: best and most impactful takeaway that I had.

207
00:12:06,000 --> 00:12:09,640
Speaker 2: The mass adoption of generative AI, happening at breakneck speed,

208
00:12:10,000 --> 00:12:14,000
Speaker 2: has spurred societies and governments around the world to get

209
00:12:14,120 --> 00:12:20,400
Speaker 2: serious about regulating AI for businesses. Compliance is complex enough already,

210
00:12:20,600 --> 00:12:23,880
Speaker 2: but throw an ever involving technology like AI into the mix,

211
00:12:24,320 --> 00:12:30,600
Speaker 2: and compliance itself becomes an exercise in adaptability. As regulators

212
00:12:30,600 --> 00:12:35,160
Speaker 2: seek greater accountability in how AI is used, businesses need

213
00:12:35,240 --> 00:12:40,160
Speaker 2: help creating governance processes comprehensive enough to comply with the law,

214
00:12:40,400 --> 00:12:43,480
Speaker 2: but agile enough to keep up with the rapid rate

215
00:12:43,520 --> 00:12:48,880
Speaker 2: of change in AI development. Regulatory scrutiny isn't the only

216
00:12:48,960 --> 00:12:54,120
Speaker 2: consideration either responsible AI governance. Of business's ability to prove

217
00:12:54,160 --> 00:12:59,000
Speaker 2: its AI models are transparent and explainable is also key

218
00:12:59,040 --> 00:13:04,080
Speaker 2: to building trust with customers, regardless of industry. In the

219
00:13:04,120 --> 00:13:08,319
Speaker 2: next part of their conversation, Laurie asked Christina what businesses

220
00:13:08,360 --> 00:13:12,760
Speaker 2: should consider when approaching AI governance. Let's listen.

221
00:13:13,600 --> 00:13:16,040
Speaker 3: So it's a particular role that businesses are playing in

222
00:13:16,120 --> 00:13:18,920
Speaker 3: AI governance, Like why is it so critical for businesses

223
00:13:18,960 --> 00:13:19,840
Speaker 3: to be part of this?

224
00:13:20,480 --> 00:13:24,959
Speaker 4: So, I think it's really critically important that businesses understand

225
00:13:25,480 --> 00:13:28,400
Speaker 4: the impacts that technology can have, both in making them

226
00:13:28,400 --> 00:13:32,240
Speaker 4: better businesses, but the impacts that those technologies can have

227
00:13:32,720 --> 00:13:37,839
Speaker 4: on the consumers that they are supporting. You know, businesses

228
00:13:37,960 --> 00:13:42,560
Speaker 4: need to be deploying AI technology that is in alignment

229
00:13:42,720 --> 00:13:44,560
Speaker 4: with the goals that they set for it and that

230
00:13:44,720 --> 00:13:47,000
Speaker 4: can be trusted, I think for us and for our

231
00:13:47,040 --> 00:13:50,839
Speaker 4: clients A lot of this comes back to trust in tech.

232
00:13:51,280 --> 00:13:56,960
Speaker 4: If you deploy something that doesn't work, that hallucinates, that discriminates,

233
00:13:57,440 --> 00:14:01,600
Speaker 4: that isn't transparent, where decisions can't be explained, then you

234
00:14:01,720 --> 00:14:05,360
Speaker 4: are going to very rapidly erode the trust at best

235
00:14:05,520 --> 00:14:08,839
Speaker 4: right of your clients and at worst for yourself. You're

236
00:14:08,880 --> 00:14:11,560
Speaker 4: going to create legal and regulatory issues for yourself as well.

237
00:14:11,600 --> 00:14:15,520
Speaker 4: So trusted technology is really important, and I think there's

238
00:14:15,559 --> 00:14:17,760
Speaker 4: a lot of pressure on businesses today to move very

239
00:14:17,840 --> 00:14:20,520
Speaker 4: rapidly and adopt technology. But if you do it without

240
00:14:20,600 --> 00:14:24,080
Speaker 4: having a program of governance in place, you're really risking

241
00:14:24,280 --> 00:14:25,240
Speaker 4: eroding that trust.

242
00:14:25,360 --> 00:14:27,560
Speaker 3: So this is really where I think a strong AI

243
00:14:27,640 --> 00:14:31,200
Speaker 3: governance comes in. Talk about from your perspective, how this

244
00:14:31,280 --> 00:14:35,160
Speaker 3: really contributes to maintaining the trust that customers and stakeholders

245
00:14:35,200 --> 00:14:36,360
Speaker 3: have in these technologies.

246
00:14:37,080 --> 00:14:39,640
Speaker 4: Yeah. Absolutely. I mean you need to have a governance

247
00:14:39,680 --> 00:14:43,680
Speaker 4: program because you need to understand that the technology, particularly

248
00:14:43,720 --> 00:14:48,280
Speaker 4: in the AI space, that you are deploying, is explainable.

249
00:14:48,320 --> 00:14:52,680
Speaker 4: You need to understand why it's making decisions and recommendations

250
00:14:52,720 --> 00:14:54,200
Speaker 4: that it's making, and you need to be able to

251
00:14:54,200 --> 00:14:56,600
Speaker 4: explain that to your consumers. I mean, you can't do

252
00:14:56,680 --> 00:14:58,760
Speaker 4: that if you don't know where your data is coming from,

253
00:14:58,760 --> 00:15:01,240
Speaker 4: what data are you using to train those models, if

254
00:15:01,240 --> 00:15:05,040
Speaker 4: you don't have a program that manages the alignment of

255
00:15:05,080 --> 00:15:08,640
Speaker 4: your AI models over time to make sure as AI

256
00:15:08,880 --> 00:15:13,360
Speaker 4: learns and evolves over uses, which is in large part

257
00:15:13,880 --> 00:15:17,680
Speaker 4: what makes it so beneficial that it stays in alignment

258
00:15:17,720 --> 00:15:21,200
Speaker 4: with the objectives that you set for the technology over time.

259
00:15:21,840 --> 00:15:25,680
Speaker 4: So you can't do that without a robust governance process

260
00:15:25,680 --> 00:15:29,160
Speaker 4: in place. So we work with clients to share our

261
00:15:29,200 --> 00:15:31,680
Speaker 4: own story here at IBM in terms of how we

262
00:15:31,720 --> 00:15:35,320
Speaker 4: put that in place, but also in our consulting practice

263
00:15:36,080 --> 00:15:40,600
Speaker 4: to help clients work with these new generative capabilities and

264
00:15:40,600 --> 00:15:43,560
Speaker 4: foundation models and the like in order to put them

265
00:15:43,560 --> 00:15:45,600
Speaker 4: to work for their business in a way that's going

266
00:15:45,640 --> 00:15:48,280
Speaker 4: to be impactful to that business but at the same

267
00:15:48,360 --> 00:15:49,440
Speaker 4: time be trusted.

268
00:15:49,840 --> 00:15:51,720
Speaker 3: So now I wanted to turn a little bit towards

269
00:15:51,720 --> 00:15:56,200
Speaker 3: watsonex governance, and so IBM recently announced their AI platform, watsonex,

270
00:15:56,440 --> 00:15:59,600
Speaker 3: which will include a governance component. Could you tell us

271
00:15:59,640 --> 00:16:01,960
Speaker 3: a little more about watsonex dot governance.

272
00:16:02,320 --> 00:16:04,560
Speaker 4: Yeah, I mean before I do that, I'll just back

273
00:16:04,640 --> 00:16:08,240
Speaker 4: up and talk about the full platform and then lean

274
00:16:08,280 --> 00:16:11,160
Speaker 4: into Watson X because I think it's important to understand

275
00:16:11,440 --> 00:16:17,360
Speaker 4: the delivery of a full suite of capabilities, to get data,

276
00:16:17,440 --> 00:16:20,360
Speaker 4: to train models, and then to govern them over their

277
00:16:20,400 --> 00:16:25,920
Speaker 4: life cycle. All of these things are really important from

278
00:16:25,960 --> 00:16:28,800
Speaker 4: the onset you need to make sure that you have,

279
00:16:29,440 --> 00:16:33,680
Speaker 4: you know, for our watsonex dot AI for example, that's

280
00:16:33,720 --> 00:16:38,280
Speaker 4: the studio to train new foundation models and generative AI

281
00:16:38,360 --> 00:16:44,000
Speaker 4: and machine learning capabilities, and we are populating that studio

282
00:16:44,360 --> 00:16:49,640
Speaker 4: with some IBM trained foundation models which we're curating and

283
00:16:49,720 --> 00:16:53,480
Speaker 4: tailoring more specifically for enterprises. So that's really important. It

284
00:16:53,520 --> 00:16:55,880
Speaker 4: comes back to the point I made earlier about business

285
00:16:55,880 --> 00:17:00,200
Speaker 4: trust and the need you know to have enterprise is

286
00:17:00,280 --> 00:17:05,600
Speaker 4: ready technologies in the AI space. And then the watsonex

287
00:17:05,600 --> 00:17:08,800
Speaker 4: dot data is a fit for purpose data store or

288
00:17:08,840 --> 00:17:12,560
Speaker 4: a data lake, and then watsonex dot gov. So that's

289
00:17:12,960 --> 00:17:17,600
Speaker 4: a particular component of the platform that my team and

290
00:17:17,680 --> 00:17:21,080
Speaker 4: the AI Ethics Board has really worked closely with the

291
00:17:21,119 --> 00:17:24,680
Speaker 4: product team on developing, and we're using it internally here

292
00:17:24,680 --> 00:17:27,880
Speaker 4: in the Chief Privacy Office as well to help us

293
00:17:28,320 --> 00:17:33,240
Speaker 4: govern our own uses of AI technology and our compliance

294
00:17:33,280 --> 00:17:39,359
Speaker 4: program here. And it essentially helps to notify you if

295
00:17:39,400 --> 00:17:42,520
Speaker 4: a model becomes biased or gets out of alignment as

296
00:17:42,560 --> 00:17:45,440
Speaker 4: you're using it over time. So companies are going to

297
00:17:45,520 --> 00:17:48,040
Speaker 4: need these capabilities. I mean they need them today to

298
00:17:48,119 --> 00:17:53,080
Speaker 4: deliver technologies with trust. They'll need them tomorrow to comply

299
00:17:53,200 --> 00:17:55,280
Speaker 4: with regulation, which is on the horizon.

300
00:17:55,520 --> 00:17:58,399
Speaker 3: And I think compliance becomes even more complex when you

301
00:17:58,440 --> 00:18:02,560
Speaker 3: consider international data protection laws and regulations. Honestly, I don't

302
00:18:02,600 --> 00:18:05,160
Speaker 3: know how anyone on any company legal team is keeping

303
00:18:05,240 --> 00:18:07,480
Speaker 3: up with this these days. But my question for you

304
00:18:07,600 --> 00:18:11,200
Speaker 3: is really how can businesses develop a strategy to maintain

305
00:18:11,240 --> 00:18:14,040
Speaker 3: compliance and to deal with it in this ever changing landscape.

306
00:18:14,119 --> 00:18:18,040
Speaker 4: It's increasingly more challenging. In fact, I saw statistic just

307
00:18:18,119 --> 00:18:22,720
Speaker 4: this morning that the regulatory obligations on companies have increased

308
00:18:22,720 --> 00:18:26,600
Speaker 4: something like seven hundred times in the last twenty years.

309
00:18:26,400 --> 00:18:31,000
Speaker 4: So it really is a huge focus area for companies.

310
00:18:31,160 --> 00:18:34,040
Speaker 4: You have to have a process in place in order

311
00:18:34,080 --> 00:18:36,600
Speaker 4: to do that, and it's not easy, particularly for a

312
00:18:36,680 --> 00:18:40,560
Speaker 4: company like IBM that it has a presence in over

313
00:18:40,560 --> 00:18:43,600
Speaker 4: one hundred and seventy countries around the world. There is

314
00:18:43,640 --> 00:18:48,520
Speaker 4: more than one hundred and fifty comprehensive privacy regulations. There

315
00:18:48,560 --> 00:18:53,080
Speaker 4: are regulations of non personal data, there are AI regulations emerging,

316
00:18:54,040 --> 00:18:58,119
Speaker 4: so you really need an operational approach to it in

317
00:18:58,240 --> 00:19:00,280
Speaker 4: order to stay compliant. But one of the things we

318
00:19:00,320 --> 00:19:02,480
Speaker 4: do as we set a baseline, and a lot of

319
00:19:02,480 --> 00:19:05,919
Speaker 4: companies do this as well, So we define a privacy baseline,

320
00:19:05,960 --> 00:19:10,760
Speaker 4: we define an AI baseline, and we ensure then as

321
00:19:10,800 --> 00:19:13,200
Speaker 4: a result of that that there are very few deviances

322
00:19:13,280 --> 00:19:16,359
Speaker 4: because it incorporates in that baseline. So that's one of

323
00:19:16,400 --> 00:19:18,560
Speaker 4: the ways we do it. Other companies, I think are

324
00:19:18,640 --> 00:19:23,840
Speaker 4: similarly situated in terms of doing that. But again, it

325
00:19:24,000 --> 00:19:26,679
Speaker 4: is a real challenge for global companies. It's one of

326
00:19:26,680 --> 00:19:30,640
Speaker 4: the reasons why we advocate for as much alignment as

327
00:19:30,720 --> 00:19:36,080
Speaker 4: possible on the international realm as well as nationally here

328
00:19:36,119 --> 00:19:39,920
Speaker 4: in the US, as much alignment as possible to make

329
00:19:40,119 --> 00:19:45,080
Speaker 4: compliance easier for easier, and not just because companies want

330
00:19:45,119 --> 00:19:48,280
Speaker 4: an easy way to comply, but the harder it is,

331
00:19:48,520 --> 00:19:52,399
Speaker 4: the less likely there will be compliance. And it's not

332
00:19:52,520 --> 00:19:58,600
Speaker 4: the objective of anybody, governments, companies, consumers to have to

333
00:19:58,760 --> 00:20:02,280
Speaker 4: set legal obligation that companies simply can't meet.

334
00:20:02,520 --> 00:20:04,560
Speaker 3: So what advice would you give to other companies who

335
00:20:04,560 --> 00:20:07,640
Speaker 3: are looking to rethink or strengthen their approach to AI. Governance.

336
00:20:07,720 --> 00:20:11,040
Speaker 4: Think you need to start with, as we did, foundational principles,

337
00:20:11,680 --> 00:20:15,119
Speaker 4: and you need to start making decisions about what technology

338
00:20:15,119 --> 00:20:17,480
Speaker 4: you're going to deploy and what technology you're not, What

339
00:20:17,480 --> 00:20:18,600
Speaker 4: are you going to use it for, and what aren't

340
00:20:18,600 --> 00:20:20,000
Speaker 4: you going to use it for? And then when you

341
00:20:20,040 --> 00:20:24,400
Speaker 4: do use it, align to those principles. That's really important.

342
00:20:24,480 --> 00:20:29,000
Speaker 4: Formalize a program, have someone within the organization, whether it's

343
00:20:29,040 --> 00:20:33,320
Speaker 4: the chief privacy officer, whether it's some other role, a

344
00:20:33,400 --> 00:20:37,840
Speaker 4: chief AI ethics officer, but have an accountable individual and

345
00:20:37,920 --> 00:20:42,080
Speaker 4: accountable organization. Do a maturity assessment, figure out where you

346
00:20:42,160 --> 00:20:44,919
Speaker 4: are and where you need to be, and really start

347
00:20:46,000 --> 00:20:50,480
Speaker 4: putting it into place today. Don't wait for regulation to

348
00:20:50,560 --> 00:20:53,200
Speaker 4: apply directly to your business, because it'll be too late.

349
00:20:54,440 --> 00:20:57,840
Speaker 3: So Smart Talks features new creators, these visionaries like yourself

350
00:20:57,840 --> 00:21:01,320
Speaker 3: who are creatively applying technology in business to drive change.

351
00:21:01,640 --> 00:21:04,320
Speaker 3: I'm curious if you see yourself as creative.

352
00:21:04,480 --> 00:21:07,800
Speaker 4: You know, I definitely do. I mean you need to

353
00:21:07,880 --> 00:21:12,520
Speaker 4: be creative when you're working in an industry that evolves

354
00:21:12,560 --> 00:21:17,040
Speaker 4: so very quickly. So you know, I started with IBM

355
00:21:17,320 --> 00:21:20,440
Speaker 4: when we were primarily a hardware company, right, and we've

356
00:21:20,520 --> 00:21:23,960
Speaker 4: changed our business so significantly over the years and the

357
00:21:24,040 --> 00:21:28,480
Speaker 4: issues that are raised with respect to each new technology,

358
00:21:28,520 --> 00:21:32,160
Speaker 4: whether it be cloud, whether it be AI now where

359
00:21:32,160 --> 00:21:33,760
Speaker 4: we're seeing a ton of issues, or you look at

360
00:21:33,760 --> 00:21:37,919
Speaker 4: emergent issues in the space of things like neurotechnologies and

361
00:21:38,000 --> 00:21:44,600
Speaker 4: quantum computers. You have to be strategic and you have

362
00:21:44,720 --> 00:21:47,960
Speaker 4: to be creative and thinking about how you can adapt

363
00:21:48,600 --> 00:21:53,840
Speaker 4: agilely quickly a company to an environment that is changing

364
00:21:53,880 --> 00:21:56,040
Speaker 4: so quickly and with.

365
00:21:56,080 --> 00:21:59,000
Speaker 3: This transformation happening at such a rapid pace. Do you

366
00:21:59,040 --> 00:22:01,320
Speaker 3: think creativity is a role in how you think about

367
00:22:01,359 --> 00:22:05,520
Speaker 3: and implement specifically a trustworthy AI strategy.

368
00:22:06,600 --> 00:22:10,640
Speaker 4: Yeah, I absolutely think it does, because again it comes

369
00:22:10,680 --> 00:22:13,840
Speaker 4: back to these capabilities, and there are ways, I guess

370
00:22:14,040 --> 00:22:17,800
Speaker 4: how you define creativity could be different, right, but I'm

371
00:22:17,840 --> 00:22:21,040
Speaker 4: thinking of creativity in the sense of sort of agility

372
00:22:21,080 --> 00:22:25,160
Speaker 4: and strategic vision and creative problem solving. I think that's

373
00:22:25,440 --> 00:22:28,560
Speaker 4: really important in the world that we're in right now,

374
00:22:28,600 --> 00:22:33,159
Speaker 4: being able to creatively problem solve with new issues that

375
00:22:33,240 --> 00:22:36,320
Speaker 4: are rising sort of every day.

376
00:22:36,960 --> 00:22:38,520
Speaker 3: And so, how do you see the role of chief

377
00:22:38,560 --> 00:22:42,200
Speaker 3: privacy officer evolving in the future as AI technology continues

378
00:22:42,240 --> 00:22:45,120
Speaker 3: to advance, Like what stuff should CPOs take to stay

379
00:22:45,119 --> 00:22:47,040
Speaker 3: ahead of all these changes that are come in their way.

380
00:22:48,320 --> 00:22:52,240
Speaker 4: So the role is evolving in most companies, I would

381
00:22:52,280 --> 00:22:57,240
Speaker 4: say pretty rapidly. Many companies are looking to chief privacy

382
00:22:57,280 --> 00:23:00,800
Speaker 4: officers who are ready understand the data that's being used

383
00:23:00,800 --> 00:23:04,320
Speaker 4: in the organization and have programs to ensure compliance with

384
00:23:04,440 --> 00:23:08,240
Speaker 4: laws that require you to manage that data in accordance

385
00:23:08,240 --> 00:23:10,880
Speaker 4: with data protection laws and the like. It's a natural

386
00:23:10,920 --> 00:23:16,920
Speaker 4: place and position for AI responsibility, and so I think

387
00:23:16,920 --> 00:23:19,600
Speaker 4: what's happening to a lot of chief privacy officers is

388
00:23:19,640 --> 00:23:23,119
Speaker 4: they're being asked to take on this AI governance responsibility

389
00:23:23,160 --> 00:23:26,680
Speaker 4: for companies and if not take it on, at least

390
00:23:26,720 --> 00:23:30,119
Speaker 4: play a very key role working with other parts of

391
00:23:30,160 --> 00:23:33,800
Speaker 4: the business in AI governance. So that really is changing.

392
00:23:34,040 --> 00:23:38,040
Speaker 4: And if chief privacy officers are in companies who maybe

393
00:23:38,119 --> 00:23:41,879
Speaker 4: haven't started thinking about AI yet, they should, so I

394
00:23:41,920 --> 00:23:45,760
Speaker 4: would encourage them to look at different resources that are

395
00:23:45,800 --> 00:23:50,159
Speaker 4: available already in AI governance space. For example, the International

396
00:23:50,200 --> 00:23:54,640
Speaker 4: Association of Privacy Professionals, which is the seventy five thousand

397
00:23:54,680 --> 00:23:59,760
Speaker 4: member professional body for the profession of Chief Privacy officers,

398
00:23:59,840 --> 00:24:04,520
Speaker 4: just recently launched an AI Governance Initiative and an AI

399
00:24:04,560 --> 00:24:08,520
Speaker 4: Governance Certification program. I sit on their advisory board. But

400
00:24:08,640 --> 00:24:11,320
Speaker 4: that's just emblematic of the fact that the field is

401
00:24:11,400 --> 00:24:12,840
Speaker 4: changing so rapidly.

402
00:24:14,359 --> 00:24:16,920
Speaker 3: And so, you know, speaking of rapid change, when you're

403
00:24:17,200 --> 00:24:19,600
Speaker 3: back here on smart Talks in twenty twenty one, you

404
00:24:19,640 --> 00:24:21,960
Speaker 3: said that the future of AI will be more transparent

405
00:24:22,000 --> 00:24:24,200
Speaker 3: and more trustworthy. You know, what do you see the

406
00:24:24,240 --> 00:24:26,280
Speaker 3: next five to ten years holding. You know, when you're

407
00:24:26,320 --> 00:24:29,240
Speaker 3: back on smart Talks in you know, twenty twenty six,

408
00:24:29,320 --> 00:24:31,000
Speaker 3: you know twenty thirty, You know what are we going

409
00:24:31,080 --> 00:24:33,359
Speaker 3: to be talking about when it comes to AI technology

410
00:24:33,359 --> 00:24:34,000
Speaker 3: and governance.

411
00:24:34,560 --> 00:24:36,719
Speaker 4: So I try to be an optimist, right And I

412
00:24:36,760 --> 00:24:40,600
Speaker 4: said that two years ago, and I think we're seeing

413
00:24:40,640 --> 00:24:45,040
Speaker 4: it now come into fruition, and there will be requirements,

414
00:24:45,840 --> 00:24:48,639
Speaker 4: whether they're coming from the US, whether they're coming from Europe,

415
00:24:48,720 --> 00:24:52,320
Speaker 4: whether they're just coming from voluntary adoption by clients of

416
00:24:52,400 --> 00:24:57,280
Speaker 4: things like the NISS Risk Management Framework, really important voluntary frameworks.

417
00:24:57,880 --> 00:25:01,760
Speaker 4: You're going to have to adopt transparent, explainable practices in

418
00:25:01,800 --> 00:25:04,240
Speaker 4: your uses of AI. So I do see that happening.

419
00:25:04,320 --> 00:25:06,760
Speaker 4: And in the next five to ten years, boy, I

420
00:25:06,800 --> 00:25:12,760
Speaker 4: think we'll see more research into trust in techniques because

421
00:25:12,760 --> 00:25:16,720
Speaker 4: we don't really know, for example, how to water mark.

422
00:25:17,480 --> 00:25:21,000
Speaker 4: We're calling for things like watermarking. There'll be more research

423
00:25:21,040 --> 00:25:26,879
Speaker 4: into how to do that. I think you'll see regulation

424
00:25:27,000 --> 00:25:29,479
Speaker 4: that's specifically going to require those types of things. So

425
00:25:29,600 --> 00:25:31,600
Speaker 4: I think again, I think the regulation is going to

426
00:25:31,680 --> 00:25:34,920
Speaker 4: drive research. It's going to drive research into these areas

427
00:25:34,960 --> 00:25:39,600
Speaker 4: that will help ensure that we can deliver new capabilities,

428
00:25:39,640 --> 00:25:43,000
Speaker 4: generated capabilities and the like with trust and explainability.

429
00:25:43,480 --> 00:25:45,520
Speaker 3: Thank you so much Christina for joining me on smart

430
00:25:45,520 --> 00:25:47,240
Speaker 3: Talks to talk about AI and governance.

431
00:25:47,920 --> 00:25:49,880
Speaker 4: Well, thank you very much for having me.

432
00:25:51,440 --> 00:25:56,159
Speaker 2: To unlock the transformative growth possible with artificial intelligence. Businesses

433
00:25:56,240 --> 00:25:59,359
Speaker 2: need to know what they wish to grow into first.

434
00:26:00,320 --> 00:26:03,159
Speaker 2: Like Christina said, the best way forward in the AI

435
00:26:03,280 --> 00:26:07,320
Speaker 2: future is for businesses to figure out their own foundational

436
00:26:07,359 --> 00:26:11,960
Speaker 2: principles around using the technology, drawing on those principles to

437
00:26:12,040 --> 00:26:15,840
Speaker 2: apply AI in a way that's ethically consistent with their

438
00:26:15,880 --> 00:26:19,560
Speaker 2: mission and complies with the legal frameworks built to hold

439
00:26:19,560 --> 00:26:24,560
Speaker 2: the technology accountable. As AI adoption grows more and more widespread,

440
00:26:24,800 --> 00:26:29,119
Speaker 2: so too will the expectation from consumers and regulators that

441
00:26:29,200 --> 00:26:34,639
Speaker 2: businesses use it responsibly. Investing independable AI governance is a

442
00:26:34,680 --> 00:26:38,639
Speaker 2: way for businesses to lay the foundations for technology that

443
00:26:38,680 --> 00:26:42,560
Speaker 2: their customers can trust while rising to the challenge of

444
00:26:42,680 --> 00:26:48,440
Speaker 2: increasing regulatory complexity. Though the emergence of AI does complicate

445
00:26:48,600 --> 00:26:53,160
Speaker 2: an already tough compliance landscape, businesses now face a creative

446
00:26:53,200 --> 00:26:57,840
Speaker 2: opportunity to set a precedent for what accountability in AI

447
00:26:58,000 --> 00:27:00,800
Speaker 2: looks like and rethink what it means means to deploy

448
00:27:01,200 --> 00:27:07,320
Speaker 2: trustworthy artificial intelligence. I'm Malcolm Gladwell. This is a paid

449
00:27:07,440 --> 00:27:11,800
Speaker 2: advertisement from IBM. Smart Talks with IBM will be taking

450
00:27:11,840 --> 00:27:15,080
Speaker 2: a short hiatus, but look for new episodes in the

451
00:27:15,119 --> 00:27:19,280
Speaker 2: coming weeks. Smart Talks with IBM is produced by Matt Romano,

452
00:27:19,840 --> 00:27:24,760
Speaker 2: David jaw Nische Venkat and Royston Deserve with Jacob Goldstein.

453
00:27:25,040 --> 00:27:29,080
Speaker 2: We're edited by Lydia jen Kott. Our engineer is Jason Gambrel.

454
00:27:29,400 --> 00:27:34,320
Speaker 2: Theme song by Gramoscope. Special thanks to Carli Migliori, Andy Kelly,

455
00:27:34,720 --> 00:27:38,960
Speaker 2: Kathy Callahan and the eight Bar and IBM teams, as

456
00:27:39,000 --> 00:27:43,120
Speaker 2: well as the Pushkin marketing team. Smart Talks with IBM

457
00:27:43,440 --> 00:27:47,720
Speaker 2: is a production of Pushkin Industries and Ruby Studio at iHeartMedia.

458
00:27:48,359 --> 00:27:52,480
Speaker 2: To find more Pushkin podcasts, listen on the iHeartRadio, app,

459
00:27:52,760 --> 00:28:00,359
Speaker 2: Apple podcasts, or wherever you listen to podcasts. H