1
00:00:00,120 --> 00:00:02,840
Speaker 1: Hey everyone, it's Robert and Joe here. Today we've got

2
00:00:02,840 --> 00:00:04,720
Speaker 1: something a little bit different to share with you. It

3
00:00:04,840 --> 00:00:08,000
Speaker 1: is a new season of the Smart Talks with IBM

4
00:00:08,119 --> 00:00:09,119
Speaker 1: podcast series.

5
00:00:09,600 --> 00:00:11,680
Speaker 2: Today we are witnessed to one of those rare moments

6
00:00:11,680 --> 00:00:14,360
Speaker 2: in history, the rise of an innovative technology with the

7
00:00:14,360 --> 00:00:18,680
Speaker 2: potential to radically transform business and society forever. The technology,

8
00:00:18,760 --> 00:00:22,200
Speaker 2: of course, is artificial intelligence, and it's the central focus

9
00:00:22,239 --> 00:00:24,800
Speaker 2: for this new season of Smart Talks with IBM.

10
00:00:25,320 --> 00:00:28,400
Speaker 1: Join hosts from your favorite Pushkin podcasts as they talk

11
00:00:28,480 --> 00:00:31,640
Speaker 1: with industry experts and leaders to explore how businesses can

12
00:00:31,680 --> 00:00:35,360
Speaker 1: integrate AI into their workflows and help drive real change

13
00:00:35,400 --> 00:00:38,160
Speaker 1: in this new era of AI. And of course, host

14
00:00:38,280 --> 00:00:40,440
Speaker 1: Malcolm Gladwell will be there to guide you through the

15
00:00:40,479 --> 00:00:42,640
Speaker 1: season and throw in his two cents as well.

16
00:00:43,120 --> 00:00:46,000
Speaker 2: Look out for new episodes of Smart Talks with IBM

17
00:00:46,400 --> 00:00:49,519
Speaker 2: every other week on the iHeartRadio app, Apple Podcasts, or

18
00:00:49,560 --> 00:00:53,360
Speaker 2: wherever you get your podcasts, and learn more at IBM

19
00:00:53,479 --> 00:00:55,480
Speaker 2: dot com slash smart Talks.

20
00:00:58,000 --> 00:01:02,040
Speaker 3: Hello, Hello, Welcome to Smart with IBM, a podcast from

21
00:01:02,080 --> 00:01:08,440
Speaker 3: Pushkin Industri's iHeartRadio and IBM. I'm Malcolm Glabwell. This season

22
00:01:08,480 --> 00:01:12,840
Speaker 3: we're continuing our conversation with new creators visionaries who are

23
00:01:12,880 --> 00:01:16,399
Speaker 3: creatively applying technology in business to drive change, but with

24
00:01:16,480 --> 00:01:21,399
Speaker 3: a focus on the transformative power of artificial intelligence and

25
00:01:21,480 --> 00:01:24,600
Speaker 3: what it means to leverage AI as a game changing

26
00:01:24,720 --> 00:01:29,720
Speaker 3: multiplier for your business. Our guest today is Christina Montgomery,

27
00:01:30,120 --> 00:01:34,200
Speaker 3: IBM's Chief Privacy and Trust Officer. She's also chair of

28
00:01:34,240 --> 00:01:39,679
Speaker 3: IBM's AI Ethics Board. In addition to overseeing IBM's privacy policy,

29
00:01:40,040 --> 00:01:45,039
Speaker 3: a core part of Christina's job involves AI governance, making

30
00:01:45,080 --> 00:01:48,880
Speaker 3: sure the way AI is used complies with the international

31
00:01:49,200 --> 00:01:55,040
Speaker 3: legal regulations customized for each industry. In today's episode, Christina

32
00:01:55,080 --> 00:01:59,920
Speaker 3: will explain why businesses need foundational principles when it comes

33
00:01:59,960 --> 00:02:04,800
Speaker 3: to using technology, why AI regulation should focus on specific

34
00:02:05,000 --> 00:02:09,280
Speaker 3: use cases over the technology itself, and share a little

35
00:02:09,280 --> 00:02:14,920
Speaker 3: bit about her landmark congressional testimony. Last May, Christina spoke

36
00:02:14,960 --> 00:02:18,640
Speaker 3: with doctor Lori Santos, host of the Pushkin podcast The

37
00:02:18,680 --> 00:02:24,160
Speaker 3: Happiness Lab, a cognitive scientist and psychology professor at Yale University.

38
00:02:24,480 --> 00:02:29,840
Speaker 3: Laurie is an expert on human happiness and cognition. Okay,

39
00:02:30,360 --> 00:02:31,639
Speaker 3: let's get to the interview.

40
00:02:34,200 --> 00:02:36,280
Speaker 4: So Christina, I'm so excited to talk to you today.

41
00:02:36,440 --> 00:02:38,640
Speaker 4: So let's start by talking a little bit about your

42
00:02:38,720 --> 00:02:41,400
Speaker 4: role at IBM. What does a Chief Privacy and Trust

43
00:02:41,440 --> 00:02:42,480
Speaker 4: officer actually do.

44
00:02:43,000 --> 00:02:46,760
Speaker 5: It's a really dynamic profession and it's not a new profession,

45
00:02:47,320 --> 00:02:50,200
Speaker 5: but the role has really changed. I mean, my role

46
00:02:50,320 --> 00:02:54,040
Speaker 5: today is broader than just helping to ensure compliance with

47
00:02:54,160 --> 00:02:58,800
Speaker 5: data protection laws globally. I'm also responsible for AI governance.

48
00:02:58,840 --> 00:03:01,799
Speaker 5: I co chair or AI Ethics Board here at IBM,

49
00:03:02,120 --> 00:03:05,160
Speaker 5: and for data clearance and data governance as well for

50
00:03:05,240 --> 00:03:08,800
Speaker 5: the company. So I have both a compliance aspect to

51
00:03:08,840 --> 00:03:11,560
Speaker 5: my role, really important on a global basis, but also

52
00:03:12,080 --> 00:03:17,000
Speaker 5: help the business to competitively differentiate because really trust is

53
00:03:17,000 --> 00:03:20,720
Speaker 5: a strategic advantage for IBM and a competitive differentiator as

54
00:03:20,720 --> 00:03:24,640
Speaker 5: a company that's been responsibly managing the most sensitive data

55
00:03:24,680 --> 00:03:27,320
Speaker 5: for our clients for more than a century now and

56
00:03:27,440 --> 00:03:30,160
Speaker 5: helping to usher new technologies into the world with trust

57
00:03:30,160 --> 00:03:33,560
Speaker 5: and transparency. And so that's also a key aspect of

58
00:03:33,600 --> 00:03:34,040
Speaker 5: my role.

59
00:03:34,800 --> 00:03:37,000
Speaker 4: And so you joined us here on smart Talks back

60
00:03:37,000 --> 00:03:39,240
Speaker 4: in twenty twenty one and you chatted with us about

61
00:03:39,240 --> 00:03:42,920
Speaker 4: IBM's approach of building trust and transparency with AI. And

62
00:03:43,040 --> 00:03:45,080
Speaker 4: that was only two years ago, but it almost feels

63
00:03:45,120 --> 00:03:47,440
Speaker 4: like in eternity has happened in the field of AI

64
00:03:47,600 --> 00:03:50,600
Speaker 4: since then, and so I'm curious how much has changed

65
00:03:50,640 --> 00:03:53,120
Speaker 4: since you were here last time. We're the things you

66
00:03:53,160 --> 00:03:55,840
Speaker 4: told us before, you know, are they still true? How

67
00:03:55,880 --> 00:03:56,680
Speaker 4: are things changing?

68
00:03:57,080 --> 00:04:00,560
Speaker 5: You're absolutely right, it feels like the world has changed

69
00:04:00,640 --> 00:04:04,320
Speaker 5: really in the last two years. But the same fundamental

70
00:04:04,360 --> 00:04:09,280
Speaker 5: principles and the same overall governance apply to IBM's program

71
00:04:09,840 --> 00:04:14,200
Speaker 5: for Data Protection and Responsible AI that we talked about

72
00:04:14,240 --> 00:04:16,920
Speaker 5: two years ago, and not much has changed there from

73
00:04:16,960 --> 00:04:20,120
Speaker 5: our perspective. And the good thing is we've put these

74
00:04:20,160 --> 00:04:24,599
Speaker 5: practices and this governance approach into place, and we have

75
00:04:24,839 --> 00:04:28,480
Speaker 5: an established way of looking at these emerging technologies. As

76
00:04:28,520 --> 00:04:31,920
Speaker 5: the technology evolves, the tech is more powerful, for sure,

77
00:04:32,080 --> 00:04:36,120
Speaker 5: foundation models are vastly larger and more capable, and our

78
00:04:36,200 --> 00:04:39,640
Speaker 5: creating in some respects new issues, but that just makes

79
00:04:39,640 --> 00:04:41,440
Speaker 5: it all the more urgent to do what we've been

80
00:04:41,480 --> 00:04:45,080
Speaker 5: doing and to put trust and transparency into place across

81
00:04:45,120 --> 00:04:47,760
Speaker 5: the business to be accountable to those principles.

82
00:04:48,960 --> 00:04:51,120
Speaker 4: And so our conversation today is really centered around this

83
00:04:51,279 --> 00:04:54,520
Speaker 4: need for new AI regulation, and part of that regulation

84
00:04:54,640 --> 00:04:57,520
Speaker 4: involves the mitigation of bias and This is something I

85
00:04:57,520 --> 00:05:00,200
Speaker 4: think about a ton as a psychologist, right, you know,

86
00:05:00,600 --> 00:05:03,719
Speaker 4: like my students and everyone who's interacting with AI is

87
00:05:03,760 --> 00:05:06,520
Speaker 4: assuming that the kind of knowledge that they're getting from

88
00:05:06,640 --> 00:05:09,359
Speaker 4: this kind of learning is accurate, right, But of course

89
00:05:09,400 --> 00:05:11,680
Speaker 4: AI is only as good as the knowledge that's going in.

90
00:05:11,720 --> 00:05:14,080
Speaker 4: And so talk to me a little bit about like

91
00:05:14,560 --> 00:05:17,039
Speaker 4: why bias occurs in AI and the level of the

92
00:05:17,080 --> 00:05:18,520
Speaker 4: problem that we're really dealing with.

93
00:05:19,360 --> 00:05:22,400
Speaker 5: Yeah, Well, obviously AI is based on data, right, It's

94
00:05:22,760 --> 00:05:27,760
Speaker 5: trained with data, and that data could be biased in

95
00:05:27,760 --> 00:05:30,200
Speaker 5: and of itself, and that's where issues could come up.

96
00:05:30,279 --> 00:05:32,160
Speaker 5: They come up in the data, they could also come

97
00:05:32,240 --> 00:05:35,760
Speaker 5: up in the output of the models themselves. So it's

98
00:05:35,880 --> 00:05:40,640
Speaker 5: really important that you build bias consideration and bias testing

99
00:05:40,760 --> 00:05:44,160
Speaker 5: into your product development cycle. And so what we've been

100
00:05:44,200 --> 00:05:46,760
Speaker 5: thinking about here at IBM and doing we had some

101
00:05:46,839 --> 00:05:49,799
Speaker 5: of our research teams delivered some of the very first

102
00:05:49,800 --> 00:05:53,159
Speaker 5: toolkits to help detect bias years ago now, right, and

103
00:05:53,200 --> 00:05:56,840
Speaker 5: deployed them to open source. And we have put into

104
00:05:56,880 --> 00:06:00,200
Speaker 5: place for our developers here at IBM and Ethics by

105
00:06:00,240 --> 00:06:03,480
Speaker 5: Design playbook that's sort of a step by step approach

106
00:06:04,200 --> 00:06:10,720
Speaker 5: which also addresses very fully bias considerations, and we provide

107
00:06:10,760 --> 00:06:13,960
Speaker 5: not only like here's a point when you should test

108
00:06:14,000 --> 00:06:16,400
Speaker 5: for it and you consider it in the data, you

109
00:06:16,440 --> 00:06:18,159
Speaker 5: have to measure it both at the data and the

110
00:06:18,200 --> 00:06:21,720
Speaker 5: model level or the outcome level, and we provide guidance

111
00:06:21,720 --> 00:06:24,400
Speaker 5: with respect to what tools can best be used to

112
00:06:24,440 --> 00:06:27,920
Speaker 5: accomplish that. So it's a really important issue. It's one

113
00:06:27,960 --> 00:06:30,920
Speaker 5: you can't just talk about. You have to provide essentially

114
00:06:31,000 --> 00:06:34,279
Speaker 5: the technology and the capabilities and the guidance to enable

115
00:06:34,320 --> 00:06:35,480
Speaker 5: people to test for it.

116
00:06:36,200 --> 00:06:38,880
Speaker 4: Recently, you had this wonderful opportunity to head to Congress

117
00:06:38,920 --> 00:06:41,960
Speaker 4: to talk about AI, and in your testimony before Congress,

118
00:06:42,000 --> 00:06:45,000
Speaker 4: you mentioned that it's often said that innovation moves too

119
00:06:45,040 --> 00:06:47,640
Speaker 4: fast for government to keep up. And this is something

120
00:06:47,640 --> 00:06:49,800
Speaker 4: that I also worry about as a psychologist. Right our

121
00:06:49,839 --> 00:06:52,760
Speaker 4: policy makers really understanding the issues that they're dealing with,

122
00:06:53,200 --> 00:06:55,440
Speaker 4: And so I'm curious how you're approaching this challenge of

123
00:06:55,480 --> 00:06:57,920
Speaker 4: adapting AI policies to keep up with the sort of

124
00:06:58,000 --> 00:07:00,760
Speaker 4: rapid pace of all the advancements were in the AI

125
00:07:00,839 --> 00:07:01,880
Speaker 4: technology itself.

126
00:07:02,839 --> 00:07:06,039
Speaker 5: I think it's really critically important that you have foundational

127
00:07:06,120 --> 00:07:11,560
Speaker 5: principles that applied to not only how you use technology,

128
00:07:11,600 --> 00:07:13,080
Speaker 5: but whether you're going to use it in the first

129
00:07:13,120 --> 00:07:14,760
Speaker 5: place and where you're going to use and apply it

130
00:07:14,760 --> 00:07:18,840
Speaker 5: across your company. And then your program from a governance perspective,

131
00:07:18,920 --> 00:07:21,440
Speaker 5: has to be agile. It has to be able to

132
00:07:21,520 --> 00:07:27,240
Speaker 5: address emerging capabilities, new training methods, etc. And part of

133
00:07:27,280 --> 00:07:31,800
Speaker 5: that involves helping to educate and instill and empower a

134
00:07:31,840 --> 00:07:35,640
Speaker 5: trustworthy culture at a company so you can spot those

135
00:07:35,680 --> 00:07:37,920
Speaker 5: issues so you can ask the right questions at the

136
00:07:38,000 --> 00:07:41,200
Speaker 5: right time if you try. We talked about during the

137
00:07:41,240 --> 00:07:45,440
Speaker 5: Senate hearing, and IBM's been talking for years about regulating

138
00:07:45,760 --> 00:07:48,920
Speaker 5: the use, not the technology itself, because if you try

139
00:07:48,920 --> 00:07:52,760
Speaker 5: to regulate technology, you're very quickly going to find out

140
00:07:53,160 --> 00:07:56,120
Speaker 5: regulation will absolutely never keep up with that.

141
00:07:56,600 --> 00:07:58,800
Speaker 4: And so in your testimony to Congress, you also talked

142
00:07:58,840 --> 00:08:02,040
Speaker 4: about this idea of a precision regulation approach for AI.

143
00:08:02,680 --> 00:08:04,880
Speaker 4: Tell me more about this. What is a precision regulation

144
00:08:04,960 --> 00:08:06,960
Speaker 4: approach and why could that be so important.

145
00:08:07,240 --> 00:08:10,200
Speaker 5: It's funny because I was able to share with Congress

146
00:08:10,840 --> 00:08:14,640
Speaker 5: our precision regulation point of view in twenty twenty three,

147
00:08:15,000 --> 00:08:17,840
Speaker 5: but that precision regulation point of view was published by

148
00:08:17,920 --> 00:08:22,600
Speaker 5: IBM in twenty twenty so we have not changed our

149
00:08:22,720 --> 00:08:27,640
Speaker 5: position that you should apply the tightest controls, the strictest

150
00:08:27,680 --> 00:08:31,920
Speaker 5: regulatory requirements to the technology where the end use and

151
00:08:32,040 --> 00:08:35,960
Speaker 5: risk of societal harm is the greatest. So that's essentially

152
00:08:36,000 --> 00:08:38,960
Speaker 5: what it is. There's lots of AI technology that's used

153
00:08:39,000 --> 00:08:42,640
Speaker 5: today that doesn't touch people, that's very low risk in nature.

154
00:08:43,160 --> 00:08:46,600
Speaker 5: And even when you think about AI that delivers a

155
00:08:46,720 --> 00:08:52,160
Speaker 5: movie recommendation versus AI that is used to diagnose cancer, right,

156
00:08:52,200 --> 00:08:55,680
Speaker 5: there's very different implications associated with those two uses of

157
00:08:55,720 --> 00:09:00,480
Speaker 5: the technology. And so essentially what precision regulation it is

158
00:09:00,480 --> 00:09:03,400
Speaker 5: is apply different rules to different risks, right, more stringent

159
00:09:03,480 --> 00:09:07,120
Speaker 5: regulation to the use cases with the greatest risk. And

160
00:09:07,160 --> 00:09:10,800
Speaker 5: then also we build that out calling for things like

161
00:09:10,840 --> 00:09:15,440
Speaker 5: transparency you see it today with content right, misinformation and

162
00:09:15,480 --> 00:09:19,600
Speaker 5: the like. We believe that consumers should always know when

163
00:09:19,600 --> 00:09:22,640
Speaker 5: they're interacting with an AI system, So be transparent, don't

164
00:09:22,720 --> 00:09:27,480
Speaker 5: hydror AI clearly define the risks. So as a country,

165
00:09:27,760 --> 00:09:30,640
Speaker 5: we need to have some clear guidance right and globally

166
00:09:30,679 --> 00:09:34,360
Speaker 5: as well in terms of which uses of AI or

167
00:09:34,480 --> 00:09:38,880
Speaker 5: higher risk where will apply higher and stricter regulation and

168
00:09:39,080 --> 00:09:41,600
Speaker 5: have sort of a common understanding of what those high

169
00:09:41,679 --> 00:09:45,440
Speaker 5: risk uses are and then demonstrate the impact in the

170
00:09:45,480 --> 00:09:50,079
Speaker 5: cases of those higher risk uses. So companies who are

171
00:09:50,160 --> 00:09:54,600
Speaker 5: using AI in spaces where they can impact people's legal rights,

172
00:09:54,640 --> 00:09:59,000
Speaker 5: for example, should have to conduct an impact assessment that

173
00:09:59,080 --> 00:10:02,880
Speaker 5: demonstrates that the technology isn't biased. So we've been pretty

174
00:10:02,880 --> 00:10:06,559
Speaker 5: clear about apply that the most stringent regulation to the

175
00:10:06,640 --> 00:10:08,000
Speaker 5: highest risk uses of AI.

176
00:10:09,760 --> 00:10:12,040
Speaker 4: And so so far we've been talking about your congressional

177
00:10:12,040 --> 00:10:14,880
Speaker 4: testimony in terms of, you know, the specific content that

178
00:10:14,920 --> 00:10:17,400
Speaker 4: you talked about, But I'm just curious on a personal level,

179
00:10:17,720 --> 00:10:19,839
Speaker 4: you know, what was that like right like, right now,

180
00:10:19,840 --> 00:10:22,120
Speaker 4: it feels like at a policy level, like there's a

181
00:10:22,200 --> 00:10:24,559
Speaker 4: kind of fever pitch going on with AI right now.

182
00:10:24,800 --> 00:10:26,240
Speaker 4: You know, what did that feel like to kind of

183
00:10:26,320 --> 00:10:28,760
Speaker 4: really have the opportunity to talk to policy makers and

184
00:10:28,800 --> 00:10:31,800
Speaker 4: sort of influence what they're thinking about AI technologies like

185
00:10:31,880 --> 00:10:32,840
Speaker 4: in the coming century.

186
00:10:32,880 --> 00:10:35,800
Speaker 5: Perhaps I was really an honor to be able to

187
00:10:35,880 --> 00:10:38,840
Speaker 5: do that and to be one of the first set

188
00:10:38,880 --> 00:10:42,960
Speaker 5: of invitees to the first hearing. And what I learned

189
00:10:43,000 --> 00:10:46,160
Speaker 5: from it essentially is, you know, really two things. The

190
00:10:46,200 --> 00:10:50,079
Speaker 5: first is really the value of authenticity. So both as

191
00:10:50,240 --> 00:10:54,200
Speaker 5: an individual and as a company, I was able to

192
00:10:54,240 --> 00:10:56,800
Speaker 5: talk about what I do, you know, I need a

193
00:10:56,800 --> 00:11:00,640
Speaker 5: lot of advanced prep, right, I talked about my job

194
00:11:00,720 --> 00:11:04,520
Speaker 5: is what IBM has been putting in place for years now.

195
00:11:05,080 --> 00:11:08,960
Speaker 5: So this isn't about creating something. This was just about

196
00:11:09,000 --> 00:11:11,320
Speaker 5: showing up and being authentic. And we were invited for

197
00:11:11,360 --> 00:11:14,200
Speaker 5: a reason. We were invited because we were one of

198
00:11:14,200 --> 00:11:18,960
Speaker 5: the earliest companies in the AI technology space. We're the

199
00:11:19,000 --> 00:11:24,439
Speaker 5: oldest technology company and we are trusted and that's an honor.

200
00:11:24,600 --> 00:11:26,840
Speaker 5: And then the second thing I came away with was

201
00:11:26,880 --> 00:11:29,240
Speaker 5: really how important this issue is to society. I don't

202
00:11:29,240 --> 00:11:34,720
Speaker 5: think I appreciated it as much until following that experience.

203
00:11:35,360 --> 00:11:39,160
Speaker 5: I had outreached from colleagues I hadn't worked with for years.

204
00:11:39,240 --> 00:11:41,440
Speaker 5: I had outreach from family members who heard me on

205
00:11:41,480 --> 00:11:44,600
Speaker 5: the radio. You know, my mother and my mother in

206
00:11:44,679 --> 00:11:47,760
Speaker 5: law and my nieces and nephews and my friends of

207
00:11:47,800 --> 00:11:49,640
Speaker 5: my kids were all like, oh, I get it, I

208
00:11:49,679 --> 00:11:50,360
Speaker 5: get what you do.

209
00:11:50,440 --> 00:11:50,640
Speaker 4: Now.

210
00:11:50,679 --> 00:11:53,160
Speaker 5: Wow, that's pretty cool, you know. So that was really

211
00:11:53,920 --> 00:11:57,080
Speaker 5: probably the best and most impactful takeaway that I had.

212
00:11:57,559 --> 00:12:01,760
Speaker 3: The mass adoption of generative AI at breakneck speed has

213
00:12:01,800 --> 00:12:06,160
Speaker 3: spurred societies and governments around the world to get serious

214
00:12:06,640 --> 00:12:11,960
Speaker 3: about regulating AI. For businesses, compliance is complex enough already,

215
00:12:12,120 --> 00:12:15,480
Speaker 3: but throw an ever involving technology like AI into the mix,

216
00:12:15,880 --> 00:12:22,160
Speaker 3: and compliance itself becomes an exercise in adaptability. As regulators

217
00:12:22,160 --> 00:12:26,720
Speaker 3: seek greater accountability in how AI is used, businesses need

218
00:12:26,760 --> 00:12:31,360
Speaker 3: help creating governance processes comprehensive enough to comply with the

219
00:12:31,440 --> 00:12:34,839
Speaker 3: law but agile enough to keep up with the rapid

220
00:12:34,920 --> 00:12:40,040
Speaker 3: rate of change in AI development. Regulatory scrutiny isn't the

221
00:12:40,120 --> 00:12:45,240
Speaker 3: only consideration either responsible AI governance of business's ability to

222
00:12:45,400 --> 00:12:50,199
Speaker 3: prove its AI models are transparent and explainable is also

223
00:12:50,320 --> 00:12:55,480
Speaker 3: key to building trust with customers, regardless of industry. In

224
00:12:55,520 --> 00:12:59,240
Speaker 3: the next part of their conversation, Laurie asked Christina what

225
00:12:59,320 --> 00:13:04,320
Speaker 3: businesses SHO should consider when approaching AI governance. Let's listen.

226
00:13:05,160 --> 00:13:07,559
Speaker 4: So it's a particular role that businesses are playing in

227
00:13:07,679 --> 00:13:10,479
Speaker 4: AI governance, Like why is it so critical for businesses

228
00:13:10,480 --> 00:13:11,480
Speaker 4: to be part of this?

229
00:13:12,040 --> 00:13:16,480
Speaker 5: So I think it's really critically important that businesses understand

230
00:13:17,040 --> 00:13:19,959
Speaker 5: the impacts that technology can have, both in making them

231
00:13:19,960 --> 00:13:23,800
Speaker 5: better businesses, but the impacts that those technologies can have

232
00:13:24,280 --> 00:13:29,400
Speaker 5: on the consumers that they are supporting. You know, businesses

233
00:13:29,520 --> 00:13:34,120
Speaker 5: need to be deploying AI technology that is in alignment

234
00:13:34,280 --> 00:13:36,160
Speaker 5: with the goals that they set for it, and that

235
00:13:36,280 --> 00:13:39,160
Speaker 5: can be trusted. I think for us and for our clients,

236
00:13:39,480 --> 00:13:42,400
Speaker 5: a lot of this comes back to trust in tech.

237
00:13:42,840 --> 00:13:48,520
Speaker 5: If you deploy something that doesn't work, that hallucinates, that discriminates,

238
00:13:49,000 --> 00:13:53,120
Speaker 5: that isn't transparent, where decisions can't be explained, then you

239
00:13:53,280 --> 00:13:56,920
Speaker 5: are going to very rapidly erode the trust at best

240
00:13:57,080 --> 00:14:00,280
Speaker 5: right of your clients and at worst for yourself if

241
00:14:00,280 --> 00:14:02,720
Speaker 5: you're going to create legal and regulatory issues for yourself

242
00:14:02,760 --> 00:14:06,680
Speaker 5: as well. So trusted technology is really important, and I

243
00:14:06,679 --> 00:14:08,880
Speaker 5: think there's a lot of pressure on businesses today to

244
00:14:08,920 --> 00:14:11,600
Speaker 5: move very rapidly and adopt technology. But if you do

245
00:14:11,640 --> 00:14:14,800
Speaker 5: it without having a program of governance in place, you're

246
00:14:14,840 --> 00:14:16,800
Speaker 5: really risking eroding that trust.

247
00:14:16,920 --> 00:14:19,120
Speaker 4: So this is really where I think a strong AI

248
00:14:19,200 --> 00:14:22,760
Speaker 4: governance comes in. Talk about from your perspective, how this

249
00:14:22,840 --> 00:14:26,720
Speaker 4: really contributes to maintaining the trust that customers and stakeholders

250
00:14:26,760 --> 00:14:27,920
Speaker 4: have in these technologies.

251
00:14:28,640 --> 00:14:31,200
Speaker 5: Yeah. Absolutely, I mean you need to have a governance

252
00:14:31,240 --> 00:14:35,240
Speaker 5: program because you need to understand that the technology, particularly

253
00:14:35,240 --> 00:14:39,840
Speaker 5: in the AI space, that you are deploying, is explainable.

254
00:14:39,880 --> 00:14:44,240
Speaker 5: You need to understand why it's making decisions and recommendations

255
00:14:44,280 --> 00:14:45,760
Speaker 5: that it's making, and you need to be able to

256
00:14:45,760 --> 00:14:48,160
Speaker 5: explain that to your consumers. I mean, you can't do

257
00:14:48,240 --> 00:14:50,320
Speaker 5: that if you don't know where your data is coming from,

258
00:14:50,320 --> 00:14:52,800
Speaker 5: what data are you using to train those models, if

259
00:14:52,800 --> 00:14:56,600
Speaker 5: you don't have a program that manages the alignment of

260
00:14:56,640 --> 00:14:59,840
Speaker 5: your AI models over time to make sure as a

261
00:15:00,160 --> 00:15:04,440
Speaker 5: I learns and evolves over uses, which is in large

262
00:15:04,480 --> 00:15:08,760
Speaker 5: part what makes it so beneficial that it stays in

263
00:15:08,800 --> 00:15:12,200
Speaker 5: alignment with the objectives that you set for the technology

264
00:15:12,200 --> 00:15:15,840
Speaker 5: over time. So you can't do that without a robust

265
00:15:16,160 --> 00:15:20,000
Speaker 5: governance process in place. So we work with clients to

266
00:15:20,120 --> 00:15:22,920
Speaker 5: share our own story here at IBM in terms of

267
00:15:22,960 --> 00:15:25,720
Speaker 5: how we put that in place, but also in our

268
00:15:25,720 --> 00:15:31,360
Speaker 5: consulting practice to help clients work with these new generative

269
00:15:31,400 --> 00:15:34,560
Speaker 5: capabilities and foundation models and the like in order to

270
00:15:34,760 --> 00:15:36,680
Speaker 5: put them to work for their business in a way

271
00:15:36,720 --> 00:15:39,480
Speaker 5: that's going to be impactful to that business but at

272
00:15:39,480 --> 00:15:41,000
Speaker 5: the same time be trusted.

273
00:15:41,400 --> 00:15:43,280
Speaker 4: So now I wanted to turn a little bit towards

274
00:15:43,280 --> 00:15:47,120
Speaker 4: watsonex governance, and so IBM recently announced their AI platform,

275
00:15:47,160 --> 00:15:50,880
Speaker 4: Watson X, which will include a governance component. Could you

276
00:15:50,920 --> 00:15:53,520
Speaker 4: tell us a little more about watsonex dot Governance.

277
00:15:53,880 --> 00:15:56,120
Speaker 5: Yeah, I mean before I do that, I'll just back

278
00:15:56,200 --> 00:15:59,760
Speaker 5: up and talk about the full platform and then lean

279
00:15:59,800 --> 00:16:02,120
Speaker 5: into to watson x because I think it's important to

280
00:16:02,200 --> 00:16:08,080
Speaker 5: understand the delivery of a full suite of capabilities, to

281
00:16:08,120 --> 00:16:11,240
Speaker 5: get data, to train models, and then to govern them

282
00:16:11,440 --> 00:16:16,720
Speaker 5: over their life cycle. All of these things are really important.

283
00:16:17,280 --> 00:16:20,360
Speaker 5: From the onset you need to make sure that you have,

284
00:16:21,000 --> 00:16:25,240
Speaker 5: you know, for our watsonex dot AI for example, that's

285
00:16:25,280 --> 00:16:29,840
Speaker 5: the studio to train new foundation models and generative AI

286
00:16:29,920 --> 00:16:35,560
Speaker 5: and machine learning capabilities, and we are populating that studio

287
00:16:35,920 --> 00:16:41,200
Speaker 5: with some IBM trained foundation models which we're curating and

288
00:16:41,280 --> 00:16:45,040
Speaker 5: tailoring more specifically for enterprises. So that's really important. It

289
00:16:45,040 --> 00:16:47,440
Speaker 5: comes back to the point I made earlier about business

290
00:16:47,440 --> 00:16:52,200
Speaker 5: trust and the need, you know, to have enterprise ready

291
00:16:53,240 --> 00:16:57,360
Speaker 5: technologies in the AI space. And then the watsonex dot

292
00:16:57,480 --> 00:17:01,040
Speaker 5: data is a fit for purpose data or a data lake,

293
00:17:01,640 --> 00:17:06,120
Speaker 5: and then watsonex dot gov. So that's a particular component

294
00:17:06,280 --> 00:17:10,240
Speaker 5: of the platform that my team and the AI Ethics

295
00:17:10,280 --> 00:17:14,359
Speaker 5: Board has really worked closely with the product team on developing,

296
00:17:14,400 --> 00:17:17,240
Speaker 5: and we're using it internally here in the Chief Privacy

297
00:17:17,280 --> 00:17:21,640
Speaker 5: Office as well to help us govern our own uses

298
00:17:21,720 --> 00:17:26,760
Speaker 5: of AI technology and our compliance program here, and it

299
00:17:26,880 --> 00:17:32,240
Speaker 5: essentially helps to notify you if a model becomes biased

300
00:17:32,280 --> 00:17:35,440
Speaker 5: or gets out of alignment as you're using it over time.

301
00:17:36,000 --> 00:17:38,639
Speaker 5: So companies are going to need these capabilities. I mean

302
00:17:38,680 --> 00:17:42,840
Speaker 5: they need them today to deliver technologies with trust. They'll

303
00:17:42,880 --> 00:17:46,159
Speaker 5: need them tomorrow to comply with regulation, which is on

304
00:17:46,240 --> 00:17:47,359
Speaker 5: the horizon.

305
00:17:47,080 --> 00:17:49,960
Speaker 4: And I think compliance becomes even more complex when you

306
00:17:50,000 --> 00:17:54,120
Speaker 4: consider international data protection laws and regulations. Honestly, I don't

307
00:17:54,160 --> 00:17:56,720
Speaker 4: know how anyone on any company's legal team is keeping

308
00:17:56,800 --> 00:17:59,040
Speaker 4: up with us these days. But my question for you

309
00:17:59,160 --> 00:18:02,760
Speaker 4: is really how can businesses develop a strategy to maintain

310
00:18:02,800 --> 00:18:05,600
Speaker 4: compliance and to deal with it in this ever changing landscape.

311
00:18:05,680 --> 00:18:09,600
Speaker 5: It's increasingly more challenging. In fact, I saw statistic just

312
00:18:09,680 --> 00:18:14,280
Speaker 5: this morning that the regulatory obligations on companies have increased

313
00:18:14,280 --> 00:18:18,160
Speaker 5: something like seven hundred times in the last twenty years.

314
00:18:17,960 --> 00:18:22,560
Speaker 5: So it really is a huge focus area for companies.

315
00:18:22,720 --> 00:18:25,600
Speaker 5: You have to have a process in place in order

316
00:18:25,640 --> 00:18:28,199
Speaker 5: to do that, and it's not easy, particularly for a

317
00:18:28,240 --> 00:18:32,120
Speaker 5: company like IBM that it has a presence in over

318
00:18:32,119 --> 00:18:35,159
Speaker 5: one hundred and seventy countries around the world. There is

319
00:18:35,200 --> 00:18:40,080
Speaker 5: more than one hundred and fifty comprehensive privacy regulations. There

320
00:18:40,119 --> 00:18:44,640
Speaker 5: are regulations of non personal data, there are AI regulations emerging,

321
00:18:45,600 --> 00:18:49,679
Speaker 5: So you really need an operational approach to it in

322
00:18:49,800 --> 00:18:51,840
Speaker 5: order to stay compliant. But one of the things we

323
00:18:51,840 --> 00:18:54,040
Speaker 5: do is we set a baseline. And a lot of

324
00:18:54,040 --> 00:18:57,480
Speaker 5: companies do this as well. So we define a privacy baseline,

325
00:18:57,520 --> 00:19:02,320
Speaker 5: we define an AI baseline, and we ensure then as

326
00:19:02,359 --> 00:19:04,760
Speaker 5: a result of that that there are very few deviances

327
00:19:04,840 --> 00:19:07,920
Speaker 5: because it incorporates in that baseline. So that's one of

328
00:19:07,960 --> 00:19:10,119
Speaker 5: the ways we do it. Other companies, I think are

329
00:19:10,200 --> 00:19:15,399
Speaker 5: similarly situated in terms of doing that. But again, it

330
00:19:15,560 --> 00:19:18,240
Speaker 5: is a real challenge for global companies. It's one of

331
00:19:18,240 --> 00:19:22,200
Speaker 5: the reasons why we advocate for as much alignment as

332
00:19:22,280 --> 00:19:27,640
Speaker 5: possible on the international realm as well as nationally here

333
00:19:27,680 --> 00:19:31,440
Speaker 5: in the US, as much alignment as possible to make

334
00:19:31,680 --> 00:19:36,640
Speaker 5: compliance easier for easier, and not just because companies want

335
00:19:36,680 --> 00:19:39,800
Speaker 5: an easy way to comply. But the harder it is,

336
00:19:40,080 --> 00:19:43,960
Speaker 5: the less likely there will be compliance. And it's not

337
00:19:44,080 --> 00:19:50,159
Speaker 5: the objective of anybody, governments, companies, consumers to have to

338
00:19:50,320 --> 00:19:53,840
Speaker 5: set legal obligations that companies simply can't meet.

339
00:19:54,080 --> 00:19:56,080
Speaker 4: So what advice would you give to other companies who

340
00:19:56,119 --> 00:19:59,159
Speaker 4: are looking to rethink or strengthen their approach to AI governance.

341
00:19:59,240 --> 00:20:02,600
Speaker 5: Think you need to start with, as we did, foundational principles,

342
00:20:03,240 --> 00:20:06,679
Speaker 5: and you need to start making decisions about what technology

343
00:20:06,680 --> 00:20:09,040
Speaker 5: you're going to deploy and what technology you're not, What

344
00:20:09,040 --> 00:20:10,160
Speaker 5: are you going to use it for, and what aren't

345
00:20:10,160 --> 00:20:11,560
Speaker 5: you going to use it for? And then when you

346
00:20:11,560 --> 00:20:15,960
Speaker 5: do use it, align to those principles. That's really important.

347
00:20:16,000 --> 00:20:20,560
Speaker 5: Formalize a program, have someone within the organization, whether it's

348
00:20:20,600 --> 00:20:24,879
Speaker 5: the chief privacy officer, whether it's some other role, a

349
00:20:24,960 --> 00:20:29,399
Speaker 5: chief AI ethics officer, but have an accountable individual and

350
00:20:29,440 --> 00:20:33,639
Speaker 5: accountable organization. Do a maturity assessment, figure out where you

351
00:20:33,720 --> 00:20:36,479
Speaker 5: are and where you need to be, and really start,

352
00:20:36,680 --> 00:20:40,760
Speaker 5: you know, putting it into place today. Don't wait for

353
00:20:41,200 --> 00:20:44,280
Speaker 5: regulation to apply directly to your business because it'll be

354
00:20:44,359 --> 00:20:44,760
Speaker 5: too late.

355
00:20:46,000 --> 00:20:49,400
Speaker 4: So Smart Talks features new creators, these visionaries like yourself

356
00:20:49,400 --> 00:20:52,920
Speaker 4: who are creatively applying technology in business to drive change.

357
00:20:53,160 --> 00:20:55,800
Speaker 4: I'm curious if you see yourself as creative.

358
00:20:56,040 --> 00:20:59,320
Speaker 5: You know, I definitely do. I mean, you need to

359
00:20:59,440 --> 00:21:04,080
Speaker 5: be creative when you're working in an industry that evolves

360
00:21:04,119 --> 00:21:08,600
Speaker 5: so very quickly. So you know, I started with IBM

361
00:21:08,880 --> 00:21:12,000
Speaker 5: when we were primarily a hardware company, right, and we've

362
00:21:12,080 --> 00:21:15,480
Speaker 5: changed our business so significantly over the years. And the

363
00:21:15,600 --> 00:21:20,040
Speaker 5: issues that are raised with respect to each new technology,

364
00:21:20,080 --> 00:21:23,720
Speaker 5: whether it be cloud, whether it be AI now where

365
00:21:23,720 --> 00:21:25,320
Speaker 5: we're seeing a ton of issues, or you look at

366
00:21:25,320 --> 00:21:29,480
Speaker 5: emergent issues in the space of things like neurotechnologies and

367
00:21:29,560 --> 00:21:36,160
Speaker 5: quantum computers. You have to be strategic and you have

368
00:21:36,280 --> 00:21:39,520
Speaker 5: to be creative and thinking about how you can adapt

369
00:21:40,160 --> 00:21:45,399
Speaker 5: agilely quickly a company to an environment that is changing

370
00:21:45,440 --> 00:21:46,320
Speaker 5: so quickly.

371
00:21:47,280 --> 00:21:50,240
Speaker 4: And with this transformation happening at such a rapid pace.

372
00:21:50,440 --> 00:21:52,399
Speaker 4: Do you think creativity plays a role in how you

373
00:21:52,440 --> 00:21:55,919
Speaker 4: think about and implement specifically a trustworthy AI strategy.

374
00:21:58,160 --> 00:22:02,560
Speaker 5: Yeah, think it does, because again, it comes back to

375
00:22:02,640 --> 00:22:05,879
Speaker 5: these capabilities, and there are ways, I guess how you

376
00:22:05,920 --> 00:22:09,640
Speaker 5: define creativity could could be different, right, But I'm thinking

377
00:22:09,640 --> 00:22:12,720
Speaker 5: of creativity in the sense of sort of agility and

378
00:22:12,760 --> 00:22:17,440
Speaker 5: strategic vision and creative problem solving. I think that's really

379
00:22:17,520 --> 00:22:20,400
Speaker 5: important in the world that we're in right now, being

380
00:22:20,520 --> 00:22:24,960
Speaker 5: able to creatively problem solve with new issues that are

381
00:22:26,000 --> 00:22:27,880
Speaker 5: rising sort of every day.

382
00:22:28,520 --> 00:22:30,080
Speaker 4: And so how do you see the role of chief

383
00:22:30,119 --> 00:22:33,760
Speaker 4: privacy officer evolving in the future as AI technology continues

384
00:22:33,800 --> 00:22:36,680
Speaker 4: to advance, Like, what step should CPOs take to stay

385
00:22:36,680 --> 00:22:38,600
Speaker 4: ahead of all these changes that are come in their way.

386
00:22:39,880 --> 00:22:43,800
Speaker 5: So the role is evolving in most companies, I would

387
00:22:43,840 --> 00:22:48,800
Speaker 5: say pretty rapidly. Many companies are looking to chief privacy

388
00:22:48,840 --> 00:22:52,360
Speaker 5: officers who are ready understand the data that's being used

389
00:22:52,359 --> 00:22:55,840
Speaker 5: in the organization and have programs to ensure compliance with

390
00:22:56,000 --> 00:22:59,720
Speaker 5: laws that require you to manage that data in accordance

391
00:22:59,760 --> 00:23:01,879
Speaker 5: with day to protection laws and the like. It's a

392
00:23:02,000 --> 00:23:08,280
Speaker 5: natural place and position for AI responsibility, and so I

393
00:23:08,280 --> 00:23:11,040
Speaker 5: think what's happening to a lot of chief privacy officers

394
00:23:11,080 --> 00:23:13,760
Speaker 5: is they're being asked to take on this AI governance

395
00:23:13,840 --> 00:23:17,760
Speaker 5: responsibility for companies and if not take it on, at

396
00:23:17,840 --> 00:23:21,600
Speaker 5: least play a very key role working with other parts

397
00:23:21,600 --> 00:23:25,359
Speaker 5: of the business in AI governance. So that really is changing.

398
00:23:25,600 --> 00:23:29,639
Speaker 5: And if chief privacy officers are in companies who maybe

399
00:23:29,680 --> 00:23:33,440
Speaker 5: haven't started thinking about AI yet, they should, So I

400
00:23:33,440 --> 00:23:37,320
Speaker 5: would encourage them to look at different resources that are

401
00:23:37,359 --> 00:23:41,720
Speaker 5: available already in AI governance space. For example, the International

402
00:23:41,760 --> 00:23:46,200
Speaker 5: Association of Privacy Professionals, which is the seventy five thousand

403
00:23:46,240 --> 00:23:51,399
Speaker 5: member professional body for the Profession of Chief Privacy Officers

404
00:23:51,440 --> 00:23:56,080
Speaker 5: just recently launched an AI Governance Initiative and an AI

405
00:23:56,119 --> 00:24:00,159
Speaker 5: Governance Certification program. I sit on their advisory board. But

406
00:24:00,200 --> 00:24:02,879
Speaker 5: that's just emblematic of the fact that the field is

407
00:24:02,960 --> 00:24:04,400
Speaker 5: changing so rapidly.

408
00:24:05,920 --> 00:24:08,480
Speaker 4: And so, you know, speaking of rapid change, when you're

409
00:24:08,760 --> 00:24:11,159
Speaker 4: back here on smart Talks in twenty twenty one, you

410
00:24:11,200 --> 00:24:13,520
Speaker 4: said that the future of AI will be more transparent

411
00:24:13,560 --> 00:24:15,720
Speaker 4: and more trustworthy. You know, what do you see the

412
00:24:15,800 --> 00:24:17,840
Speaker 4: next five to ten years holding. You know, when you're

413
00:24:17,880 --> 00:24:20,760
Speaker 4: back on smart Talks in you know, twenty twenty six,

414
00:24:20,880 --> 00:24:22,560
Speaker 4: you know twenty thirty, you know what are we going

415
00:24:22,640 --> 00:24:24,920
Speaker 4: to be talking about when it comes to AI technology

416
00:24:24,920 --> 00:24:25,560
Speaker 4: and governance.

417
00:24:26,119 --> 00:24:28,280
Speaker 5: So I try to be an optimist, right and I

418
00:24:28,320 --> 00:24:32,159
Speaker 5: said that two years ago, and I think we're seeing

419
00:24:32,200 --> 00:24:36,600
Speaker 5: it now come into fruition. And there will be requirements,

420
00:24:37,400 --> 00:24:40,200
Speaker 5: whether they're coming from the US, whether they're coming from Europe,

421
00:24:40,280 --> 00:24:43,879
Speaker 5: whether they're just coming from voluntary adoption by clients of

422
00:24:43,960 --> 00:24:48,679
Speaker 5: things like the NISS Risk Management Framework, really important voluntary frameworks.

423
00:24:49,440 --> 00:24:53,040
Speaker 5: You're going to have to adopt transparent and explainable practices

424
00:24:53,200 --> 00:24:55,800
Speaker 5: in your uses of AI. So I do see that happening.

425
00:24:55,880 --> 00:24:58,320
Speaker 5: And in the next five to ten years, boy, I

426
00:24:58,359 --> 00:25:04,320
Speaker 5: think we'll see more research into trust and techniques because

427
00:25:04,320 --> 00:25:08,280
Speaker 5: we don't really know, for example, how to water mark.

428
00:25:09,040 --> 00:25:12,040
Speaker 5: We were calling for things like watermarking. There'll be more

429
00:25:12,119 --> 00:25:17,399
Speaker 5: research into how to do that. I think you'll see

430
00:25:17,680 --> 00:25:20,880
Speaker 5: you regulation that's specifically going to require those types of things.

431
00:25:20,920 --> 00:25:23,080
Speaker 5: So I think again, I think the regulation is going

432
00:25:23,119 --> 00:25:25,920
Speaker 5: to drive research. It's going to drive research into these

433
00:25:26,000 --> 00:25:31,159
Speaker 5: areas that will help ensure that we can deliver new capabilities,

434
00:25:31,200 --> 00:25:34,560
Speaker 5: generated capabilities and the like with trust and explainability.

435
00:25:35,040 --> 00:25:37,080
Speaker 4: Thank you so much Christina for joining me on smart

436
00:25:37,080 --> 00:25:38,760
Speaker 4: Talks to talk about AI and governance.

437
00:25:39,480 --> 00:25:41,440
Speaker 5: Well, thank you very much for having me.

438
00:25:43,000 --> 00:25:47,720
Speaker 3: To unlock the transformative growth possible with artificial intelligence. Businesses

439
00:25:47,800 --> 00:25:50,919
Speaker 3: need to know what they wish to grow into first.

440
00:25:51,880 --> 00:25:54,720
Speaker 3: Like Christina said, the best way forward in the AI

441
00:25:54,840 --> 00:25:58,880
Speaker 3: future is for businesses to figure out their own foundational

442
00:25:58,920 --> 00:26:03,520
Speaker 3: principles around using the technology, drawing on those principles to

443
00:26:03,600 --> 00:26:07,400
Speaker 3: apply AI in a way that's ethically consistent with their

444
00:26:07,440 --> 00:26:11,119
Speaker 3: mission and complies with I legal frameworks built to hold

445
00:26:11,119 --> 00:26:16,159
Speaker 3: the technology accountable. As AI adoption grows more and more widespread,

446
00:26:16,359 --> 00:26:20,679
Speaker 3: so too will the expectation from consumers and regulators that

447
00:26:20,760 --> 00:26:26,199
Speaker 3: businesses use it responsibly. Investing independable AI governance is a

448
00:26:26,240 --> 00:26:30,200
Speaker 3: way for businesses to lay the foundations for technology that

449
00:26:30,240 --> 00:26:34,119
Speaker 3: their customers can trust while rising to the challenge of

450
00:26:34,240 --> 00:26:40,040
Speaker 3: increasing regulatory complexity. Though the emergence of AI does complicate

451
00:26:40,160 --> 00:26:44,720
Speaker 3: an already tough compliance landscape, businesses now face a creative

452
00:26:44,760 --> 00:26:49,399
Speaker 3: opportunity to set a precedent for what accountability in AI

453
00:26:49,560 --> 00:26:53,520
Speaker 3: looks like and rethink what it means to deploy trustworthy

454
00:26:53,960 --> 00:26:59,760
Speaker 3: artificial intelligence. I'm Malcolm Gladwell. This is a paid advertisement

455
00:27:00,200 --> 00:27:03,480
Speaker 3: from IBM. Smart Talks with IBM will be taking a

456
00:27:03,480 --> 00:27:07,640
Speaker 3: short hiatus, but look for new episodes in the coming weeks.

457
00:27:08,280 --> 00:27:11,680
Speaker 3: Smart Talks with IBM is produced by Matt Romano, David

458
00:27:11,800 --> 00:27:16,840
Speaker 3: jaw nische Venkat and Royston Deserve with Jacob Colstein. We're

459
00:27:16,920 --> 00:27:20,640
Speaker 3: edited by Lydia Jane Kott. Our engineer is Jason Gambrel.

460
00:27:20,960 --> 00:27:25,879
Speaker 3: Theme song by Gramoscope. Special thanks to Carli Migliore, Andy Kelly,

461
00:27:26,280 --> 00:27:30,040
Speaker 3: Kathy Callahan and the Eight Bar and ib M teams,

462
00:27:30,359 --> 00:27:34,199
Speaker 3: as well as the Pushkin marketing team. Smart Talks with

463
00:27:34,280 --> 00:27:37,960
Speaker 3: IBM is a production of Pushkin Industries and Ruby Studio

464
00:27:38,359 --> 00:27:42,880
Speaker 3: at iHeartMedia. To find more Pushkin podcasts, listen on the

465
00:27:42,920 --> 00:27:51,080
Speaker 3: iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts.