1
00:00:00,120 --> 00:00:02,840
Speaker 1: Hey everyone, it's Robert and Joe here. Today we've got

2
00:00:02,840 --> 00:00:04,720
Speaker 1: something a little bit different to share with you. It

3
00:00:04,840 --> 00:00:08,000
Speaker 1: is a new season of the Smart Talks with IBM

4
00:00:08,119 --> 00:00:09,119
Speaker 1: podcast series.

5
00:00:09,600 --> 00:00:11,680
Speaker 2: Today we are witnessed to one of those rare moments

6
00:00:11,680 --> 00:00:14,360
Speaker 2: in history, the rise of an innovative technology with the

7
00:00:14,360 --> 00:00:18,680
Speaker 2: potential to radically transform business and society forever. The technology,

8
00:00:18,760 --> 00:00:22,200
Speaker 2: of course, is artificial intelligence, and it's the central focus

9
00:00:22,239 --> 00:00:24,800
Speaker 2: for this new season of Smart Talks with IBM.

10
00:00:25,320 --> 00:00:28,400
Speaker 1: Join hosts from your favorite Pushkin podcasts as they talk

11
00:00:28,480 --> 00:00:31,640
Speaker 1: with industry experts and leaders to explore how businesses can

12
00:00:31,680 --> 00:00:35,360
Speaker 1: integrate AI into their workflows and help drive real change

13
00:00:35,400 --> 00:00:38,160
Speaker 1: in this new era of AI. And of course, host

14
00:00:38,280 --> 00:00:40,440
Speaker 1: Malcolm Gladwell will be there to guide you through the

15
00:00:40,479 --> 00:00:42,640
Speaker 1: season and throw in his two cents as well.

16
00:00:43,120 --> 00:00:46,120
Speaker 2: Look out for new episodes of Smart Talks with IBM

17
00:00:46,400 --> 00:00:49,519
Speaker 2: every other week on the iHeartRadio app, Apple Podcasts, or

18
00:00:49,560 --> 00:00:53,360
Speaker 2: wherever you get your podcasts, and learn more at IBM

19
00:00:53,479 --> 00:00:55,480
Speaker 2: dot com slash smart talks.

20
00:00:56,200 --> 00:00:59,800
Speaker 3: Hello, Hello, Welcome to Smart Talks with IBM, a podcast

21
00:00:59,880 --> 00:01:05,840
Speaker 3: from Pushkin Industries, iHeartRadio and IBM. I'm Malcolm Gladwell. This

22
00:01:05,959 --> 00:01:10,920
Speaker 3: season we're continuing our conversation with new creators visionaries who

23
00:01:10,920 --> 00:01:15,080
Speaker 3: are creatively applying technology in business to drive change, but

24
00:01:15,160 --> 00:01:19,720
Speaker 3: with a focus on the transformative power of artificial intelligence

25
00:01:20,040 --> 00:01:22,760
Speaker 3: and what it means to leverage AI as a game

26
00:01:22,880 --> 00:01:27,759
Speaker 3: changing multiplier for your business. Our guest today is Jeff Boutier,

27
00:01:28,200 --> 00:01:31,720
Speaker 3: head of Product and Growth at hugging Face, the leading

28
00:01:31,840 --> 00:01:37,039
Speaker 3: open source and open science artificial intelligence platform. An engineer

29
00:01:37,040 --> 00:01:40,920
Speaker 3: by background, he has a self professed obsession with the

30
00:01:40,959 --> 00:01:45,560
Speaker 3: business of technology. Recently, IBM and hugging Face announced the

31
00:01:45,600 --> 00:01:50,200
Speaker 3: collaboration bringing together hugging faces repositories of open source AI

32
00:01:50,320 --> 00:01:55,160
Speaker 3: models with IBM's Watson X platform. It's a move that

33
00:01:55,200 --> 00:01:59,600
Speaker 3: gives businesses even more access to AI while staying true

34
00:01:59,600 --> 00:02:05,320
Speaker 3: to IBA's long standing philosophy of supporting open source technology.

35
00:02:06,200 --> 00:02:10,080
Speaker 3: With open source, businesses can build better AI models that

36
00:02:10,160 --> 00:02:14,680
Speaker 3: suit their specific needs using their own proprietary data while

37
00:02:14,720 --> 00:02:19,799
Speaker 3: browsing a ready catalog of pre trained models. In today's episode,

38
00:02:20,000 --> 00:02:23,120
Speaker 3: you'll hear why open source is so crucial to the

39
00:02:23,200 --> 00:02:27,800
Speaker 3: advancement of AI, how IBM's Watson X interacts with open

40
00:02:27,800 --> 00:02:32,280
Speaker 3: source AI, and Jeff's thoughts on why this singular omnipotent

41
00:02:32,560 --> 00:02:36,600
Speaker 3: AI model is a myth. Jeff spoke with Tim Harford,

42
00:02:36,840 --> 00:02:41,119
Speaker 3: host of the Pushkin podcast Cautionary Tales, a longtime columnist

43
00:02:41,120 --> 00:02:44,600
Speaker 3: at The Financial Times, where he writes the Undercover Economist.

44
00:02:44,880 --> 00:02:48,680
Speaker 3: Tim is also a BBC broadcaster with his show More

45
00:02:48,840 --> 00:02:52,520
Speaker 3: or Less. Okay, let's get to the interview.

46
00:03:00,040 --> 00:03:02,840
Speaker 4: I am a Jeff Boudier and I'm a product director

47
00:03:02,919 --> 00:03:04,279
Speaker 4: at Hugging Face.

48
00:03:05,120 --> 00:03:08,640
Speaker 5: So I'm immediately intrigue. Hugging Face. Is this a reference

49
00:03:08,639 --> 00:03:10,240
Speaker 5: to the Alien movie or something else.

50
00:03:11,560 --> 00:03:15,400
Speaker 4: It is not, and it may be not obvious to

51
00:03:15,520 --> 00:03:19,200
Speaker 4: a listener, but hugging Face is the name of that

52
00:03:19,320 --> 00:03:22,160
Speaker 4: cute emoji, you know, the one that's smiling with his

53
00:03:22,320 --> 00:03:25,520
Speaker 4: two hands extended like that to give you a big hug.

54
00:03:25,760 --> 00:03:29,000
Speaker 4: That's hugging Face. So basically, we name the company after

55
00:03:29,120 --> 00:03:30,480
Speaker 4: an emoji.

56
00:03:31,360 --> 00:03:33,760
Speaker 5: And it is I saw your website and it is

57
00:03:33,800 --> 00:03:36,880
Speaker 5: a very friendly emoji. So that's that's nice. So tell

58
00:03:36,960 --> 00:03:39,240
Speaker 5: us a little bit about hugging Face and about what

59
00:03:39,320 --> 00:03:39,880
Speaker 5: you do there.

60
00:03:40,320 --> 00:03:44,800
Speaker 4: Of course, hugging Face is the leading open platform for

61
00:03:45,160 --> 00:03:49,720
Speaker 4: AI builders and it's the place that all of the

62
00:03:50,000 --> 00:03:55,440
Speaker 4: AI researchers use to share their work, their new AI

63
00:03:55,520 --> 00:04:00,560
Speaker 4: models and collaborate around them. It's the place where the

64
00:04:00,720 --> 00:04:05,880
Speaker 4: data scientists go and find those pre train models and

65
00:04:06,040 --> 00:04:09,680
Speaker 4: access them and use them and work with them. And

66
00:04:09,760 --> 00:04:14,440
Speaker 4: increasingly it's the place where developers are coming to turn

67
00:04:14,560 --> 00:04:19,840
Speaker 4: all of these AI models and datasets into their own applications,

68
00:04:19,880 --> 00:04:21,039
Speaker 4: their own features.

69
00:04:21,720 --> 00:04:24,440
Speaker 5: So it's like the I don't know, the Facebook group

70
00:04:24,520 --> 00:04:27,360
Speaker 5: or the Reddit or the Twitter for people who are

71
00:04:27,400 --> 00:04:31,680
Speaker 5: interested in particularly generative language AI or all kinds of

72
00:04:31,760 --> 00:04:33,280
Speaker 5: artificial intelligence.

73
00:04:33,520 --> 00:04:37,520
Speaker 4: All kinds of AI really, and of course generative AI

74
00:04:37,720 --> 00:04:42,679
Speaker 4: is this new wave that has caught the world by storm.

75
00:04:43,040 --> 00:04:45,679
Speaker 4: But on a hiking face you can find any kind

76
00:04:45,800 --> 00:04:50,120
Speaker 4: of model. The new sort of transformers models to do

77
00:04:50,200 --> 00:04:54,920
Speaker 4: anything from translation or if you wanted to transcribe what

78
00:04:54,960 --> 00:04:58,760
Speaker 4: I'm saying into text, then you would use a transformer model.

79
00:04:58,800 --> 00:05:01,600
Speaker 4: If you wanted to then take that text and make

80
00:05:01,640 --> 00:05:05,560
Speaker 4: a summary, that would be another transformer model. If you

81
00:05:05,640 --> 00:05:10,080
Speaker 4: wanted to create a nice little thumbnail for this podcast

82
00:05:10,120 --> 00:05:14,000
Speaker 4: by typing a sentence, that would be another type of model.

83
00:05:14,560 --> 00:05:17,400
Speaker 4: So all these models you can find, there's actually three

84
00:05:17,480 --> 00:05:22,120
Speaker 4: hundred thousands that are free and publicly accessible. You can

85
00:05:22,200 --> 00:05:25,920
Speaker 4: find them on our website at Hikingphase dot co and

86
00:05:26,040 --> 00:05:28,880
Speaker 4: use them using our open source libraries.

87
00:05:29,720 --> 00:05:32,680
Speaker 5: And so this this is fascinating. So there are three

88
00:05:32,800 --> 00:05:36,200
Speaker 5: hundred thousand models. Now when you say model, I'm thinking

89
00:05:36,200 --> 00:05:39,560
Speaker 5: in my head, Oh, it's kind of like a computer program.

90
00:05:39,600 --> 00:05:42,640
Speaker 5: There were three hundred thousand computer programs. Is that roughly

91
00:05:42,720 --> 00:05:43,719
Speaker 5: right or it not?

92
00:05:43,839 --> 00:05:49,920
Speaker 4: Really, it's a general idea. A model is a giant

93
00:05:51,080 --> 00:05:56,480
Speaker 4: set of numbers that are working together to sift through

94
00:05:57,120 --> 00:06:00,320
Speaker 4: some inputs that you're going to give it. So think

95
00:06:00,400 --> 00:06:04,880
Speaker 4: of it of a big black box filled with numbers,

96
00:06:05,800 --> 00:06:10,960
Speaker 4: and you give it as an input, maybe some text,

97
00:06:11,320 --> 00:06:15,240
Speaker 4: maybe a prompt, so you're asking you're giving an instruction

98
00:06:15,480 --> 00:06:18,120
Speaker 4: to the model, or maybe you give it an image

99
00:06:18,160 --> 00:06:22,640
Speaker 4: as an input, and then it will sift through that

100
00:06:22,760 --> 00:06:26,800
Speaker 4: information thanks to all of these numbers, which we call

101
00:06:26,839 --> 00:06:31,320
Speaker 4: in the field parameters, and it will produce an output.

102
00:06:31,839 --> 00:06:34,400
Speaker 4: So when I told you, hey, we can transcribe this

103
00:06:34,640 --> 00:06:38,560
Speaker 4: conversation into text, the input would have been the conversation

104
00:06:39,160 --> 00:06:41,839
Speaker 4: in an audio file, and then the output would have

105
00:06:41,880 --> 00:06:44,880
Speaker 4: been the text of the transcription. If you want to

106
00:06:44,920 --> 00:06:48,960
Speaker 4: create a thumbnail for this podcast episode, then the input

107
00:06:49,000 --> 00:06:51,880
Speaker 4: would be what we call the prompt, which is really

108
00:06:51,880 --> 00:06:56,400
Speaker 4: a text description like a Frenchman in San Francisco talking

109
00:06:56,440 --> 00:07:02,080
Speaker 4: about machine learning, and the output would be completely original image.

110
00:07:02,640 --> 00:07:05,960
Speaker 4: So that's how I think about what an AI model is,

111
00:07:06,440 --> 00:07:10,600
Speaker 4: and I think what we're starting to realize is that

112
00:07:11,480 --> 00:07:15,640
Speaker 4: this is becoming the new way of building technology in

113
00:07:15,720 --> 00:07:19,679
Speaker 4: the world. It has been for the field of dealing,

114
00:07:19,840 --> 00:07:23,760
Speaker 4: understanding generating text for quite some time, but now it's

115
00:07:23,880 --> 00:07:27,800
Speaker 4: sort of moving across every field of technology. We have

116
00:07:28,200 --> 00:07:32,360
Speaker 4: models to create images, as I say, but also to

117
00:07:32,480 --> 00:07:37,960
Speaker 4: generate new proteins to make predictions on numerical data. So

118
00:07:38,040 --> 00:07:42,520
Speaker 4: every kind of field of machine learning is now using

119
00:07:43,960 --> 00:07:47,840
Speaker 4: this new type of models. But what's interesting is that

120
00:07:48,440 --> 00:07:51,960
Speaker 4: if you're say a product manager at a tech company

121
00:07:52,000 --> 00:07:54,440
Speaker 4: and you say, hey, I want to build a feature

122
00:07:55,080 --> 00:07:58,560
Speaker 4: that does this. A few years ago, the approach would

123
00:07:58,560 --> 00:08:02,680
Speaker 4: have been to ask software developer to write a thousand

124
00:08:02,760 --> 00:08:06,200
Speaker 4: lines of code in order to build a prototype. And

125
00:08:06,280 --> 00:08:09,720
Speaker 4: the new way of doing things today is to go

126
00:08:09,880 --> 00:08:14,400
Speaker 4: look for an off the shelf pre train model that

127
00:08:14,480 --> 00:08:18,560
Speaker 4: does a pretty good job at solving exactly that problem,

128
00:08:18,680 --> 00:08:21,800
Speaker 4: so you can create a prototype of that feature fast.

129
00:08:21,800 --> 00:08:24,360
Speaker 4: So it's a new approach of building tech.

130
00:08:24,560 --> 00:08:27,640
Speaker 5: I'm not a programmer. But I'm aware that there was

131
00:08:27,880 --> 00:08:30,440
Speaker 5: this idea of open source code and now we have

132
00:08:30,520 --> 00:08:33,480
Speaker 5: open source models. So what does it mean for something

133
00:08:33,520 --> 00:08:34,400
Speaker 5: to be open source?

134
00:08:35,040 --> 00:08:40,760
Speaker 4: Open source AI actually means a lot of different specific things.

135
00:08:41,480 --> 00:08:45,640
Speaker 4: It's the open source implementation of the model. So if

136
00:08:45,679 --> 00:08:49,959
Speaker 4: you use the Hugging Phase transformers library to use a model,

137
00:08:50,000 --> 00:08:54,560
Speaker 4: you're using an open source code library to use that model, just.

138
00:08:55,040 --> 00:08:57,680
Speaker 5: To end up on that. The transformers, these are these

139
00:08:57,760 --> 00:09:01,000
Speaker 5: kind of ways of turning a picture of a dog

140
00:09:01,160 --> 00:09:03,800
Speaker 5: into a text output that says, hey, this is a

141
00:09:03,840 --> 00:09:06,400
Speaker 5: picture of a dog, or this is a French text

142
00:09:06,440 --> 00:09:09,280
Speaker 5: and with the transformers helping you turn it into English text,

143
00:09:09,360 --> 00:09:11,280
Speaker 5: or it's doing all of these things that you've been describing.

144
00:09:11,320 --> 00:09:15,280
Speaker 5: That's the transformer is the kind of the engine at

145
00:09:15,320 --> 00:09:15,880
Speaker 5: the heart of.

146
00:09:15,840 --> 00:09:20,719
Speaker 4: That, yes, exactly. And we call them transformers because they

147
00:09:20,720 --> 00:09:24,800
Speaker 4: correspond to this new way of building machine learning models

148
00:09:25,200 --> 00:09:29,720
Speaker 4: that was introduced by Google actually with a very important

149
00:09:29,800 --> 00:09:32,800
Speaker 4: paper called Attention Is All You Need and that was

150
00:09:32,840 --> 00:09:37,839
Speaker 4: published in twenty seventeen by researchers out of Google Deep Mind.

151
00:09:38,760 --> 00:09:42,560
Speaker 5: Well that's just six years so new.

152
00:09:43,320 --> 00:09:46,600
Speaker 4: It is very new, and ever since the pace of

153
00:09:46,840 --> 00:09:52,280
Speaker 4: innovation of like new model architectures has really really accelerated,

154
00:09:52,600 --> 00:09:57,360
Speaker 4: but it really started from this inflection point that came

155
00:09:57,480 --> 00:10:01,760
Speaker 4: from this paper, and it's implement in what is now

156
00:10:01,800 --> 00:10:07,600
Speaker 4: called transformer models, the transformer that has conquered every area

157
00:10:07,720 --> 00:10:09,440
Speaker 4: of machine learning since.

158
00:10:09,640 --> 00:10:12,680
Speaker 5: Okay, so say churned you up. So you've got this

159
00:10:12,800 --> 00:10:17,160
Speaker 5: library of Transformer models and that open source and that

160
00:10:17,200 --> 00:10:19,679
Speaker 5: means that means what anyone can use them for free,

161
00:10:20,480 --> 00:10:22,360
Speaker 5: or that anybody can implement them for free. What does

162
00:10:22,360 --> 00:10:24,040
Speaker 5: it mean?

163
00:10:24,200 --> 00:10:27,160
Speaker 4: So again, there's lots that go into it, but the

164
00:10:27,200 --> 00:10:31,600
Speaker 4: most important thing is for the model itself to be

165
00:10:31,840 --> 00:10:35,960
Speaker 4: available so that a data scientists or an engineer can

166
00:10:36,360 --> 00:10:40,719
Speaker 4: download them and use them. And also there are a

167
00:10:40,760 --> 00:10:45,600
Speaker 4: lot of considerations about how you make them accessible, and

168
00:10:45,679 --> 00:10:49,600
Speaker 4: a very important one is whether or not you give

169
00:10:49,840 --> 00:10:54,880
Speaker 4: access to the training data, all the information that went

170
00:10:55,080 --> 00:10:59,800
Speaker 4: into training that model and teaching it to do what

171
00:11:00,080 --> 00:11:01,000
Speaker 4: it's trained to do.

172
00:11:01,200 --> 00:11:04,160
Speaker 5: So I might have fed millions of words into a

173
00:11:04,280 --> 00:11:07,360
Speaker 5: into a language transformer, or I might have fed millions

174
00:11:07,400 --> 00:11:10,000
Speaker 5: of photographs into a into a picture transformer.

175
00:11:10,080 --> 00:11:13,559
Speaker 4: Yeah, yes, and now it's trillions and that and the

176
00:11:13,880 --> 00:11:17,600
Speaker 4: accessibility of that training data is very very important.

177
00:11:18,520 --> 00:11:24,319
Speaker 5: What's the relationship between the hugging Face libraries and GitHub, which,

178
00:11:25,440 --> 00:11:29,720
Speaker 5: if I understand GitHub correctly, it's this the repository of

179
00:11:29,760 --> 00:11:33,720
Speaker 5: open source code, lots and lots of lines of code

180
00:11:33,720 --> 00:11:37,440
Speaker 5: and routines and programs that are shared and updated and

181
00:11:38,360 --> 00:11:41,760
Speaker 5: tracked and they're all available on GitHub, which sounds similar

182
00:11:41,800 --> 00:11:44,000
Speaker 5: to what you're doing with hugging Face for AI. So

183
00:11:44,040 --> 00:11:46,959
Speaker 5: what what what is the interaction or the relationship there?

184
00:11:47,559 --> 00:11:50,000
Speaker 4: Yeah, I think you nailed it on the head there.

185
00:11:50,040 --> 00:11:54,199
Speaker 4: So hugging phase is to AI where GitHub is to code. Right,

186
00:11:54,200 --> 00:12:00,920
Speaker 4: there's this central platform where AI builders can go and

187
00:12:01,000 --> 00:12:06,160
Speaker 4: collaborate around AI artifacts, which are models and datasets. So

188
00:12:06,200 --> 00:12:10,640
Speaker 4: it's quite different than software, but we play this central

189
00:12:10,920 --> 00:12:15,920
Speaker 4: role in the community to share and collaborate and access

190
00:12:16,040 --> 00:12:20,240
Speaker 4: all of those artifacts for AI like it had offers

191
00:12:20,240 --> 00:12:20,800
Speaker 4: for code.

192
00:12:22,040 --> 00:12:24,960
Speaker 5: And that community must be incredibly important. I mean, the

193
00:12:25,000 --> 00:12:27,600
Speaker 5: open sources is nothing if you don't have a community

194
00:12:27,640 --> 00:12:30,000
Speaker 5: of people working on it. So how have you been

195
00:12:30,040 --> 00:12:33,160
Speaker 5: able to foster and nurture that community?

196
00:12:33,760 --> 00:12:37,120
Speaker 4: Well, I think it goes to the origins of the

197
00:12:37,200 --> 00:12:41,320
Speaker 4: transformer model and hugging and face role into that. So

198
00:12:41,960 --> 00:12:46,560
Speaker 4: when the first sort of open model came out. It

199
00:12:46,640 --> 00:12:49,800
Speaker 4: was called Bird and it came out of Google. The

200
00:12:49,840 --> 00:12:54,400
Speaker 4: only way you could access it was to use a

201
00:12:54,440 --> 00:12:58,880
Speaker 4: tool called TensorFlow. But it happened that most of the

202
00:12:59,320 --> 00:13:05,400
Speaker 4: AI community was using a different tool called PyTorch. And

203
00:13:05,480 --> 00:13:10,560
Speaker 4: something that Hugging Face did is to make that new

204
00:13:10,640 --> 00:13:17,040
Speaker 4: model Bert accessible to all PyTorch user and they did

205
00:13:17,040 --> 00:13:21,040
Speaker 4: it in open source. It was a project called Bert's

206
00:13:21,240 --> 00:13:24,079
Speaker 4: pre Trained PyTorch or Burt PyTorch pre Trained.

207
00:13:24,600 --> 00:13:26,720
Speaker 5: So this is like being able to play my Zelda

208
00:13:26,760 --> 00:13:30,800
Speaker 5: game on an Xbox or a PlayStation, right or am

209
00:13:30,840 --> 00:13:32,480
Speaker 5: I not really understanding what's going on?

210
00:13:32,920 --> 00:13:35,280
Speaker 4: No, That's exactly what it is. And the thing is

211
00:13:35,480 --> 00:13:39,440
Speaker 4: everybody was using the game Boy and so it became

212
00:13:39,800 --> 00:13:44,560
Speaker 4: a very popular and from there the community sort of

213
00:13:44,640 --> 00:13:48,000
Speaker 4: gathered to make all of the other models that were

214
00:13:48,040 --> 00:13:52,520
Speaker 4: then published by AI researchers available through that library, which

215
00:13:52,679 --> 00:13:57,640
Speaker 4: was quickly renamed from bert pre Trained PyTorch into transformers

216
00:13:57,679 --> 00:14:02,160
Speaker 4: to welcome like all of these different new models, and

217
00:14:02,520 --> 00:14:07,960
Speaker 4: today that's open source library. Transformers is what all AI

218
00:14:08,000 --> 00:14:11,840
Speaker 4: builders are using when they want to access those models,

219
00:14:11,880 --> 00:14:14,239
Speaker 4: see how they work, and build upon them.

220
00:14:15,120 --> 00:14:18,280
Speaker 5: What's striking about this field is that it's changing so fast,

221
00:14:18,280 --> 00:14:22,080
Speaker 5: it's improving so quickly, So how do open source models

222
00:14:22,800 --> 00:14:26,680
Speaker 5: keep up with that? How do they get iterated and improved?

223
00:14:26,800 --> 00:14:29,760
Speaker 4: Actually, it's not so much that open source is keeping

224
00:14:29,840 --> 00:14:32,800
Speaker 4: up with it. It's actually open source that is driving,

225
00:14:33,520 --> 00:14:37,000
Speaker 4: that is driving this piece of change. And that's because

226
00:14:37,680 --> 00:14:43,040
Speaker 4: with open source and open research data, scientists researchers can

227
00:14:43,160 --> 00:14:46,840
Speaker 4: build upon each other's work, they can reproduce each other's work,

228
00:14:47,160 --> 00:14:51,280
Speaker 4: they can access each other's work using our open source libraries,

229
00:14:51,280 --> 00:14:53,600
Speaker 4: et cetera. So, in a sense, it's not really that

230
00:14:54,080 --> 00:14:58,680
Speaker 4: open source AI is a new idea. It's rather the opposite.

231
00:14:58,840 --> 00:15:02,960
Speaker 4: There's been a blip of time in which closed source

232
00:15:03,200 --> 00:15:06,920
Speaker 4: AI seemed to be the dominant way, but it's really

233
00:15:07,480 --> 00:15:11,200
Speaker 4: a blip. In fact, you know, none of the incredible

234
00:15:11,240 --> 00:15:15,840
Speaker 4: advances that we're marvel about today would be possible without

235
00:15:16,040 --> 00:15:19,040
Speaker 4: open source. We're standing upon the shoulders of fifty years

236
00:15:19,040 --> 00:15:23,360
Speaker 4: of research and open source software. So I think that

237
00:15:23,480 --> 00:15:26,360
Speaker 4: that's really important. If it wasn't for that, we'll probably

238
00:15:26,360 --> 00:15:31,240
Speaker 4: be fifty years away from having these amazing experiences like

239
00:15:31,400 --> 00:15:37,200
Speaker 4: CHTGBT or stable diffusion, et cetera. So it's really open

240
00:15:37,280 --> 00:15:41,600
Speaker 4: source that is fueling this pace of change, all these

241
00:15:41,640 --> 00:15:45,160
Speaker 4: new models, all these new capabilities. To give you an example,

242
00:15:45,480 --> 00:15:49,960
Speaker 4: so Meta released the Lama large language model just a

243
00:15:50,040 --> 00:15:54,360
Speaker 4: few months ago, and ever since there's been this Cambrian

244
00:15:54,480 --> 00:15:58,920
Speaker 4: explosion of variations and improvements upon the original models, and

245
00:15:58,960 --> 00:16:02,080
Speaker 4: today there are there are thousands of them that we

246
00:16:02,520 --> 00:16:07,960
Speaker 4: host and track and evaluate. So yeah, open source is

247
00:16:08,000 --> 00:16:11,640
Speaker 4: really the gas and the engine for that.

248
00:16:12,920 --> 00:16:15,760
Speaker 3: Jeff just made it clear that it is open source,

249
00:16:16,000 --> 00:16:20,040
Speaker 3: not closed, that sets the pace for AI innovation. If

250
00:16:20,040 --> 00:16:24,600
Speaker 3: that's true, then forward thinking businesses shouldn't shy from leveraging

251
00:16:24,680 --> 00:16:29,040
Speaker 3: open source AI to solve their own proprietary challenges. But

252
00:16:29,240 --> 00:16:34,160
Speaker 3: how businesses can face serious obstacles when trying to adopt

253
00:16:34,400 --> 00:16:38,960
Speaker 3: open source technologies, like complying with government regulation or making

254
00:16:39,000 --> 00:16:43,240
Speaker 3: sure their customers data stays protected. In the next part

255
00:16:43,280 --> 00:16:47,560
Speaker 3: of their conversation, Jeff and Tim discuss how IBM's collaboration

256
00:16:47,760 --> 00:16:51,880
Speaker 3: with hugging Face empowers businesses to tap into the open

257
00:16:51,920 --> 00:16:56,240
Speaker 3: source AI community and how the watsonex platform can enable

258
00:16:56,280 --> 00:17:00,160
Speaker 3: them to customize those AI models to their needs.

259
00:17:00,760 --> 00:17:03,280
Speaker 5: Just wants to ask about the partnership between hugging Face

260
00:17:03,360 --> 00:17:05,919
Speaker 5: and an IBM, how did that come.

261
00:17:05,800 --> 00:17:13,560
Speaker 4: About, Well, it came through a conversation, a conversation between

262
00:17:13,920 --> 00:17:20,359
Speaker 4: our CEO, Clement Delangue and Bill Higgins IBM, who's really

263
00:17:20,400 --> 00:17:25,280
Speaker 4: really close to all the amazing research work and open

264
00:17:25,320 --> 00:17:30,680
Speaker 4: source work that's happening at IBM, and that conversation sort

265
00:17:30,680 --> 00:17:35,600
Speaker 4: of sparked the evidence that we needed to do something together.

266
00:17:36,200 --> 00:17:40,200
Speaker 4: We share a lot of values in terms of the

267
00:17:40,240 --> 00:17:45,000
Speaker 4: importance of open source, which is fundamental to us, with

268
00:17:45,359 --> 00:17:50,200
Speaker 4: the importance of doing things in an ethics first way

269
00:17:50,280 --> 00:17:55,399
Speaker 4: to enable the community to incorporate ethical considerations in how

270
00:17:55,880 --> 00:18:01,120
Speaker 4: they're building AI. And we sort of have a different

271
00:18:01,440 --> 00:18:05,480
Speaker 4: audience to start with, which is all the AI builders

272
00:18:05,600 --> 00:18:10,240
Speaker 4: use hiking phase today to access all the models we

273
00:18:10,320 --> 00:18:14,239
Speaker 4: talked about, to use them using our open source and

274
00:18:14,320 --> 00:18:18,679
Speaker 4: build with them. And IBM has this incredible history of

275
00:18:18,800 --> 00:18:24,320
Speaker 4: working with enterprise companies and enabling them to make use

276
00:18:24,359 --> 00:18:28,360
Speaker 4: of that technology in a way that's compliant with everything

277
00:18:28,440 --> 00:18:32,159
Speaker 4: that an enterprise requires, and so being able to marry

278
00:18:32,200 --> 00:18:36,360
Speaker 4: these two things together is an amazing opportunity. And now

279
00:18:36,400 --> 00:18:40,679
Speaker 4: we can enable the largest corporations that have sort of

280
00:18:40,880 --> 00:18:46,280
Speaker 4: complex requirements in order to deploy machine learning systems and

281
00:18:47,080 --> 00:18:50,480
Speaker 4: give them an easy experience to take advantage of all

282
00:18:50,520 --> 00:18:53,639
Speaker 4: the latest and greatest that AI has to offer through

283
00:18:53,680 --> 00:18:55,760
Speaker 4: our platform.

284
00:18:55,840 --> 00:18:59,439
Speaker 5: Let's talk about this idea of a single model or

285
00:18:59,440 --> 00:19:02,960
Speaker 5: a variety of models, because what I've been hearing you say.

286
00:19:03,520 --> 00:19:05,360
Speaker 5: You've been saying, oh, there are lots of models. There

287
00:19:05,400 --> 00:19:09,480
Speaker 5: are hundreds of thousands of models available on hugging Face.

288
00:19:09,640 --> 00:19:13,000
Speaker 5: But you've also said there's a single thing, the transformer,

289
00:19:13,640 --> 00:19:18,080
Speaker 5: and they're all transformers. So if they're all basically the

290
00:19:18,160 --> 00:19:22,879
Speaker 5: same thing, why can't you just build one super clever

291
00:19:22,920 --> 00:19:24,000
Speaker 5: model that can do everything.

292
00:19:26,119 --> 00:19:31,080
Speaker 4: That's a really interesting idea and very much a new idea.

293
00:19:31,880 --> 00:19:35,800
Speaker 4: The reason we have over a million repositories three hundred

294
00:19:35,840 --> 00:19:40,000
Speaker 4: thousand free inaccessible models on a hiking Face platform is

295
00:19:40,040 --> 00:19:44,159
Speaker 4: that models are typically trained to do one thing, and

296
00:19:44,200 --> 00:19:47,719
Speaker 4: they're typically trained to do one thing with specific types

297
00:19:47,800 --> 00:19:53,960
Speaker 4: of data. And what became new and evident in the

298
00:19:54,040 --> 00:19:56,720
Speaker 4: research that came out over the last couple of years

299
00:19:57,320 --> 00:20:01,080
Speaker 4: is that if you train a big enough model with

300
00:20:01,680 --> 00:20:06,119
Speaker 4: enough data, then those models start to have sort of

301
00:20:06,320 --> 00:20:10,080
Speaker 4: general capabilities. You can ask them to do different things,

302
00:20:10,359 --> 00:20:13,840
Speaker 4: you can even train them to respond to instructions. So

303
00:20:13,960 --> 00:20:18,280
Speaker 4: with the same model you can say, hey, summarize this paragraph,

304
00:20:18,600 --> 00:20:22,119
Speaker 4: translate this into English, start a conversation in French, and

305
00:20:22,359 --> 00:20:25,919
Speaker 4: pivot to German. And so these are general sort of

306
00:20:26,080 --> 00:20:32,600
Speaker 4: language capabilities. And I think when ch GBT came online

307
00:20:33,200 --> 00:20:38,359
Speaker 4: and the world sort of discovered these new capabilities, there was,

308
00:20:38,920 --> 00:20:41,840
Speaker 4: at least for a short period, this sort of idea,

309
00:20:42,000 --> 00:20:45,840
Speaker 4: this sort of myth that the endgame of all this

310
00:20:46,840 --> 00:20:50,600
Speaker 4: is maybe one or a handful of models that are

311
00:20:50,800 --> 00:20:55,000
Speaker 4: so much better than anything else than exists, that they

312
00:20:55,000 --> 00:20:57,640
Speaker 4: can do anything that we can ask them to do,

313
00:20:58,440 --> 00:21:01,919
Speaker 4: and that's the only model that we will need. And I,

314
00:21:02,160 --> 00:21:06,440
Speaker 4: for one, think it is a myth. I don't think

315
00:21:06,480 --> 00:21:10,560
Speaker 4: it is practical for a variety of reasons. Say you're

316
00:21:11,000 --> 00:21:15,159
Speaker 4: writing an email and you have like this great suggestion

317
00:21:15,280 --> 00:21:19,560
Speaker 4: of text to sort of complete your sentence. Well, that's AI.

318
00:21:19,720 --> 00:21:22,520
Speaker 4: That's a large language model, that's a transformer model that

319
00:21:22,560 --> 00:21:25,240
Speaker 4: does that. So there are a ton of existing use

320
00:21:25,240 --> 00:21:28,880
Speaker 4: cases like this, and these use cases are powered by

321
00:21:29,680 --> 00:21:32,640
Speaker 4: specific models that have been trained to do one thing

322
00:21:32,760 --> 00:21:35,840
Speaker 4: well and to do it fast. If you wanted to

323
00:21:36,000 --> 00:21:42,440
Speaker 4: apply these sort of all knowing, powerful oracle type of model,

324
00:21:42,960 --> 00:21:47,000
Speaker 4: you would not be able to serve millions of customers

325
00:21:47,040 --> 00:21:49,720
Speaker 4: through a search engine. You will not be able to

326
00:21:51,359 --> 00:21:55,480
Speaker 4: complete people's sentences because the amount of money that you

327
00:21:55,520 --> 00:21:58,800
Speaker 4: would need, the number of computers that you would need

328
00:21:59,000 --> 00:22:04,000
Speaker 4: to run such of of service just exceeds what is

329
00:22:04,040 --> 00:22:09,960
Speaker 4: available on the planet. So one reason for which it's

330
00:22:10,040 --> 00:22:14,960
Speaker 4: not a practical scenario is that it's just very expensive

331
00:22:15,640 --> 00:22:18,800
Speaker 4: to run those very very large models.

332
00:22:19,160 --> 00:22:21,280
Speaker 5: What I'm hearing is it's like, look, if you want

333
00:22:21,320 --> 00:22:25,040
Speaker 5: to screw in a screw you need a screwdriver. You

334
00:22:25,080 --> 00:22:29,080
Speaker 5: don't want an entire tool shed full of tools if

335
00:22:29,160 --> 00:22:31,359
Speaker 5: the task is to screw in a screwdriver, and sure

336
00:22:31,400 --> 00:22:34,600
Speaker 5: you could bring the tool shed that are all the tools.

337
00:22:34,640 --> 00:22:38,680
Speaker 5: There's a screwdriver there, but it's not necessary. It's incredibly expensive,

338
00:22:38,680 --> 00:22:43,480
Speaker 5: it's incredibly cumbersome, and that cost exists even though maybe

339
00:22:43,560 --> 00:22:46,280
Speaker 5: is the user who's just typing in a into a

340
00:22:46,280 --> 00:22:49,040
Speaker 5: prompt box. The user may not see it, but it's

341
00:22:49,080 --> 00:22:50,040
Speaker 5: still very real.

342
00:22:51,320 --> 00:22:54,840
Speaker 4: That's right. And then another one is performance. So taking

343
00:22:54,920 --> 00:22:58,120
Speaker 4: the screwdriver example, so and by the way, like we're

344
00:22:58,160 --> 00:23:00,919
Speaker 4: not quite there at this moment where we have this

345
00:23:01,080 --> 00:23:04,600
Speaker 4: all knowing, powerful oracle that is still sort of a

346
00:23:04,720 --> 00:23:08,720
Speaker 4: sci fi scenario. But we have screwdrivers, but we also

347
00:23:08,840 --> 00:23:13,439
Speaker 4: have the leatherman, right, the multitole Swiss army knife, And

348
00:23:13,480 --> 00:23:16,280
Speaker 4: that's sort of the moment that we are in today.

349
00:23:16,359 --> 00:23:19,960
Speaker 4: But now, if I'm trying to open up my computer,

350
00:23:20,560 --> 00:23:23,840
Speaker 4: turns out that it requires a specific kind of screw

351
00:23:24,000 --> 00:23:28,159
Speaker 4: like these tiny little tork screws, and having a torque

352
00:23:28,160 --> 00:23:31,560
Speaker 4: screw driver will get me much further than trying to

353
00:23:31,680 --> 00:23:34,760
Speaker 4: use my leatherman, where maybe I'll get the knife blade

354
00:23:34,800 --> 00:23:37,880
Speaker 4: and it will mess up the screw and maybe eventually

355
00:23:37,880 --> 00:23:40,520
Speaker 4: I'll get to what I need. But my point is

356
00:23:40,640 --> 00:23:45,760
Speaker 4: that if you take a very specifically trained model for

357
00:23:45,840 --> 00:23:49,320
Speaker 4: a particular problem, it will work much better. It will

358
00:23:49,320 --> 00:23:54,120
Speaker 4: give you better results than a very very generalistic, big

359
00:23:54,200 --> 00:23:57,159
Speaker 4: model that can do a lot of things. And so

360
00:23:57,240 --> 00:24:01,080
Speaker 4: for things like search engines or things things like translation,

361
00:24:01,440 --> 00:24:06,080
Speaker 4: for things that are very specific, companies are much better

362
00:24:06,160 --> 00:24:11,040
Speaker 4: off using smaller, more efficient models that produce better results.

363
00:24:10,840 --> 00:24:15,359
Speaker 5: That's really interesting. And presumably then being able to know

364
00:24:15,400 --> 00:24:18,160
Speaker 5: which model to use, or being able to know who

365
00:24:18,200 --> 00:24:22,000
Speaker 5: to ask which model to use, becomes a very important capability.

366
00:24:22,840 --> 00:24:26,400
Speaker 4: Yes, and that's what we're trying to make easy through

367
00:24:26,400 --> 00:24:27,159
Speaker 4: our platform.

368
00:24:28,520 --> 00:24:32,520
Speaker 5: So tell me about how this works with IBM's what's

369
00:24:32,520 --> 00:24:36,120
Speaker 5: an X platform. How do you see Hugging faces customers

370
00:24:36,160 --> 00:24:37,040
Speaker 5: benefiting from that.

371
00:24:38,920 --> 00:24:43,000
Speaker 4: The end goal is to make it really easy for

372
00:24:43,119 --> 00:24:47,600
Speaker 4: what's and ex customers to make use of all the

373
00:24:47,680 --> 00:24:51,080
Speaker 4: great models and libraries that we talked about, all the

374
00:24:52,080 --> 00:24:55,600
Speaker 4: three hundred thousand models are today on Hugging Face. And

375
00:24:55,800 --> 00:25:00,040
Speaker 4: to do this we need to really collaborate deeply with

376
00:25:00,040 --> 00:25:03,480
Speaker 4: with the IBM teams that build the What's and X

377
00:25:03,520 --> 00:25:08,720
Speaker 4: platform so that our libraries, our open source our models

378
00:25:09,119 --> 00:25:13,760
Speaker 4: are well integrated into the platform. If you're a single user,

379
00:25:13,760 --> 00:25:16,240
Speaker 4: if you are a data science student and you want

380
00:25:16,280 --> 00:25:19,040
Speaker 4: to use a model, is we make it super easy? Right.

381
00:25:19,080 --> 00:25:21,480
Speaker 4: We have our open source library, you can download the

382
00:25:21,520 --> 00:25:24,119
Speaker 4: model on your computer and run with it then. But

383
00:25:24,640 --> 00:25:31,080
Speaker 4: in enterprises there is a vast complexity of infrastructure and

384
00:25:31,280 --> 00:25:35,560
Speaker 4: rules around what people can do and how the data

385
00:25:35,680 --> 00:25:40,159
Speaker 4: can be accessed, and all this complexity is sort of

386
00:25:40,560 --> 00:25:44,239
Speaker 4: solved by the Whatson X platform.

387
00:25:44,920 --> 00:25:48,840
Speaker 5: This season of the Smart Talks podcast features what we're

388
00:25:48,880 --> 00:25:51,760
Speaker 5: calling new creators. Do you see yourself as being a

389
00:25:51,800 --> 00:25:52,800
Speaker 5: creative person.

390
00:25:54,840 --> 00:25:57,320
Speaker 4: I think it's a requirement for the job. I mean,

391
00:25:57,320 --> 00:26:02,080
Speaker 4: we're in such a new and rapidly evolving industry that

392
00:26:02,359 --> 00:26:06,359
Speaker 4: we have to be creative in order to invent the

393
00:26:06,440 --> 00:26:11,000
Speaker 4: business models the use cases of tomorrow. My role within

394
00:26:11,040 --> 00:26:16,040
Speaker 4: the company is really to create the business around all

395
00:26:16,240 --> 00:26:20,199
Speaker 4: the great work of our science and open source and

396
00:26:20,320 --> 00:26:24,439
Speaker 4: product team, and by and large, the business model of

397
00:26:24,600 --> 00:26:29,560
Speaker 4: AI within the whole ecosystem is still something that companies

398
00:26:29,600 --> 00:26:34,600
Speaker 4: are trying to figure out. So creativity is really important

399
00:26:34,720 --> 00:26:38,520
Speaker 4: to really have the conversation with companies, understand what they're

400
00:26:38,520 --> 00:26:40,600
Speaker 4: trying to do, and then build the right kind of solution.

401
00:26:41,200 --> 00:26:45,400
Speaker 4: So that's like where creativity comes into play.

402
00:26:46,160 --> 00:26:50,359
Speaker 5: And one of the things that you've you've been talking

403
00:26:50,400 --> 00:26:53,879
Speaker 5: about is just this growing number of models, this growing

404
00:26:53,960 --> 00:27:00,359
Speaker 5: number of capabilities, this growing number of use cases almostly

405
00:27:00,480 --> 00:27:06,359
Speaker 5: exciting but also I think completely bewildering for most people

406
00:27:07,359 --> 00:27:12,000
Speaker 5: who are trying to navigate their way through this maze

407
00:27:12,040 --> 00:27:15,320
Speaker 5: of possibilities that is growing faster than they can even

408
00:27:15,560 --> 00:27:19,560
Speaker 5: learn about it. So how are you helping people navigate

409
00:27:19,760 --> 00:27:22,240
Speaker 5: and make choices in that environment and how does the

410
00:27:22,280 --> 00:27:24,280
Speaker 5: partnership with IBM help with that.

411
00:27:27,000 --> 00:27:30,880
Speaker 4: Well? As I said, our vision is that AI machine

412
00:27:30,960 --> 00:27:36,000
Speaker 4: learning is becoming the default way of creating technology and

413
00:27:36,040 --> 00:27:39,920
Speaker 4: that means like every product, app, service that you're going

414
00:27:39,960 --> 00:27:43,520
Speaker 4: to be using is going to be using AI to

415
00:27:43,640 --> 00:27:48,639
Speaker 4: do whatever it is better faster. And I guess there

416
00:27:48,640 --> 00:27:52,760
Speaker 4: are two competing visions of the world coming from that.

417
00:27:52,840 --> 00:27:59,040
Speaker 4: There is this vision of the oracle, all powerful model

418
00:27:59,119 --> 00:28:03,280
Speaker 4: that can do every thing, and our vision is different.

419
00:28:03,400 --> 00:28:07,600
Speaker 4: Our vision is that every single company will be able

420
00:28:08,640 --> 00:28:13,440
Speaker 4: to create their own models that they own, that they

421
00:28:13,480 --> 00:28:17,880
Speaker 4: can use, that they control. And that's the division that

422
00:28:17,920 --> 00:28:22,080
Speaker 4: we're trying to bring to life through our open source

423
00:28:22,160 --> 00:28:26,640
Speaker 4: tools that make this work easy through our platform where

424
00:28:26,680 --> 00:28:29,160
Speaker 4: you can find all those pre train models are shared

425
00:28:29,200 --> 00:28:32,560
Speaker 4: by the community. So we really want to empower companies

426
00:28:32,640 --> 00:28:35,720
Speaker 4: to build their own stuff, not to outsource all the

427
00:28:35,840 --> 00:28:40,280
Speaker 4: intelligence to a third party. And the What's on next

428
00:28:40,640 --> 00:28:47,160
Speaker 4: platform from IBM gives those tools to enterprise companies. So

429
00:28:47,200 --> 00:28:52,040
Speaker 4: that's you can use the open source models that hiking

430
00:28:52,080 --> 00:28:56,520
Speaker 4: Face offers, then you can improve them with your own

431
00:28:56,600 --> 00:29:01,040
Speaker 4: data without sharing that data to a third party, and

432
00:29:01,080 --> 00:29:04,959
Speaker 4: then you could do all of this work in compliance

433
00:29:05,320 --> 00:29:09,920
Speaker 4: with whatever governance requirements that you have for your company,

434
00:29:10,000 --> 00:29:14,240
Speaker 4: maybe your finance services company and you have a specific

435
00:29:14,280 --> 00:29:18,640
Speaker 4: set of rules, maybe your healthcare company, and you have

436
00:29:18,840 --> 00:29:25,800
Speaker 4: very strong privacy requirements for patients data maybe your tech company,

437
00:29:25,880 --> 00:29:30,640
Speaker 4: and you have your your customers, your users' personal information,

438
00:29:31,320 --> 00:29:33,280
Speaker 4: so you need to be able to do this work

439
00:29:33,440 --> 00:29:34,680
Speaker 4: respecting all of that.

440
00:29:35,720 --> 00:29:37,640
Speaker 5: Jeff Breier, thank you very much.

441
00:29:38,320 --> 00:29:40,080
Speaker 4: Thanks so much, Tam. It's fun.

442
00:29:41,680 --> 00:29:44,640
Speaker 3: To create the AI models of the future. We're going

443
00:29:44,680 --> 00:29:47,160
Speaker 3: to need open source. That means it's a place for

444
00:29:47,320 --> 00:29:50,320
Speaker 3: business in the open source community to harness the game

445
00:29:50,440 --> 00:29:55,960
Speaker 3: changing potential of AI innovation. Like Jeff said, businesses face

446
00:29:56,200 --> 00:29:59,880
Speaker 3: unique challenges they need to solve at scale without pri

447
00:30:00,040 --> 00:30:04,000
Speaker 3: upper support systems. Tapping into open source AI at enterprise

448
00:30:04,080 --> 00:30:07,960
Speaker 3: level is daunting finding the right size model for the job,

449
00:30:08,280 --> 00:30:12,880
Speaker 3: fine tuning its purpose, all while addressing governance acquirements around

450
00:30:12,920 --> 00:30:18,920
Speaker 3: data privacy and ethics. So for businesses, IBM's collaboration with

451
00:30:19,000 --> 00:30:22,800
Speaker 3: hugging Face is a market progress because it signifies that

452
00:30:22,960 --> 00:30:27,400
Speaker 3: business can tap into open source AI while preserving enterprise

453
00:30:27,480 --> 00:30:32,640
Speaker 3: level integrity. Businesses should embrace the open source community and

454
00:30:32,720 --> 00:30:36,480
Speaker 3: the AI future, much like hugging Face and its emoji

455
00:30:36,560 --> 00:30:41,080
Speaker 3: namesake suggests. I'm Malcolm Gladwell. This is a paid advertisement

456
00:30:41,200 --> 00:30:45,920
Speaker 3: from IBM. Smart Talks with IBM is produced by Matt Romano,

457
00:30:46,320 --> 00:30:50,600
Speaker 3: David jaw Nisha Ncat and Royston Deserve with Jacob Goldstein.

458
00:30:50,960 --> 00:30:54,880
Speaker 3: We're edited by Lydia gen Kott. Our engineers are Jason Gambrel,

459
00:30:55,320 --> 00:31:00,760
Speaker 3: Sarah Brugaier and Ben Tolliday. Theme song by Gramoscope. Special

460
00:31:00,800 --> 00:31:04,560
Speaker 3: thanks to Carli Migliore, Andy Kelly, Kathy Callahan and the

461
00:31:04,560 --> 00:31:07,360
Speaker 3: eight Bar and IBM teams, as well as the Pushkin

462
00:31:07,600 --> 00:31:10,880
Speaker 3: marketing team. Smart Talks with IBM is a production of

463
00:31:11,000 --> 00:31:16,080
Speaker 3: Pushkin Industries and Ruby Studio at iHeartMedia. To find more

464
00:31:16,080 --> 00:31:20,840
Speaker 3: Pushkin podcasts, listen on the iHeartRadio app, Apple Podcasts, or

465
00:31:20,880 --> 00:31:34,280
Speaker 3: wherever you listen to podcasts,