1
00:00:07,400 --> 00:00:12,360
Speaker 1: Welcome to Strictly Business, Variety's podcast featuring conversations with industry

2
00:00:12,440 --> 00:00:16,279
Speaker 1: leaders about the business of media and entertainment. I'm Todd

3
00:00:16,320 --> 00:00:20,520
Speaker 1: Spangler with Variety Today. Our guest is Elizabeth Stone, chief

4
00:00:20,560 --> 00:00:24,079
Speaker 1: Technology Officer of Netflix. She first joined the company in

5
00:00:24,120 --> 00:00:27,360
Speaker 1: twenty twenty and became VP of Data and Insights before

6
00:00:27,400 --> 00:00:31,120
Speaker 1: she was named CTO in October twenty twenty three. And

7
00:00:31,200 --> 00:00:34,840
Speaker 1: what's unusual for a tech executive. Stone is a background

8
00:00:34,920 --> 00:00:38,440
Speaker 1: in economics with a PhD from Stanford, and she once

9
00:00:38,479 --> 00:00:55,280
Speaker 1: worked as a trader at Merrill Lynch. Let me just

10
00:00:55,320 --> 00:00:59,200
Speaker 1: start by asking what is the scope of Netflix's technology operations?

11
00:01:00,120 --> 00:01:05,040
Speaker 1: The team I'm imagining this gigantic wall sized whiteboard to

12
00:01:05,120 --> 00:01:06,400
Speaker 1: keep track of everything.

13
00:01:07,760 --> 00:01:12,640
Speaker 2: It's something like that on Sundays. So my team includes

14
00:01:12,840 --> 00:01:17,040
Speaker 2: engineering as well as a data and insights team. So

15
00:01:17,080 --> 00:01:20,959
Speaker 2: that's data science, data engineering, consumer research, a lot of

16
00:01:20,959 --> 00:01:26,720
Speaker 2: different flavors of data oriented roles. And we are thousands

17
00:01:26,760 --> 00:01:31,679
Speaker 2: of people strong at Netflix, So about that three thousand

18
00:01:31,720 --> 00:01:33,000
Speaker 2: people at this point.

19
00:01:33,560 --> 00:01:37,640
Speaker 1: What are the particular challenges of managing this team globally?

20
00:01:37,840 --> 00:01:40,280
Speaker 1: Is it the distributed nature of the team, is it

21
00:01:40,400 --> 00:01:43,120
Speaker 1: the diversity of the projects.

22
00:01:42,800 --> 00:01:47,319
Speaker 2: It's probably a combination. So the team is largely based

23
00:01:47,360 --> 00:01:50,400
Speaker 2: in the United States, that we have a mixture of

24
00:01:50,440 --> 00:01:54,720
Speaker 2: people California remote work. There's of course the size and

25
00:01:54,920 --> 00:01:59,040
Speaker 2: scale of the team, which is a challenge making sure

26
00:01:59,120 --> 00:02:01,800
Speaker 2: that we're all lined on our big bets and heading

27
00:02:01,840 --> 00:02:05,040
Speaker 2: in the same direction and building things that work together.

28
00:02:06,000 --> 00:02:08,400
Speaker 2: But I would say the greatest complexity comes from the

29
00:02:08,520 --> 00:02:12,440
Speaker 2: fact that our team supports all areas of the business.

30
00:02:13,240 --> 00:02:18,040
Speaker 2: So you think about the core consumer member product experience,

31
00:02:18,720 --> 00:02:23,639
Speaker 2: more recently adding support for ads, for games, for live content.

32
00:02:24,400 --> 00:02:26,480
Speaker 2: We also do a lot of work to support the

33
00:02:26,520 --> 00:02:32,320
Speaker 2: content and studio production organizations with technology solutions and insights,

34
00:02:33,120 --> 00:02:37,359
Speaker 2: and then core infrastructure or support platforms, So how we

35
00:02:37,400 --> 00:02:41,440
Speaker 2: think about the data that we're using and the products

36
00:02:41,440 --> 00:02:44,919
Speaker 2: that are available for engineers to build upon. So when

37
00:02:44,960 --> 00:02:47,959
Speaker 2: you look end to end, there's lots of different topics

38
00:02:48,000 --> 00:02:50,760
Speaker 2: and areas of the business that we need to be

39
00:02:50,880 --> 00:02:53,839
Speaker 2: deep in, and it means we've got partnerships across all

40
00:02:53,880 --> 00:02:58,160
Speaker 2: of Netflix. So I find a lot of both complexity

41
00:02:58,200 --> 00:02:59,120
Speaker 2: and fun in that.

42
00:03:00,919 --> 00:03:04,680
Speaker 1: Now, I guess what was the most surprising or unexpected

43
00:03:04,680 --> 00:03:08,480
Speaker 1: thing that you discovered when you joined Netflix in twenty twenty,

44
00:03:09,560 --> 00:03:12,760
Speaker 1: what's different about Netflix and was it difficult to adapt

45
00:03:13,200 --> 00:03:14,760
Speaker 1: to the Netflix culture?

46
00:03:14,760 --> 00:03:17,640
Speaker 2: And we can talk about that. So it definitely struve

47
00:03:17,720 --> 00:03:20,600
Speaker 2: me at the time as a very different culture than

48
00:03:20,680 --> 00:03:24,799
Speaker 2: places I'd been before. So some of the highlights are

49
00:03:25,000 --> 00:03:29,760
Speaker 2: how little structured process there was, especially in twenty twenty

50
00:03:30,000 --> 00:03:35,280
Speaker 2: compared to previous companies, so much lighter in operating practices,

51
00:03:35,680 --> 00:03:39,480
Speaker 2: people practices. There was a lot of deep application of

52
00:03:39,560 --> 00:03:43,040
Speaker 2: judgment throughout the company rather than rules. So of course

53
00:03:43,120 --> 00:03:46,280
Speaker 2: read Hastings at this point well known book known rules,

54
00:03:46,360 --> 00:03:49,800
Speaker 2: Rules was basically what it was like to join Netflix

55
00:03:49,880 --> 00:03:53,440
Speaker 2: at the time. That felt so different than companies i'd

56
00:03:53,480 --> 00:03:56,520
Speaker 2: been at that really had quite a bit of structure

57
00:03:56,760 --> 00:03:59,360
Speaker 2: or approval process or things that you needed to go

58
00:03:59,400 --> 00:04:02,400
Speaker 2: through to help you make a decision. And it was

59
00:04:02,400 --> 00:04:05,040
Speaker 2: a welcome change for me. So I don't know that

60
00:04:05,120 --> 00:04:07,280
Speaker 2: it was a surprise as much as something that I

61
00:04:07,360 --> 00:04:10,160
Speaker 2: was heading to intentionally. So as I was in the

62
00:04:10,200 --> 00:04:14,440
Speaker 2: interview process at Netflix, it sounded like a fascinating and

63
00:04:14,560 --> 00:04:17,720
Speaker 2: great fit for me with what I was looking for.

64
00:04:17,760 --> 00:04:19,359
Speaker 2: So I found a lot of that to be true

65
00:04:19,880 --> 00:04:23,920
Speaker 2: when I got here four years later. Having been here,

66
00:04:24,120 --> 00:04:26,880
Speaker 2: we're a much bigger company than even we were when

67
00:04:26,920 --> 00:04:30,560
Speaker 2: I joined in April twenty twenty, so thousands bigger as

68
00:04:30,560 --> 00:04:33,360
Speaker 2: a company. Many people have joined in the last few years.

69
00:04:34,080 --> 00:04:36,599
Speaker 2: And so with that type of scale, we do need

70
00:04:36,920 --> 00:04:39,880
Speaker 2: flavors of good process, so things that are oh, we're

71
00:04:39,920 --> 00:04:42,160
Speaker 2: going to actually be much more effective in how we

72
00:04:42,200 --> 00:04:45,480
Speaker 2: work because we have some guidance around how we think

73
00:04:45,520 --> 00:04:48,599
Speaker 2: about certain problems or principles or ways we think about

74
00:04:49,080 --> 00:04:52,000
Speaker 2: bringing in and growing talent. And I think that's been

75
00:04:52,040 --> 00:04:55,000
Speaker 2: a good evolution. So the culture in terms of what

76
00:04:55,040 --> 00:04:57,960
Speaker 2: we value and that we try to avoid process that's

77
00:04:58,040 --> 00:05:04,120
Speaker 2: not useful or that remove using good judgment sustains. But

78
00:05:04,160 --> 00:05:06,560
Speaker 2: we've had to change some of the operating practices just

79
00:05:06,600 --> 00:05:08,279
Speaker 2: so we stay effective as a company.

80
00:05:09,279 --> 00:05:11,880
Speaker 1: I mean, I'm sure there've always been rules of thumb

81
00:05:11,960 --> 00:05:16,920
Speaker 1: and sort of best practices as you mentioned, but now

82
00:05:16,920 --> 00:05:18,880
Speaker 1: you're maybe codifying them a little more.

83
00:05:19,400 --> 00:05:22,920
Speaker 2: Yeah. So maybe a good example is how we think

84
00:05:22,960 --> 00:05:27,520
Speaker 2: about planning our biggest priorities for the year, and that,

85
00:05:28,000 --> 00:05:31,320
Speaker 2: of course we've always had planning and it was less formal,

86
00:05:31,440 --> 00:05:34,560
Speaker 2: it was maybe less structured. Teams would locally say like,

87
00:05:34,640 --> 00:05:37,479
Speaker 2: what's most important for us for next year and let's

88
00:05:37,480 --> 00:05:40,880
Speaker 2: focus on those efforts. At this type of scale and

89
00:05:40,880 --> 00:05:44,120
Speaker 2: complexity across Netflix, we need to have a sense across

90
00:05:44,160 --> 00:05:47,240
Speaker 2: the whole company of what's most important. So one of

91
00:05:47,279 --> 00:05:49,800
Speaker 2: the things that's evolved since I've been here is more

92
00:05:49,920 --> 00:05:55,200
Speaker 2: leadership conversation and communication around here's what's top of mind

93
00:05:55,240 --> 00:05:57,960
Speaker 2: for us this year. And that provides some structure for

94
00:05:58,000 --> 00:06:00,560
Speaker 2: how many many teams think about and this is what

95
00:06:00,600 --> 00:06:04,400
Speaker 2: we should do to nest into these company priorities, so

96
00:06:04,440 --> 00:06:07,120
Speaker 2: that even that is not an introduction of a rule

97
00:06:07,240 --> 00:06:09,640
Speaker 2: or you must do the following thing, but it's guidance

98
00:06:09,839 --> 00:06:12,919
Speaker 2: that I think allows people to feel more aligned across

99
00:06:12,960 --> 00:06:13,880
Speaker 2: thousands of people.

100
00:06:14,320 --> 00:06:18,040
Speaker 1: Okay, so we're about the midpoint of the year. What

101
00:06:18,080 --> 00:06:23,080
Speaker 1: are the big priorities for the company this year? And

102
00:06:23,160 --> 00:06:25,040
Speaker 1: how do you measure that? How do you refine that?

103
00:06:25,520 --> 00:06:28,479
Speaker 2: So a lot of it is what's very visible externally,

104
00:06:28,560 --> 00:06:33,240
Speaker 2: which is expanding the world of entertainment that Netflix offers,

105
00:06:33,600 --> 00:06:37,240
Speaker 2: and even expanding aspects of our business model, which is

106
00:06:37,240 --> 00:06:39,800
Speaker 2: where ADS comes into play. That we now an AD

107
00:06:39,880 --> 00:06:43,960
Speaker 2: supported plan offering for members. And when I talk about

108
00:06:43,960 --> 00:06:49,360
Speaker 2: expanding the world of entertainment, that's continued innovation, improvement, on

109
00:06:49,680 --> 00:06:53,920
Speaker 2: what's long been our core competency or core service film TV,

110
00:06:54,400 --> 00:06:58,000
Speaker 2: and then through recommendations and our product really matching members

111
00:06:58,040 --> 00:07:02,320
Speaker 2: with great content, but expanded that to include games, both

112
00:07:02,400 --> 00:07:07,320
Speaker 2: mobile and early efforts in cloud, as well as live content,

113
00:07:07,480 --> 00:07:10,920
Speaker 2: which is very name for Netflix. So when I think

114
00:07:10,960 --> 00:07:14,360
Speaker 2: about some of those new bets across ads, games and

115
00:07:14,400 --> 00:07:16,720
Speaker 2: live and how that builds on top of the foundation

116
00:07:16,880 --> 00:07:19,280
Speaker 2: we've built a lot of, that is the focus for

117
00:07:19,400 --> 00:07:23,360
Speaker 2: company priorities this year, and so that's where a lot

118
00:07:23,360 --> 00:07:26,560
Speaker 2: of teams are focused in being able to adapt current

119
00:07:27,040 --> 00:07:30,320
Speaker 2: ways of thinking and working to this expansion of entertainment

120
00:07:30,760 --> 00:07:33,800
Speaker 2: and even needing to think about how the product needs

121
00:07:33,840 --> 00:07:36,520
Speaker 2: to evolve to reflect how much Netflix is offering at

122
00:07:36,520 --> 00:07:37,160
Speaker 2: this point.

123
00:07:37,400 --> 00:07:39,240
Speaker 1: So you've had to bring on a whole bunch of

124
00:07:39,240 --> 00:07:44,440
Speaker 1: people with different skill sets, domain expertising in games, advertising

125
00:07:44,640 --> 00:07:45,600
Speaker 1: and live right.

126
00:07:45,960 --> 00:07:48,560
Speaker 2: Definitely, Yeah, that's been a big part of our growth.

127
00:07:48,880 --> 00:07:52,160
Speaker 1: So obviously you mentioned live on Netflix just announced you've

128
00:07:52,160 --> 00:07:57,400
Speaker 1: got to Christmas NFL games. I'm sure plans already underway

129
00:07:57,440 --> 00:08:03,240
Speaker 1: to get that as and fully bulletproof is possible. Let

130
00:08:03,320 --> 00:08:04,880
Speaker 1: me just bring up, you know, the Love is Blind

131
00:08:04,920 --> 00:08:08,320
Speaker 1: live reunion from market last year. That's that's a long

132
00:08:08,360 --> 00:08:11,000
Speaker 1: time ago now, But you know what specifically did you

133
00:08:11,080 --> 00:08:14,840
Speaker 1: learn what specifically went wrong? Without without maybe getting to

134
00:08:14,920 --> 00:08:16,200
Speaker 1: in the technical weeds.

135
00:08:16,480 --> 00:08:20,440
Speaker 2: Yeah, it's a distant memory, thankfully, because it's a painful

136
00:08:20,680 --> 00:08:24,280
Speaker 2: one for many of us on the team. I think

137
00:08:24,320 --> 00:08:27,200
Speaker 2: the thing about Love is Blind is that it was

138
00:08:28,240 --> 00:08:32,040
Speaker 2: a good reminder of how complicated it really is to

139
00:08:32,160 --> 00:08:36,320
Speaker 2: deliver live well at scale. And it can seem like

140
00:08:36,400 --> 00:08:39,640
Speaker 2: it should be an easy thing because broadcasters around the

141
00:08:39,679 --> 00:08:43,120
Speaker 2: world do this every day, but to do that in

142
00:08:43,160 --> 00:08:47,239
Speaker 2: the streaming context, so delivering that content over the Internet

143
00:08:47,800 --> 00:08:50,840
Speaker 2: is a different type of challenge, and being able to

144
00:08:50,920 --> 00:08:55,520
Speaker 2: adapt the way Netflix has optimized streaming video on demand

145
00:08:55,640 --> 00:08:59,560
Speaker 2: for streaming live content is complex. So I think Love

146
00:08:59,600 --> 00:09:03,000
Speaker 2: is Blind and really put a microscope on that, and

147
00:09:03,080 --> 00:09:06,120
Speaker 2: I think we've gained a ton of amazing learnings around

148
00:09:06,200 --> 00:09:11,000
Speaker 2: just how to ensure greater reliability, stability for members who

149
00:09:11,040 --> 00:09:14,400
Speaker 2: are watching, and even operational aspects of things. So live

150
00:09:14,440 --> 00:09:18,400
Speaker 2: has a big operations component to how we think about

151
00:09:18,559 --> 00:09:22,600
Speaker 2: managing productions in real time and getting that content to

152
00:09:22,760 --> 00:09:28,320
Speaker 2: a member's phone or TV screen in almost immediate real

153
00:09:28,400 --> 00:09:32,400
Speaker 2: time nature. So that's been something that I think we

154
00:09:32,520 --> 00:09:35,520
Speaker 2: knew in concept, we know in reality now and we've

155
00:09:35,520 --> 00:09:38,679
Speaker 2: had a couple very successful events since then, which I

156
00:09:38,720 --> 00:09:41,400
Speaker 2: think reflects some of those learnings, things like the Tom

157
00:09:41,440 --> 00:09:45,079
Speaker 2: Brady roast. So that gives us some preparation for NFL,

158
00:09:45,200 --> 00:09:47,840
Speaker 2: but it's still a big challenge ahead of us that

159
00:09:47,880 --> 00:09:50,000
Speaker 2: the team has to be really focused on now.

160
00:09:50,120 --> 00:09:54,839
Speaker 1: So you came from the data science side. Obviously you've

161
00:09:54,880 --> 00:09:58,720
Speaker 1: played a big role in compiling and developing and compiling

162
00:09:58,920 --> 00:10:03,599
Speaker 1: the Netflix Engagement Report, which is a huge project. What

163
00:10:03,720 --> 00:10:09,160
Speaker 1: are the biggest challenges in getting that assembled? And you

164
00:10:09,200 --> 00:10:13,320
Speaker 1: know is what are the gatting factors to creating it

165
00:10:13,360 --> 00:10:15,920
Speaker 1: in a you know, this big spreadsheet that you're putting

166
00:10:15,920 --> 00:10:18,319
Speaker 1: out for us a year now. Yeah, in some ways,

167
00:10:18,360 --> 00:10:21,760
Speaker 1: it was really the shift towards realizing the value of

168
00:10:21,800 --> 00:10:24,600
Speaker 1: transparency and engagement that was the biggest part of the

169
00:10:24,640 --> 00:10:29,080
Speaker 1: conversation versus the challenge of putting the data together itself.

170
00:10:29,440 --> 00:10:31,680
Speaker 1: I think one of the things that has been a

171
00:10:31,720 --> 00:10:34,600
Speaker 1: focus for us is ensuring the accuracy of that data.

172
00:10:34,720 --> 00:10:38,960
Speaker 2: Of course, so tons of thousands of titles and thinking

173
00:10:39,040 --> 00:10:42,120
Speaker 2: about hours watched around the globe to make sure that

174
00:10:42,160 --> 00:10:44,800
Speaker 2: we log that information and are able to report it

175
00:10:44,840 --> 00:10:48,840
Speaker 2: accurately is an important bar for us to be able

176
00:10:48,880 --> 00:10:53,520
Speaker 2: to clear. But that's also something that our data scientists,

177
00:10:53,600 --> 00:10:57,400
Speaker 2: analyst data engineers deal with as a challenge daily to

178
00:10:57,640 --> 00:11:00,080
Speaker 2: ensure really high quality data and that we're able to

179
00:11:00,160 --> 00:11:03,360
Speaker 2: leverage it. So really the big changes deciding that we

180
00:11:03,360 --> 00:11:07,160
Speaker 2: would share that outside of the virtual walls of Netflix.

181
00:11:08,720 --> 00:11:10,880
Speaker 1: We're like a little bit over a year, I guess

182
00:11:10,920 --> 00:11:16,240
Speaker 1: in the concerted role out of paid sharing for Netflix.

183
00:11:16,400 --> 00:11:19,360
Speaker 1: What what's been the biggest insight about that? I know

184
00:11:19,880 --> 00:11:23,200
Speaker 1: Greg Peters has talked at various times about you know,

185
00:11:23,280 --> 00:11:26,000
Speaker 1: iterating it, you know, doing you know a b testing

186
00:11:26,000 --> 00:11:28,720
Speaker 1: to see what works the best here? Is this really

187
00:11:28,880 --> 00:11:32,319
Speaker 1: almost more human psychology and you know, in the economics

188
00:11:32,360 --> 00:11:38,120
Speaker 1: world than you know, making sure something is working correctly

189
00:11:38,240 --> 00:11:40,520
Speaker 1: From a technical context.

190
00:11:40,480 --> 00:11:44,120
Speaker 2: It's probably a combination. And that's the beauty of economics

191
00:11:44,200 --> 00:11:47,320
Speaker 2: that it's both you know, what are the right algorithms

192
00:11:47,400 --> 00:11:51,160
Speaker 2: or product features to really make this successful, but also

193
00:11:51,320 --> 00:11:54,640
Speaker 2: understanding that it was a big change for consumers and

194
00:11:54,720 --> 00:11:57,680
Speaker 2: landing that well, to acknowledge that people had a different

195
00:11:57,760 --> 00:12:01,199
Speaker 2: expectation of being able to share netlik books no matter

196
00:12:01,240 --> 00:12:03,760
Speaker 2: what our terms and conditions said, and we needed to

197
00:12:03,920 --> 00:12:07,800
Speaker 2: change expectations and that was challenging to do, and it

198
00:12:07,920 --> 00:12:11,360
Speaker 2: also varied across markets and how to communicate that well.

199
00:12:11,920 --> 00:12:14,320
Speaker 2: So I think it was kind of a double challenge

200
00:12:14,320 --> 00:12:17,880
Speaker 2: of the technical and then you know, the human. We're

201
00:12:17,880 --> 00:12:20,679
Speaker 2: trying to deliver a great experience and certainly want to

202
00:12:20,720 --> 00:12:23,880
Speaker 2: have long term relationships with our members, so we wanted

203
00:12:23,880 --> 00:12:25,679
Speaker 2: to make sure we didn't put that at risk as

204
00:12:25,760 --> 00:12:26,760
Speaker 2: part of the change.

205
00:12:27,280 --> 00:12:31,360
Speaker 1: Did you and so for in this case, in this project,

206
00:12:31,360 --> 00:12:34,440
Speaker 1: did you bring you know, your economics background to bear

207
00:12:35,400 --> 00:12:38,640
Speaker 1: in certain ways, maybe that you know a CTO that

208
00:12:38,679 --> 00:12:41,600
Speaker 1: had come up through and you know, software or hardware

209
00:12:41,640 --> 00:12:42,800
Speaker 1: engineering side might have.

210
00:12:43,400 --> 00:12:45,840
Speaker 2: Yeah, that's a good question. I mean I feel like

211
00:12:45,960 --> 00:12:49,920
Speaker 2: I use my economics background on almost a daily basis,

212
00:12:49,960 --> 00:12:53,160
Speaker 2: and I'm sure it applied in this context of account sharing.

213
00:12:53,760 --> 00:12:57,880
Speaker 2: So examples are things like how we frame the problem,

214
00:12:58,000 --> 00:13:00,840
Speaker 2: like what we're trying to achieve, to find what success

215
00:13:00,960 --> 00:13:05,440
Speaker 2: looks like, figuring out how to optimize what the different

216
00:13:05,480 --> 00:13:08,520
Speaker 2: solutions are. When we're constrained by certain things, so we

217
00:13:08,600 --> 00:13:13,439
Speaker 2: might have technical constraints, we might have consumer expectation constraints.

218
00:13:13,440 --> 00:13:16,160
Speaker 2: You make trade offs every day and how do we

219
00:13:16,240 --> 00:13:18,640
Speaker 2: want to think about should we go with path A

220
00:13:18,880 --> 00:13:21,000
Speaker 2: or path B? What are the pros and cons of

221
00:13:21,040 --> 00:13:24,160
Speaker 2: those things, and having a structured way to make decisions.

222
00:13:24,520 --> 00:13:27,720
Speaker 2: That's where a lot of my economics background comes into play,

223
00:13:28,320 --> 00:13:32,280
Speaker 2: and that account sharing is one example of then what's

224
00:13:32,320 --> 00:13:34,920
Speaker 2: the strategy and the best way to execute on something

225
00:13:35,280 --> 00:13:37,760
Speaker 2: in a way that has a lot of intellectual rigor

226
00:13:37,840 --> 00:13:41,160
Speaker 2: and how we think about delivering it. So economics has

227
00:13:41,200 --> 00:13:45,360
Speaker 2: been useful there, and it's also useful from a technical perspective.

228
00:13:45,480 --> 00:13:49,160
Speaker 2: So while I grew up with more of an economics background,

229
00:13:49,880 --> 00:13:52,480
Speaker 2: it's a deeply technical field that has a lot of

230
00:13:52,559 --> 00:13:54,400
Speaker 2: you know, I think of it as applied math for

231
00:13:54,480 --> 00:13:57,280
Speaker 2: a certain set of problems, and a lot of how

232
00:13:57,320 --> 00:14:01,200
Speaker 2: we think about account sharing is an optimistation problem too,

233
00:14:02,080 --> 00:14:05,160
Speaker 2: and especially a data problem. So I think the skills

234
00:14:05,200 --> 00:14:08,760
Speaker 2: have applied there as well as many other problem spaces

235
00:14:08,760 --> 00:14:09,360
Speaker 2: in Netflix.

236
00:14:10,600 --> 00:14:13,200
Speaker 1: You know, when I talk to technology executive sometimes you

237
00:14:13,280 --> 00:14:15,960
Speaker 1: hear I hear them say something along the lines of,

238
00:14:16,000 --> 00:14:18,640
Speaker 1: you know, our job to make this totally invisible. This

239
00:14:19,280 --> 00:14:21,800
Speaker 1: stuff should just work, it should be intuitive to use,

240
00:14:21,880 --> 00:14:24,480
Speaker 1: and we need to like fate into the background. Right,

241
00:14:24,920 --> 00:14:27,640
Speaker 1: is that is that a philosophy that resonates with you

242
00:14:27,760 --> 00:14:30,360
Speaker 1: a Netflix or you know, I guess another way to

243
00:14:30,360 --> 00:14:32,840
Speaker 1: ask the questions, when do you choose to promote some

244
00:14:32,960 --> 00:14:36,520
Speaker 1: kind of new feature call attention to the value that

245
00:14:36,560 --> 00:14:40,160
Speaker 1: you're providing to you know, is in a consumer facing

246
00:14:40,200 --> 00:14:43,680
Speaker 1: way the research that courts, you know, one way or

247
00:14:43,800 --> 00:14:44,080
Speaker 1: the other.

248
00:14:44,720 --> 00:14:48,280
Speaker 2: Yeah, So a portion of my team, that consumer insights team,

249
00:14:48,400 --> 00:14:52,840
Speaker 2: does a lot of research that helps to inform the

250
00:14:52,920 --> 00:14:57,920
Speaker 2: experience we deliver, and some of it comes from what's

251
00:14:57,960 --> 00:15:01,120
Speaker 2: the value of the features? So when you come to

252
00:15:01,160 --> 00:15:03,680
Speaker 2: the Netflix product, like do I feel like I'm getting

253
00:15:03,720 --> 00:15:06,880
Speaker 2: the things I need in order to make great entertainment choices?

254
00:15:07,360 --> 00:15:09,760
Speaker 2: So that could be things like is it easy for

255
00:15:09,800 --> 00:15:11,640
Speaker 2: me to tell from the trailer and the way we

256
00:15:11,720 --> 00:15:14,880
Speaker 2: describe the title what it's about and whether it's going

257
00:15:14,920 --> 00:15:17,240
Speaker 2: to be a good fit for me. But then there's

258
00:15:17,280 --> 00:15:19,640
Speaker 2: also aspects of the research that are about how we

259
00:15:19,800 --> 00:15:24,080
Speaker 2: organize information on the homepage to make it easy to navigate,

260
00:15:24,160 --> 00:15:26,840
Speaker 2: and this is something that we're testing now externally with

261
00:15:26,880 --> 00:15:30,560
Speaker 2: some members of changing how we structure the page so

262
00:15:30,600 --> 00:15:34,720
Speaker 2: it's more intuitive and it's simpler. It's easier to feature

263
00:15:34,760 --> 00:15:37,360
Speaker 2: a specific title in a dynamic way. With all that

264
00:15:37,440 --> 00:15:39,760
Speaker 2: information in one place, you don't have to do quite

265
00:15:39,760 --> 00:15:43,320
Speaker 2: so much scanning of the page. And so there's probably

266
00:15:43,360 --> 00:15:47,080
Speaker 2: a combination. There's very high value features that we learn

267
00:15:47,160 --> 00:15:51,040
Speaker 2: from research are worth including, even if they add complexity

268
00:15:51,120 --> 00:15:54,000
Speaker 2: to the product. But we also try to resist that

269
00:15:54,080 --> 00:15:57,680
Speaker 2: with some simplicity of navigation and discovery on the page.

270
00:15:58,120 --> 00:16:01,160
Speaker 2: And I think a lot of the magic from there's

271
00:16:01,200 --> 00:16:05,080
Speaker 2: a ton of complexity sitting behind that homepage that hopefully members,

272
00:16:05,120 --> 00:16:08,520
Speaker 2: as you said, have no idea that it requires that

273
00:16:08,640 --> 00:16:11,760
Speaker 2: much technical detail in order to deliver something that just

274
00:16:11,880 --> 00:16:14,240
Speaker 2: works every time you turn on Netflix.

275
00:16:15,200 --> 00:16:18,440
Speaker 3: Don't even think about swiping away. We'll be right back

276
00:16:18,680 --> 00:16:28,200
Speaker 3: with Netflix's Elizabeth Stone after this break, and we're back

277
00:16:28,280 --> 00:16:32,160
Speaker 3: with more from Netflix CTO Elizabeth Stone.

278
00:16:32,480 --> 00:16:36,920
Speaker 1: Yeah, so when you do, you know, user requirements gathering

279
00:16:37,040 --> 00:16:40,840
Speaker 1: kind of kind of a thing that can that can

280
00:16:41,120 --> 00:16:44,920
Speaker 1: be sort of the problem. There is self reporting, right,

281
00:16:45,000 --> 00:16:48,000
Speaker 1: people say they want this thing and that's not really

282
00:16:48,040 --> 00:16:51,200
Speaker 1: what they want or need, right, So the challenge is

283
00:16:51,240 --> 00:16:54,640
Speaker 1: figuring out what features people actually will want and even

284
00:16:54,680 --> 00:16:57,080
Speaker 1: they don't even know what that is. I mean is

285
00:16:57,080 --> 00:17:00,480
Speaker 1: that you know, looking looking for an invisible had kind

286
00:17:00,480 --> 00:17:01,120
Speaker 1: of a problem.

287
00:17:01,480 --> 00:17:05,280
Speaker 2: Yeah. So this is where some of the power of

288
00:17:05,320 --> 00:17:07,720
Speaker 2: the way we have the team structured comes into play,

289
00:17:07,800 --> 00:17:11,440
Speaker 2: which is we're able to marry the user research where

290
00:17:11,440 --> 00:17:13,920
Speaker 2: we actually are out there talking to members or would

291
00:17:13,960 --> 00:17:19,240
Speaker 2: be members, with behavioral data. So experimentation where we actually

292
00:17:19,280 --> 00:17:22,080
Speaker 2: build some of these features that we've heard from members

293
00:17:22,080 --> 00:17:25,240
Speaker 2: would be valuable. But before we roll them out everywhere,

294
00:17:25,280 --> 00:17:28,520
Speaker 2: we will run ab experiments that help us understand is

295
00:17:28,560 --> 00:17:32,239
Speaker 2: it actually helpful? So are people able to find and

296
00:17:32,280 --> 00:17:35,200
Speaker 2: watch a title more quickly or do they engage more

297
00:17:35,240 --> 00:17:38,120
Speaker 2: with the service? And that would be another input. So

298
00:17:38,160 --> 00:17:40,600
Speaker 2: the shorthand for that is we marry some of the

299
00:17:41,400 --> 00:17:44,760
Speaker 2: attitudinal data that we hear from consumers with the behavioral

300
00:17:45,000 --> 00:17:47,320
Speaker 2: that we see in some of our other data to

301
00:17:47,400 --> 00:17:50,040
Speaker 2: get to the best decision for the member experience.

302
00:17:50,520 --> 00:17:52,800
Speaker 1: Do you have a recent example of that, you know,

303
00:17:52,880 --> 00:17:57,280
Speaker 1: what's a big win on the feature front that really

304
00:17:58,400 --> 00:18:01,320
Speaker 1: move the needle in terms of either user engagement or

305
00:18:02,000 --> 00:18:03,120
Speaker 1: customer satisfaction.

306
00:18:03,640 --> 00:18:06,840
Speaker 2: Yeah, so I think one example would be the addition

307
00:18:07,000 --> 00:18:10,400
Speaker 2: of my Netflix, which is on mobile, and it's something

308
00:18:10,440 --> 00:18:13,760
Speaker 2: that we're now exploring for the TV experience, which is

309
00:18:13,800 --> 00:18:16,679
Speaker 2: a place to keep track of titles that you've watched

310
00:18:16,720 --> 00:18:20,080
Speaker 2: for continue watching titles that you've liked or loved. So

311
00:18:20,160 --> 00:18:23,360
Speaker 2: given that thumbs up or double thumbs up and are

312
00:18:23,400 --> 00:18:26,480
Speaker 2: able to kind of make sense of the catalogs. We

313
00:18:26,560 --> 00:18:28,679
Speaker 2: have so many titles, So where's a place where I

314
00:18:28,720 --> 00:18:32,000
Speaker 2: can find things that are especially interesting to me and

315
00:18:32,080 --> 00:18:35,600
Speaker 2: have it feel even more personalized to my interests? And

316
00:18:35,680 --> 00:18:38,520
Speaker 2: so that was something that we launched probably at some

317
00:18:38,560 --> 00:18:41,960
Speaker 2: point last year, and that solved the consumer need and

318
00:18:41,960 --> 00:18:44,240
Speaker 2: it was actually successful with a lot of use of

319
00:18:44,280 --> 00:18:47,440
Speaker 2: that way to navigate the product or the content catalog.

320
00:18:47,960 --> 00:18:52,359
Speaker 1: So is it the is it the practice or the expectation?

321
00:18:52,480 --> 00:18:57,159
Speaker 1: I guess that everything you do gets bubbled back to

322
00:18:57,240 --> 00:19:01,119
Speaker 1: a business outcome. In other words, is there do you

323
00:19:01,200 --> 00:19:04,520
Speaker 1: have to prove that an initiative is going to yield

324
00:19:04,640 --> 00:19:07,040
Speaker 1: some incremental benefit or for not?

325
00:19:07,560 --> 00:19:13,160
Speaker 2: This is a great economics problem. So we don't have

326
00:19:13,400 --> 00:19:17,520
Speaker 2: endless resources, So we want to spend our time and

327
00:19:17,720 --> 00:19:20,119
Speaker 2: energy and dollars on things that are going to be

328
00:19:20,200 --> 00:19:24,080
Speaker 2: most impactful for Netflix. Not all of those things are

329
00:19:24,160 --> 00:19:28,280
Speaker 2: easy to measure, and so a good rule of thumb

330
00:19:28,359 --> 00:19:31,600
Speaker 2: is really thinking about what's going to deliver a better

331
00:19:31,680 --> 00:19:36,280
Speaker 2: member experience or a better creator experience, and some of

332
00:19:36,280 --> 00:19:39,199
Speaker 2: that can be very deeply data informed, like we have

333
00:19:39,359 --> 00:19:41,359
Speaker 2: signs that it's going to be a very big return

334
00:19:41,400 --> 00:19:44,880
Speaker 2: on investments for members and creators, and therefore for Netflix

335
00:19:45,560 --> 00:19:48,359
Speaker 2: and other times it's judgment and we have to be

336
00:19:48,440 --> 00:19:50,639
Speaker 2: able to marry the data we have on what's going

337
00:19:50,680 --> 00:19:53,000
Speaker 2: to be impactful with our judgment of what's going to

338
00:19:53,040 --> 00:19:56,479
Speaker 2: be a good outcome for the business. So I wouldn't

339
00:19:56,480 --> 00:19:58,960
Speaker 2: say you need to prove that from the beginning in

340
00:19:59,080 --> 00:20:01,919
Speaker 2: order to invest in something, but you do need to

341
00:20:02,000 --> 00:20:07,640
Speaker 2: have a coherent hypothesis for the path to value. And

342
00:20:07,680 --> 00:20:11,440
Speaker 2: I think that this is really important, especially for technology

343
00:20:11,520 --> 00:20:15,159
Speaker 2: teams who are innovating, where some things are measurable and

344
00:20:15,200 --> 00:20:17,320
Speaker 2: some things are bets that you say, let's head down

345
00:20:17,320 --> 00:20:20,919
Speaker 2: this path. I do research like bind new technology, and

346
00:20:21,000 --> 00:20:23,600
Speaker 2: if it doesn't work out, we can change the direction.

347
00:20:24,200 --> 00:20:26,200
Speaker 2: But if you had to measure that from the beginning,

348
00:20:26,240 --> 00:20:28,679
Speaker 2: you might not take some of the bigger swings that

349
00:20:28,760 --> 00:20:30,440
Speaker 2: end up very impactful over time.

350
00:20:30,840 --> 00:20:35,120
Speaker 1: Okay, now, the group used to run data and consumer insights.

351
00:20:35,160 --> 00:20:35,800
Speaker 1: Is that what it is?

352
00:20:36,160 --> 00:20:36,840
Speaker 2: Yes, that's right.

353
00:20:37,320 --> 00:20:42,240
Speaker 1: Yeah, the company maintains that as a standalone basically service

354
00:20:42,520 --> 00:20:46,760
Speaker 1: internal service organization, right, And I think you've said that

355
00:20:47,400 --> 00:20:50,800
Speaker 1: it's set up this way to avoid confirmation bias. But

356
00:20:50,920 --> 00:20:53,800
Speaker 1: the flip side of that is, you know, how do

357
00:20:53,800 --> 00:20:58,159
Speaker 1: you align that with the business itself? What are the

358
00:20:58,200 --> 00:21:02,439
Speaker 1: incentives to you know, give the business team what they need?

359
00:21:02,440 --> 00:21:06,639
Speaker 1: Are there in economic terms, right, what is the incentive

360
00:21:06,680 --> 00:21:10,280
Speaker 1: to have that team deliver business unit?

361
00:21:10,920 --> 00:21:13,560
Speaker 2: Yeah, so this is a big part of the identity

362
00:21:13,600 --> 00:21:16,440
Speaker 2: and charter of the data and insights team. I do

363
00:21:16,480 --> 00:21:21,879
Speaker 2: think it is a superpower that Netflix has a centralized

364
00:21:22,000 --> 00:21:26,320
Speaker 2: data and insights team that can be objective about how

365
00:21:26,359 --> 00:21:30,480
Speaker 2: we think about data and research. I actually don't frame

366
00:21:30,520 --> 00:21:33,920
Speaker 2: it as a service organization as much as a really

367
00:21:33,960 --> 00:21:38,880
Speaker 2: strong strategic partner to all of our business stakeholders. So

368
00:21:38,920 --> 00:21:42,040
Speaker 2: that means, of course, yes, sometimes as stakeholder has a

369
00:21:42,119 --> 00:21:44,440
Speaker 2: question and we answer that question or help them get

370
00:21:44,480 --> 00:21:47,040
Speaker 2: to a better decision, and other times we should be

371
00:21:47,119 --> 00:21:50,679
Speaker 2: proactively saying here's an insight, here's an idea for an

372
00:21:50,680 --> 00:21:53,479
Speaker 2: innovation that could be useful for the business. So you

373
00:21:53,520 --> 00:21:56,399
Speaker 2: get both a put if you frame as being strategic

374
00:21:56,520 --> 00:22:00,240
Speaker 2: partners versus purely service. And one of the ways that

375
00:22:00,280 --> 00:22:03,359
Speaker 2: we talk amongst the team is you get almost this

376
00:22:04,080 --> 00:22:06,959
Speaker 2: identity crisis and that success on the team. And what

377
00:22:07,000 --> 00:22:09,440
Speaker 2: I mean by that is you're part of the data

378
00:22:09,480 --> 00:22:13,360
Speaker 2: and insights organization. There's an excellencing craft and a functional

379
00:22:13,359 --> 00:22:16,840
Speaker 2: expertise that comes from that. But you're also a member

380
00:22:17,359 --> 00:22:20,120
Speaker 2: of the business areas that support So you're a member

381
00:22:20,200 --> 00:22:22,640
Speaker 2: of the cross functional ads team or the cross functional

382
00:22:22,680 --> 00:22:26,480
Speaker 2: games team and so on, and so your success is

383
00:22:26,520 --> 00:22:29,560
Speaker 2: that business area's success. So we're able, if we do

384
00:22:29,640 --> 00:22:34,760
Speaker 2: this right, to be objective and really proactive, forwardlooking thinkers

385
00:22:34,880 --> 00:22:38,040
Speaker 2: using data and research, but we're also delivering what the

386
00:22:38,080 --> 00:22:40,760
Speaker 2: business needs at the same time, and we have to

387
00:22:40,760 --> 00:22:43,720
Speaker 2: get there through the ways that we operate basically, so

388
00:22:43,880 --> 00:22:49,280
Speaker 2: really prioritizing strong partnership, communication, joint pilanning across the teams.

389
00:22:49,720 --> 00:22:51,240
Speaker 2: And that's a big part of my role in the

390
00:22:51,280 --> 00:22:54,400
Speaker 2: team's role to make sure we're balancing those incentives properly.

391
00:22:55,400 --> 00:22:57,520
Speaker 1: Now, as you know, there are a lot of admirers

392
00:22:57,520 --> 00:23:03,240
Speaker 1: of Netflix and the technology organization. You know, no less

393
00:23:03,240 --> 00:23:06,439
Speaker 1: than Bob Iger has publicly said several times, right like

394
00:23:06,520 --> 00:23:12,000
Speaker 1: we need to get technology like like Netflix has and

395
00:23:12,080 --> 00:23:15,919
Speaker 1: implements and iterates. So has Bob Iger called you up

396
00:23:15,920 --> 00:23:17,680
Speaker 1: to our for your job.

397
00:23:17,720 --> 00:23:19,880
Speaker 2: I'm just kidding, No.

398
00:23:21,160 --> 00:23:26,840
Speaker 1: Netflix, I guess the question is, does that, you know,

399
00:23:27,040 --> 00:23:29,320
Speaker 1: being out front like that and you know Netflix is

400
00:23:29,320 --> 00:23:32,280
Speaker 1: seen as a pioneer, there is there pressure in the

401
00:23:32,480 --> 00:23:34,679
Speaker 1: uh you know, to stay at the forefront of the

402
00:23:34,720 --> 00:23:39,320
Speaker 1: industry like that or are there misconceptions about what Netflix

403
00:23:39,359 --> 00:23:40,600
Speaker 1: is and how it works?

404
00:23:41,080 --> 00:23:44,680
Speaker 2: And you tell me, I don't think that there's misperceptions

405
00:23:44,680 --> 00:23:47,520
Speaker 2: about it. I think Netflix is super powerful. A long

406
00:23:47,600 --> 00:23:52,159
Speaker 2: time has been the way that technology can be so

407
00:23:52,320 --> 00:23:57,040
Speaker 2: valuable for entertainment and can bring an innovation to entertainment

408
00:23:57,080 --> 00:24:01,920
Speaker 2: that's really appealing to consumers. So that is a strength

409
00:24:02,280 --> 00:24:04,959
Speaker 2: I agree with Bob Biger that Netflix has. I think

410
00:24:05,000 --> 00:24:08,880
Speaker 2: it's a strength that is very important that we maintain.

411
00:24:09,320 --> 00:24:13,120
Speaker 2: I don't know that that translates to pressure to stay

412
00:24:13,160 --> 00:24:17,280
Speaker 2: on the edge versus that's how the team thinks and operates,

413
00:24:17,400 --> 00:24:23,600
Speaker 2: which is seeking impactful innovation, not seeking to be on

414
00:24:23,720 --> 00:24:26,320
Speaker 2: the bleeding edge of innovation for the sake of it,

415
00:24:26,400 --> 00:24:30,040
Speaker 2: but because it delivers something great for members or creators

416
00:24:30,080 --> 00:24:32,800
Speaker 2: and for Netflix in the end. And I think that

417
00:24:32,960 --> 00:24:36,320
Speaker 2: part of the culture at Netflix, especially in the technology team,

418
00:24:36,440 --> 00:24:39,280
Speaker 2: is to think about what's best for the business and

419
00:24:39,320 --> 00:24:42,480
Speaker 2: how can technology contribute to that, And that's a motivator

420
00:24:42,640 --> 00:24:46,359
Speaker 2: for the team versus feeling like, you know, we've got

421
00:24:46,359 --> 00:24:49,719
Speaker 2: people nipping at our heels and we feel the pressure

422
00:24:49,760 --> 00:24:52,159
Speaker 2: to stay ahead. But you know, we're also a business

423
00:24:52,200 --> 00:24:54,560
Speaker 2: and we're hoping to be successful, which means you do

424
00:24:54,640 --> 00:24:58,600
Speaker 2: have to have some real drive to do an even

425
00:24:58,680 --> 00:25:01,520
Speaker 2: better job than what the company doing. So I hope

426
00:25:01,520 --> 00:25:02,320
Speaker 2: we keep that drive.

427
00:25:02,800 --> 00:25:07,119
Speaker 1: Yeah, well, speaking of you know, doing technology development for

428
00:25:07,400 --> 00:25:12,040
Speaker 1: technology development, say, there's AI. Obviously that's a big conversation

429
00:25:12,520 --> 00:25:15,399
Speaker 1: people are having. And I've been part of you know,

430
00:25:16,200 --> 00:25:21,440
Speaker 1: Netflix's technology stack for years, right the machine learning algorithms

431
00:25:21,520 --> 00:25:25,080
Speaker 1: for recommendations for example, how do you think of AI?

432
00:25:25,359 --> 00:25:27,560
Speaker 1: I mean is this you know, as you point out

433
00:25:27,600 --> 00:25:30,679
Speaker 1: that the goal is to serve the business here, what

434
00:25:30,760 --> 00:25:33,280
Speaker 1: are the new areas of opportunity for Netflix as it

435
00:25:33,320 --> 00:25:36,639
Speaker 1: pertains to AI? Writ large and you know you can

436
00:25:36,680 --> 00:25:39,520
Speaker 1: address whatever particular areas underte that.

437
00:25:40,240 --> 00:25:43,600
Speaker 2: As you said, we have leveraged machine learning and even

438
00:25:43,680 --> 00:25:47,000
Speaker 2: AI for a very long time. It's the core of

439
00:25:47,080 --> 00:25:51,520
Speaker 2: our personalized recommendations where we're able to match the right

440
00:25:51,560 --> 00:25:53,720
Speaker 2: title with the right member at the right time. So

441
00:25:53,800 --> 00:25:57,040
Speaker 2: in that sense, it's a technology we're very familiar with.

442
00:25:57,240 --> 00:26:00,480
Speaker 2: And there's also been use cases for machine learning AI

443
00:26:00,880 --> 00:26:03,600
Speaker 2: on the production side, So how we think about things

444
00:26:03,720 --> 00:26:07,080
Speaker 2: like post production visual effects is a good example, how

445
00:26:07,119 --> 00:26:12,119
Speaker 2: we think about localization of content subs stubs doing that

446
00:26:12,240 --> 00:26:15,280
Speaker 2: very efficiently as an organization, So that's all very familiar

447
00:26:15,280 --> 00:26:18,480
Speaker 2: to us. I think of the generative AI wave that

448
00:26:18,520 --> 00:26:22,480
Speaker 2: we're living now as a step function in that technology,

449
00:26:22,640 --> 00:26:26,119
Speaker 2: and we're exploring it to figure out how do we

450
00:26:26,119 --> 00:26:29,159
Speaker 2: think about integrating this in the product to improve the

451
00:26:29,200 --> 00:26:32,680
Speaker 2: member experience, or think about ways that we can use

452
00:26:32,720 --> 00:26:36,480
Speaker 2: it to enable creators and bring their visions to life

453
00:26:36,520 --> 00:26:40,200
Speaker 2: in an even better way than previously. And in that sense,

454
00:26:40,240 --> 00:26:43,400
Speaker 2: I consider it sort of a continuation of an innovation

455
00:26:43,560 --> 00:26:46,280
Speaker 2: journey we've been on. There's just a new tool that

456
00:26:46,320 --> 00:26:49,200
Speaker 2: has a lot of excitement around it, and so we're

457
00:26:49,200 --> 00:26:51,639
Speaker 2: trying to be thoughtful about the aspects of the business

458
00:26:51,680 --> 00:26:54,560
Speaker 2: where that can actually be useful for members and creators.

459
00:26:55,160 --> 00:26:57,000
Speaker 1: Have you I mean, I'm sure this has come up,

460
00:26:57,040 --> 00:27:00,320
Speaker 1: but what about an AI chatbot? It would suggest thing

461
00:27:00,440 --> 00:27:03,679
Speaker 1: to watch on Netflix? What are the pros cons of that?

462
00:27:03,920 --> 00:27:07,000
Speaker 1: Why haven't we seen that from you guys, are really

463
00:27:07,040 --> 00:27:07,720
Speaker 1: anybody else?

464
00:27:08,200 --> 00:27:11,120
Speaker 2: Yeah, it's something that we're working on now to think

465
00:27:11,160 --> 00:27:14,280
Speaker 2: about what is I'll describe it instead of as a chatbot,

466
00:27:14,320 --> 00:27:18,720
Speaker 2: but a more interactive discovery experience, So a way to

467
00:27:18,800 --> 00:27:22,760
Speaker 2: think about organizing the catalog, which is so big, into

468
00:27:22,840 --> 00:27:26,400
Speaker 2: something that feels more tractable for members, to help them

469
00:27:26,440 --> 00:27:29,919
Speaker 2: discover things and reshape in the moment our understanding of

470
00:27:29,920 --> 00:27:32,439
Speaker 2: what a member is looking for. So we're working on

471
00:27:32,560 --> 00:27:35,840
Speaker 2: prototypes of that and would love to be out trying

472
00:27:35,880 --> 00:27:39,480
Speaker 2: that experience with members sometime soon. Good.

473
00:27:39,960 --> 00:27:42,520
Speaker 1: All right, well look forward to that. One other thing

474
00:27:42,720 --> 00:27:45,520
Speaker 1: on AI, I mean that was a flashpoint in the

475
00:27:45,680 --> 00:27:48,680
Speaker 1: two guild strikes last year, the Hollywood strikes, the writers

476
00:27:48,680 --> 00:27:52,119
Speaker 1: and the actors, you know. I mean, what do you

477
00:27:52,160 --> 00:27:56,360
Speaker 1: say to the creative people you know in Netflix and

478
00:27:56,600 --> 00:28:00,240
Speaker 1: externally about you know, people who are afraid that this

479
00:28:00,359 --> 00:28:03,560
Speaker 1: technology is just going to swallow Hollywood and you know,

480
00:28:04,119 --> 00:28:07,840
Speaker 1: it's all just going to be computer generated at some

481
00:28:07,920 --> 00:28:08,720
Speaker 1: point in the future.

482
00:28:09,320 --> 00:28:12,639
Speaker 2: My personal perspective on this is that the technology is

483
00:28:12,680 --> 00:28:16,600
Speaker 2: not going to replace human creativity. So the way Netflix

484
00:28:16,680 --> 00:28:20,040
Speaker 2: is approaching generative AI is as a way to enable

485
00:28:20,160 --> 00:28:23,720
Speaker 2: creator visions, like I mentioned, not what's the path for

486
00:28:24,040 --> 00:28:27,159
Speaker 2: replacement using this technology? And I think that makes a

487
00:28:27,200 --> 00:28:30,679
Speaker 2: big difference for how we approach problems and the partnership

488
00:28:30,720 --> 00:28:33,560
Speaker 2: we continue to have with creators as we use the tech.

489
00:28:34,720 --> 00:28:37,280
Speaker 1: I mean, do you think it's just a question of

490
00:28:37,320 --> 00:28:40,600
Speaker 1: time and education and getting people familiar with the tools

491
00:28:40,600 --> 00:28:42,920
Speaker 1: and what the technology can really enable.

492
00:28:43,400 --> 00:28:46,560
Speaker 2: I think we're already seeing so a lot of productions

493
00:28:46,600 --> 00:28:51,680
Speaker 2: where we don't even directly manage the production, it's partner

494
00:28:51,720 --> 00:28:55,040
Speaker 2: managed productions are leveraging these tools in many ways from

495
00:28:55,080 --> 00:28:58,520
Speaker 2: other vendors. And you see that even in adaptations of

496
00:28:58,640 --> 00:29:02,560
Speaker 2: tools that creators are very familiar with, and now there's

497
00:29:02,600 --> 00:29:05,840
Speaker 2: generative AI aspects of them, and so it's starting to

498
00:29:05,880 --> 00:29:08,440
Speaker 2: become more commonly used as we start to see what

499
00:29:08,440 --> 00:29:11,760
Speaker 2: are ways we could get to better quality, how could

500
00:29:11,760 --> 00:29:14,280
Speaker 2: we bring visions to life more easily with things like

501
00:29:14,360 --> 00:29:18,680
Speaker 2: pre visualization tools. So I think that that's naturally happening

502
00:29:18,800 --> 00:29:21,400
Speaker 2: as people are trying things out and getting more comfortable

503
00:29:21,480 --> 00:29:24,760
Speaker 2: with what the capabilities are. And in that sense, it's

504
00:29:24,800 --> 00:29:27,640
Speaker 2: not you know, Netflix pushing that forward, it's actually the

505
00:29:27,720 --> 00:29:30,440
Speaker 2: industry where we're seeing some of those shifts. Happen already.

506
00:29:30,840 --> 00:29:35,120
Speaker 1: Okay, Elizabeth, thank you uh so much. Just to close

507
00:29:35,160 --> 00:29:39,520
Speaker 1: off the conversation here, you know, for the year ahead,

508
00:29:39,840 --> 00:29:42,600
Speaker 1: what what are the key milestones, what are the key

509
00:29:43,400 --> 00:29:47,400
Speaker 1: goalposts other than the Christmas Day NFL games going off

510
00:29:47,440 --> 00:29:49,240
Speaker 1: with that hit? What what are the what are the

511
00:29:49,320 --> 00:29:51,040
Speaker 1: key things you're looking for? I mean, is it just

512
00:29:51,480 --> 00:29:54,719
Speaker 1: continuation of scale on the ad side, and you know,

513
00:29:54,960 --> 00:29:57,600
Speaker 1: making sure games is optimized and I know there's a

514
00:29:57,680 --> 00:30:02,080
Speaker 1: multi platform games initiative that's cooking also. I mean, are

515
00:30:02,080 --> 00:30:05,240
Speaker 1: there any big things you would call out in the

516
00:30:05,280 --> 00:30:06,400
Speaker 1: next say twelve months.

517
00:30:06,760 --> 00:30:09,160
Speaker 2: I think you've covered a lot so of course live

518
00:30:09,240 --> 00:30:15,040
Speaker 2: including NFL games, ads, as I mentioned, thinking about innovation

519
00:30:15,240 --> 00:30:18,880
Speaker 2: of our user interface, so we'll be out there trying

520
00:30:18,920 --> 00:30:22,400
Speaker 2: a new version of what that homepage discovery experience is.

521
00:30:22,880 --> 00:30:27,120
Speaker 2: And then you know, we're very focused on growth globally

522
00:30:27,320 --> 00:30:31,080
Speaker 2: and delivering for members in countries around the world, and

523
00:30:31,120 --> 00:30:35,360
Speaker 2: there's a lot of room for delivering for those members.

524
00:30:35,360 --> 00:30:38,240
Speaker 2: So there's some markets where a lot of people have

525
00:30:38,320 --> 00:30:40,640
Speaker 2: Netflix already, the US is one of them, but there's

526
00:30:40,640 --> 00:30:44,000
Speaker 2: many markets where we're really early in our journey. So

527
00:30:44,120 --> 00:30:47,120
Speaker 2: I think over the next twelve months we're also going

528
00:30:47,160 --> 00:30:49,720
Speaker 2: to be trying to deliver for those more global member base.

529
00:30:50,080 --> 00:30:53,440
Speaker 1: Yeah, that's sort of conversationally I think you mentioned. Is

530
00:30:53,480 --> 00:30:55,960
Speaker 1: there a timeline for when you're going to test that

531
00:30:56,080 --> 00:30:56,520
Speaker 1: or roll down?

532
00:30:57,200 --> 00:30:59,720
Speaker 2: Yeah, we're still working on that internally to make sure

533
00:30:59,720 --> 00:31:03,080
Speaker 2: that it's really good experience for members before we have

534
00:31:03,200 --> 00:31:04,480
Speaker 2: members trying it themselves.

535
00:31:04,760 --> 00:31:07,200
Speaker 1: All right, Well, I like one last question for you.

536
00:31:07,280 --> 00:31:07,560
Speaker 2: Listen.

537
00:31:08,320 --> 00:31:12,320
Speaker 1: You, when you've talked about the Netflix culture in the past,

538
00:31:12,360 --> 00:31:16,960
Speaker 1: you've described it as not natural human behavior in some cases.

539
00:31:17,040 --> 00:31:21,040
Speaker 1: You know, the directness with which you know the company

540
00:31:21,040 --> 00:31:23,880
Speaker 1: demands that you talk to people and about their work

541
00:31:23,920 --> 00:31:26,360
Speaker 1: and their products, and you know there's of course the

542
00:31:26,440 --> 00:31:28,720
Speaker 1: keeper test, which is you know, I guess that that

543
00:31:28,960 --> 00:31:32,000
Speaker 1: is uncomfortable for a lot of people. Was it difficult

544
00:31:32,000 --> 00:31:35,520
Speaker 1: for you to adapt? Do you have to continually adapt

545
00:31:35,520 --> 00:31:36,280
Speaker 1: to this kind of thing?

546
00:31:36,680 --> 00:31:40,360
Speaker 2: It's not difficult to adapt to when everyone's doing it.

547
00:31:40,960 --> 00:31:43,960
Speaker 2: So what I find about Netflix is if you find

548
00:31:43,960 --> 00:31:47,840
Speaker 2: that your peers are being candid with you, what you're

549
00:31:47,880 --> 00:31:51,000
Speaker 2: doing well, where you could be doing better, you feel

550
00:31:51,040 --> 00:31:54,760
Speaker 2: like you're invested in each other's success. People who report

551
00:31:54,800 --> 00:31:57,200
Speaker 2: to me give me feedback they expect for me to

552
00:31:57,240 --> 00:31:59,640
Speaker 2: give them feedback on how things are going and how

553
00:31:59,640 --> 00:32:02,880
Speaker 2: they can be even more successful. It comes quite naturally

554
00:32:03,000 --> 00:32:05,880
Speaker 2: if you're in that environment. I found that other companies

555
00:32:05,920 --> 00:32:08,920
Speaker 2: where that's not the natural way of working, it's much

556
00:32:09,000 --> 00:32:11,480
Speaker 2: harder because you might want to be direct or you

557
00:32:11,560 --> 00:32:14,760
Speaker 2: might want to receive that candid feedback, but it's just

558
00:32:14,920 --> 00:32:17,560
Speaker 2: not the normal practice. And that's where I find greater

559
00:32:17,680 --> 00:32:20,400
Speaker 2: discomfort in it. So, I mean, it doesn't make it

560
00:32:20,480 --> 00:32:23,040
Speaker 2: easy at Netflix. It means it's a muscle that you

561
00:32:23,160 --> 00:32:27,480
Speaker 2: have to constantly train because it's so important for how

562
00:32:27,520 --> 00:32:30,720
Speaker 2: strong the talent here is. So we remind people all

563
00:32:30,760 --> 00:32:33,280
Speaker 2: the time and give and receive feedback. We're in one

564
00:32:33,320 --> 00:32:35,920
Speaker 2: of those cycles now, and then it should be something

565
00:32:35,960 --> 00:32:37,920
Speaker 2: we're doing continuously, all.

566
00:32:37,920 --> 00:32:40,760
Speaker 1: Right, Elizabeth Stone, thank you so much, appreciate your time.

567
00:32:41,360 --> 00:32:44,120
Speaker 1: Have a great day and good luck with good luck

568
00:32:44,160 --> 00:32:44,320
Speaker 1: in that.

569
00:32:44,640 --> 00:32:46,120
Speaker 2: Thank you so much, nice talking to you.

570
00:32:49,680 --> 00:32:52,600
Speaker 3: Thanks for listening. Be sure to leave us a review

571
00:32:52,760 --> 00:32:56,360
Speaker 3: at Apple Podcasts or Amazon Music. We love to hear

572
00:32:56,400 --> 00:32:59,560
Speaker 3: from listeners. Please go to Variety dot com and sign

573
00:32:59,600 --> 00:33:03,600
Speaker 3: up for the free weekly Strictly Business newsletter, and don't

574
00:33:03,640 --> 00:33:07,120
Speaker 3: forget to tune in next week for another episode of

575
00:33:07,240 --> 00:33:08,160
Speaker 3: Strictly Business,