WEBVTT - Sleepwalkers at CES 0:00:04.600 --> 0:00:08.320 Sleepwalkers is a production of I Heart Radio and Unusual Productions. 0:00:15.240 --> 0:00:18.320 I'm Lain and I'm Kara Price. Welcome to a special 0:00:18.360 --> 0:00:34.880 bonus episode of Sleepwalkers from the Consumer Electronic show, So Kara, 0:00:35.000 --> 0:00:37.920 I'd never been to Las Vegas before, which is the 0:00:38.000 --> 0:00:41.360 difference between us. I've been to Vegas too many times. Well, 0:00:41.479 --> 0:00:43.720 I could tell, and it did feel good to be 0:00:43.800 --> 0:00:45.960 in good hands with an old Vegas hand like you. 0:00:46.360 --> 0:00:48.880 One of the new things though for me was slats, 0:00:48.880 --> 0:00:52.000 which I don't normally play. I think so consciously. I 0:00:52.040 --> 0:00:54.400 was thinking about what Tristan Harris talked about in the 0:00:54.400 --> 0:00:57.000 first season Sleepwalkers. You know he was that former googler 0:00:57.240 --> 0:00:59.640 who told us that Instagram is actually supposed to feel 0:00:59.640 --> 0:01:02.360 a lot like slot machines. Well, that's right. Tristan studied 0:01:02.400 --> 0:01:05.360 at the Stanford Persuasion Lab and told us about how 0:01:05.720 --> 0:01:10.320 casino architecture has influenced the development of highly addictive type 0:01:10.360 --> 0:01:13.440 products like Instagram. So it's interesting for me to actually 0:01:13.440 --> 0:01:16.720 see Vegas and the bright lights and the impossibility of 0:01:16.880 --> 0:01:20.600 escape firsthand, not to mention the replicas of the Empire 0:01:20.640 --> 0:01:23.520 State Building, the Canals of Venice, the Colisseum of Rome, 0:01:23.680 --> 0:01:25.520 and you know, I was lucky enough to see the 0:01:25.520 --> 0:01:27.640 Seattle Space Needle for the first time. I just didn't 0:01:27.640 --> 0:01:30.240 know that it was in Las Vegas. But that's not 0:01:30.280 --> 0:01:33.120 why we were there. We were there for CEES, the 0:01:33.160 --> 0:01:37.040 Consumer Electronics Show, and this episode we're actually going to 0:01:37.160 --> 0:01:39.480 talk about some of the coolest things we saw there, 0:01:39.520 --> 0:01:42.039 but we're going to focus more on the innovations that 0:01:42.040 --> 0:01:44.959 are at the intersection of technology and humanity rather than 0:01:45.000 --> 0:01:48.240 talk about you know, infamous toilet paper dispensers. One of 0:01:48.240 --> 0:01:50.680 the big reasons we went is because we were invited 0:01:50.720 --> 0:01:54.480 by WaveMaker, which is an agency part of w p P, 0:01:55.160 --> 0:01:58.280 to do an interview on stage, a live podcast, so 0:01:58.320 --> 0:02:00.960 to speak with Matt and on a hand, who is 0:02:01.000 --> 0:02:04.440 head of product at ARC Publishing and ARC Publishing is 0:02:04.520 --> 0:02:07.280 part of the Washington Post. Yeah, and ARC is also 0:02:07.400 --> 0:02:10.880 an interesting case of AI and action because they're forward 0:02:10.880 --> 0:02:13.880 thinking in terms of increasing the visibility of content through 0:02:13.880 --> 0:02:18.560 personalization and optimizing everything from headlines to photo selection, all 0:02:18.680 --> 0:02:21.560 using machine learning, and those are things that really matter 0:02:21.680 --> 0:02:24.400 for journalists and readers. Yeah, and this use of AI 0:02:24.480 --> 0:02:27.040 stands out to me because it provides a solution to 0:02:27.160 --> 0:02:30.040 real problem. How do you get eyeballs on the right 0:02:30.080 --> 0:02:33.080 content when there's just so much. That said, the issue 0:02:33.080 --> 0:02:36.639 of personalization does also raise questions about what happens when 0:02:36.680 --> 0:02:39.200 machines start to know us better than we know ourselves. 0:02:39.600 --> 0:02:42.079 Not to mention, what are the appropriate limits of how 0:02:42.120 --> 0:02:45.960 companies use AI and dature about us. Yeah, AI can 0:02:46.000 --> 0:02:51.040 definitely streamline processes by detecting patterns that you know, human 0:02:51.080 --> 0:02:54.440 beings cannot see, or it can allow you to do 0:02:54.480 --> 0:02:57.000 things at the scale like tag hundreds of thousands of 0:02:57.080 --> 0:03:01.560 articles that again, human beings just cannot do. So greater 0:03:01.600 --> 0:03:04.680 efficiency is on one side of the spectrum and extremely 0:03:04.720 --> 0:03:06.960 attractive to people, but on the other side you have 0:03:07.080 --> 0:03:10.040 issues of taking humans out of the loop, like the 0:03:10.080 --> 0:03:13.960 black box problem and authenticity in a world of deep fakes. 0:03:14.000 --> 0:03:17.359 So a question for businesses and users of technology is 0:03:17.400 --> 0:03:21.040 sort of when does AI add to our experience and 0:03:21.080 --> 0:03:24.000 when does it maybe hold us back or take advantage 0:03:24.000 --> 0:03:27.040 of us. You know, for example, from seeing news stories 0:03:27.040 --> 0:03:29.680 that we should see, but maybe the algorithm doesn't think 0:03:29.720 --> 0:03:32.080 we want to see it or that we won't click 0:03:32.120 --> 0:03:34.720 on it. Right in the old days when everyone received 0:03:34.760 --> 0:03:37.400 a print use of paper on their doorstep, everyone has 0:03:37.440 --> 0:03:40.640 the same front page and the same headlines. Nowadays, when 0:03:40.640 --> 0:03:43.080 you log onto a news website or onto social media, 0:03:43.520 --> 0:03:47.160 everybody has a different version of the world, and that 0:03:47.320 --> 0:03:50.280 is obviously positive for driving engagement, but may not be 0:03:50.320 --> 0:03:53.520 so positive in terms of having conversations with the same 0:03:53.520 --> 0:03:56.600 facts about the same stories. Equally, we have to ask 0:03:56.800 --> 0:03:59.200 do we want articles where the headline has been written 0:03:59.200 --> 0:04:02.160 by an algorithm or do we prefer headlines written by 0:04:02.200 --> 0:04:04.800 a person. And that's something we talked about with Matt 0:04:05.160 --> 0:04:08.520 because ARC actually tested the headline writing technology. Let's talk 0:04:08.560 --> 0:04:13.800 to Matt seriously. Let's cut to the chase. Are really 0:04:13.920 --> 0:04:18.320 came out of a collaboration trying to better understand what 0:04:18.760 --> 0:04:22.040 actual journalists needed. Can you talk a little bit more. Yeah, 0:04:22.120 --> 0:04:24.240 at the very beginning, you know, we were just trying 0:04:24.240 --> 0:04:27.000 to solve problems for ourselves seven or eight years ago. 0:04:27.400 --> 0:04:28.520 You know, we knew and he had to make some 0:04:28.520 --> 0:04:32.080 pretty fundamental transformation to the post and to really prepare 0:04:32.080 --> 0:04:34.760 ourselves for the digital future. We didn't have the right 0:04:34.800 --> 0:04:36.760 tools to do it, and we couldn't really find the 0:04:36.839 --> 0:04:38.920 right tools on the market either. What we did was 0:04:39.400 --> 0:04:40.800 spent a lot of time with the journalists and the 0:04:40.920 --> 0:04:42.840 editors trying to figure out what it was that make 0:04:42.880 --> 0:04:45.359 their lives easier. It's trying to figure out how do 0:04:45.400 --> 0:04:47.440 you make journalists work better, how can they publish faster? 0:04:47.880 --> 0:04:49.320 What are the little things you can do inside of 0:04:49.320 --> 0:04:51.320 a product to make it easier for them to write 0:04:51.360 --> 0:04:54.600 stories or publish From there? About four years ago is 0:04:54.640 --> 0:04:57.520 when we started evolving it into a commercial offering. Today 0:04:57.600 --> 0:05:00.360 we're running hundreds of websites around the world. We're in 0:05:00.400 --> 0:05:04.360 about twenty different countries. We're running companies like BP their 0:05:04.400 --> 0:05:07.040 internal communications as well as some of their marketing. We're 0:05:07.120 --> 0:05:09.640 running large broadcasters and all their live video and body 0:05:09.680 --> 0:05:11.440 and of course we're still running a lot of newspapers 0:05:11.440 --> 0:05:14.080 and news publishers like The Post and many others around 0:05:14.080 --> 0:05:17.160 the world, Lucky and publishing. You know that AI and 0:05:17.240 --> 0:05:21.919 artificial intelligence are made in headlines, and there was a 0:05:22.040 --> 0:05:25.039 story in the Financial Times last year we said fort 0:05:25.880 --> 0:05:31.520 of AI startups use no AI whatsoever. So I bet 0:05:31.600 --> 0:05:35.279 it's probably higher. But when we talk about using AI, 0:05:35.400 --> 0:05:37.400 or when you talk about using AI, what do we 0:05:37.400 --> 0:05:40.479 actually mean? So it can span the range of technologies 0:05:40.480 --> 0:05:43.600 from something like machine learning, which is basically a way 0:05:43.600 --> 0:05:45.760 to use algorithms, to take large sets of data and 0:05:45.800 --> 0:05:48.520 either uncovered patterns in it or try to model a 0:05:48.520 --> 0:05:52.040 way to predict a certain outcome to technologies like computer vision, 0:05:52.040 --> 0:05:53.960 which you can use to look at images or video 0:05:54.320 --> 0:05:56.839 and extract information about them by recognizing patterns and trying 0:05:56.839 --> 0:05:59.920 to identify objects inside of them, and so a lot 0:05:59.920 --> 0:06:01.720 of those technologies, then when you put them together, you 0:06:01.760 --> 0:06:04.400 can form some really interesting workflows that you know in 0:06:04.400 --> 0:06:06.080 the past you might have had to use humans to 0:06:06.080 --> 0:06:09.040 do that, you can actually do much more simple automatically. 0:06:09.520 --> 0:06:12.640 Was there a particular business challenge or challenge at the 0:06:12.640 --> 0:06:15.560 Washington Post that you couldn't have solved if you hadn't 0:06:15.600 --> 0:06:19.880 been using AI. Any story that we write on Washington Post, 0:06:19.920 --> 0:06:22.880 we're mapping to a set of two or three hundred topics. 0:06:22.880 --> 0:06:24.800 Maybe an example of one of those might be like 0:06:25.000 --> 0:06:28.680 congressional policy or narcotics crime. What you're trying to do 0:06:28.720 --> 0:06:30.800 is say, if I look at all this content, I'm 0:06:30.839 --> 0:06:33.080 not just pulling specific words out of it. I'm actually 0:06:33.080 --> 0:06:35.520 trying to figure out what is this content about, what 0:06:35.640 --> 0:06:38.120 is the fundamental concept of this content. So you pick 0:06:38.160 --> 0:06:40.680 a set of articles, let's say a hundred thousand news 0:06:40.760 --> 0:06:43.040 articles in the case of this example for the post, 0:06:43.960 --> 0:06:47.000 and at first you use humans it's called micro labor 0:06:47.120 --> 0:06:49.160 to do this training set. And the goal is you're 0:06:49.160 --> 0:06:52.320 building an algorithm based on a set of real data. 0:06:52.480 --> 0:06:54.760 And so the humans are going there and saying this article, yeah, 0:06:54.760 --> 0:06:57.200 this is about congressional policy. Why because I know it is, 0:06:57.240 --> 0:06:59.360 I read it, that's what it's about. This one's about 0:06:59.440 --> 0:07:02.960 narcotics time, and this one's about soccer. And so you 0:07:03.000 --> 0:07:06.680 train all these articles against that algorithm until finally the 0:07:06.720 --> 0:07:10.640 algorithm is basically sufficiently advanced to predict a new article 0:07:10.680 --> 0:07:13.360 that you put into it and determine an outcome with 0:07:13.400 --> 0:07:16.320 the same high probability of success that you were able 0:07:16.360 --> 0:07:19.600 to with humans training it. Now, every time a journalist 0:07:19.920 --> 0:07:22.840 saves or publishes a story, we're able to parse over 0:07:22.880 --> 0:07:25.600 all the content inside that story. Then we can predict 0:07:25.680 --> 0:07:28.200 the strength at which it's likely to belong to that topic. 0:07:28.840 --> 0:07:32.920 How do you create a better user experience, in your case, 0:07:32.960 --> 0:07:36.480 news experience for an individual or consumer With that metadat 0:07:36.520 --> 0:07:38.080 you can do a lot of interesting things. We can 0:07:38.120 --> 0:07:40.680 figure out that hey, this is something that they're interested 0:07:40.720 --> 0:07:42.200 in reading, perhaps they'd like to read more, and it 0:07:42.240 --> 0:07:45.920 actually serves the signal into our recommendation algorithms. From your perspective, 0:07:46.000 --> 0:07:50.840 where can businesses sort of harness the power of machine 0:07:50.920 --> 0:07:55.320 learning to really hone in on who their customer is 0:07:55.400 --> 0:07:59.200 and what that customer wants. We want to deliver more 0:07:59.240 --> 0:08:01.080 content to our readers who want to help them find 0:08:01.080 --> 0:08:04.760 more content that we've created. We have about nine journalists 0:08:04.760 --> 0:08:07.360 at the Washington Post. We write something like, you know, 0:08:07.360 --> 0:08:10.040 three or four hundred original stories a day, so there's 0:08:10.040 --> 0:08:12.360 a lot of content there. To get readers to all 0:08:12.360 --> 0:08:14.840 that different content and have them continue moving through your 0:08:14.960 --> 0:08:17.360 content you spend a lot of money to produce is 0:08:17.360 --> 0:08:20.200 really challenging, and so that's a great use case for personalization. 0:08:20.840 --> 0:08:22.680 But where you can make it really come alive is 0:08:23.040 --> 0:08:26.880 by having more sophisticated metadata, more sophisticated information about that content. 0:08:26.960 --> 0:08:29.080 It's more likely to bring readers to it, and so 0:08:29.160 --> 0:08:31.400 that's where these machine learning models really come in handy. 0:08:32.160 --> 0:08:34.400 I think part of what's fun about this conversation is 0:08:34.440 --> 0:08:37.520 there's a lot of cases out there where average users, 0:08:38.200 --> 0:08:39.920 you know, they imagine they see something like that. You 0:08:39.960 --> 0:08:42.160 see the boots on Instagram and you think, oh my god, 0:08:42.240 --> 0:08:47.120 like these companies must crazy, like you know, indiscernible for magic, right, like, 0:08:47.200 --> 0:08:49.160 there must be some crazy model out there doing this, 0:08:49.200 --> 0:08:51.960 and perhaps there is. But in a lot of ways, 0:08:52.640 --> 0:08:57.640 you know, your users aren't necessarily as aware of the 0:08:57.679 --> 0:09:01.400 advertising ecosystem, the data ecosystem, and how these things tied 0:09:01.440 --> 0:09:04.240 together between platforms and sites, and I think, as like 0:09:04.280 --> 0:09:07.360 industry professionals, we always kind of underestimate that fact. And 0:09:07.400 --> 0:09:10.040 so the net effect is that users are completely surprised 0:09:10.040 --> 0:09:11.720 by this. They think you must be doing something completely 0:09:11.800 --> 0:09:13.880 unheard of to achieve it, when in fact, you know 0:09:14.080 --> 0:09:16.599 it could be really simple data sharing. And so the 0:09:16.640 --> 0:09:18.160 reason I think that's important is then when you do 0:09:18.200 --> 0:09:21.080 build technologies that actually utilize some of these more sophisticated 0:09:21.120 --> 0:09:24.600 methods to build data sets, you have to be aware 0:09:24.640 --> 0:09:27.000 that your users. You know, first of all, your users 0:09:27.000 --> 0:09:29.520 aren't gonna necessarily anticipate the outcomes that you can create, 0:09:30.120 --> 0:09:31.400 and if you don't do a good job on the 0:09:31.400 --> 0:09:33.559 product side of making sure that you really think through 0:09:33.600 --> 0:09:36.000 the use case and how you're leveraging technology to solve it, 0:09:36.400 --> 0:09:39.360 you can generate unexpected outcomes. You know, there was the 0:09:39.400 --> 0:09:43.600 example of a retailer who produced advertising flyers that were 0:09:43.600 --> 0:09:46.520 able to predict folks who are pregnant, right, even if 0:09:47.000 --> 0:09:49.120 some of those folks didn't necessarily know that themselves yet 0:09:49.200 --> 0:09:51.800 or hadn't shared it with with their family or their spouses. 0:09:52.280 --> 0:09:56.480 And so that was a case really of both the 0:09:56.559 --> 0:10:00.680 company and the consumer being shocked by outcomes we're generating 0:10:00.679 --> 0:10:03.520 exactly right. I mean, the you know, the algorithm doesn't 0:10:03.520 --> 0:10:06.360 do anything magic, but that's a case of you know, 0:10:06.400 --> 0:10:08.520 putting together in that case, like a marketing program, where 0:10:08.559 --> 0:10:10.920 you don't really think through what's the possible data that 0:10:10.920 --> 0:10:13.280 this could produce and what are my users? What do 0:10:13.320 --> 0:10:15.280 they already know about this data? You know, you need 0:10:15.360 --> 0:10:17.320 to think really hard about your users and what they 0:10:17.360 --> 0:10:18.880 want and what they're trying to achieve, and what the 0:10:18.960 --> 0:10:21.959 dangers are and leveraging this technology. It's no different than 0:10:22.200 --> 0:10:24.679 in that way than any previous technology solutions you might 0:10:24.679 --> 0:10:26.520 have used to build a great product for people, and 0:10:26.520 --> 0:10:28.800 it can be misused just as easily. Funny enough, the 0:10:28.840 --> 0:10:32.360 first episode of Sleepwalkers season one open with a story 0:10:32.480 --> 0:10:37.840 of Washington Post employee Gillian Brockell, who, to your point 0:10:37.840 --> 0:10:41.520 about pregnancy and data stuffered a miscarriage but continue to 0:10:41.559 --> 0:10:46.320 receive targeted ads for pregnancy goods after a miscarriage, and 0:10:46.360 --> 0:10:49.640 she wrote this openless is the technology companies saying please 0:10:49.640 --> 0:10:52.200 stop targeting me. But that raised a big question for us, 0:10:52.240 --> 0:10:55.040 which is what happens when the algorithms go wrong? Yeah, 0:10:55.200 --> 0:10:57.440 I'd almost be more specific with the way that you 0:10:57.520 --> 0:11:00.200 say that, and like the algorithm didn't go wrong, right, 0:11:00.240 --> 0:11:02.559 but like the implementation of it and the product that 0:11:02.600 --> 0:11:05.320 they built around it did because it wasn't really correctly conceived. 0:11:05.360 --> 0:11:07.160 And we have to make sure that like what you're 0:11:07.200 --> 0:11:09.160 trying to do automatically fits really well with what your 0:11:09.240 --> 0:11:11.559 users are trying to accomplish, doesn't happen in a way 0:11:11.559 --> 0:11:14.600 that's not expected. Is a well designed product, you know, 0:11:14.640 --> 0:11:17.079 So in that specific case, yeah, I mean it always 0:11:17.120 --> 0:11:19.240 starts and ends with kind of good product design. If 0:11:19.520 --> 0:11:21.760 you're not doing that, just like any other tool, you 0:11:21.800 --> 0:11:23.920 can misuse it. One of the other things we did 0:11:23.960 --> 0:11:27.640 on the show was we used a language generator to 0:11:27.720 --> 0:11:31.360 can't with pickup lines based on a data set of 0:11:31.400 --> 0:11:36.480 all none of them were actually I don't have kind 0:11:36.480 --> 0:11:39.560 of things like you are a thing and I love you, 0:11:39.559 --> 0:11:41.199 you know, which is now the name of the book 0:11:41.360 --> 0:11:44.320 by the woman who. Yeah, woman Jell Shane her wrote 0:11:44.320 --> 0:11:46.079 a book about it, and then she also did these 0:11:46.080 --> 0:11:48.800 things like AI recipes, like one was for chocolate chocolate 0:11:48.880 --> 0:11:52.400 chocolate chicken cake. So there is funny things and Shakespeare 0:11:52.559 --> 0:11:55.960 on it, and I didn't revealed two things. One is 0:11:56.040 --> 0:12:00.199 when you turn these deep learning algorithms onto big data sets, 0:12:00.520 --> 0:12:02.959 they reveal passions you might not necessarily be aware of, 0:12:03.000 --> 0:12:05.000 like with a lot of chicken and quite a lot 0:12:05.040 --> 0:12:07.880 of chocolates. On the other hand, like these were clearly 0:12:08.360 --> 0:12:10.840 not something human would ever make. So how do you 0:12:10.880 --> 0:12:13.800 think about the line between doing fun things in AI 0:12:13.960 --> 0:12:16.680 and doing stuff which is valuable for business and also 0:12:16.800 --> 0:12:19.040 not getting lost in the uncanny valley. So a good 0:12:19.080 --> 0:12:21.559 example of this for instances. We spent some time at 0:12:21.559 --> 0:12:24.520 the Post trying to build a headline generation algorithm we 0:12:24.520 --> 0:12:28.640 could automatically create headlines for stories. And you know, the 0:12:28.679 --> 0:12:31.240 idea I think at the beginning wasn't necessarily that, you know, 0:12:31.280 --> 0:12:33.760 journalists are never read headlines again, but we'd be able 0:12:33.760 --> 0:12:35.800 to create some alternative headlines in different ways to think 0:12:35.840 --> 0:12:38.280 about a story. Our intention was, let's see if we 0:12:38.320 --> 0:12:39.800 can come up with something so that we can create 0:12:39.840 --> 0:12:42.360 several different variants of a headline. Part of our software 0:12:42.400 --> 0:12:45.440 platform we include content testing framework. So one of the 0:12:45.480 --> 0:12:48.040 things that we can do is say, for a given story, 0:12:48.120 --> 0:12:50.480 let's have three different headlines for it, Let's run a 0:12:50.520 --> 0:12:53.280 test as soon as it publishest of the audience is 0:12:53.320 --> 0:12:55.640 going to get each variant, and then as people start 0:12:55.720 --> 0:12:57.120 to click one more than the other, we're gonna shift 0:12:57.120 --> 0:12:59.600 the burden of traffic to the most successful variant. And that, 0:12:59.640 --> 0:13:02.240 I would them, by itself works really well. If you know, 0:13:02.280 --> 0:13:04.120 folks in the audience here were to look at the 0:13:04.160 --> 0:13:06.520 homepage of our site right now, there's probably two or 0:13:06.559 --> 0:13:08.480 three stories that are running those types of tests where 0:13:08.480 --> 0:13:11.040 different people would se different headlines or different images, or 0:13:11.240 --> 0:13:14.680 in fact maybe actually just complutely different stories, and those 0:13:14.720 --> 0:13:17.320 tests will resolve in like fifteen or twenty minutes. So 0:13:17.400 --> 0:13:19.280 that works well enough by itself. But then we realized, well, 0:13:19.280 --> 0:13:20.960 we could probably create more of these tests if only 0:13:20.960 --> 0:13:23.439 we could automatically create headlines for them. We could just 0:13:23.480 --> 0:13:25.600 be running these tests all the time for every single story. 0:13:26.080 --> 0:13:28.840 But what we found was, you know, not exactly so 0:13:29.320 --> 0:13:31.760 if the idea was to save journalists time and doing 0:13:31.800 --> 0:13:33.679 that in the end, I mean, you'd have to come 0:13:33.760 --> 0:13:36.240 up with something that's fairly solid and ready to publish out. 0:13:36.600 --> 0:13:38.720 We were able to create something that allowed you know, 0:13:38.800 --> 0:13:41.200 journalists basically have different formulations that they could play with 0:13:41.240 --> 0:13:43.000 and maybe gave them some ideas of what to create, 0:13:43.240 --> 0:13:44.839 but it still require people to look at it. In 0:13:44.880 --> 0:13:48.720 the end, how can businesses work better with their engineers, 0:13:48.840 --> 0:13:53.920 with their tech teams to sort of create and not 0:13:54.280 --> 0:13:57.240 stay siloed in a way that like somebody who works 0:13:57.240 --> 0:13:59.840 in marketing feels like, well, you know, there's actually this 0:14:00.040 --> 0:14:01.520 need that I have, but I don't know who to 0:14:01.559 --> 0:14:03.600 talk to about it, and that don't really know what 0:14:03.640 --> 0:14:05.800 to do. It's an awesome question to me, Like one 0:14:05.800 --> 0:14:06.920 of the best things that you can do as a 0:14:06.960 --> 0:14:10.080 business is to put those people together, sometimes even physically. 0:14:10.200 --> 0:14:12.920 So when we started this project, you know, we literally 0:14:12.960 --> 0:14:17.560 co located engineers, product people directly inside the news room 0:14:17.600 --> 0:14:19.760 to sit with the folks who are doing this work. Now, 0:14:19.760 --> 0:14:22.480 when it comes to a I M L, you remember, 0:14:22.480 --> 0:14:24.640 these are just tools. These are tools to make work easier. 0:14:24.680 --> 0:14:27.160 Their tools in a lot of cases for automation and efficiency. 0:14:27.560 --> 0:14:29.920 There's some problems that can't be solved without it. In 0:14:30.000 --> 0:14:32.000 the end, though, you know, you're still trying to solve 0:14:32.040 --> 0:14:34.040 some business problem, and most of those involved some sort 0:14:34.040 --> 0:14:35.560 of users that you need to get to know. So 0:14:36.200 --> 0:14:38.440 you know, even at the post we had data sciencests 0:14:38.440 --> 0:14:40.640 who were on those teams embedded in the news room 0:14:40.680 --> 0:14:42.680 as well. You know, they weren't kind of seated somewhere 0:14:42.720 --> 0:14:44.960 else thinking of problems on their own. There's a time 0:14:45.000 --> 0:14:47.440 and a place for creating room for prototyping sometimes that 0:14:47.480 --> 0:14:50.440 has to happen to especially with really advanced technologies. But 0:14:50.480 --> 0:14:54.000 beyond prototyping, putting those teams together is super crucial. So 0:14:54.040 --> 0:14:57.160 how do you make sure, speaking metaphorically, you write a 0:14:57.200 --> 0:15:00.160 good brief to your AI team, well, your engineering team. 0:15:00.200 --> 0:15:03.200 I still think, you know, start with the problem that 0:15:03.200 --> 0:15:05.120 you're trying to solve. Like, if you're going in thinking 0:15:05.200 --> 0:15:08.120 let's use AI to solve something, I think you're probably 0:15:08.720 --> 0:15:10.960 starting the problem the wrong way, and start by framing 0:15:11.000 --> 0:15:13.240 up the problem in business terms. For people, you'd be surprised, 0:15:13.280 --> 0:15:15.800 I think how much you know engineers and product folks 0:15:15.800 --> 0:15:17.880 really actually prefer to get that first before they start 0:15:17.920 --> 0:15:19.880 diving into what's the technology that I'm getting used to 0:15:19.880 --> 0:15:23.160 solve this problem? With a buzz around AI, especially right now. 0:15:23.920 --> 0:15:25.480 You know, people tend to go into it. I think 0:15:25.480 --> 0:15:28.080 thinking this is kind of something that's pretty close to magic. 0:15:28.560 --> 0:15:30.440 We just use AI. It's going to solve these problems 0:15:30.440 --> 0:15:32.320 that we haven't been able to solve some other way. 0:15:32.480 --> 0:15:34.280 And that's not really the right way to approach it. 0:15:34.560 --> 0:15:37.119 But a I will be transformative, I think for organizations 0:15:37.120 --> 0:15:39.640 that apply it the right way, with the product mindset, 0:15:39.720 --> 0:15:41.320 with a good knowledge of the problem that they're trying 0:15:41.320 --> 0:15:44.600 to solve, with empathy for their users. When we're doing 0:15:44.680 --> 0:15:48.239 our research for this panel, as an article in Bloomberg 0:15:48.240 --> 0:15:53.520 News saying that Jeff Bezos is pestimally very invested in 0:15:53.520 --> 0:15:56.480 the product, and then you called him Jeff in conversation, 0:15:56.720 --> 0:16:01.160 I found very impress So we will unders people sending 0:16:01.160 --> 0:16:04.400 me an email already without obviously telling us the contented 0:16:04.440 --> 0:16:07.640 your meetings. How does his vision imbue what you do? 0:16:08.160 --> 0:16:11.440 So certainly for us it's boon to have him owning 0:16:11.440 --> 0:16:14.000 the company. I think that's one of the greatest things 0:16:14.040 --> 0:16:16.360 is you know, obviously at the Washington Post were known 0:16:16.400 --> 0:16:19.040 for an amazing newsroom, but we've also spent a lot 0:16:19.040 --> 0:16:21.480 of time investing in our engineering team that started to 0:16:21.520 --> 0:16:23.720 some extent before you know, we were purchased by Jeff, 0:16:23.760 --> 0:16:26.000 but certainly after we purchased us. It opened a lot 0:16:26.040 --> 0:16:28.000 of new doors for us, and it gets people excited 0:16:28.040 --> 0:16:29.120 to come and work for us and some of the 0:16:29.120 --> 0:16:32.080 problems that we're trying to solve. It really inspires people 0:16:32.160 --> 0:16:34.160 to be able to build like a platform like we built, 0:16:34.760 --> 0:16:36.920 you know, within a newspaper company. I think would have 0:16:36.960 --> 0:16:39.280 been hired to Fathom probably ten years ago, but I 0:16:39.280 --> 0:16:41.040 mean today we really can say that, you know, we're 0:16:41.040 --> 0:16:44.320 a content company and we're a technology company. And I 0:16:44.360 --> 0:16:46.680 think part of that starts with him in the leadership 0:16:46.720 --> 0:17:04.920 that he provides. More Sleepwalkers after the break, So, Kara, 0:17:05.080 --> 0:17:08.720 that was our conversation on stage with Matt monahantan CS 0:17:08.800 --> 0:17:11.960 in early January. It was interesting because we hear so 0:17:12.040 --> 0:17:15.800 much about tech companies becoming publishers, whether it's Facebook, YouTube, 0:17:15.880 --> 0:17:19.600 or Twitter, but we hear less about publishers becoming tech companies. 0:17:20.080 --> 0:17:22.280 I guess that's where Jeff Basos as an owner is 0:17:22.320 --> 0:17:25.120 what we might call a differentiator. So I was personally 0:17:25.160 --> 0:17:28.480 struck with Matt's experiments with the headline generator. You know, 0:17:28.520 --> 0:17:30.960 for the time being, it doesn't work well enough to 0:17:31.000 --> 0:17:34.760 be a commercial product, but I think it will soon. 0:17:34.840 --> 0:17:37.280 You know, look at autocomplete when you send a Gmail 0:17:37.359 --> 0:17:41.200 like Sincerely Comma. You know I get those all the time, 0:17:41.640 --> 0:17:44.960 and it works. You know, in an apocalyptic reading, that 0:17:45.040 --> 0:17:47.520 means that machines will take over our lives and there 0:17:47.520 --> 0:17:49.919 will be no work left for humans. We won't have 0:17:49.960 --> 0:17:52.119 to come up with smart headlines. But I think in 0:17:52.160 --> 0:17:56.320 a more optimistic reading, using algorithms to generate writing suggestions 0:17:56.320 --> 0:18:00.840 could actually enable originality. Reminds me of that Chinese science 0:18:00.840 --> 0:18:03.200 fiction writer who you and I have talked about named 0:18:03.280 --> 0:18:08.120 Chen show Fund, who actually used an algorithm to create 0:18:08.280 --> 0:18:11.800 ideas for his own work, and he used it when 0:18:11.840 --> 0:18:14.200 he had writer's block. He wasn't using it to replace 0:18:14.680 --> 0:18:17.400 his creative skill. He was using it as an enhancement tool. 0:18:17.440 --> 0:18:19.639 And I think that's really interesting. Yeah, And in season 0:18:19.640 --> 0:18:21.920 one of sleep Walkers, we spoke to a filmmaker called 0:18:21.960 --> 0:18:24.760 Oscar Shop who actually shot a whole film written by 0:18:24.800 --> 0:18:28.880 Ai called Sunspring. Oscar and Chen turned the technology into 0:18:28.920 --> 0:18:31.359 a tool that actually serves their purposes. You know, you 0:18:31.400 --> 0:18:34.359 can develop all kinds of technology in a vacuum. About 0:18:34.359 --> 0:18:37.040 the technology that really serves people and fills a need. 0:18:37.400 --> 0:18:39.760 Is the technology that sticks around, you know, speaking of 0:18:39.760 --> 0:18:42.800 technology that really sticks around, and then some technology that 0:18:43.160 --> 0:18:46.399 might not stick around. There's so much stuff on the 0:18:46.440 --> 0:18:49.879 floor of CS you and I have never been to 0:18:49.880 --> 0:18:52.159 see us before. I think we were very overwhelmed by 0:18:52.160 --> 0:18:54.880 what we saw and excited. It was kind of inspected 0:18:54.920 --> 0:18:58.679 gudgets paradise, and obviously, as someone who's obsessed with technology 0:18:58.880 --> 0:19:01.639 and consumer technology, I would have bought every single thing 0:19:01.640 --> 0:19:04.320 I thought you tried to buy. I did try to 0:19:04.359 --> 0:19:07.080 buy that. I mean that keyboard with the mouse burnt 0:19:07.119 --> 0:19:11.400 into the keyboard. How much did this cost? I almost 0:19:11.400 --> 0:19:17.040 bought a laser cool laser patch for my back, which 0:19:17.119 --> 0:19:21.560 placebo or not made me look very hot. But no, 0:19:21.760 --> 0:19:24.520 in all seriousness, you know, there are things that were 0:19:24.560 --> 0:19:27.320 on the floor that are kind of amazing when you 0:19:27.359 --> 0:19:30.320 think about it, Like from this company called Pillow Health. 0:19:30.680 --> 0:19:33.359 They've developed this device called Priya that looks like a 0:19:33.359 --> 0:19:35.520 little face, a cute little face, as they always do, 0:19:36.040 --> 0:19:39.840 and it's basically a pill dispenser that is voice and 0:19:39.880 --> 0:19:43.080 face activated, so anybody could have one. I could have one, 0:19:43.080 --> 0:19:45.919 You could have one. But I think they've developed it 0:19:46.040 --> 0:19:50.080 mostly for elderly people who have many pills that they 0:19:50.080 --> 0:19:54.080 have to take throughout the day, and who's children or 0:19:54.640 --> 0:19:57.800 health aids want to be able to control when their 0:19:57.840 --> 0:20:01.360 medicine is dispensed. And I think for someone who might 0:20:01.480 --> 0:20:07.040 have memory impairment, physical impairment, the idea that someone who 0:20:07.160 --> 0:20:10.920