WEBVTT - Inside YouTube's Battle Against the Internet's Darkest Corners 0:00:00.240 --> 0:00:03.200 Let me tell you something. There's a full out war 0:00:03.400 --> 0:00:07.240 going on on this subject right now. The bunch of 0:00:07.280 --> 0:00:10.400 filthy sonamite perverts and if you don't like to get 0:00:10.400 --> 0:00:13.960 out of here, they're at war with us tonight. This 0:00:14.000 --> 0:00:16.880 is a video from YouTube of Believe It or Not, 0:00:17.120 --> 0:00:21.639 a sermon from Steven Anderson. He's a pastor in temper Arizona. 0:00:22.040 --> 0:00:26.000 He's young, he's bearded, wearing a tie, working up a sweat, 0:00:26.440 --> 0:00:30.880 pretty much screaming at his audience. It's intense. He's pacing 0:00:30.920 --> 0:00:33.760 back and forth in front of a podium. At one 0:00:33.840 --> 0:00:36.000 point he even jumps up on the podium, raising his 0:00:36.120 --> 0:00:40.440 voice even more. He's vigorously wagging his finger. You know what, 0:00:40.600 --> 0:00:44.519 we have hundreds of people, hundreds of people here that 0:00:44.680 --> 0:00:47.600 will not compromise. And if you're not one of them, 0:00:47.640 --> 0:00:50.519 the get out. I don't want to hang around with 0:00:50.560 --> 0:00:53.720 a bunch of fag hags at a bunch of queer 0:00:53.760 --> 0:00:59.520 baits and a bunch of feminints. Get out. Look no 0:00:59.720 --> 0:01:03.040 one in a million years what I thought when I 0:01:03.120 --> 0:01:11.720 was a child. Stephen Anderson uploads his sermons online several 0:01:11.760 --> 0:01:15.720 times a week, sometimes multiple times a day. The one 0:01:15.760 --> 0:01:19.520 we're watching right now has over seventy thousand views. It 0:01:19.560 --> 0:01:23.120 doesn't have any ads running before it or beside it. Well. 0:01:23.200 --> 0:01:25.440 A couple of weeks ago, reporters for The Times of 0:01:25.520 --> 0:01:28.319 London were watching another YouTube video from the same pastor 0:01:28.800 --> 0:01:31.240 and they saw right next to him and add from 0:01:31.240 --> 0:01:36.000 the cosmetics maker Lourel promoting a British charity, obviously not 0:01:36.240 --> 0:01:39.000 the kind of thing Loria would want to be associated with. 0:01:39.560 --> 0:01:42.959 The newspaper found several other ads from household names running 0:01:43.280 --> 0:01:46.120 on videos, like one from an Egyptian cleric who had 0:01:46.160 --> 0:01:49.080 been banned from the UK for terrorism. There was one 0:01:49.160 --> 0:01:52.360 from David Duke, a former Ku Klux Klan leader. The 0:01:52.480 --> 0:01:55.760 pastor Stephen Anderson, his videos appeared next to ads from 0:01:55.960 --> 0:01:59.600 Nissan and the Guardian newspaper. When The Times called these 0:01:59.640 --> 0:02:02.560 advert tiser's, several of them said they were shocked and 0:02:02.600 --> 0:02:06.040 they immediately yanked their ads from YouTube. And over the 0:02:06.080 --> 0:02:09.200 past two weeks that story is exploded into a crisis 0:02:09.280 --> 0:02:12.520 for the world's largest video service and its owner Google. 0:02:27.480 --> 0:02:30.600 I am Akito and I'm mark Berg and this week 0:02:30.639 --> 0:02:34.120 on Decrypted, we're plunging into one of the thorniest issues 0:02:34.160 --> 0:02:37.240 in the modern Internet. How do you police the unwieldy, 0:02:37.480 --> 0:02:41.560 ever expanding mess that's the Worldwide Web. And what can Google, 0:02:41.680 --> 0:02:45.200 a company that believes in letting nearly anything and everything 0:02:45.520 --> 0:02:48.680 live online, do about it? Can they keep their paying 0:02:48.680 --> 0:02:52.560 advertisers away from the Internet's darkest corners. Stay with us 0:02:52.600 --> 0:02:54.720 as we take you on a tour of the latest 0:02:54.720 --> 0:02:59.440 attempts to apply artificial intelligence to solve Google's crisis. At 0:02:59.480 --> 0:03:02.360 stake here is the future of online advertising, which is 0:03:02.440 --> 0:03:05.640 the fuel for the Internet. Google and another rival, Facebook 0:03:05.800 --> 0:03:08.639 have taken more and more of the share of all 0:03:08.680 --> 0:03:22.600 the advertising dollars spent online. So Mark, you're our Google reporter, 0:03:22.919 --> 0:03:25.520 which looks like a pretty fun job. It means you 0:03:25.560 --> 0:03:30.000 get to write about basically everything. Yep. It's a Google's 0:03:30.040 --> 0:03:32.840 parent company, which you know, is called Alphabet. Now. They 0:03:32.840 --> 0:03:36.760 sell phones, they sell laptops, they sell smart speakers, they 0:03:36.880 --> 0:03:40.440 sell smart thermostats. There's two healthcare companies in there, and 0:03:40.520 --> 0:03:43.120 you have the drones and the robots and the self 0:03:43.200 --> 0:03:47.040 driving cars. You know, despite all this cool future SI stuff, 0:03:47.080 --> 0:03:50.040 Alphabet still makes the bulk of its money, actually eighty 0:03:50.200 --> 0:03:54.280 percent last year from selling ads. That amounts to over 0:03:54.440 --> 0:03:58.680 seventy nine billion dollars in two and YouTube. It's a 0:03:58.680 --> 0:04:01.160 really big video site is one of the fastest growing 0:04:01.160 --> 0:04:07.000 parts of that business. Thanks to the avalanche that the 0:04:07.040 --> 0:04:10.160 Times of London set off, the growth of that revenue 0:04:10.200 --> 0:04:12.920 could be in jeopardy. Now, before we get into the 0:04:13.000 --> 0:04:15.920 thick of this controversy, we should take a step back 0:04:15.920 --> 0:04:19.560 and explain how YouTube advertising actually works. It doesn't really 0:04:19.600 --> 0:04:22.640 happen through the traditional sales process and TV where a 0:04:22.680 --> 0:04:25.200 company would say we want our ads to run in 0:04:25.240 --> 0:04:28.800 front of a commercial for Gray's Anatomy. Instead, the brands 0:04:28.839 --> 0:04:33.680 could target really specific slices of YouTube's viewers. They could say, 0:04:33.839 --> 0:04:36.479 we want to target men in their twenties who really 0:04:36.520 --> 0:04:40.120 like Nascar, or we really want to reach teenage girls 0:04:40.120 --> 0:04:43.080 who want makeup advice, and to do that on a 0:04:43.160 --> 0:04:46.520 really big scale. The ad spots are bought and sold 0:04:46.600 --> 0:04:51.600 through google super fast computerized auction system. It all happens automatically. 0:04:52.360 --> 0:04:55.240 To date, advertisers have pretty much accepted this trade off. 0:04:55.560 --> 0:04:58.080 They get all the benefits of running ads with YouTube 0:04:58.080 --> 0:05:00.640 and it's huge audience, and they stept the risk that 0:05:00.720 --> 0:05:02.640 some of their ads could just show up next to 0:05:02.760 --> 0:05:05.560 pretty much anything, which is why when the Times of 0:05:05.600 --> 0:05:09.000 London's story first broke. I call around a few advertisers 0:05:09.000 --> 0:05:10.360 here in the US, and a lot of them said 0:05:10.400 --> 0:05:13.760 the same thing. WI this New A Monday, which was 0:05:13.839 --> 0:05:16.720 a week after all this drama started, we sent our 0:05:16.720 --> 0:05:20.000 reporter Shelley Hagen to an advertising event in New York, 0:05:21.040 --> 0:05:24.600 and despite some of the high profile announcements of advertisers 0:05:24.600 --> 0:05:27.479 pulling their money from YouTube, a lot of the people 0:05:27.560 --> 0:05:33.040 she talked to seemed pretty sanguine about it too. Um. 0:05:33.080 --> 0:05:39.000 I mean, you do a lot of algorithmic betting and curation, 0:05:39.400 --> 0:05:44.000 but you still have you know, ads that could turn up, 0:05:44.480 --> 0:05:48.520 you know, next to offensive content. Uh. I don't think 0:05:48.560 --> 0:05:51.280 it's it's it's always been a problem in u g C. 0:05:52.040 --> 0:05:55.880 Uh media. That's Anna Thomas as a product marketer at 0:05:55.800 --> 0:05:59.479 Adobe and yeah you g C. That stands for user 0:05:59.560 --> 0:06:02.640 generated content, which is what makes up a vast majority 0:06:02.680 --> 0:06:05.000 of the videos on YouTube. There are videos that are 0:06:05.040 --> 0:06:07.640 posted by anyone instead of saying the polished videos that 0:06:07.680 --> 0:06:11.400 we at Bloomberg would publish. It's what makes YouTube unique 0:06:11.480 --> 0:06:15.000 and compelling and often kind of strange. It's great for 0:06:15.040 --> 0:06:17.800 advertisers who wants to reach younger audiences who don't watch 0:06:17.839 --> 0:06:20.160 a lot of traditional TV and At the same time, 0:06:20.200 --> 0:06:23.239 we at Bloomberg have all kinds of strict guidelines about 0:06:23.240 --> 0:06:26.680 what's okay and what's not okay to say. We certainly 0:06:26.760 --> 0:06:29.120 would never use the kind of epithets that we heard 0:06:29.120 --> 0:06:31.839 from the pastor earlier in the show, even if we 0:06:32.000 --> 0:06:34.839 personally believe them, which I must say I do not. 0:06:35.480 --> 0:06:39.559 But users on YouTube can say pretty much whatever they want, 0:06:39.960 --> 0:06:41.920 and this means that content is going to be a 0:06:41.920 --> 0:06:46.919 lot less predictable, and so it's something that ultimately you 0:06:46.960 --> 0:06:51.360 algorithms get smartto and you can actually prevent that from happening. 0:06:51.480 --> 0:06:57.080 But um, yeah, I don't see it as a huge deal. Um. 0:06:57.200 --> 0:07:01.160 And most advertisers are aware of these things. Not everyone 0:07:01.200 --> 0:07:04.800 agrees with Anna. Right after meeting Anna, Shelley visited a 0:07:04.839 --> 0:07:08.760 big name in the advertising world. His name's Rob Norman, 0:07:08.920 --> 0:07:12.240 and he's the chief Digital officer at Group M, which 0:07:12.280 --> 0:07:15.480 is the media buying arm of the ad agency w PP. 0:07:16.720 --> 0:07:18.600 We've had a lot of conversations with Google in the 0:07:18.640 --> 0:07:21.960 last couple of weeks, as you might imagine, from the 0:07:22.000 --> 0:07:24.760 people that look after oppossess on a daily basis, up 0:07:24.760 --> 0:07:30.480 to the most senior commercial levels of Google, and we've 0:07:30.720 --> 0:07:33.600 listened to their ideas. We put some ideas of our 0:07:33.600 --> 0:07:38.360 own on the table, and we I hope that some 0:07:38.480 --> 0:07:42.440 of those are implementable in the very near future. It's 0:07:42.480 --> 0:07:44.840 a big deal when a guy like Rob is this concerned. 0:07:45.600 --> 0:07:48.800 W p PS parent companies spent around five billion dollars 0:07:48.840 --> 0:07:50.840 on Google last year, and they plan to spend more 0:07:50.960 --> 0:07:54.360 this year. For most people, the idea of an head 0:07:54.520 --> 0:07:58.440 sitting next to a jihadi video one incidence of that 0:07:58.600 --> 0:08:07.280 is once too many are always a good number. We 0:08:07.280 --> 0:08:11.320 should know that these problems happened pretty rarely. One agency 0:08:11.440 --> 0:08:14.320 executive told me that of the more than hundreds of 0:08:14.360 --> 0:08:18.119 millions of impressions or ads that they serve online, only 0:08:18.200 --> 0:08:21.360 two landed on what they would call questionable content. And 0:08:21.400 --> 0:08:24.120 as Rob noted to Shelly, this is not a new 0:08:24.160 --> 0:08:27.000 problem for Google. Right so, more than a decade ago, 0:08:27.080 --> 0:08:28.840 and it was just a search company, it introduced a 0:08:28.880 --> 0:08:32.200 system called ad Sense. It was an algorithm that scoured 0:08:32.240 --> 0:08:35.040 texts on the websites to connect them with relevant ads, 0:08:35.840 --> 0:08:37.920 and we still we still talk about this story. But 0:08:38.200 --> 0:08:40.600 one time The New York Post ran a story about 0:08:40.640 --> 0:08:43.920 a gruesome murder where the killer had chopped up the 0:08:43.960 --> 0:08:46.679 body and stuffed it in a garbage bag and right 0:08:46.720 --> 0:08:49.640 next to that story and add placed by Google for 0:08:49.760 --> 0:08:54.040 plastic bags. Google has since put quite a bit of 0:08:54.120 --> 0:08:56.720 effort to curb this problem. It even set up a 0:08:56.800 --> 0:09:01.439 kind of white gloves service called Google Preferred, where advertisers 0:09:01.480 --> 0:09:04.079 pay more to have their ads run on the most 0:09:04.120 --> 0:09:08.000 popular and what Google says is brand safe content. But 0:09:08.120 --> 0:09:10.960 obviously the problem is persisted, and it really reached a 0:09:11.000 --> 0:09:19.480 flashpoint this past week. Coming off Google's advertising crisis goes global. 0:09:19.760 --> 0:09:22.040 Big brands in the US joined the UK and the 0:09:22.120 --> 0:09:26.240 backlash over terrorist videos. So A T and T announced 0:09:26.280 --> 0:09:29.280 it was pulling its ads from YouTube and other Google 0:09:29.360 --> 0:09:33.800 channels that weren't searched. Verison Johnson Johnson quickly followed. Then 0:09:33.840 --> 0:09:36.080 there were a bunch more. You know, we had reporters 0:09:36.120 --> 0:09:37.960 called up in Europe, and there were brands that pulled 0:09:37.960 --> 0:09:41.160 their ads, and then Walmart, Disney, Starbucks did the same 0:09:41.200 --> 0:09:44.000 after reporters show that their ads were running next to 0:09:44.000 --> 0:09:48.200 YouTube videos with pretty blatantly racist messages. Here's Rob Norman 0:09:48.320 --> 0:09:51.880 from group em again. I think there may be a 0:09:52.040 --> 0:09:56.920 group of advertisers that decide that YouTube and other forms 0:09:56.960 --> 0:10:00.760 of user generating content in particular simply to hot to handle. 0:10:01.360 --> 0:10:05.720 I think those that do may withdraw over a longer 0:10:05.800 --> 0:10:08.080 period of time. I think there'll be many others that 0:10:08.120 --> 0:10:11.920 do come back, but they'll come back when they're satisfied 0:10:12.360 --> 0:10:18.080 that the controls are in place that lets them believe 0:10:18.240 --> 0:10:20.680 in the safety of their brands in that environment more 0:10:20.720 --> 0:10:24.640 than they've been moved to believe in the last few days. 0:10:32.600 --> 0:10:36.480 This brings us to the question why now. In part, 0:10:36.640 --> 0:10:40.200 it's our political climate since the US election. Big tech 0:10:40.240 --> 0:10:43.560 companies like Google, Facebook, Twitter, and face a ton of 0:10:43.559 --> 0:10:47.920 criticism over the content they host, especially those articles with 0:10:48.120 --> 0:10:51.800 latent misinformation, you know the fake news problem, which we 0:10:52.000 --> 0:10:55.880 covered on this podcast back in November. And marketers like 0:10:56.000 --> 0:10:59.520 rob are really sensitive about this stuff. It's clearly an 0:10:59.520 --> 0:11:03.560 issue because as soon as the corporate reputation is damaged 0:11:03.600 --> 0:11:08.719 by anything, that's a significant issue for that corporation. It's 0:11:08.760 --> 0:11:11.840 a significant issue also because some of the content would 0:11:11.840 --> 0:11:19.960 be offensive to almost anyone in America. You know, there's 0:11:20.200 --> 0:11:23.439 one more factor. There's this kind of growing angst in 0:11:23.520 --> 0:11:26.600 the advertising world about how much of the market Google 0:11:26.640 --> 0:11:29.199 and Facebook are eating up. The word I always hear 0:11:29.600 --> 0:11:33.760 do a belie more than forty six percent of all 0:11:33.880 --> 0:11:38.120 digital ad sales worldwide last year went to Google and Facebook. 0:11:38.240 --> 0:11:42.200 According to e Marketer, that share is even higher in 0:11:42.240 --> 0:11:44.640 the US. A lot of people have told me that 0:11:44.679 --> 0:11:47.359 the YouTube boycott has a really good chance for advertiser 0:11:47.400 --> 0:11:50.560 of the pile on. After it began, Martin Sorel, the 0:11:50.720 --> 0:11:54.240 head of w PP, who's like Rob Norman's Superboss, send 0:11:54.440 --> 0:11:57.319 a statement blasting Google and Facebook. He said they were 0:11:57.559 --> 0:12:02.480 quote masquerading as technology company. Martin Sorrell said that these 0:12:02.520 --> 0:12:06.559 companies place ads, so they should act more like media 0:12:06.640 --> 0:12:10.840 companies and actually take responsibility for the content that they're hosting, 0:12:11.120 --> 0:12:15.720 even if they actually didn't make that content themselves. So 0:12:15.840 --> 0:12:17.880 I think the time has sort of come where the 0:12:18.000 --> 0:12:21.640 harm has become so salient and so significant that companies 0:12:21.679 --> 0:12:23.480 are going to one way or another, they are going 0:12:23.559 --> 0:12:25.480 to have to start to get control over the really 0:12:25.520 --> 0:12:28.480 bad things that are happening on their networks. So that 0:12:28.559 --> 0:12:31.440 sounds an awful lot like Martin Sorrell, but it's actually 0:12:31.520 --> 0:12:35.400 someone from a very different profession. My name is Honi Freed. 0:12:35.760 --> 0:12:39.320 I am a professor of computer science at Dartmouth College 0:12:39.760 --> 0:12:43.880 and a senior advisor to the Counter Extremism project. Before 0:12:43.920 --> 0:12:47.360 he worked on counter terrorism, Honey worked on a different mission. 0:12:47.720 --> 0:12:51.520 He built algorithms to rid the web of child pornography. 0:12:51.600 --> 0:12:55.360 So he helped create a technology that extracted unique identifiers 0:12:55.360 --> 0:12:58.520 on images, what he called a signature, and that sticks 0:12:58.520 --> 0:13:00.600 with them as they move across the Internet, even if 0:13:00.600 --> 0:13:03.680 they're modified. And what that allows is to do is 0:13:03.720 --> 0:13:06.600 to build up this database of known bad content, in 0:13:06.600 --> 0:13:09.040 that case, it was child pornography, and then we just 0:13:09.120 --> 0:13:12.280 sit at the pipe of a Facebook or Twitter or 0:13:12.440 --> 0:13:15.600 a Google and every image that comes in gets scanned 0:13:15.720 --> 0:13:19.040 and compared against this database of known child pornography, and 0:13:19.040 --> 0:13:23.040 then anything that is a hit against that database gets flagged, removed, 0:13:23.080 --> 0:13:27.520 and reported. Similar software runs online for things like detecting 0:13:27.600 --> 0:13:31.360 spam and viruses, and over the past year, Honey has 0:13:31.400 --> 0:13:35.000 taken that same framework and applied it the detecting child 0:13:35.080 --> 0:13:39.480 pornography and content that promotes terrorism. He's also started to 0:13:39.559 --> 0:13:44.240 move from still images online to audio and video files. Video, 0:13:44.400 --> 0:13:48.080 it turns out, is much more complicated, and while that 0:13:48.240 --> 0:13:50.679 sounds like a simple extension, it is turns out to 0:13:50.720 --> 0:13:54.960 be a very very difficult engineering task because a video 0:13:55.120 --> 0:13:58.800 standard video is twenty four still images for every second, 0:13:59.400 --> 0:14:01.840 so imagine a three minute video. You were talking about 0:14:01.880 --> 0:14:05.000 thousands and thousands of images, and so being able to 0:14:05.040 --> 0:14:09.400 do that efficiently and accurately um is very very challenging 0:14:09.480 --> 0:14:13.120 engineering problem. Just consider the sheer volume of video that 0:14:13.240 --> 0:14:16.640 Google has to deal with. Here's Eric Schmidt, the former 0:14:16.720 --> 0:14:19.720 CEO and now chairman of Alphabet, talking about this on 0:14:19.800 --> 0:14:22.800 Fox Business last week, YouTube, for example, went from about 0:14:22.840 --> 0:14:26.640 a hundred million hours watched per day to one billion 0:14:26.720 --> 0:14:30.680 hours of YouTube watched globally every day. That's an extraordinary 0:14:30.720 --> 0:14:35.040 platform and an extraordinary responsibility. Honey, though, says Google has 0:14:35.160 --> 0:14:38.480 used this as a convenient excuse for not yet solving 0:14:38.480 --> 0:14:42.080 the problem of offensive content. So, first of all, YouTube 0:14:42.120 --> 0:14:44.360 does a pretty good job of taking down content that 0:14:44.400 --> 0:14:48.880 are violations of touch all property and copyrights. That's Google's 0:14:48.880 --> 0:14:51.880 Content i D program. It's a software system they built 0:14:51.920 --> 0:14:55.480 that automatically pulls videos that violate copyright claims. Are actually 0:14:55.480 --> 0:14:57.400 pretty effective at it, and they're effective at it because 0:14:57.400 --> 0:14:59.600 there's a financial interest for them to do it. Um 0:14:59.600 --> 0:15:02.080 They get suit when they don't do it, and now 0:15:02.120 --> 0:15:06.280 there's a financial interest for them to eliminate content. Um 0:15:05.840 --> 0:15:08.760 Um based on the advertising problems that they've been seeing 0:15:08.760 --> 0:15:14.440 over the last few week. Three days after the Times 0:15:14.440 --> 0:15:17.600 of London story came out, Matt Britain, whose Google's top 0:15:17.600 --> 0:15:20.800 business executive in Europe, was speaking in an AS conference. 0:15:20.960 --> 0:15:23.560 He was really contrite. He said the company was terribly 0:15:23.560 --> 0:15:26.560 sorry and that a fix was coming. Later that day, 0:15:26.840 --> 0:15:29.880 Google laid out some of the fixes in a blog post. 0:15:29.960 --> 0:15:34.240 Google promised a slew of new controls for advertisers. Yeah, 0:15:34.240 --> 0:15:37.440 they'd have more levers to avoid quote unquote higher risk 0:15:37.480 --> 0:15:42.160 content videos with religious or political themes. The company promised 0:15:42.160 --> 0:15:46.360 to hire significant numbers of people and developed new AI 0:15:46.480 --> 0:15:50.640 tools to fix the problems of ads next questionable content soon. 0:15:50.880 --> 0:15:54.280 Google promised these sorts of incidents would be resolved in 0:15:54.360 --> 0:15:56.880 less than a few hours. Many people we talked to 0:15:56.960 --> 0:15:59.640 said that if anybody could do this, it would be Google. 0:16:00.080 --> 0:16:02.400 They've certainly got a lot of money and they've been 0:16:02.400 --> 0:16:05.120 working on this technology for a long time. Our editor 0:16:05.160 --> 0:16:10.360 Alistair Barr talked to this guy over the weekend. Um Meat, Ma, 0:16:10.440 --> 0:16:15.280 Dannie and I working machine learning, applying machine learning, developing 0:16:15.280 --> 0:16:21.640 machine learning techniques currently to security at Cisco. Before that, 0:16:21.680 --> 0:16:25.520 Omid worked at Google. I was there three years ago 0:16:25.600 --> 0:16:29.000 up to three years ago. For three years so UM 0:16:29.040 --> 0:16:32.000 and I was at in YouTube research, so we were 0:16:32.040 --> 0:16:35.640 again developing machine learning techniques and the rest of the 0:16:35.680 --> 0:16:39.880 folks were doing machine learning visual analysis audio analysis is 0:16:39.920 --> 0:16:44.560 speech analysis techniques UH and applying it mostly to YouTube videos. 0:16:44.920 --> 0:16:48.360 So this was super advanced research, and O Meat spent 0:16:48.440 --> 0:16:51.880 a lot of that time on, of all things, video 0:16:51.880 --> 0:16:56.440 games like Minecraft. Google's researchers wanted to ensure that the 0:16:56.560 --> 0:16:59.680 videos uploaded to YouTube were what they claimed to be, 0:17:01.200 --> 0:17:10.879 so videos classified as Minecraft would be indeed Minecraft. They 0:17:10.920 --> 0:17:14.679 also didn't work on detecting and tagging images inside YouTube 0:17:14.720 --> 0:17:17.920 videos like look there's a cat or that's a blue sky. 0:17:18.320 --> 0:17:22.719 There are different ways of analysis. The video gets dissected 0:17:22.880 --> 0:17:25.760 or like a surgery. Basically, all these channels get dissected. 0:17:26.320 --> 0:17:29.480 So why would Google invest so much of its research 0:17:29.520 --> 0:17:34.800 horsepower into this. Better we can tag automatically, the better 0:17:34.840 --> 0:17:39.639 ads we can put recommendation, the relevance can get improved. 0:17:40.440 --> 0:17:45.040 Related videos list on the right side and song. In twelve, 0:17:45.200 --> 0:17:48.679 O Meat and his colleagues published the research paper showing 0:17:48.720 --> 0:17:53.080 they had a ninety nine accuracy rate in classifying video 0:17:53.119 --> 0:17:56.560 content on YouTube. Basically, for the work we had done. 0:17:56.600 --> 0:18:00.920 Not just this that our team, the YouTube the research 0:18:01.000 --> 0:18:05.120 team was awarded. For example, we could take vacation together. 0:18:05.800 --> 0:18:09.119 They went to Hawaii and Google didn't want to comment 0:18:09.240 --> 0:18:12.720 for this episode, but Mark, you got your hands on 0:18:12.760 --> 0:18:14.960 an email the company sent out to some of its 0:18:14.960 --> 0:18:19.920 advertising agencies last week after the company's initial blog post 0:18:20.000 --> 0:18:23.000 failed to stem the YouTube boycott YEP. In that note, 0:18:23.040 --> 0:18:26.560 Google gave a few more details about the new capabilities 0:18:26.560 --> 0:18:31.080 its policies could bring. Here's one example. Under its old policies, 0:18:31.160 --> 0:18:34.560 Google would have allowed advertising if someone in a YouTube 0:18:34.640 --> 0:18:37.560 video was wearing a T shirt with let's say, offensive 0:18:37.680 --> 0:18:41.280 language or an offensive slogan. Under the new policies, it 0:18:41.320 --> 0:18:44.439 would disabled ads. Some of the experts we talked to, 0:18:44.520 --> 0:18:47.280 so this is not an easy fix to implement. And 0:18:47.440 --> 0:18:51.240 here's honey Free, the Dartmouth professor. The tools that we 0:18:51.320 --> 0:18:55.240 have developed what are generally called robust hashing. UM don't 0:18:55.280 --> 0:18:58.879 go after content like oh, I've seen this flag or 0:18:58.920 --> 0:19:01.760 this logo or the symbol, and somewhere in the image 0:19:02.040 --> 0:19:04.280 it goes after this is the same image that I've 0:19:04.320 --> 0:19:07.280 seen before. This is the same video, the same audio recording. 0:19:07.320 --> 0:19:10.399 So it's very very specific. So the good news about 0:19:10.440 --> 0:19:14.600 that is that we can find that content extremely efficiently 0:19:14.680 --> 0:19:17.280 and very accurately. And when I say accurately, I mean 0:19:17.680 --> 0:19:20.760 we can work at Internet scale with billions of uploads 0:19:20.800 --> 0:19:23.680 today and make very very few mistakes. And that's really 0:19:23.680 --> 0:19:26.280 good because that means you can fully automate the system. 0:19:26.320 --> 0:19:29.520 So this works for things that have already been flagged 0:19:29.560 --> 0:19:33.399 as inappropriate content, like let's say, a particular clip of 0:19:33.480 --> 0:19:37.359 child pornography gets taken down, but gets re uploaded to 0:19:37.400 --> 0:19:40.040 the Internet a couple of days later. What gets tricky 0:19:40.119 --> 0:19:44.240 is when a computer sees something entirely new, something that 0:19:44.320 --> 0:19:48.280 hasn't been tagged yet as appropriate or inappropriate. And while 0:19:48.320 --> 0:19:51.040 there is technology to do that, that technology is not 0:19:51.119 --> 0:19:55.280 good enough to work fully automatically, extremely efficiently with no 0:19:55.400 --> 0:19:57.600 human intervention. And what I mean by that is that 0:19:57.840 --> 0:20:00.159 it will inevitably make mistakes, and it will make two 0:20:00.240 --> 0:20:02.719 cut types of mistakes. It will allow things to go 0:20:02.800 --> 0:20:06.000 through that you don't want to UM maybe we tolerate that, 0:20:06.119 --> 0:20:08.639 but more troublingly, it will filter out things that you 0:20:08.680 --> 0:20:11.280 don't want it to filter out. So without human intervention, 0:20:11.800 --> 0:20:15.760 that technology today UM does not work at scale, at 0:20:15.840 --> 0:20:19.520 Internet scale. So Google is one of many companies working 0:20:19.520 --> 0:20:23.080 on an AI approach called deep learning, which is more 0:20:23.160 --> 0:20:26.280 or less a system where algorithms sort of teach themselves. 0:20:26.760 --> 0:20:29.879 So Google's researchers once fed a ton of videos of 0:20:30.080 --> 0:20:32.679 cats to a computer without telling the computer what a 0:20:32.720 --> 0:20:35.840 cat was. Over time, the computers basically learned to identify look, 0:20:35.880 --> 0:20:38.240 there's a cut, and they could in theory, do the 0:20:38.320 --> 0:20:41.320 same thing with a Nazi flag or another hate symbol. 0:20:41.640 --> 0:20:45.280 Our editor Alistair also spoke to another researcher who has 0:20:45.280 --> 0:20:50.720 worked on these problems extensively. I'm j Bolo. I'm a 0:20:50.840 --> 0:20:56.320 professor of computer science at the University of Rochester. J 0:20:56.480 --> 0:20:59.800 Bo worked on developing AI systems that could detect hate 0:20:59.800 --> 0:21:03.439 mes sages on social media like Facebook and Twitter. He 0:21:03.480 --> 0:21:07.359 also uses AI tools for other ends. So we have 0:21:07.600 --> 0:21:13.280 used social media too, for example, analyze the progress of 0:21:13.320 --> 0:21:18.000 the presidential election, and many of our findings helped to 0:21:18.080 --> 0:21:23.000 explain why Trump won election. And we'll also use social 0:21:23.040 --> 0:21:27.760 media to track under age drinking problem, which is a 0:21:27.800 --> 0:21:33.040 big problem in the United States. We also use Instagram 0:21:33.080 --> 0:21:37.879 to track drug use and drug dealers, and this was 0:21:38.040 --> 0:21:42.560 actually in collaboration with the New York State Attorney General's office. 0:21:45.359 --> 0:21:48.800 His research also used advanced AI to come through Instagram 0:21:48.800 --> 0:21:51.959 looking for things like weed and pills and even bonds. 0:21:52.800 --> 0:21:56.880 Pretty amazing, yeah, and a little creepy, But Jaba said 0:21:56.920 --> 0:22:01.160 that video presents a far greater obstacle for AI researchers. 0:22:03.480 --> 0:22:07.920 A video is not just a bunch of uh individual 0:22:08.000 --> 0:22:12.640 frames uh when you watch a video. The most important 0:22:12.680 --> 0:22:16.600 aspect or unique aspect of video is the motion expect 0:22:17.160 --> 0:22:23.159 So things move, people move. That requires UM motion based 0:22:23.640 --> 0:22:31.760 analytical methods too. For example, understand that the changing expression 0:22:31.840 --> 0:22:38.320 on the person's face or an action performed by a person, oh, 0:22:38.480 --> 0:22:43.399 you know, something like an explosion, Oh, you know, people 0:22:43.480 --> 0:22:48.159 fighting each other. It's a challenge. So it's a big 0:22:48.240 --> 0:22:52.240 challenge for Google because it has YouTube, but Facebook and 0:22:52.280 --> 0:22:55.159 Twitter and other companies are also pouring a lot of resources, 0:22:55.200 --> 0:22:58.000 and the online video on Facebook and Twitter also doing 0:22:58.040 --> 0:23:00.760 live video, which has its own set of challenge. Right, 0:23:01.000 --> 0:23:04.359 So that's one challenge figuring out what's inside a video. 0:23:04.720 --> 0:23:07.760 But still Google's algorithms are getting pretty good at that. 0:23:08.400 --> 0:23:11.200 Here's an even thorniar problem. How do you make judgments 0:23:11.200 --> 0:23:14.439 about the gray areas whether a video is outright offensive 0:23:14.720 --> 0:23:18.520 or hate speech? Or incendiary. Google has typically tried to 0:23:18.560 --> 0:23:21.560 steer clear of that. O Meats at his research team 0:23:21.600 --> 0:23:24.199 at YouTube didn't spend a lot of time trying to 0:23:24.200 --> 0:23:28.119 classify political videos because that puts Google into it a 0:23:28.119 --> 0:23:33.680 comfortable position right making editorial judgment. Google thinks of YouTube 0:23:33.720 --> 0:23:37.240 like online search. It's just search for video. If it 0:23:37.359 --> 0:23:41.320 curious that it can't be the subjective, neutral technology platform 0:23:41.400 --> 0:23:44.800 where all information is free. A few advertisers, if any, 0:23:45.119 --> 0:23:47.320 would want to run ads in front of that kind 0:23:47.359 --> 0:23:51.919 of video that we watched earlier, with a screaming pastor 0:23:51.920 --> 0:23:55.000 putting down gay people. Right. But but some online news 0:23:55.000 --> 0:23:58.199 services like maybe Vice News or even like CBS might 0:23:58.240 --> 0:24:00.920 want to run a new segment on that that pastor, 0:24:01.359 --> 0:24:17.280 and advertisers probably want to run ads on that. So 0:24:17.400 --> 0:24:19.680 remember the note we told you that Google sent out 0:24:19.680 --> 0:24:22.919 to advertisers. In that Google said that there are forty 0:24:23.040 --> 0:24:26.960 eight videos flagged as offensive by The Times and other papers. 0:24:27.280 --> 0:24:30.399 Forty four of them would immediately disabled ads under the 0:24:30.440 --> 0:24:34.000 new policies, but the other four could still show ads. 0:24:34.119 --> 0:24:37.320 Google had decided that these videos didn't violate their terms 0:24:37.320 --> 0:24:40.280 of service. Still, with the new controls, advertisers would have 0:24:40.400 --> 0:24:43.920 clearer options for opting out of videos like this. Here's 0:24:43.960 --> 0:24:59.439 one of those four. It starts out with a prayer 0:24:59.600 --> 0:25:03.080 and Arabic asking God to have mercy upon the soul 0:25:03.160 --> 0:25:07.919 of martyrs, and then it transitions into a song. The 0:25:08.080 --> 0:25:12.159 language is pretty graphic. You'll hear phrases like will wage 0:25:12.200 --> 0:25:16.520 wars against them, will return the rape truth, we won't 0:25:16.560 --> 0:25:21.160 accept any occupied land, the ground will erupt and burn them, 0:25:21.359 --> 0:25:24.640 will liberate elk Uods and all goods for our listeners 0:25:24.680 --> 0:25:27.160 who don't know. Is the Arabic name for the city 0:25:27.200 --> 0:25:31.800 of Jerusalem. Okay, so it's clear that this is about 0:25:31.880 --> 0:25:35.760 the Israeli Palestine conflict. Um. You know, it's pretty graphic language, 0:25:35.800 --> 0:25:38.840 like we said, with a lot of metaphors violence. When 0:25:38.840 --> 0:25:42.680