1
00:00:00,080 --> 00:00:02,960
Speaker 1: There are increasing moves to stop AI companies from scraping

2
00:00:03,000 --> 00:00:05,800
Speaker 1: websites for free, which is what's been going on. Cloud

3
00:00:05,840 --> 00:00:08,479
Speaker 1: Fair is a website infrastructure company that looks after about

4
00:00:08,480 --> 00:00:12,240
Speaker 1: sixteen percent of the global Internet traffic. It has decided

5
00:00:12,400 --> 00:00:15,640
Speaker 1: to block the AI crawlers as a default. Steffanie Cohen

6
00:00:15,960 --> 00:00:19,680
Speaker 1: is Cloudfair's chief strategy officer with US high Stephanie.

7
00:00:20,040 --> 00:00:21,239
Speaker 2: Hi, lovely to be here.

8
00:00:21,280 --> 00:00:23,240
Speaker 1: It's great to chat to you. What proportion of your

9
00:00:23,400 --> 00:00:25,560
Speaker 1: sites that you guys are in charge of do you

10
00:00:25,760 --> 00:00:28,479
Speaker 1: think want to keep AI crawlers off?

11
00:00:30,080 --> 00:00:33,520
Speaker 2: Yeah? I think what if you look at cloud Flair's

12
00:00:33,520 --> 00:00:35,880
Speaker 2: Internet presence a cloud flare, more than twenty percent of

13
00:00:35,880 --> 00:00:38,279
Speaker 2: the Internet sits behind cloud Flare, And what we know

14
00:00:38,360 --> 00:00:40,199
Speaker 2: about the people that sit behind cloud flair is they

15
00:00:40,240 --> 00:00:43,040
Speaker 2: want control over who is scraping their site. They want

16
00:00:43,080 --> 00:00:45,120
Speaker 2: to know who's on their site, they want to know

17
00:00:45,120 --> 00:00:46,640
Speaker 2: what they're planning to do on that site. And then

18
00:00:46,640 --> 00:00:49,360
Speaker 2: they want to have the option to either block, allow,

19
00:00:49,600 --> 00:00:51,839
Speaker 2: or charge AI crawlers for being on their site.

20
00:00:53,159 --> 00:00:55,280
Speaker 1: If they charge, how much do they charge?

21
00:00:56,960 --> 00:00:59,760
Speaker 2: So all of the content creators on cloud flair site

22
00:00:59,760 --> 00:01:02,320
Speaker 2: gets to determine whether or not they allow the crawlers

23
00:01:02,320 --> 00:01:03,840
Speaker 2: on their site and what they're going to charge. And

24
00:01:03,880 --> 00:01:06,840
Speaker 2: the way that it works is that that communication happens

25
00:01:06,880 --> 00:01:11,760
Speaker 2: privately between the site owner and the AI crawler. So

26
00:01:11,800 --> 00:01:14,040
Speaker 2: each site will determine their own price and at the

27
00:01:14,080 --> 00:01:16,399
Speaker 2: same time, each crawler will also determine their own price,

28
00:01:16,440 --> 00:01:19,000
Speaker 2: and when the market meets, that's where we'll begin to

29
00:01:19,040 --> 00:01:21,120
Speaker 2: see a sustainable ecosystem flourish.

30
00:01:21,240 --> 00:01:25,040
Speaker 1: I see and so roughly on average what's being charged.

31
00:01:27,160 --> 00:01:30,959
Speaker 2: So we're really at the beginning of creating this sustainable ecosystem.

32
00:01:31,040 --> 00:01:32,520
Speaker 2: The first thing that we needed to do is make

33
00:01:32,560 --> 00:01:35,280
Speaker 2: sure that content creators had control and so that's why

34
00:01:35,319 --> 00:01:38,040
Speaker 2: we change the defaults on the cloud Flare network. We've

35
00:01:38,080 --> 00:01:40,200
Speaker 2: now changed it so that for new domains on the

36
00:01:40,200 --> 00:01:43,440
Speaker 2: cloud Flare network, AI crawlers are blocked by default, and

37
00:01:43,480 --> 00:01:46,000
Speaker 2: that will allow this market to come together. But as

38
00:01:46,040 --> 00:01:47,600
Speaker 2: you know, there's been a lot of deals done with

39
00:01:47,720 --> 00:01:52,080
Speaker 2: very large AI companies with very large publishers, and what

40
00:01:52,240 --> 00:01:53,920
Speaker 2: we're going to be able to do on our marketplace

41
00:01:54,000 --> 00:01:57,040
Speaker 2: is bring the long tail of both AI crawlers and

42
00:01:57,200 --> 00:01:59,520
Speaker 2: content creators together and we'll see what that produces.

43
00:01:59,600 --> 00:02:03,520
Speaker 1: Okay, So if okay, so We're obviously not going to

44
00:02:03,520 --> 00:02:04,840
Speaker 1: find out how much is being charged, and I can

45
00:02:04,920 --> 00:02:07,760
Speaker 1: understand it might be commercially sensitive, but if everybody in

46
00:02:07,800 --> 00:02:10,639
Speaker 1: the Internet was to charge the AI crawlers for the

47
00:02:10,680 --> 00:02:14,240
Speaker 1: AI crawlers using it, would this actually lead to the

48
00:02:14,400 --> 00:02:18,320
Speaker 1: likes of media companies being able to fund to make

49
00:02:18,360 --> 00:02:20,520
Speaker 1: a decent amount of money off it to fund the

50
00:02:20,520 --> 00:02:23,680
Speaker 1: gap that they're seeing in declining advertising rates at the moment.

51
00:02:25,200 --> 00:02:27,560
Speaker 2: So, as you know, what's going on on the Internet

52
00:02:27,600 --> 00:02:29,960
Speaker 2: is that people are reading the derivative rather than reading

53
00:02:30,000 --> 00:02:32,840
Speaker 2: the ration. And when that happens, it's very difficult to

54
00:02:33,000 --> 00:02:36,360
Speaker 2: monetize through ads, subscriptions, or just through the satisfaction of

55
00:02:36,400 --> 00:02:39,000
Speaker 2: knowing that someone is reading your content. And so we

56
00:02:39,160 --> 00:02:42,160
Speaker 2: think this is the first step, but we think there'll

57
00:02:42,160 --> 00:02:46,480
Speaker 2: be a sustainable business model that includes being paid for crawling.

58
00:02:46,560 --> 00:02:49,480
Speaker 2: But there are probably other business models that we can't

59
00:02:49,480 --> 00:02:52,799
Speaker 2: even see are going to emerge based on what's going

60
00:02:52,800 --> 00:02:55,480
Speaker 2: on with AI. But what we know is that in

61
00:02:55,600 --> 00:02:58,960
Speaker 2: order for AI to thrive, content has to thrive. So

62
00:02:59,040 --> 00:03:01,360
Speaker 2: this is not just about protecting content creators, but this

63
00:03:01,480 --> 00:03:05,160
Speaker 2: is also about creating a world where AI innovation can

64
00:03:05,200 --> 00:03:05,880
Speaker 2: thrive as well.

65
00:03:06,320 --> 00:03:08,440
Speaker 1: Do you think everybody on the Internet has to do

66
00:03:08,520 --> 00:03:10,960
Speaker 1: this in order for it to work, because otherwise the

67
00:03:11,000 --> 00:03:13,280
Speaker 1: II is just going to scripe the freewab science, isn't it.

68
00:03:15,000 --> 00:03:16,960
Speaker 2: I think that what has to happen is if you look,

69
00:03:17,000 --> 00:03:20,120
Speaker 2: there have been very large AI companies and very large

70
00:03:20,160 --> 00:03:23,200
Speaker 2: content creators that have that have deals. But the only

71
00:03:23,240 --> 00:03:25,800
Speaker 2: way that those deals work is if others don't get

72
00:03:25,840 --> 00:03:28,720
Speaker 2: access to that content for free. So what everyone has

73
00:03:28,720 --> 00:03:31,360
Speaker 2: to have is the ability to have control over who's

74
00:03:31,360 --> 00:03:33,679
Speaker 2: on their site. But they may decide that it makes

75
00:03:33,720 --> 00:03:36,200
Speaker 2: sense to let someone through because they already have signed

76
00:03:36,200 --> 00:03:38,880
Speaker 2: a deal with them, or because they are sending traffic

77
00:03:39,000 --> 00:03:42,040
Speaker 2: back to them. So I think what matters is that

78
00:03:42,120 --> 00:03:43,880
Speaker 2: you have the control that you need so that you

79
00:03:43,920 --> 00:03:45,880
Speaker 2: can set the deals that you want. Maybe some of

80
00:03:45,920 --> 00:03:48,600
Speaker 2: them will be exclusive, Maybe some of them you'll you'll

81
00:03:48,640 --> 00:03:50,960
Speaker 2: charge many people some maybe you won't charge some at

82
00:03:51,000 --> 00:03:54,040
Speaker 2: all because you're getting a bunch of traffic from those crawlers.

83
00:03:54,120 --> 00:03:56,360
Speaker 1: So it may it may end up in a situation

84
00:03:56,400 --> 00:03:59,720
Speaker 1: where one of the crawlers is the best one of

85
00:03:59,720 --> 00:04:02,280
Speaker 1: the eye AI providers is the best AI provided because

86
00:04:02,320 --> 00:04:05,520
Speaker 1: it has access to the best websites. Yeah, whereas other

87
00:04:05,600 --> 00:04:08,240
Speaker 1: others may be crappy because they haven't got the good access.

88
00:04:08,400 --> 00:04:09,920
Speaker 1: Is that a potential future.

89
00:04:11,080 --> 00:04:13,960
Speaker 2: I think there's a real optimistic view of the future

90
00:04:14,000 --> 00:04:18,360
Speaker 2: where this is where original, high quality content thrives because

91
00:04:18,400 --> 00:04:21,919
Speaker 2: to your point, AI crawlers want access to that content,

92
00:04:22,120 --> 00:04:24,240
Speaker 2: and so we do think that there is a real

93
00:04:24,320 --> 00:04:27,400
Speaker 2: opportunity here for us to create an ecosystem where there's

94
00:04:27,480 --> 00:04:30,159
Speaker 2: demand for high quality original content, which we think is

95
00:04:30,200 --> 00:04:32,839
Speaker 2: good for AI but also good for human beings.

96
00:04:32,920 --> 00:04:35,120
Speaker 1: Ah, this is fascinating. Hey, thank you, Steffan. We appreciate it.

97
00:04:35,200 --> 00:04:39,240
Speaker 1: Stiffnaney Cohen cloud Fair, chief strategy Officer. For more from

98
00:04:39,279 --> 00:04:42,599
Speaker 1: Hither duplessy Ellen Drive, listen live to news talks it'd

99
00:04:42,600 --> 00:04:45,760
Speaker 1: be from four pm weekdays, or follow the podcast on

100
00:04:45,880 --> 00:04:46,680
Speaker 1: iHeartRadio