1 00:00:02,730 --> 00:00:05,590 Speaker 1: Bloomberg Audio Studios. Podcasts. 2 00:00:05,910 --> 00:00:06,430 Speaker 2: Radio. 3 00:00:06,850 --> 00:00:07,270 Speaker 3: News. 4 00:00:18,570 --> 00:00:21,439 Speaker 4: Hello and welcome to another episode of the Oddbots Podcast. 5 00:00:21,500 --> 00:00:22,320 Speaker 4: I'm Tracey Alloway. 6 00:00:22,380 --> 00:00:23,560 Speaker 1: And I'm Joe Weisenthal. 7 00:00:23,880 --> 00:00:27,060 Speaker 4: Joe, it's October. We're recording this on October 29th. 8 00:00:27,610 --> 00:00:27,870 Speaker 5: Yeah. 9 00:00:27,970 --> 00:00:30,450 Speaker 4: It is also Black October. Have you heard this term? 10 00:00:30,590 --> 00:00:30,990 Speaker 5: Yeah. 11 00:00:31,250 --> 00:00:33,170 Speaker 1: What does that mean? Have stocks gone down? 12 00:00:33,450 --> 00:00:34,090 Speaker 2: No. 13 00:00:34,350 --> 00:00:36,910 Speaker 4: Your mind immediately goes to markets. There are other worlds 14 00:00:36,990 --> 00:00:38,599 Speaker 4: out there. We're going to be talking about the world 15 00:00:38,620 --> 00:00:41,460 Speaker 4: of cybersecurity today because one of the reasons people are 16 00:00:41,500 --> 00:00:44,960 Speaker 4: talking about Black October is we've had a number of 17 00:00:45,060 --> 00:00:48,960 Speaker 4: recent security issues, the most prominent of which has been 18 00:00:49,370 --> 00:00:53,089 Speaker 4: a bunch of vulnerabilities discovered in PHP, which is a 19 00:00:53,130 --> 00:00:57,110 Speaker 4: coding language that apparently is used by 70% of websites 20 00:00:57,190 --> 00:01:00,410 Speaker 4: in existence. We can get into that stat later. But 21 00:01:00,450 --> 00:01:04,030 Speaker 4: the really interesting thing about this vulnerability being uncovered is 22 00:01:05,030 --> 00:01:08,530 Speaker 4: AI was used to uncover it. And so the big 23 00:01:08,610 --> 00:01:11,660 Speaker 4: question coming back to AI, there's a concern now that 24 00:01:12,250 --> 00:01:14,880 Speaker 4: you have good actors who are maybe using AI to 25 00:01:14,940 --> 00:01:19,230 Speaker 4: discover old vulnerabilities and trying to fix them. But you 26 00:01:19,250 --> 00:01:21,990 Speaker 4: could also have bad actors who are using AI to 27 00:01:22,050 --> 00:01:25,030 Speaker 4: exploit those same vulnerabilities. So we're back to the question 28 00:01:25,060 --> 00:01:28,820 Speaker 4: of whether AI makes the internet safer or more dangerous. 29 00:01:29,140 --> 00:01:30,980 Speaker 1: It makes it more dangerous. No, I have no idea. 30 00:01:31,360 --> 00:01:35,060 Speaker 1: Ever since essentially Anthropic announced that they had this model 31 00:01:35,080 --> 00:01:38,720 Speaker 1: called Mythos and now it's Project Glasswing, which was rolled 32 00:01:38,760 --> 00:01:41,640 Speaker 1: out to companies to use for defensive purposes before it 33 00:01:41,680 --> 00:01:44,410 Speaker 1: was available to the general public, there has been much 34 00:01:44,569 --> 00:01:49,300 Speaker 1: more awareness of cybersecurity issues as it relates to AI. 35 00:01:49,380 --> 00:01:52,360 Speaker 1: And there are many things that people debate about AI 36 00:01:52,440 --> 00:01:55,740 Speaker 1: and some of these bigger existential questions. Pretty much everyone 37 00:01:55,820 --> 00:01:59,950 Speaker 1: agrees cybersecurity good, more cybersecurity good. I will say I 38 00:01:59,950 --> 00:02:02,850 Speaker 1: know very little about the world of cybersecurity. I don't 39 00:02:02,910 --> 00:02:07,170 Speaker 1: know what, for example, a cybersecurity company actually sells. I 40 00:02:07,190 --> 00:02:10,389 Speaker 1: don't know how if I were buying cybersecurity, I would 41 00:02:10,430 --> 00:02:12,750 Speaker 1: know whether it's working or whether no one had tried 42 00:02:12,770 --> 00:02:15,190 Speaker 1: to hack me or not or something like that. I 43 00:02:15,250 --> 00:02:17,710 Speaker 1: need to learn a lot more, but it's absolutely true 44 00:02:17,730 --> 00:02:19,549 Speaker 1: about the uptick in the volume of hacks. 45 00:02:19,630 --> 00:02:22,030 Speaker 4: There are hackers out there right now taking notes going, 46 00:02:22,090 --> 00:02:25,620 Speaker 4: Joe doesn't know anything about cybersecurity, probably doesn't have a VPN. 47 00:02:25,840 --> 00:02:26,139 Speaker 5: I don't. 48 00:02:26,180 --> 00:02:28,620 Speaker 1: That's why my vibe coding projects, I don't want to ever, 49 00:02:28,760 --> 00:02:31,220 Speaker 1: I have yet to build anything with an account because 50 00:02:31,280 --> 00:02:33,120 Speaker 1: I don't want to like have to think about that. 51 00:02:33,180 --> 00:02:35,560 Speaker 1: So it's just like, I don't want to count information 52 00:02:35,620 --> 00:02:39,420 Speaker 1: with anyone. But I have as a user of technical products, 53 00:02:40,240 --> 00:02:43,100 Speaker 1: over the last several weeks, you get more emails saying like, 54 00:02:43,510 --> 00:02:46,510 Speaker 1: Our thing was compromised. You're probably fine, but maybe you 55 00:02:46,530 --> 00:02:49,880 Speaker 1: should refresh some keys or whatever. This is happening. This 56 00:02:49,960 --> 00:02:52,880 Speaker 1: is real. And everyone really needs to learn more. And 57 00:02:52,900 --> 00:02:56,930 Speaker 1: if you think about like black swans and Wall Street 58 00:02:56,970 --> 00:02:59,550 Speaker 1: and all that stuff, you know, there's been for years 59 00:02:59,590 --> 00:03:02,470 Speaker 1: a lot of talk about cyber as a major risk 60 00:03:02,530 --> 00:03:02,809 Speaker 1: out there. 61 00:03:02,910 --> 00:03:03,010 Speaker 2: Yeah. 62 00:03:03,350 --> 00:03:05,610 Speaker 4: All right, well, we do, in fact, have the perfect guest. 63 00:03:05,770 --> 00:03:08,489 Speaker 4: We're going to be speaking to two people from the 64 00:03:08,530 --> 00:03:12,059 Speaker 4: company Calif, one of the companies that actually uncovered this 65 00:03:12,120 --> 00:03:16,200 Speaker 4: PHP vulnerability. The other one was a company called Wiz. 66 00:03:16,300 --> 00:03:19,660 Speaker 4: We have Tai Yun. He is the CEO over at Calif, 67 00:03:19,840 --> 00:03:22,430 Speaker 4: which is a hacking firm, and we can get into 68 00:03:22,630 --> 00:03:25,230 Speaker 4: exactly what that means, as well as Bruce Dang. He 69 00:03:25,350 --> 00:03:28,110 Speaker 4: is an engineer at Calif as well. So, Tai and Bruce, 70 00:03:28,130 --> 00:03:29,630 Speaker 4: thank you so much for coming on All Thoughts. 71 00:03:30,150 --> 00:03:31,829 Speaker 2: We're glad to be here. So. 72 00:03:32,530 --> 00:03:35,910 Speaker 4: Just to begin with, because hackers kind of, they always 73 00:03:35,940 --> 00:03:38,460 Speaker 4: have cool achievements that they like to talk about. Why 74 00:03:38,480 --> 00:03:40,240 Speaker 4: don't we just go ahead and start with who you 75 00:03:40,520 --> 00:03:43,480 Speaker 4: are and what are the coolest things you've done in 76 00:03:43,520 --> 00:03:44,540 Speaker 4: your hacking career? 77 00:03:45,900 --> 00:03:49,820 Speaker 3: Yeah. So I started my career at Microsoft almost 20 78 00:03:49,820 --> 00:03:53,590 Speaker 3: years ago in the Microsoft Security Response Center. These are 79 00:03:53,610 --> 00:03:56,870 Speaker 3: the people that do patch Tuesdays. On the second Tuesday 80 00:03:56,950 --> 00:03:59,230 Speaker 3: of every month, Microsoft releases an update. So I worked 81 00:03:59,290 --> 00:04:02,690 Speaker 3: in that team a long time ago. Probably the biggest 82 00:04:02,710 --> 00:04:04,390 Speaker 3: thing I did there and that I could talk about 83 00:04:04,470 --> 00:04:09,250 Speaker 3: is I helped reverse engineer the Stuxnet virus, which is 84 00:04:09,330 --> 00:04:12,750 Speaker 3: one of the one that was related to the supposedly 85 00:04:12,790 --> 00:04:17,489 Speaker 3: the Iranian nuclear program was used to sabotage the Iranian 86 00:04:17,510 --> 00:04:18,230 Speaker 3: nuclear program. 87 00:04:18,430 --> 00:04:20,710 Speaker 4: This is the thing that people describe it as the 88 00:04:20,750 --> 00:04:24,660 Speaker 4: world's first like known digital cyber weapon, right? 89 00:04:24,960 --> 00:04:26,580 Speaker 2: Yeah, yeah, yeah, yeah. 90 00:04:26,940 --> 00:04:29,960 Speaker 3: I'm sure there were many others before, but yeah, that's 91 00:04:29,980 --> 00:04:33,710 Speaker 3: the first public one, yeah. I worked there for almost 92 00:04:33,730 --> 00:04:37,010 Speaker 3: a decade. And then I left to do a startup 93 00:04:37,089 --> 00:04:40,990 Speaker 3: with my friends. And then I joined Apple. I worked 94 00:04:41,089 --> 00:04:46,110 Speaker 3: on iOS security. And then Ty managed to convince me 95 00:04:46,230 --> 00:04:48,690 Speaker 3: to follow the Church of AI. 96 00:04:49,390 --> 00:04:50,550 Speaker 2: And so here I am. 97 00:04:50,810 --> 00:04:54,020 Speaker 5: Yeah, it took me four years to convince Bruce. So, 98 00:04:54,060 --> 00:04:57,260 Speaker 5: you know, when you go to a website or you 99 00:04:57,300 --> 00:05:03,080 Speaker 5: go to Amazon, even Facebook, Instagram, your computer will share 100 00:05:03,110 --> 00:05:06,650 Speaker 5: information with the server, right? Yeah. And that would be, 101 00:05:07,910 --> 00:05:13,050 Speaker 5: the information would travel through many, many other servers, like 102 00:05:13,270 --> 00:05:17,570 Speaker 5: your ISP, your Wi-Fi routers. So you want to encrypt 103 00:05:17,690 --> 00:05:20,310 Speaker 5: that information to make sure that only the server can 104 00:05:20,370 --> 00:05:24,440 Speaker 5: see your data, nobody else. And there's a mechanism for that. 105 00:05:24,510 --> 00:05:28,400 Speaker 5: It's called the SSL protocol, or the later version is 106 00:05:28,440 --> 00:05:34,150 Speaker 5: called like TOS, like transport layer protocol. security protocol. So, um, 107 00:05:34,370 --> 00:05:37,789 Speaker 5: I love those protocols and, um, back in the day, 108 00:05:37,890 --> 00:05:41,989 Speaker 5: together with, uh, friends, we found a bunch of ways 109 00:05:42,070 --> 00:05:46,680 Speaker 5: to break the protocols, you know, to decrypt the data. And, um, 110 00:05:47,200 --> 00:05:49,960 Speaker 5: it's pretty cool. It's actually changed the internet, how the 111 00:05:49,980 --> 00:05:55,140 Speaker 5: internet works, because we basically, um, showed that in the 112 00:05:55,180 --> 00:05:59,990 Speaker 5: past that like two protocols, like SSL Versus 3.0, that 113 00:06:00,029 --> 00:06:04,740 Speaker 5: was invented by Netscape. You remember Netscape? Yeah, yeah. 30 114 00:06:04,740 --> 00:06:08,460 Speaker 5: years ago. And then people still use it because it's 115 00:06:08,580 --> 00:06:12,160 Speaker 5: still working somewhat. But we show that, oh, it's not 116 00:06:12,200 --> 00:06:16,700 Speaker 5: working very correctly. If you use SSL 3.0, then people 117 00:06:16,740 --> 00:06:20,560 Speaker 5: can decrypt your data. So we hope the internet really, 118 00:06:20,650 --> 00:06:26,230 Speaker 5: really removes SSL 3.0. And that's one thing. The other 119 00:06:26,270 --> 00:06:29,810 Speaker 5: thing that I really enjoy doing was, you know, people 120 00:06:29,850 --> 00:06:33,210 Speaker 5: think that in the past you have compression, you have 121 00:06:33,230 --> 00:06:35,609 Speaker 5: some data and you want to compress the data and 122 00:06:35,630 --> 00:06:38,910 Speaker 5: you have encryption. And both of them actually randomize your 123 00:06:38,930 --> 00:06:41,270 Speaker 5: data in some way, right? If you look at a 124 00:06:41,550 --> 00:06:44,979 Speaker 5: zip file, you open a zip file, look at the 125 00:06:45,040 --> 00:06:49,160 Speaker 5: raw binaries, buy data in the zip file, look random 126 00:06:49,180 --> 00:06:54,289 Speaker 5: to you. Because... all the redundant information has been removed 127 00:06:54,310 --> 00:06:56,990 Speaker 5: and encoded in a way to reduce the size of 128 00:06:57,070 --> 00:07:00,770 Speaker 5: the file. And so one of the basic question in 129 00:07:01,890 --> 00:07:06,370 Speaker 5: computer security was, would you compress data and then you 130 00:07:06,410 --> 00:07:11,880 Speaker 5: encrypt or you encrypt then compress? And I took some 131 00:07:12,000 --> 00:07:16,840 Speaker 5: crypto class at Stanford University and I remember my professor, 132 00:07:17,620 --> 00:07:20,340 Speaker 5: he asked this question in one of the homework assignments. 133 00:07:21,340 --> 00:07:25,520 Speaker 5: And the obvious answer back then was you don't want 134 00:07:25,580 --> 00:07:28,800 Speaker 5: to compress and then you don't want to encrypt and 135 00:07:28,820 --> 00:07:33,000 Speaker 5: compress because encryption randomize your data. It means that compression 136 00:07:33,020 --> 00:07:36,070 Speaker 5: would be less effective. So the right answer is you 137 00:07:36,110 --> 00:07:40,090 Speaker 5: compress and then you encrypt. And that was the correct 138 00:07:40,170 --> 00:07:43,270 Speaker 5: answer on my homework assignment. I think I took that 139 00:07:43,350 --> 00:07:48,420 Speaker 5: class in 2011. And then three years later, we found 140 00:07:48,450 --> 00:07:52,210 Speaker 5: that it's wrong. If you do that, then people can 141 00:07:52,270 --> 00:07:55,210 Speaker 5: actually decrypt your data. And the way it works is 142 00:07:55,250 --> 00:07:59,510 Speaker 5: pretty cool. I can explain to you. So compression actually reveals, 143 00:08:00,250 --> 00:08:02,570 Speaker 5: if you look at a compressed file, you know how 144 00:08:02,690 --> 00:08:06,680 Speaker 5: well it is compressed, right? Because it's softer. If it's smaller, 145 00:08:06,960 --> 00:08:09,980 Speaker 5: the compression is very good because there's a lot of 146 00:08:10,000 --> 00:08:14,060 Speaker 5: redundant information. That's why compression works well. It means that 147 00:08:14,200 --> 00:08:16,610 Speaker 5: if you look at a compression alone, the size of 148 00:08:16,650 --> 00:08:21,010 Speaker 5: the compression, a compressed file, you can tell whether the 149 00:08:21,050 --> 00:08:26,710 Speaker 5: file has a lot of redundant information. And encryption hides 150 00:08:26,950 --> 00:08:29,990 Speaker 5: the length. So it doesn't hide the size of the 151 00:08:30,310 --> 00:08:33,480 Speaker 5: encrypted data. When you encrypt a message, you can still 152 00:08:33,580 --> 00:08:36,480 Speaker 5: see that this message is maybe this long or this 153 00:08:36,520 --> 00:08:40,580 Speaker 5: message may be this long, right? Because encryption doesn't try 154 00:08:40,620 --> 00:08:43,280 Speaker 5: to hide that. There's some people try to do that. 155 00:08:44,080 --> 00:08:46,900 Speaker 5: So it means that if you compress and then you encrypt, 156 00:08:47,320 --> 00:08:50,670 Speaker 5: it means that I can see where your message has 157 00:08:50,710 --> 00:08:54,700 Speaker 5: some redundant information, has a lot of redundant information. And 158 00:08:54,720 --> 00:08:57,580 Speaker 5: that alone is a leakage already, right? That you don't 159 00:08:57,600 --> 00:09:00,600 Speaker 5: want to review. And I gave you an example. Let's 160 00:09:00,620 --> 00:09:03,160 Speaker 5: say that you take a message, like a random message 161 00:09:03,179 --> 00:09:06,280 Speaker 5: that I gave you, and then you put your password 162 00:09:06,340 --> 00:09:09,750 Speaker 5: together with that message. Let's just say that there's a 163 00:09:09,790 --> 00:09:11,910 Speaker 5: reason for you to do that. You put your password 164 00:09:11,950 --> 00:09:14,370 Speaker 5: together with the message I gave you and you compress together. 165 00:09:15,130 --> 00:09:18,900 Speaker 5: And then you encrypt it. with a key that I 166 00:09:18,940 --> 00:09:23,700 Speaker 5: don't know. And you reveal the ciphertext, the result encrypted 167 00:09:23,860 --> 00:09:26,620 Speaker 5: message to me. So supposedly I don't have the key. 168 00:09:26,730 --> 00:09:29,650 Speaker 5: I shouldn't be able to read the message, right? But 169 00:09:29,730 --> 00:09:33,470 Speaker 5: because I give you a message and you get my 170 00:09:33,510 --> 00:09:38,250 Speaker 5: message put together with your password, you compress together. If 171 00:09:38,330 --> 00:09:43,130 Speaker 5: the compression works very well, then I know whether my 172 00:09:43,170 --> 00:09:45,850 Speaker 5: message has some redundancy with your password. 173 00:09:46,580 --> 00:09:49,600 Speaker 4: You can reverse engineer between the two relationships. 174 00:09:49,679 --> 00:09:54,040 Speaker 5: Yeah, exactly. And you do that repeatedly. Eventually, we can 175 00:09:54,090 --> 00:09:59,730 Speaker 5: recover your password. And that's exactly what we found on 176 00:09:59,890 --> 00:10:04,820 Speaker 5: the web encryption protocols. In the past, that's exactly what 177 00:10:04,840 --> 00:10:09,180 Speaker 5: they did. They take my data, they concatenate your password, 178 00:10:09,720 --> 00:10:13,340 Speaker 5: they compress together, and then they send over the wire. 179 00:10:14,080 --> 00:10:17,700 Speaker 5: And then as an attacker, I can see the length 180 00:10:18,330 --> 00:10:22,120 Speaker 5: And that's enough for me to decrypt. And the solution 181 00:10:22,200 --> 00:10:26,579 Speaker 5: was really interesting, because they decided to turn off compression. 182 00:10:27,340 --> 00:10:30,699 Speaker 5: There's no solution. And then eventually, I was at Google 183 00:10:30,740 --> 00:10:35,170 Speaker 5: back then. I was born in Saigon, Vietnam. I grew 184 00:10:35,250 --> 00:10:38,590 Speaker 5: up there. Then I got a job in Mountain View 185 00:10:39,070 --> 00:10:43,530 Speaker 5: for a consultancy, like what we do. So I moved 186 00:10:43,550 --> 00:10:46,700 Speaker 5: to Mountain View in 2011. And then I joined Google. 187 00:10:47,550 --> 00:10:53,630 Speaker 5: later that year. And I did cryptography at Google, basically 188 00:10:53,710 --> 00:10:58,410 Speaker 5: help people do cryptography correctly, like try to avoid the 189 00:10:58,450 --> 00:11:02,880 Speaker 5: mistakes that we found. I was there for 12 years. 190 00:11:03,780 --> 00:11:07,700 Speaker 5: I work on like very last number of products. Google 191 00:11:07,760 --> 00:11:11,320 Speaker 5: has a lot of products, right? Gmail, Chrome browser that 192 00:11:11,340 --> 00:11:13,569 Speaker 5: we are using right now. And there's a lot of 193 00:11:13,610 --> 00:11:17,010 Speaker 5: people out there that want to hack Google and our products. 194 00:11:17,030 --> 00:11:17,209 Speaker 2: Yeah. 195 00:11:17,690 --> 00:11:21,490 Speaker 5: So my job was to hack these products before we 196 00:11:21,530 --> 00:11:24,979 Speaker 5: released them to the public. And I was there for 197 00:11:24,980 --> 00:11:30,280 Speaker 5: 12 years and then I left to create Calif. And yeah. 198 00:11:30,440 --> 00:11:34,440 Speaker 1: That's fantastic. I'm already fascinated by this conversation. I want 199 00:11:34,480 --> 00:11:37,230 Speaker 1: to get into obviously what Calif does specifically. I want 200 00:11:37,250 --> 00:11:40,090 Speaker 1: to get into some of the latest vulnerabilities. But before 201 00:11:40,170 --> 00:11:42,110 Speaker 1: we do that, you know, both of you have had 202 00:11:42,830 --> 00:11:47,130 Speaker 1: long careers that by and large predate what we now 203 00:11:47,270 --> 00:11:53,050 Speaker 1: call AI. Are the risks associated with AI a sort 204 00:11:53,100 --> 00:11:58,280 Speaker 1: of continuation on some path that cybersecurity have well understood, 205 00:11:59,020 --> 00:12:01,760 Speaker 1: or are we in a truly novel area for how 206 00:12:01,820 --> 00:12:06,540 Speaker 1: we should we be thinking about cybersecurity risk? 207 00:12:07,200 --> 00:12:10,500 Speaker 3: I think it's a combination of the two. We've always 208 00:12:10,580 --> 00:12:15,069 Speaker 3: known that there are security vulnerabilities in systems. And so 209 00:12:15,110 --> 00:12:18,030 Speaker 3: we have security engineers who are hackers. 210 00:12:17,650 --> 00:12:20,350 Speaker 2: Who try to find and exploit these vulnerabilities. 211 00:12:20,970 --> 00:12:24,109 Speaker 3: So the system has always been vulnerable to some degree, 212 00:12:24,130 --> 00:12:27,550 Speaker 3: to a varying degree. What AI has done is it's 213 00:12:27,610 --> 00:12:30,470 Speaker 3: kind of accelerated the speed in which you identify and 214 00:12:30,510 --> 00:12:34,380 Speaker 3: exploit these vulnerabilities to a level where it's almost like superhuman. 215 00:12:34,660 --> 00:12:37,740 Speaker 3: You know, like 20 years ago, Microsoft would release, let's say, 216 00:12:37,800 --> 00:12:41,960 Speaker 3: maybe 20 updates every month. Now you're looking at like, 217 00:12:42,059 --> 00:12:46,990 Speaker 3: you know, 1,400 updates per month. And that number is 218 00:12:47,010 --> 00:12:49,569 Speaker 3: going to continue to grow. Just to give you an 219 00:12:49,650 --> 00:12:55,329 Speaker 3: idea of the scale of things. And now, for example, 220 00:12:55,570 --> 00:12:59,040 Speaker 3: to have certain systems, it may take you, let's say, 221 00:12:59,140 --> 00:13:02,780 Speaker 3: three months with lots of people. Now, that may take 222 00:13:02,820 --> 00:13:06,160 Speaker 3: you a few days with just one person and a 223 00:13:06,210 --> 00:13:07,670 Speaker 3: very capable AI model. 224 00:13:08,170 --> 00:13:09,530 Speaker 2: That's the reality that we're in today. 225 00:13:09,550 --> 00:13:14,989 Speaker 3: It even changed the way how Attackers have certain types 226 00:13:15,030 --> 00:13:19,270 Speaker 3: of systems, like in the banking sector, that our customers 227 00:13:19,309 --> 00:13:22,010 Speaker 3: are in. You can tell some stories about that time. 228 00:13:22,210 --> 00:13:23,809 Speaker 3: But that's kind of what I think. I think it's 229 00:13:23,830 --> 00:13:26,290 Speaker 3: the systems have always been vulnerable to some degree. 230 00:13:26,510 --> 00:13:31,079 Speaker 2: And now it's the AI just... accelerates that and makes 231 00:13:31,120 --> 00:13:31,960 Speaker 2: it much more visible. 232 00:13:32,480 --> 00:13:35,579 Speaker 3: And it's also made it easier for people to identify 233 00:13:35,620 --> 00:13:37,300 Speaker 3: these vulnerabilities as well. 234 00:13:37,400 --> 00:13:40,390 Speaker 2: And so, and then, you know, obviously there are pros 235 00:13:40,429 --> 00:13:42,510 Speaker 2: and cons to that. But yeah, you can go into 236 00:13:42,530 --> 00:13:42,830 Speaker 2: that later. 237 00:13:58,940 --> 00:14:02,520 Speaker 4: Why don't we talk about discovering the PHP vulnerability then? 238 00:14:02,540 --> 00:14:06,949 Speaker 4: Because I think this was a joint project between Calif, Wiz, 239 00:14:07,230 --> 00:14:10,910 Speaker 4: and I think it was OpenAI. How did you find this? 240 00:14:10,990 --> 00:14:13,090 Speaker 4: What was the process? And then what was the decision? 241 00:14:13,110 --> 00:14:14,689 Speaker 4: I know you guys have gotten a little bit of 242 00:14:14,730 --> 00:14:20,130 Speaker 4: pushback about publicizing it before it was fixed. So what 243 00:14:20,150 --> 00:14:23,630 Speaker 4: was the process of actually putting out the statement saying 244 00:14:23,670 --> 00:14:24,410 Speaker 4: we found this thing? 245 00:14:24,830 --> 00:14:30,080 Speaker 5: We're really interested in finding vulnerabilities that have wide impact. 246 00:14:30,780 --> 00:14:36,210 Speaker 5: And we want to understand these vulnerabilities to help inform decisions, 247 00:14:36,690 --> 00:14:40,710 Speaker 5: defense decisions, how to fix and how to mitigate them 248 00:14:40,750 --> 00:14:45,370 Speaker 5: for once and forever. On that background, we joined the 249 00:14:45,690 --> 00:14:51,760 Speaker 5: OpenAI's Pass the Planet program. This is where OpenAI funds 250 00:14:51,780 --> 00:14:56,640 Speaker 5: companies like Calif to go out and help open source 251 00:14:56,680 --> 00:15:02,820 Speaker 5: maintainers to identify and fix, mitigate vulnerabilities in their software. 252 00:15:03,980 --> 00:15:08,600 Speaker 5: And we have this engineer, like Ryan Osteen, who is, 253 00:15:09,280 --> 00:15:13,080 Speaker 5: he's like a PHP historian. Yesterday, we went to a 254 00:15:13,120 --> 00:15:18,470 Speaker 5: conference in San Francisco. We basically presented like, you know, 255 00:15:18,570 --> 00:15:23,750 Speaker 5: PHP 2 shell, and Ryan walked everybody over like the 256 00:15:23,750 --> 00:15:28,780 Speaker 5: 25 years of history of PHP vulnerabilities. So we have 257 00:15:28,800 --> 00:15:32,810 Speaker 5: this engineer, Ryan, and this is how we work, basically. 258 00:15:34,010 --> 00:15:37,310 Speaker 5: Our engineer, they're interested in certain things, and then they 259 00:15:37,350 --> 00:15:40,130 Speaker 5: just go and hack those things. And, you know, as 260 00:15:40,190 --> 00:15:42,810 Speaker 5: business people, we try to see, okay, how can we 261 00:15:43,270 --> 00:15:46,080 Speaker 5: convince some other people to pay us for this work? 262 00:15:46,680 --> 00:15:50,960 Speaker 5: And that's exactly what we did with OpenAI. We convinced OpenAI, okay, 263 00:15:50,980 --> 00:15:54,980 Speaker 5: we're going to hack PHP. Can you fund our work? 264 00:15:55,590 --> 00:15:58,430 Speaker 5: And they said, yes, they gave us tokens and money. 265 00:15:58,990 --> 00:16:01,990 Speaker 5: And then we go and convince like PHP Foundation, which 266 00:16:02,050 --> 00:16:05,990 Speaker 5: is the organization behind PHP, that let's work together, you know, 267 00:16:06,030 --> 00:16:09,310 Speaker 5: we can help you. And it's actually pretty interesting because 268 00:16:09,330 --> 00:16:12,710 Speaker 5: I mentioned like they're like a history of 25 years 269 00:16:12,870 --> 00:16:17,990 Speaker 5: of like PHP vulnerabilities. And the person that is behind 270 00:16:18,310 --> 00:16:22,110 Speaker 5: all of these hacks is like Stephen Escher. He's also 271 00:16:22,170 --> 00:16:24,880 Speaker 5: at Calif. He joined us, I think, six months ago. 272 00:16:25,600 --> 00:16:28,420 Speaker 5: Back in the day, he's the PHP security guy. Basically, 273 00:16:28,450 --> 00:16:33,390 Speaker 5: he hacked and helped, like, harden PHP. So we didn't 274 00:16:33,430 --> 00:16:37,030 Speaker 5: know anybody at the PHP Foundation. So we asked Stefan 275 00:16:37,130 --> 00:16:40,790 Speaker 5: to introduce us. And because, you know, even though Stefan 276 00:16:41,430 --> 00:16:45,050 Speaker 5: stopped working on PHP security for a while now, his 277 00:16:45,130 --> 00:16:50,090 Speaker 5: name still rings, you know, very... loudly in this circle 278 00:16:50,110 --> 00:16:50,710 Speaker 5: of the world. 279 00:16:50,890 --> 00:16:54,370 Speaker 4: I saw the comic that you guys put up about this. 280 00:16:54,390 --> 00:16:55,730 Speaker 4: It was kind of funny. 281 00:16:55,950 --> 00:16:56,270 Speaker 3: Yes. 282 00:16:56,510 --> 00:16:59,210 Speaker 4: He's like Indiana Jones type character. 283 00:16:59,290 --> 00:17:03,500 Speaker 5: Yeah. So he connected us and then we met with 284 00:17:03,720 --> 00:17:07,939 Speaker 5: Volker Dusch at the PHP Foundation. And yeah, that's how 285 00:17:08,000 --> 00:17:13,320 Speaker 5: we got started. I think we started in July 30. 286 00:17:13,320 --> 00:17:16,380 Speaker 5: And then immediately after one or two days, you know, 287 00:17:16,420 --> 00:17:22,350 Speaker 5: Ryan found the first vulnerability, and he was interested in 288 00:17:22,810 --> 00:17:29,859 Speaker 5: finding a vulnerability similar to one vulnerability that Stefan disclosed 289 00:17:29,859 --> 00:17:35,619 Speaker 5: 22 years ago. And to his surprise, there's still a 290 00:17:35,660 --> 00:17:39,780 Speaker 5: bunch of them like that in the PHP software right now. 291 00:17:41,040 --> 00:17:44,489 Speaker 5: And so that's how we found it. And then we 292 00:17:44,530 --> 00:17:48,200 Speaker 5: reported it to the PHP Foundation. because we know immediately 293 00:17:48,260 --> 00:17:52,450 Speaker 5: that is a critical vulnerability that could affect a large 294 00:17:52,490 --> 00:17:56,210 Speaker 5: part of the internet. Then the Peace Preach Foundation actually 295 00:17:56,290 --> 00:18:00,070 Speaker 5: shared with us that they actually found this vulnerability, the 296 00:18:00,109 --> 00:18:04,179 Speaker 5: same vulnerability, two years ago. But they didn't fix it. 297 00:18:04,960 --> 00:18:09,379 Speaker 5: Because when they found it, they discovered that the fix 298 00:18:09,740 --> 00:18:14,360 Speaker 5: is very expensive. And there's just maybe a few options 299 00:18:14,859 --> 00:18:18,670 Speaker 5: to fix it. and none of them were actually perfect 300 00:18:19,330 --> 00:18:25,870 Speaker 5: because they need to change a lot of functionalities within PHP. 301 00:18:26,790 --> 00:18:32,689 Speaker 5: And those changes may introduce new bugs, new vulnerabilities, or 302 00:18:32,750 --> 00:18:36,570 Speaker 5: new reliability issue. And people would be very upset if 303 00:18:36,650 --> 00:18:42,409 Speaker 5: they upgrade PHP and their website stop working. So in 304 00:18:42,430 --> 00:18:47,780 Speaker 5: the end, back in like 2024, The PHP Foundation decided 305 00:18:47,920 --> 00:18:52,380 Speaker 5: not to fix it because there was no exploit. No 306 00:18:52,520 --> 00:18:55,200 Speaker 5: one could manage to show that this is actually a 307 00:18:55,240 --> 00:18:59,380 Speaker 5: real issue. And that's exactly what they asked us. They 308 00:18:59,420 --> 00:19:01,860 Speaker 5: come back, they ask like, do you have an exploit? 309 00:19:02,880 --> 00:19:05,419 Speaker 5: Then the next day we share with them the first 310 00:19:05,560 --> 00:19:11,000 Speaker 5: exploit that showed them that, yeah, someone doesn't need to 311 00:19:11,060 --> 00:19:13,530 Speaker 5: know anything. can hack your website remotely. 312 00:19:13,730 --> 00:19:16,830 Speaker 4: So just to be clear, the use of AI in 313 00:19:16,930 --> 00:19:20,680 Speaker 4: all of this, it wasn't necessarily that important in finding 314 00:19:20,720 --> 00:19:24,959 Speaker 4: the vulnerability. It was more important in developing an actual 315 00:19:25,080 --> 00:19:27,119 Speaker 4: exploit for a known issue. 316 00:19:27,460 --> 00:19:29,790 Speaker 5: We think like that's probably the right way to think 317 00:19:29,850 --> 00:19:33,149 Speaker 5: about it. But I think even for finding the vulnerability 318 00:19:33,230 --> 00:19:35,730 Speaker 5: without AI, it would take a lot more time, right? 319 00:19:36,550 --> 00:19:41,860 Speaker 5: So you can imagine like PHP was created like maybe 320 00:19:41,859 --> 00:19:45,270 Speaker 5: 30 years ago. So it's a huge code base with 321 00:19:45,330 --> 00:19:49,920 Speaker 5: a lot of features. And as a human engineer, you 322 00:19:49,960 --> 00:19:53,840 Speaker 5: have to dig into this code base and it will 323 00:19:53,859 --> 00:19:56,520 Speaker 5: take like months, right? And along the way, people may 324 00:19:56,560 --> 00:20:00,209 Speaker 5: give up. But the fact that PHP Foundation filed the 325 00:20:00,230 --> 00:20:06,090 Speaker 5: bug in 2024 proved that humans can actually file the bugs, right? 326 00:20:07,070 --> 00:20:11,650 Speaker 5: And the problem was the exploitation, like you said. And 327 00:20:11,830 --> 00:20:15,899 Speaker 5: for exploitation, like AI plays a major role in it. 328 00:20:16,600 --> 00:20:20,180 Speaker 5: It's like becomes an engineering problem where our engineers would 329 00:20:20,220 --> 00:20:22,280 Speaker 5: tell AI, you know, this is how I want to 330 00:20:22,300 --> 00:20:26,880 Speaker 5: design our exploit and you go and do it. And then, 331 00:20:27,060 --> 00:20:30,600 Speaker 5: you know, prove to us that it works correctly. Yeah. 332 00:20:31,000 --> 00:20:35,030 Speaker 1: Let me ask a philosophical question. Is there a reason 333 00:20:35,090 --> 00:20:40,950 Speaker 1: to believe that defensive cybersecurity can stay ahead of offensive cybersecurity, 334 00:20:41,010 --> 00:20:43,170 Speaker 1: just generally, and then we can get into some details. 335 00:20:43,550 --> 00:20:48,530 Speaker 1: But philosophically, is that something that we can reasonably hope for? 336 00:20:49,150 --> 00:20:50,030 Speaker 2: I think so, yes. 337 00:20:50,310 --> 00:20:50,670 Speaker 1: How come? 338 00:20:51,170 --> 00:20:55,670 Speaker 3: Because you can use the AI to build certain types 339 00:20:55,859 --> 00:21:00,810 Speaker 3: of systems in ways that may be very challenging. for 340 00:21:00,930 --> 00:21:06,290 Speaker 3: attackers to exploit. Today, you can use AI to build 341 00:21:06,590 --> 00:21:11,730 Speaker 3: whatever systems you want very quickly. You can experiment, get 342 00:21:11,770 --> 00:21:16,640 Speaker 3: feedback very quickly. It can also process data very, very 343 00:21:16,680 --> 00:21:18,980 Speaker 3: quickly and reason about that data. 344 00:21:19,660 --> 00:21:22,120 Speaker 2: And so I think it is possible. 345 00:21:22,240 --> 00:21:26,400 Speaker 3: I think it may not be immediate, but over time, 346 00:21:26,460 --> 00:21:31,220 Speaker 3: we're going to perhaps use AI to help us write 347 00:21:31,260 --> 00:21:35,680 Speaker 3: more secure software or build more secure systems. And eventually, 348 00:21:35,700 --> 00:21:41,080 Speaker 3: I think that should counterbalance the advantages. 349 00:21:40,680 --> 00:21:45,810 Speaker 2: That the attackers have. Perhaps I'm being optimistic, but I 350 00:21:45,850 --> 00:21:47,369 Speaker 2: do foresee this happening. 351 00:21:47,910 --> 00:21:50,770 Speaker 5: I can add something there. So if you look at 352 00:21:50,790 --> 00:21:54,580 Speaker 5: the history of humanity, our history is the history of 353 00:21:54,650 --> 00:21:59,480 Speaker 5: like weaponize our tools, right? If you find a powerful tool, 354 00:21:59,600 --> 00:22:02,120 Speaker 5: then someone would find a way to use that tool 355 00:22:03,000 --> 00:22:03,520 Speaker 5: as a weapon. 356 00:22:03,820 --> 00:22:04,060 Speaker 4: Yeah. 357 00:22:04,300 --> 00:22:10,040 Speaker 5: So, and attackers always are early adopters of tools, right? 358 00:22:10,540 --> 00:22:16,040 Speaker 5: New technology. Defenders move like at much slower pace than attackers. 359 00:22:16,810 --> 00:22:20,350 Speaker 5: But fundamentally, we believe there are more good guys than 360 00:22:20,410 --> 00:22:25,310 Speaker 5: bad guys out there. And eventually, the good guys would 361 00:22:25,350 --> 00:22:30,990 Speaker 5: catch up and build things that help everybody defend off 362 00:22:31,190 --> 00:22:38,520 Speaker 5: the bad guys. And I don't think in general, cybersecurity, 363 00:22:38,740 --> 00:22:40,460 Speaker 5: if you see the news all the time, you see 364 00:22:40,700 --> 00:22:44,920 Speaker 5: we got data breach every day almost, even before AI. 365 00:22:45,510 --> 00:22:48,190 Speaker 5: So you may feel like it's not working at all. 366 00:22:48,930 --> 00:22:51,450 Speaker 5: But if you're in the field, you can see the progress, right? 367 00:22:51,550 --> 00:22:55,370 Speaker 5: In the past, like, 25 years, there's a tremendous progress 368 00:22:55,430 --> 00:22:58,730 Speaker 5: in cybersecurity. Of course, people still got hacked because this 369 00:22:58,830 --> 00:23:04,150 Speaker 5: is difficult. Imagine you have a very complex system. Like, 370 00:23:04,210 --> 00:23:07,260 Speaker 5: imagine you have a house with lots of doors and 371 00:23:07,280 --> 00:23:11,000 Speaker 5: lots of ways to get in. And cybersecurity must, like, 372 00:23:11,040 --> 00:23:13,160 Speaker 5: make sure that everything is safe. At the same time, 373 00:23:13,220 --> 00:23:16,770 Speaker 5: your house or your Office must remain open for business. 374 00:23:17,609 --> 00:23:20,660 Speaker 5: So it's actually a pretty difficult problem. At the same time, 375 00:23:20,680 --> 00:23:25,060 Speaker 5: without the progress that the industry made in the past 376 00:23:25,060 --> 00:23:29,860 Speaker 5: 25 years, things could be much worse. And I think 377 00:23:29,900 --> 00:23:32,820 Speaker 5: the same thing would be happening here. It's going to 378 00:23:32,859 --> 00:23:37,010 Speaker 5: take a while for defenders to catch up. Right now, 379 00:23:37,510 --> 00:23:40,770 Speaker 5: AI helps attackers more than defenders. 380 00:23:41,540 --> 00:23:44,140 Speaker 4: Is the general idea that we're just in this sort 381 00:23:44,160 --> 00:23:48,730 Speaker 4: of awkward phase where a lot of people in cybersecurity 382 00:23:48,770 --> 00:23:51,850 Speaker 4: always refer to technical debt, right? This idea that you 383 00:23:51,890 --> 00:23:56,170 Speaker 4: haven't upgraded a system for ages, like the PHP vulnerability 384 00:23:56,230 --> 00:23:57,950 Speaker 4: we were just discussing, and so now you have to 385 00:23:57,990 --> 00:24:01,560 Speaker 4: pay back that technical debt. The idea that we're just 386 00:24:01,600 --> 00:24:03,879 Speaker 4: in an awkward moment where AI is going to be 387 00:24:03,960 --> 00:24:08,170 Speaker 4: able to exploit existing flaws in the system is But 388 00:24:08,300 --> 00:24:11,600 Speaker 4: maybe we eventually move to a place where we upgrade 389 00:24:11,660 --> 00:24:12,879 Speaker 4: the entire system. 390 00:24:13,100 --> 00:24:14,820 Speaker 1: We reduce our outstanding technical debt. 391 00:24:14,840 --> 00:24:16,980 Speaker 4: We reduce the outstanding technical debt. Exactly. 392 00:24:17,320 --> 00:24:19,030 Speaker 5: That's an optimistic way to look at it. 393 00:24:19,280 --> 00:24:19,780 Speaker 2: Okay. 394 00:24:20,540 --> 00:24:23,020 Speaker 5: You know, we found that there are systems that we 395 00:24:23,359 --> 00:24:27,370 Speaker 5: thought that, okay, that don't have a lot of technical debt. 396 00:24:27,869 --> 00:24:31,910 Speaker 5: That turned out that they just like dumpster fire like 397 00:24:32,150 --> 00:24:37,260 Speaker 5: everybody else. That was a very big surprise to me personally. 398 00:24:37,800 --> 00:24:41,540 Speaker 5: We know there are places, there are bad problems, and 399 00:24:41,820 --> 00:24:44,619 Speaker 5: we know that if we just point AI at that thing, 400 00:24:44,640 --> 00:24:47,219 Speaker 5: there will be a lot of bugs coming out. But 401 00:24:47,240 --> 00:24:50,520 Speaker 5: there are also places we thought that could be okay 402 00:24:50,740 --> 00:24:55,139 Speaker 5: because they received a lot of attention over the years already. 403 00:24:55,160 --> 00:24:58,060 Speaker 5: But it turns out that there's still a lot of 404 00:24:58,080 --> 00:25:02,180 Speaker 5: bugs there as well. And a lot means a lot more, 405 00:25:02,220 --> 00:25:05,640 Speaker 5: like 10x more than we expected. And of course there 406 00:25:05,660 --> 00:25:10,419 Speaker 5: are still like a few small places where there was 407 00:25:10,880 --> 00:25:13,720 Speaker 5: fewer bugs back then before AI. Now there's still fewer 408 00:25:13,800 --> 00:25:17,500 Speaker 5: bugs still now, but they're like very small, like very 409 00:25:17,640 --> 00:25:21,419 Speaker 5: like lit, like they are the exception, not the norms. 410 00:25:21,840 --> 00:25:26,310 Speaker 5: So that's actually a big surprise because I don't think 411 00:25:26,369 --> 00:25:31,409 Speaker 5: anybody was surprised to find a critical vulnerability in PHP. 412 00:25:32,830 --> 00:25:36,990 Speaker 5: because PHP wasn't known for like race security. So of 413 00:25:37,030 --> 00:25:40,230 Speaker 5: course at the PHP 2.0 scale, then that's a different matter. 414 00:25:40,290 --> 00:25:43,830 Speaker 5: But if you find another vulnerability in PHP today, then 415 00:25:44,090 --> 00:25:47,350 Speaker 5: that's not a surprise. But if you go and, you know, 416 00:25:48,590 --> 00:25:51,929 Speaker 5: find a vulnerability or maybe a hundred vulnerabilities, a thousand 417 00:25:51,950 --> 00:25:55,430 Speaker 5: vulnerabilities in the Chrome browser for example, that's a surprise. 418 00:25:56,250 --> 00:25:59,330 Speaker 5: And that's exactly what happened. Like people found like a 419 00:25:59,350 --> 00:26:03,560 Speaker 5: thousand vulnerabilities in Chrome and you know, I was at 420 00:26:03,580 --> 00:26:07,760 Speaker 5: Google and I knew, and then the entire industry like 421 00:26:07,920 --> 00:26:12,080 Speaker 5: assumed that and knew that, wow, there was a lot 422 00:26:12,100 --> 00:26:16,870 Speaker 5: of investment in Chrome security. And not just like, you know, remove, 423 00:26:17,109 --> 00:26:20,969 Speaker 5: like eliminate vulnerabilities, but making sure that even if you 424 00:26:20,990 --> 00:26:24,470 Speaker 5: have vulnerabilities, it's difficult to exploit these vulnerabilities. This is 425 00:26:24,510 --> 00:26:28,050 Speaker 5: what we call mitigations. You accept that you have vulnerabilities, 426 00:26:28,670 --> 00:26:32,700 Speaker 5: but then you make sure that like attackers couldn't use 427 00:26:32,720 --> 00:26:36,979 Speaker 5: these vulnerabilities to control your computer or your browser. We 428 00:26:37,010 --> 00:26:40,970 Speaker 5: thought that, you know, the defense layers were pretty good 429 00:26:41,670 --> 00:26:47,390 Speaker 5: and the rate of vulnerabilities were low. But then after AI, 430 00:26:47,710 --> 00:26:53,379 Speaker 5: we just discovered that either like we, as security engineers, 431 00:26:54,280 --> 00:26:58,310 Speaker 5: suck at our job. We couldn't find these things. Or 432 00:26:58,690 --> 00:27:01,530 Speaker 5: they're just like too much code that we didn't have 433 00:27:01,609 --> 00:27:02,930 Speaker 5: enough time to look into. 434 00:27:03,350 --> 00:27:08,320 Speaker 1: So this raises another question. Like, is it possible to 435 00:27:08,359 --> 00:27:11,939 Speaker 1: have a provably secure browser or a piece of software? 436 00:27:12,000 --> 00:27:15,080 Speaker 1: You ship the piece of software and you can prove 437 00:27:15,180 --> 00:27:18,300 Speaker 1: maybe in the same way that mathematicians produce things and 438 00:27:18,320 --> 00:27:21,139 Speaker 1: this thing called lean or whatever. Does cyber have an 439 00:27:21,200 --> 00:27:25,219 Speaker 1: equivalent where you could say, This is provably secure. And 440 00:27:25,270 --> 00:27:28,609 Speaker 1: we know that no matter how good AI gets, it 441 00:27:28,650 --> 00:27:31,670 Speaker 1: will never find a vulnerability in this piece of software. 442 00:27:32,470 --> 00:27:35,800 Speaker 2: Oh, man, this is a loaded question, Joe. 443 00:27:36,020 --> 00:27:37,060 Speaker 1: I thought it was a simple question. 444 00:27:38,640 --> 00:27:38,860 Speaker 2: Yeah. 445 00:27:39,940 --> 00:27:42,650 Speaker 3: I don't know, to be honest, you know, like browsers 446 00:27:42,750 --> 00:27:45,330 Speaker 3: are like, you know, its own operating system, like very, 447 00:27:45,369 --> 00:27:49,830 Speaker 3: very complex operating systems. Yeah. And today doing, you know, 448 00:27:49,850 --> 00:27:53,670 Speaker 3: like formally verified software, you know, like it doesn't work 449 00:27:54,180 --> 00:27:58,520 Speaker 3: at that scale. It's typically smaller, smaller software components. 450 00:27:58,960 --> 00:28:02,040 Speaker 1: So there is such thing as formal verification when it 451 00:28:02,119 --> 00:28:03,240 Speaker 1: comes to software. 452 00:28:03,300 --> 00:28:03,540 Speaker 3: Yes. 453 00:28:03,780 --> 00:28:06,090 Speaker 1: You could have small piece of software in which you 454 00:28:06,140 --> 00:28:09,760 Speaker 1: formally prove in the same way that It takes mathematicians 455 00:28:09,800 --> 00:28:13,340 Speaker 1: seven lines to prove the two plus two equals four. 456 00:28:13,840 --> 00:28:17,240 Speaker 1: There is some sort of equivalent for software where you 457 00:28:17,260 --> 00:28:21,639 Speaker 1: can prove these lines of code are secure, but the 458 00:28:22,220 --> 00:28:26,119 Speaker 1: question is the scale of even something like a browser 459 00:28:26,180 --> 00:28:29,050 Speaker 1: is just so enormous. That's a very difficult thing to do. 460 00:28:29,230 --> 00:28:29,470 Speaker 4: Yes. 461 00:28:30,369 --> 00:28:33,990 Speaker 3: It's something that I think is being researched right now 462 00:28:34,350 --> 00:28:38,489 Speaker 3: because I think this is one of the paths forward 463 00:28:38,600 --> 00:28:44,460 Speaker 3: to build more secure systems. For example, we've had systems 464 00:28:44,520 --> 00:28:47,960 Speaker 3: where the core of it is formally verified. It's an 465 00:28:48,000 --> 00:28:51,680 Speaker 3: operating system that's formally verified, but then there are applications 466 00:28:51,720 --> 00:28:56,800 Speaker 3: running on these things. Those applications... that have issues. So 467 00:28:57,420 --> 00:29:01,489 Speaker 3: it's complicated. And these are small systems, right? And you're 468 00:29:01,510 --> 00:29:04,430 Speaker 3: talking about browser. Browser is like many billions of lines 469 00:29:04,470 --> 00:29:07,590 Speaker 3: of code. And so that's, yeah, I mean, I can 470 00:29:07,610 --> 00:29:09,490 Speaker 3: tell you how big Chrome is. I mean, they have 471 00:29:09,530 --> 00:29:12,830 Speaker 3: like zillions of people working on it. And, you know, 472 00:29:12,950 --> 00:29:15,969 Speaker 3: like lots of people working on Chrome security all these years. But, 473 00:29:16,230 --> 00:29:18,530 Speaker 3: you know, it's, but long story short, there are such things, 474 00:29:18,850 --> 00:29:21,230 Speaker 3: but to make it work on a browser or at 475 00:29:21,270 --> 00:29:25,120 Speaker 3: a very, very large system, Like, this is, we don't 476 00:29:25,580 --> 00:29:26,740 Speaker 3: have anything like that today. 477 00:29:26,900 --> 00:29:27,260 Speaker 4: Got it. 478 00:29:27,380 --> 00:29:31,380 Speaker 5: I think there's a deeper reason here. You know, the 479 00:29:31,440 --> 00:29:35,170 Speaker 5: reason like software is eating the world. You can see like, 480 00:29:35,250 --> 00:29:37,830 Speaker 5: you know, the top, the biggest companies in the world 481 00:29:37,850 --> 00:29:42,080 Speaker 5: are software companies, right? It's because software is easy to change. 482 00:29:42,820 --> 00:29:46,700 Speaker 5: And imagine, you know, that allows like software companies to 483 00:29:46,840 --> 00:29:51,380 Speaker 5: adapt and to learn much quicker than the rest of 484 00:29:51,420 --> 00:29:53,700 Speaker 5: the world. You have a product, if that doesn't work, 485 00:29:54,020 --> 00:29:56,900 Speaker 5: then tomorrow, tonight, you can change it. And that's actually 486 00:29:56,940 --> 00:30:00,820 Speaker 5: allows the software world, you know, to become, uh, to 487 00:30:00,840 --> 00:30:05,160 Speaker 5: dominate the world basically. So basically fundamentally software is easy 488 00:30:05,200 --> 00:30:08,400 Speaker 5: to change and people want to change it constantly. And 489 00:30:08,440 --> 00:30:12,270 Speaker 5: that's the reason why, you know, uh, the tech companies 490 00:30:12,330 --> 00:30:15,770 Speaker 5: are doing so well. So if you ask like, okay, 491 00:30:15,910 --> 00:30:20,130 Speaker 5: now let's not change this like frequently, uh, then that 492 00:30:20,190 --> 00:30:21,390 Speaker 5: goes against the interest. 493 00:30:21,470 --> 00:30:21,730 Speaker 2: Right. 494 00:30:22,590 --> 00:30:26,310 Speaker 5: And that's not going to happen. So it means that 495 00:30:26,550 --> 00:30:32,030 Speaker 5: even if you have a way to formally verify everything, 496 00:30:32,210 --> 00:30:33,890 Speaker 5: that would take a lot of time unless you have 497 00:30:34,050 --> 00:30:40,140 Speaker 5: some sort of AI help you do the formal verification automatically. Otherwise, 498 00:30:40,180 --> 00:30:42,520 Speaker 5: it's not going to happen because people want to change 499 00:30:42,560 --> 00:30:46,420 Speaker 5: their software as quick as possible because that's how they 500 00:30:46,900 --> 00:30:47,400 Speaker 5: do business. 501 00:31:03,740 --> 00:31:06,700 Speaker 4: This is sort of a related question, but does AI 502 00:31:06,820 --> 00:31:10,640 Speaker 4: start to change, I guess, the way or like the 503 00:31:10,700 --> 00:31:13,700 Speaker 4: waterfall of the way things get fixed? So nowadays it's 504 00:31:13,720 --> 00:31:18,110 Speaker 4: like a vulnerability is discovered. The software company issues an 505 00:31:18,170 --> 00:31:21,690 Speaker 4: alert and a patch. They disclose it and then people 506 00:31:21,730 --> 00:31:26,190 Speaker 4: go off and fix it. If exploits are coming faster, 507 00:31:26,530 --> 00:31:29,310 Speaker 4: does that process start to change in any way? 508 00:31:29,930 --> 00:31:32,600 Speaker 5: Yeah, it changed a lot. Basically, back in the day, 509 00:31:32,760 --> 00:31:36,940 Speaker 5: we have like two kinds of vulnerabilities, like zero-day vulnerabilities 510 00:31:37,180 --> 00:31:40,080 Speaker 5: that nobody knows about except the person who filed it. 511 00:31:40,820 --> 00:31:46,830 Speaker 5: And then end-day vulnerabilities, the vulnerabilities have been disclosed and 512 00:31:47,050 --> 00:31:50,310 Speaker 5: maybe fixed by the vendors and maybe users have upgraded 513 00:31:50,470 --> 00:31:53,730 Speaker 5: or not. But now we also have something we call 514 00:31:53,810 --> 00:32:00,870 Speaker 5: like minus one-day vulnerabilities. So these vulnerabilities... happen when, you know, 515 00:32:01,410 --> 00:32:06,550 Speaker 5: imagine like, let's take the PHP wannabe as an example. 516 00:32:07,090 --> 00:32:10,340 Speaker 5: Let's say that, you know, because PHP Foundation and PHP 517 00:32:10,410 --> 00:32:15,040 Speaker 5: itself is an open source project, everything happens on GitHub, right? 518 00:32:15,520 --> 00:32:19,280 Speaker 5: And imagine, let's imagine that we reported a bug and 519 00:32:19,520 --> 00:32:25,660 Speaker 5: PHP fixed the bug publicly on GitHub. The moment we 520 00:32:25,740 --> 00:32:31,380 Speaker 5: found the bug and then PHP produce a patch for 521 00:32:31,420 --> 00:32:38,040 Speaker 5: that bug, that very moment, that information became public. Somebody 522 00:32:38,140 --> 00:32:43,260 Speaker 5: who is watching the PHP repository on GitHub, they could 523 00:32:43,420 --> 00:32:48,650 Speaker 5: turn that away into an exploit within a few hours. 524 00:32:49,390 --> 00:32:52,930 Speaker 5: Of course, the vulnerability that we found was more complicated 525 00:32:52,970 --> 00:32:56,590 Speaker 5: than that. But you can imagine that in principle, the 526 00:32:56,630 --> 00:33:01,120 Speaker 5: window between the passwords being developed, it's not ready yet, 527 00:33:01,500 --> 00:33:05,500 Speaker 5: you know, for anybody. And before it's become ready for 528 00:33:05,620 --> 00:33:10,130 Speaker 5: end user to upgrade, there's a window, right? In the past, 529 00:33:10,530 --> 00:33:15,170 Speaker 5: you know, the attacker has to wait until the past 530 00:33:15,210 --> 00:33:18,850 Speaker 5: become available. And then it would take them a long 531 00:33:18,890 --> 00:33:22,530 Speaker 5: time to develop and exploit. So user can have longer 532 00:33:22,590 --> 00:33:27,520 Speaker 5: time to upgrade. Now that window is collapsing. It is 533 00:33:27,580 --> 00:33:31,400 Speaker 5: collapsing everywhere, not just in PHP, right? In Windows, in 534 00:33:31,560 --> 00:33:36,640 Speaker 5: Mac OS, in anything. Because right now there are people 535 00:33:37,100 --> 00:33:41,360 Speaker 5: who have like what we call like end day factory. 536 00:33:41,560 --> 00:33:46,340 Speaker 5: You know, they're just waiting for patches from Microsoft, from Apple, 537 00:33:46,420 --> 00:33:51,780 Speaker 5: from important critical software. Like I'm sure tens of people 538 00:33:51,840 --> 00:33:57,500 Speaker 5: are watching PHP today. After what happened, they're watching PHP, Python, Java, 539 00:33:58,000 --> 00:34:02,130 Speaker 5: all of those important languages. And if there is any 540 00:34:02,210 --> 00:34:05,130 Speaker 5: sign of like, oh, there's a new vulnerability, they're going 541 00:34:05,150 --> 00:34:10,050 Speaker 5: to like, you know, their AI would go pro and then, 542 00:34:10,140 --> 00:34:15,050 Speaker 5: you know, bundling tokens to identify that vulnerability, reproduce it, 543 00:34:15,440 --> 00:34:21,110 Speaker 5: be an exploit for it. So that changed the... the practice, right? 544 00:34:21,330 --> 00:34:21,529 Speaker 2: Yeah. 545 00:34:21,790 --> 00:34:26,030 Speaker 1: I'm curious what you thought after the hugging face incident, 546 00:34:26,110 --> 00:34:28,830 Speaker 1: because after that moment, there were sort of two schools 547 00:34:28,890 --> 00:34:31,069 Speaker 1: of thought that I thought there's, or that I perceived. 548 00:34:31,320 --> 00:34:34,620 Speaker 1: There's one that was like AI creature, there was this 549 00:34:34,640 --> 00:34:38,660 Speaker 1: sort of, I guess maybe the anthropomorphic Asian view, AI creatures, 550 00:34:38,680 --> 00:34:42,799 Speaker 1: they want to escape their cages. They're devious, they're tricksters. 551 00:34:42,860 --> 00:34:45,360 Speaker 1: They want to show, come up with a way to 552 00:34:45,440 --> 00:34:48,359 Speaker 1: cheat on the right answers, but cheat on the way 553 00:34:48,540 --> 00:34:51,100 Speaker 1: that the greater, you know, all these stories. And then 554 00:34:51,120 --> 00:34:54,290 Speaker 1: there are other people that were just like, guys, build 555 00:34:54,330 --> 00:34:57,089 Speaker 1: a stronger sandbox. Stop talking about all this stuff. Just 556 00:34:57,110 --> 00:35:00,080 Speaker 1: build a stronger sandbox. Jensen Wong is kind of in 557 00:35:00,120 --> 00:35:03,020 Speaker 1: that category. I loved his interview with Ezra Klein, where 558 00:35:03,080 --> 00:35:06,520 Speaker 1: he was basically like, what's your problem? Just make models 559 00:35:06,580 --> 00:35:09,580 Speaker 1: that don't hack. He's like, just don't ship it. Fix 560 00:35:09,600 --> 00:35:12,299 Speaker 1: your model. Don't let them hack. And I love that 561 00:35:12,540 --> 00:35:15,279 Speaker 1: because it's just like, just fix it. And the AI 562 00:35:15,300 --> 00:35:17,299 Speaker 1: people are like, no, no, no, we can't. You don't understand. 563 00:35:17,340 --> 00:35:20,050 Speaker 1: We haven't solved alignment. It's very tricky. And he's like, 564 00:35:20,130 --> 00:35:23,589 Speaker 1: just fix the model. And I'm sort of curious, as 565 00:35:23,670 --> 00:35:26,370 Speaker 1: people who have come out of the traditional cyber world, 566 00:35:26,650 --> 00:35:30,319 Speaker 1: but are now in the AI world, Where do you 567 00:35:30,400 --> 00:35:32,779 Speaker 1: land on this question? Can you just like, guys, just 568 00:35:32,840 --> 00:35:35,359 Speaker 1: fix the sandbox, or is there something deeper here? 569 00:35:35,660 --> 00:35:38,200 Speaker 5: I don't think that it's simple to fix the sandbox. 570 00:35:38,430 --> 00:35:41,129 Speaker 5: That's the problem. People think that, you know, maybe people 571 00:35:41,150 --> 00:35:44,090 Speaker 5: are not insecure. They think that there's a simple solution 572 00:35:44,110 --> 00:35:47,670 Speaker 5: to cybersecurity. Just do this thing and you're safe. And 573 00:35:47,690 --> 00:35:50,630 Speaker 5: that's how we have so many problems, you know. Vendors 574 00:35:50,730 --> 00:35:54,210 Speaker 5: go out and tell people, just buy this box, this 575 00:35:54,290 --> 00:35:57,739 Speaker 5: magic box. and put it in your network and you'll 576 00:35:57,760 --> 00:36:00,560 Speaker 5: be safe. No, it's not like that. Even that box 577 00:36:00,680 --> 00:36:04,230 Speaker 5: has problems. If you look into like, there's a famous 578 00:36:04,870 --> 00:36:08,509 Speaker 5: idea in security, like product security is not the same 579 00:36:08,570 --> 00:36:12,710 Speaker 5: as security product. Like security products may not have product security. 580 00:36:13,410 --> 00:36:17,470 Speaker 5: Even like the firewalls that you buy, they may have problems. 581 00:36:17,930 --> 00:36:22,130 Speaker 5: And they do. And what I mean here is, so... 582 00:36:22,700 --> 00:36:25,520 Speaker 5: What we found, like the Hugging Face and OpenAI incident 583 00:36:25,560 --> 00:36:31,040 Speaker 5: was fascinating, right? To watch. And it's difficult. Enterprise security 584 00:36:31,080 --> 00:36:35,400 Speaker 5: is very difficult. Anybody saying like, oh, just buy this thing, 585 00:36:35,460 --> 00:36:38,100 Speaker 5: just do that thing, you're going to be safe. No, 586 00:36:38,420 --> 00:36:43,120 Speaker 5: it's just like very complicated. And the reason is like, 587 00:36:43,320 --> 00:36:45,320 Speaker 5: and then people say, oh, just fix the model. And 588 00:36:45,360 --> 00:36:46,660 Speaker 5: that's a comeback to the- And. 589 00:36:46,620 --> 00:36:48,940 Speaker 1: Just fix the model, just make one that doesn't hack. 590 00:36:49,390 --> 00:36:51,910 Speaker 5: That comes back to the incentive again. And because if 591 00:36:51,930 --> 00:36:56,259 Speaker 5: you look at like how AI companies, and Jensen must 592 00:36:56,300 --> 00:36:58,420 Speaker 5: be careful here, right? Because if AI company is not 593 00:36:58,460 --> 00:37:00,600 Speaker 5: making money, he's not making money as well. 594 00:37:00,620 --> 00:37:01,200 Speaker 2: Yeah, of course. 595 00:37:01,460 --> 00:37:06,600 Speaker 5: And the AI companies, they're making money by selling tokens. 596 00:37:07,060 --> 00:37:09,740 Speaker 5: And most of the tokens are used for coding today, 597 00:37:09,760 --> 00:37:13,489 Speaker 5: like programming, building things, right? And if you build, if 598 00:37:13,530 --> 00:37:15,950 Speaker 5: you want to be really good at coding, you need 599 00:37:16,530 --> 00:37:21,540 Speaker 5: an agent on AI that understands code understand technical problems, 600 00:37:21,880 --> 00:37:26,430 Speaker 5: can debug things, can control computers. But that's basically the 601 00:37:26,469 --> 00:37:30,870 Speaker 5: definition of a hacker. So you cannot really escape this. 602 00:37:30,890 --> 00:37:33,509 Speaker 5: You cannot have a cake and eat it too, right? 603 00:37:33,640 --> 00:37:36,620 Speaker 5: You need to be good at coding. Then you have 604 00:37:36,640 --> 00:37:41,000 Speaker 5: an AI that is good at hacking. So unless you 605 00:37:41,020 --> 00:37:44,279 Speaker 5: don't want spend more tokens, or you don't want to 606 00:37:44,320 --> 00:37:49,200 Speaker 5: get into the coding business anymore, you focus on scientific discovery, 607 00:37:49,719 --> 00:37:55,880 Speaker 5: physical AI, whatever, then maybe you can train a model 608 00:37:56,060 --> 00:38:00,149 Speaker 5: that is not good at hacking anymore. But as long 609 00:38:00,310 --> 00:38:06,070 Speaker 5: as you want to automate knowledge work, then eventually your 610 00:38:06,530 --> 00:38:09,110 Speaker 5: AI must be able to use computers and control computers 611 00:38:09,130 --> 00:38:13,130 Speaker 5: to understand technical problems. And that's the reason of someone 612 00:38:13,150 --> 00:38:16,609 Speaker 5: like Bruce. So it's unavoidable. 613 00:38:17,050 --> 00:38:19,649 Speaker 4: So actually, on this note, I'm curious, can you just 614 00:38:19,690 --> 00:38:23,489 Speaker 4: talk about what it's like being a hacker in the 615 00:38:23,610 --> 00:38:28,600 Speaker 4: age of AI versus, say, even three or four years ago, right? Like, 616 00:38:28,680 --> 00:38:31,700 Speaker 4: it's crazy how quickly stuff has shifted. But how has 617 00:38:31,739 --> 00:38:33,420 Speaker 4: your day to day changed? 618 00:38:34,160 --> 00:38:37,270 Speaker 3: Yeah, I'll just use some background. So I wrote a 619 00:38:37,310 --> 00:38:41,750 Speaker 3: textbook with some friends on reverse engineering. It used to 620 00:38:41,770 --> 00:38:44,290 Speaker 3: be like, you know, like you try to like reversing 621 00:38:44,350 --> 00:38:49,110 Speaker 3: is the practice of understanding software without any source code 622 00:38:49,130 --> 00:38:49,880 Speaker 3: or hardware systems. 623 00:38:50,730 --> 00:38:54,029 Speaker 2: That used to be like, I wrote this textbook. And 624 00:38:54,050 --> 00:38:55,130 Speaker 2: then I think with the. 625 00:38:55,070 --> 00:38:57,410 Speaker 3: Advent of AI, I think what, like, you know, whatever, 626 00:38:57,770 --> 00:39:00,450 Speaker 3: nine months ago, as of nine months ago, like that 627 00:39:00,730 --> 00:39:03,029 Speaker 3: entire book is deprecating. It takes like, you know, two 628 00:39:03,090 --> 00:39:05,270 Speaker 3: prompts or one prompt and then bam, it will generate 629 00:39:05,290 --> 00:39:08,040 Speaker 3: the textbook for you. And it's even better, you know, 630 00:39:08,100 --> 00:39:09,879 Speaker 3: and it took us, you know, a long time to write. 631 00:39:10,560 --> 00:39:13,060 Speaker 3: Just that gives you an idea. And then I talk 632 00:39:13,080 --> 00:39:16,219 Speaker 3: to friends who've been doing this for decades, right? 633 00:39:16,239 --> 00:39:17,719 Speaker 2: They're like specialists in this area, right? 634 00:39:18,560 --> 00:39:20,680 Speaker 3: And they tell me that, oh yeah, man, like I spend, 635 00:39:20,719 --> 00:39:24,219 Speaker 3: you know, whatever, $ 5 on some open source model and 636 00:39:24,260 --> 00:39:27,810 Speaker 3: it reverse engineered the firmware on, you know, some Canon 637 00:39:27,930 --> 00:39:31,180 Speaker 3: or like you know, HP printer and it hacks it, Bruce. 638 00:39:31,300 --> 00:39:33,140 Speaker 3: You know, and whereas before it would take me, you know, 639 00:39:33,180 --> 00:39:35,390 Speaker 3: like months to figure some stuff out. 640 00:39:35,410 --> 00:39:38,090 Speaker 2: You know, that's just to give. 641 00:39:37,969 --> 00:39:43,430 Speaker 3: You like a glimpse into what's happening, like how quickly 642 00:39:43,469 --> 00:39:44,140 Speaker 3: things change. 643 00:39:44,880 --> 00:39:48,700 Speaker 2: Well, a friend, you know, like this guy, Matt Swish, he's, 644 00:39:48,719 --> 00:39:49,939 Speaker 2: you know, and he's this dude. 645 00:39:50,120 --> 00:39:52,060 Speaker 4: He's a friend of the show. 646 00:39:52,300 --> 00:39:54,960 Speaker 1: A longtime friend of the show, multi-time guest. 647 00:39:55,000 --> 00:39:55,360 Speaker 5: Yeah, keep going. 648 00:39:55,400 --> 00:39:58,719 Speaker 4: We used to hang out in Abu Dhabi a lot. 649 00:39:58,910 --> 00:40:01,070 Speaker 2: Oh, okay, yeah. Like, I'm no Matt Fawkes. 650 00:40:01,130 --> 00:40:03,950 Speaker 3: I mean, like, now he's like, he gave a talk 651 00:40:04,050 --> 00:40:06,130 Speaker 3: with our team the other day, and he showed his rack, 652 00:40:06,150 --> 00:40:11,690 Speaker 3: you know, in his apartment, and he's sitting there, you know, like, 653 00:40:11,790 --> 00:40:17,060 Speaker 3: hacking stuff, like, while he's sitting there talking. It's just insane. 654 00:40:17,620 --> 00:40:20,259 Speaker 3: We have people that, let's say, for example, we've been 655 00:40:20,280 --> 00:40:24,320 Speaker 3: hacking the iPhone. To build a full chain on the iPhone, 656 00:40:24,800 --> 00:40:28,120 Speaker 3: it's not an easy task. Sometimes it takes several people 657 00:40:28,400 --> 00:40:33,080 Speaker 3: to do this. And our team was able to do 658 00:40:33,120 --> 00:40:36,299 Speaker 3: this within. 659 00:40:35,600 --> 00:40:36,120 Speaker 2: A few days. 660 00:40:36,750 --> 00:40:37,050 Speaker 3: Wow. 661 00:40:37,090 --> 00:40:39,330 Speaker 2: You know, as a one person. And we have multiple 662 00:40:39,350 --> 00:40:39,950 Speaker 2: of these chains. 663 00:40:40,550 --> 00:40:45,719 Speaker 3: It really compresses the time, like the exploitation, the time 664 00:40:45,760 --> 00:40:48,600 Speaker 3: to build exploits. 665 00:40:49,960 --> 00:40:52,339 Speaker 2: We see customers getting hacked. 666 00:40:52,520 --> 00:40:56,740 Speaker 3: By AI, you know, in the financials, in the banks, basically. 667 00:40:56,760 --> 00:41:00,109 Speaker 3: You know, defenders cannot, you know, actually, Ty, why don't 668 00:41:00,130 --> 00:41:01,650 Speaker 3: you tell these stories? 669 00:41:01,730 --> 00:41:03,450 Speaker 2: And, you know, just a little bit, this has happened 670 00:41:03,469 --> 00:41:06,650 Speaker 2: in the last few weeks, you know. Yeah. how things changed. 671 00:41:07,190 --> 00:41:10,590 Speaker 5: Yeah, basically we saw like, you know, in the OpenAI 672 00:41:10,630 --> 00:41:15,220 Speaker 5: and HackingFace incident, there's one detail that is very interesting. 673 00:41:15,239 --> 00:41:19,360 Speaker 5: I'm not sure if people noticed, but OpenAI noticed that 674 00:41:19,560 --> 00:41:23,880 Speaker 5: like the people who handled incidents on the OpenAI side, 675 00:41:23,900 --> 00:41:26,010 Speaker 5: they said that they have handled a lot of incidents, 676 00:41:26,070 --> 00:41:28,750 Speaker 5: but this is the first time they saw that, you know, 677 00:41:28,770 --> 00:41:33,820 Speaker 5: an incident where you have like a hundred people points 678 00:41:34,000 --> 00:41:37,430 Speaker 5: within your networks that were compromised at the same time. 679 00:41:38,150 --> 00:41:41,170 Speaker 5: Like there were a hundred like hackers within your system. 680 00:41:42,050 --> 00:41:44,989 Speaker 5: Normally you have maybe one or two, right? Now you 681 00:41:45,030 --> 00:41:47,410 Speaker 5: have a hundred. And I saw that I was like, wow, 682 00:41:47,450 --> 00:41:51,000 Speaker 5: that was pretty interesting, but maybe because OpenAI have lots 683 00:41:51,020 --> 00:41:54,420 Speaker 5: of tokens, so their agents can just go, you know, 684 00:41:54,480 --> 00:41:58,299 Speaker 5: hack everything. And then just two weeks ago, we have 685 00:41:58,400 --> 00:42:02,489 Speaker 5: like two clients in the banking industry. And they got 686 00:42:02,530 --> 00:42:04,830 Speaker 5: hacked and they called us to help them with like 687 00:42:05,030 --> 00:42:08,750 Speaker 5: the incident response. We came in because we do hacking 688 00:42:08,810 --> 00:42:11,049 Speaker 5: as well. We know like usually it took us like 689 00:42:11,090 --> 00:42:13,739 Speaker 5: maybe a month or something. Once we come in a 690 00:42:13,760 --> 00:42:16,440 Speaker 5: network and then took a month for us to spread out. 691 00:42:17,180 --> 00:42:19,239 Speaker 5: For the AI, it took like, they came in, they 692 00:42:19,280 --> 00:42:22,780 Speaker 5: brought inside the network on Monday. By Tuesday, they're everywhere. 693 00:42:23,440 --> 00:42:23,820 Speaker 3: Everywhere. 694 00:42:24,219 --> 00:42:27,160 Speaker 5: And like, it's like cancer is spread like at a 695 00:42:27,180 --> 00:42:30,680 Speaker 5: machine speed. And it was like mind blowing, right? Because 696 00:42:30,739 --> 00:42:33,540 Speaker 5: how could you defend against that? Like in the past, 697 00:42:33,940 --> 00:42:36,920 Speaker 5: we were telling our clients, you know, you have to 698 00:42:36,980 --> 00:42:41,279 Speaker 5: invest in detection. Well, how can you help yourself? If 699 00:42:41,630 --> 00:42:43,750 Speaker 5: you see it, you can detect it, but there's a 700 00:42:43,790 --> 00:42:44,310 Speaker 5: hundred of them. 701 00:42:45,270 --> 00:42:45,690 Speaker 3: Noted. 702 00:42:45,890 --> 00:42:48,430 Speaker 4: We have a hundred hackers in our system right now. 703 00:42:48,489 --> 00:42:49,370 Speaker 4: It's not really helpful. 704 00:42:49,570 --> 00:42:50,010 Speaker 3: Exactly. 705 00:42:50,130 --> 00:42:51,650 Speaker 5: How could you do it, right? And then you have 706 00:42:51,670 --> 00:42:53,770 Speaker 5: to shut down everything. But shut down is not an 707 00:42:53,850 --> 00:42:56,890 Speaker 5: option because you are a bank. So that's actually true. 708 00:42:57,260 --> 00:43:00,520 Speaker 5: requires us to sit down and think about how to 709 00:43:00,540 --> 00:43:04,420 Speaker 5: redesign the defense system. What we have right now is 710 00:43:04,460 --> 00:43:08,100 Speaker 5: not working very well. But to come back to your question, 711 00:43:08,180 --> 00:43:12,370 Speaker 5: as a hacker, how does it feel? I love history. 712 00:43:12,850 --> 00:43:15,730 Speaker 5: So I always come back to history to see, did 713 00:43:15,770 --> 00:43:21,250 Speaker 5: it happen before? It actually did happen with chess, for example. 714 00:43:21,670 --> 00:43:26,410 Speaker 5: You remember in 1997, You know, IBM Deep Blue defeated 715 00:43:27,010 --> 00:43:31,910 Speaker 5: Garry Kasparov. That was a big moment, right? And then 716 00:43:31,910 --> 00:43:36,230 Speaker 5: 20 years later, AlphaGo defeated like Lizzie Doe in the 717 00:43:36,250 --> 00:43:40,020 Speaker 5: game of Go. But those two moments are actually quite different. 718 00:43:40,700 --> 00:43:44,780 Speaker 5: Like the first moment, the Deep Blue moment, after Deep Blue, 719 00:43:45,380 --> 00:43:49,160 Speaker 5: the best chess player wasn't a computer because Deep Blue 720 00:43:49,219 --> 00:43:52,450 Speaker 5: was a mainframe, right? Not something that you can run 721 00:43:52,510 --> 00:43:56,489 Speaker 5: on your computer. And it took 10 years for someone 722 00:43:56,570 --> 00:44:00,509 Speaker 5: to go from Deep Blue to develop a computer software 723 00:44:00,790 --> 00:44:03,590 Speaker 5: that you can run on your computer that defeated everybody. 724 00:44:04,850 --> 00:44:07,350 Speaker 5: But the 10 years, it took 10 years. During that 725 00:44:07,350 --> 00:44:10,359 Speaker 5: 10 years, the best chess player was a centaur player. 726 00:44:10,739 --> 00:44:13,700 Speaker 5: This is a term coined by Garry Kasparov. It's a 727 00:44:13,719 --> 00:44:19,239 Speaker 5: combination of computer and human. It took 10 years. For AlphaGo, 728 00:44:19,360 --> 00:44:24,160 Speaker 5: it took like three years. Like after AlphaGo, Then 2019, 729 00:44:24,160 --> 00:44:27,920 Speaker 5: somebody come up with a computer software that defeated anybody 730 00:44:28,219 --> 00:44:32,280 Speaker 5: without any human input at all. At Calif, we think like, 731 00:44:32,340 --> 00:44:36,710 Speaker 5: you know, we are enter the era of the central hackers. Basically, 732 00:44:36,750 --> 00:44:40,590 Speaker 5: the best hacker right now is a combination of AI 733 00:44:40,670 --> 00:44:44,370 Speaker 5: and humans. And, you know, our work and our friends' 734 00:44:44,430 --> 00:44:48,190 Speaker 5: work as well show that AI is powerful. It can 735 00:44:48,210 --> 00:44:53,180 Speaker 5: hack anything. but it still requires a human expert to 736 00:44:53,200 --> 00:44:55,960 Speaker 5: arrive at it. And we don't know how long this 737 00:44:55,980 --> 00:45:00,920 Speaker 5: is going to last, maybe three months. Maybe we come 738 00:45:00,940 --> 00:45:04,400 Speaker 5: back here and by the end of the year, we 739 00:45:04,420 --> 00:45:06,880 Speaker 5: tell you we retire now. There's nothing else for us 740 00:45:06,920 --> 00:45:10,770 Speaker 5: to do. Maybe it lasts like three years. And this 741 00:45:10,790 --> 00:45:13,270 Speaker 5: is actually pretty exciting to find out. That's exactly what 742 00:45:13,310 --> 00:45:17,210 Speaker 5: we want to do. And we think like, That's like 743 00:45:17,230 --> 00:45:21,710 Speaker 5: a pessimistic view, right? AI is going to do everything. 744 00:45:22,550 --> 00:45:26,569 Speaker 5: Let's just retire and hate AI. Or you can see 745 00:45:26,630 --> 00:45:30,760 Speaker 5: AI as a tool. It's like humanity just discovered fire. 746 00:45:31,640 --> 00:45:35,020 Speaker 5: And without fire, we wouldn't be here today. So, of course, 747 00:45:35,080 --> 00:45:37,960 Speaker 5: we have to think about how to control fire. At 748 00:45:38,000 --> 00:45:40,419 Speaker 5: the same time, a very important question that we need 749 00:45:40,440 --> 00:45:45,250 Speaker 5: to ask is, what should we cook, right? That's very important, 750 00:45:45,370 --> 00:45:47,750 Speaker 5: and we want to find out What can we cook 751 00:45:47,770 --> 00:45:51,050 Speaker 5: with AI and how can we do it in a 752 00:45:51,090 --> 00:45:53,150 Speaker 5: safe way that helps everybody? 753 00:45:53,450 --> 00:45:55,370 Speaker 4: Protein that makes our brains get bigger. 754 00:45:55,469 --> 00:45:56,830 Speaker 3: Yeah. All right. 755 00:45:56,890 --> 00:45:59,660 Speaker 4: Well, Ty and Bruce from Calif, thank you so much 756 00:45:59,680 --> 00:46:00,839 Speaker 4: for coming on All Thoughts. 757 00:46:00,880 --> 00:46:03,660 Speaker 1: That was a lot of fun. You're both excellent. Thank 758 00:46:03,700 --> 00:46:04,120 Speaker 1: you both. 759 00:46:04,400 --> 00:46:04,720 Speaker 5: Thanks. 760 00:46:05,040 --> 00:46:06,000 Speaker 2: Thanks, Joe. Thanks, Tracy. 761 00:46:06,060 --> 00:46:06,360 Speaker 5: Thank you. 762 00:46:06,380 --> 00:46:06,680 Speaker 2: Bye-bye. 763 00:46:06,700 --> 00:46:10,040 Speaker 3: Bye-bye. 764 00:46:19,600 --> 00:46:23,980 Speaker 4: Joe, I thought that was a fantastic conversation. Not exactly uplifting. 765 00:46:24,440 --> 00:46:29,439 Speaker 4: I was surprised at Ty's response. I thought maybe people 766 00:46:29,480 --> 00:46:32,040 Speaker 4: would say we're in this weird, awkward moment where a 767 00:46:32,080 --> 00:46:35,569 Speaker 4: lot of legacy issues are being discovered. But once AI 768 00:46:35,780 --> 00:46:39,290 Speaker 4: actually gets deployed and starts redesigning some of the architecture, 769 00:46:39,810 --> 00:46:42,890 Speaker 4: it'll be a lot more resilient. Apparently that's not the case. 770 00:46:42,930 --> 00:46:44,110 Speaker 4: That's a little bit worrying. 771 00:46:44,410 --> 00:46:46,790 Speaker 1: I agree. There was so much in there that was interesting. 772 00:46:46,989 --> 00:46:50,450 Speaker 1: And like I could, you know, all this stuff about how, yes, 773 00:46:50,469 --> 00:46:53,440 Speaker 1: you could have provably secure software, but it's small. You 774 00:46:53,460 --> 00:46:55,740 Speaker 1: can't even do it for something that says the browser. 775 00:46:56,219 --> 00:46:58,719 Speaker 1: I love that last point about how we're in the 776 00:46:58,800 --> 00:47:02,900 Speaker 1: centaur age where the strongest hacker is the computer working 777 00:47:02,920 --> 00:47:05,760 Speaker 1: with a human, but that may not be there for 778 00:47:05,800 --> 00:47:08,150 Speaker 1: much longer. Three months, a year. We have no idea, 779 00:47:08,230 --> 00:47:12,190 Speaker 1: but the presumption should be that it won't last very long. etc. 780 00:47:12,469 --> 00:47:16,239 Speaker 4: Also, the idea of like a computer, an agent that 781 00:47:16,280 --> 00:47:21,160 Speaker 4: can code is inherently a hacker. And so you can't 782 00:47:21,200 --> 00:47:23,590 Speaker 4: just tell it not to hack. 783 00:47:23,610 --> 00:47:25,430 Speaker 1: Just teach it not to hack. What's wrong with you? 784 00:47:25,469 --> 00:47:28,129 Speaker 1: Just teach it not to hack. If coding is going 785 00:47:28,150 --> 00:47:30,400 Speaker 1: to be big business, which it is, you're going to 786 00:47:30,420 --> 00:47:31,460 Speaker 1: be building a lot of good hackers. 787 00:47:31,520 --> 00:47:31,840 Speaker 5: Yeah. 788 00:47:31,980 --> 00:47:34,759 Speaker 4: And so the hackers proliferate. On that note, shall we 789 00:47:34,780 --> 00:47:35,120 Speaker 4: leave it there? 790 00:47:35,200 --> 00:47:35,700 Speaker 1: Let's leave it there. 791 00:47:35,940 --> 00:47:38,360 Speaker 4: This has been another episode of the All Thoughts Podcast. 792 00:47:38,460 --> 00:47:41,180 Speaker 4: I'm Traci Allaway. You can follow me at Traci Allaway. 793 00:47:41,360 --> 00:47:44,430 Speaker 1: And I'm Joe Wiesenthal. You can follow me at The Stalwart. 794 00:47:44,510 --> 00:47:48,109 Speaker 1: Follow our producers, Carmen Rodriguez at Carmen Arman, Dashiell Bennett 795 00:47:48,210 --> 00:47:51,370 Speaker 1: at Dashbot, Cale Brooks at Cale Brooks. and Kevin Lozano 796 00:47:51,489 --> 00:47:54,069 Speaker 1: at Kevin Lloyd Lozano. And of course, you can chat 797 00:47:54,090 --> 00:47:58,469 Speaker 1: about all of these topics 24-7 in our Discord, discord.gg 798 00:47:58,510 --> 00:48:01,430 Speaker 1: slash oddlots. And for more Oddlots content, you can check 799 00:48:01,469 --> 00:48:04,339 Speaker 1: out our daily newsletter, which you can find at bloomberg.com 800 00:48:04,380 --> 00:48:05,110 Speaker 1: slash oddlots. 801 00:48:05,280 --> 00:48:07,480 Speaker 4: And if you enjoyed this conversation, if you like it 802 00:48:07,540 --> 00:48:09,560 Speaker 4: when we talk about hacking, then please leave us a 803 00:48:09,620 --> 00:48:12,670 Speaker 4: positive review on your favorite podcast platform. And if you're 804 00:48:12,690 --> 00:48:15,510 Speaker 4: watching on video, then please leave a comment or like, 805 00:48:15,710 --> 00:48:17,110 Speaker 4: or better yet, subscribe. 806 00:48:17,350 --> 00:48:19,890 Speaker 1: And remember, if you're a Bloomberg subscriber, you can listen 807 00:48:19,930 --> 00:48:22,990 Speaker 1: to all of our episodes absolutely ad-free. All you need 808 00:48:23,010 --> 00:48:25,700 Speaker 1: to do is find the Bloomberg channel on Apple Podcasts 809 00:48:26,239 --> 00:48:27,480 Speaker 1: and follow the instructions there. 810 00:48:27,980 --> 00:48:45,859 Speaker 4: Thanks for joining. Thank you.