1 00:00:00,880 --> 00:00:05,040 S1: Unsupervised Learning is a podcast about trends and ideas in cybersecurity, 2 00:00:05,080 --> 00:00:09,960 S1: national security, AI, technology and society, and how best to 3 00:00:10,000 --> 00:00:17,520 S1: upgrade ourselves to be ready for what's coming. So I 4 00:00:17,520 --> 00:00:19,640 S1: want to talk today about how to think about AI 5 00:00:19,640 --> 00:00:23,200 S1: and cybersecurity, and specifically how to think about and what 6 00:00:23,239 --> 00:00:27,360 S1: to build regarding AI for cybersecurity. There are a million 7 00:00:27,360 --> 00:00:30,159 S1: different directions you could go, and you only have so 8 00:00:30,160 --> 00:00:32,159 S1: much time and so many resources. So I want to 9 00:00:32,159 --> 00:00:35,400 S1: give a possible direction here. So I want to frame 10 00:00:35,400 --> 00:00:39,760 S1: everything today by asking and answering two questions. One, how 11 00:00:39,760 --> 00:00:43,440 S1: does software and security change when we add agents, and 12 00:00:43,440 --> 00:00:46,800 S1: how should we prioritize our efforts on adding AI and 13 00:00:46,800 --> 00:00:50,120 S1: agents to our cybersecurity program? So these are the main 14 00:00:50,120 --> 00:00:52,279 S1: ideas I want to talk about that. I'll kind of 15 00:00:52,320 --> 00:00:55,600 S1: bring us around to answering those questions, and these ideas 16 00:00:55,600 --> 00:00:57,680 S1: build on each other. So we're going to take one 17 00:00:57,680 --> 00:01:00,600 S1: at a time. The first one is the concept of 18 00:01:00,760 --> 00:01:04,200 S1: intelligence pipelines, which is a way to visualize workflows within 19 00:01:04,200 --> 00:01:07,440 S1: a business. Second one is Theory of Constraints, which talks 20 00:01:07,440 --> 00:01:10,759 S1: about how systems struggle to do things at scale and 21 00:01:10,800 --> 00:01:13,560 S1: like where the blocker is. The third one is something 22 00:01:13,560 --> 00:01:16,560 S1: I call AI state management, which is how I see 23 00:01:16,680 --> 00:01:21,080 S1: I actually replacing most software. And the final one builds 24 00:01:21,080 --> 00:01:24,640 S1: on all of those. And that's the AI security attack 25 00:01:24,640 --> 00:01:28,319 S1: and defense framework. So let's start with something I call 26 00:01:28,319 --> 00:01:32,720 S1: intelligence pipelines. So I started thinking about intelligence pipelines in 27 00:01:32,720 --> 00:01:37,880 S1: the context of how AI will replace human workers. So 28 00:01:37,880 --> 00:01:43,480 S1: these are explainable visualized workflows that show how business processes 29 00:01:43,480 --> 00:01:47,360 S1: require human level intelligence. So imagine like you have someone 30 00:01:47,360 --> 00:01:50,600 S1: named Mark and they work at a company called claim. Right. 31 00:01:50,600 --> 00:01:53,040 S1: And this is what his day looks like, right. He 32 00:01:53,040 --> 00:01:55,480 S1: does all these different tasks that you see in this diagram. 33 00:01:56,200 --> 00:02:00,320 S1: The brain icons are where his human action is required. 34 00:02:00,320 --> 00:02:02,760 S1: And let's say this is a world without AI agents. 35 00:02:02,760 --> 00:02:05,280 S1: So this is where this is human work. This is 36 00:02:05,280 --> 00:02:08,360 S1: things that only humans can do. And Mark is a 37 00:02:08,360 --> 00:02:12,400 S1: really good employee because he could do 124 of these 38 00:02:12,400 --> 00:02:17,079 S1: claims per week, and he has an 87% quality score. 39 00:02:17,160 --> 00:02:19,680 S1: And it's really hard to find people like Mark. So 40 00:02:19,680 --> 00:02:21,600 S1: he makes a lot of money. They give him nice 41 00:02:21,600 --> 00:02:25,960 S1: perks because they don't want Mark to leave, because 124 42 00:02:25,960 --> 00:02:31,680 S1: claims a week 87% quality. So that's insurance. This one's 43 00:02:31,680 --> 00:02:35,359 S1: for a medical company. This is called Badspot. And Badspot 44 00:02:35,360 --> 00:02:40,639 S1: is a company that reviews moles in person using licensed dermatologists. Now, 45 00:02:40,680 --> 00:02:44,120 S1: Kim is one of the dermatologists here, and she sees 46 00:02:44,160 --> 00:02:49,560 S1: 212 patients a week, has 92% accuracy. And again, it's 47 00:02:49,560 --> 00:02:52,320 S1: really hard to find employees as good as Kim. And 48 00:02:52,320 --> 00:02:54,560 S1: the key point here is that the only reason we 49 00:02:54,560 --> 00:02:59,200 S1: have jobs at all is because of these blue icons, right? 50 00:02:59,680 --> 00:03:03,640 S1: These blue brain icons are the specialty. They are the 51 00:03:03,680 --> 00:03:09,240 S1: geniuses of like general human intelligence that somebody can do 52 00:03:09,639 --> 00:03:12,560 S1: what Mark could do and Kim can do. Right? Is 53 00:03:12,560 --> 00:03:16,200 S1: it a new spot? Analyze the mole. Was it dangerous before? 54 00:03:16,200 --> 00:03:18,720 S1: You have to do these manual checks. This is human work. 55 00:03:18,720 --> 00:03:21,800 S1: This is why we all have jobs, right? Otherwise, this 56 00:03:21,800 --> 00:03:24,600 S1: would have just been like a script or automation. We've 57 00:03:24,600 --> 00:03:27,840 S1: had programming and automation for years and years and years, decades. 58 00:03:28,639 --> 00:03:30,960 S1: But there are certain things that only humans can do, 59 00:03:30,960 --> 00:03:33,119 S1: and that's what these blue icons are. So I just 60 00:03:33,120 --> 00:03:36,840 S1: want to stress that again, the only reason we have 61 00:03:36,840 --> 00:03:40,880 S1: jobs is because somebody is paying us to be one 62 00:03:40,880 --> 00:03:44,520 S1: of these blue icons. We as humans are inside of 63 00:03:44,520 --> 00:03:47,000 S1: a workflow that looks like this. And I'm trying to 64 00:03:47,000 --> 00:03:49,440 S1: get you to think of your job in terms of 65 00:03:49,440 --> 00:03:52,440 S1: these workflows, because this is exactly how AI is going 66 00:03:52,480 --> 00:03:55,190 S1: to think about your job, right when it comes to 67 00:03:55,230 --> 00:03:59,150 S1: optimize it. Air quotes optimize. We don't normally think of 68 00:03:59,230 --> 00:04:03,310 S1: our jobs or our pipelines or our workflows in terms 69 00:04:03,310 --> 00:04:06,470 S1: of a visual flow like this. This is exactly how 70 00:04:06,510 --> 00:04:10,390 S1: McKinsey is going to think about it, or KPMG or 71 00:04:10,430 --> 00:04:15,150 S1: any random consultancy that comes in to optimize air quotes. 72 00:04:15,750 --> 00:04:19,789 S1: Your company and your department and your team and your workflows. 73 00:04:20,270 --> 00:04:22,390 S1: They're going to say, okay, tell me exactly what it 74 00:04:22,390 --> 00:04:25,349 S1: is that you actually do. And they're going to produce 75 00:04:25,350 --> 00:04:27,590 S1: some sort of visual. It's going to look something like 76 00:04:27,589 --> 00:04:32,550 S1: this for your job. Your team's job, your department. And 77 00:04:32,589 --> 00:04:35,710 S1: actually a collection of them will be your company. So 78 00:04:35,710 --> 00:04:37,670 S1: you have to start thinking about this. That's why this 79 00:04:37,670 --> 00:04:40,430 S1: is the first concept here. So this one's for a 80 00:04:40,430 --> 00:04:44,109 S1: military intelligence company. So you have to look at satellite 81 00:04:44,110 --> 00:04:46,469 S1: photos and do a bunch of analysis and then create 82 00:04:46,510 --> 00:04:49,710 S1: narratives around them and write the report. So just like 83 00:04:49,710 --> 00:04:52,430 S1: the other workflows, you see the different pieces and you 84 00:04:52,430 --> 00:04:57,390 S1: see why they require human right? This is intelligence. You know, analysis. 85 00:04:58,029 --> 00:05:00,510 S1: And Amir is the best at his job because he 86 00:05:00,510 --> 00:05:03,789 S1: could do 12 of these assessments per week, and his 87 00:05:03,830 --> 00:05:07,390 S1: accuracy is really high. It's 84%. This is an attacker 88 00:05:07,910 --> 00:05:11,990 S1: organization based out of Eastern Europe. Let's say this is 89 00:05:11,990 --> 00:05:15,950 S1: their primary attack workflow. Again, this is a company called 90 00:05:15,950 --> 00:05:18,950 S1: cyber Attacks or whatever. So they find targets. They run 91 00:05:18,950 --> 00:05:22,830 S1: recon tools. Again, these are things that manual security testers 92 00:05:22,830 --> 00:05:26,870 S1: will have been doing for quite some time. And again, 93 00:05:26,910 --> 00:05:30,109 S1: you can't just script everything right. You can script a 94 00:05:30,110 --> 00:05:31,950 S1: lot of this stuff, but a lot of it requires 95 00:05:31,950 --> 00:05:36,470 S1: the blue icon of human intelligence and human thought. And 96 00:05:36,470 --> 00:05:39,430 S1: that's again why we are employed. So you've got to 97 00:05:39,430 --> 00:05:42,510 S1: find and filter targets because they can only do so 98 00:05:42,510 --> 00:05:45,150 S1: many assessments. Right. They got to run a bunch of 99 00:05:45,150 --> 00:05:48,310 S1: recon tools. They attempt to exploit what they find, and 100 00:05:48,310 --> 00:05:52,750 S1: then they try to do various tasks afterwards, like expand, uh, 101 00:05:52,750 --> 00:05:57,310 S1: you know, get initial access, sell the initial access, you know, uh, 102 00:05:57,430 --> 00:06:01,110 S1: gain persistence laterally, move things like that. So the core 103 00:06:01,110 --> 00:06:04,630 S1: idea here is that human work can be broken down 104 00:06:04,630 --> 00:06:09,430 S1: into workflows like this. Humans don't normally see things this way, 105 00:06:09,710 --> 00:06:13,430 S1: but I can guarantee you this is how companies coming 106 00:06:13,430 --> 00:06:17,670 S1: to replace you will see things. And unfortunately that's going 107 00:06:17,670 --> 00:06:21,350 S1: to include your management. Right. That's going to include the 108 00:06:21,350 --> 00:06:24,109 S1: C team and the board. They're going to be like, 109 00:06:24,110 --> 00:06:26,950 S1: why can't we just do this with AI. So there 110 00:06:26,990 --> 00:06:29,429 S1: are going to be thinking about how to make this 111 00:06:29,430 --> 00:06:32,750 S1: kind of diagram for the work that you do, for 112 00:06:32,750 --> 00:06:34,630 S1: the work that we all do. And by the way, 113 00:06:34,630 --> 00:06:39,510 S1: like a year ago, uh, AI consulting. Yeah, this is 2024, 114 00:06:39,630 --> 00:06:43,550 S1: I believe might have been 2023. I'm pretty sure it's 2024. 115 00:06:44,029 --> 00:06:50,589 S1: McKinsey's consulting for AI was already 40% of their business, 40%. 116 00:06:50,630 --> 00:06:53,990 S1: This is according to New York Times. So this thing 117 00:06:53,990 --> 00:06:56,390 S1: is coming fast for us. So that was the first concept. 118 00:06:56,430 --> 00:06:58,910 S1: Next one is Theory of Constraints, which a lot of 119 00:06:58,910 --> 00:07:02,550 S1: you probably have already heard of. And it relates directly 120 00:07:02,550 --> 00:07:05,909 S1: to the intelligence pipelines we just talked about. So the 121 00:07:05,910 --> 00:07:08,230 S1: Theory of Constraints basically says that you should stop trying 122 00:07:08,230 --> 00:07:12,670 S1: to optimize everything simultaneously, because not all problems are actually 123 00:07:12,670 --> 00:07:15,670 S1: hurting you the same amount. It says the biggest thing 124 00:07:15,710 --> 00:07:18,870 S1: hurting you is the slowest point in your overall workflow. 125 00:07:19,390 --> 00:07:22,630 S1: Or as Goldratt puts it, your overall output is equal 126 00:07:22,630 --> 00:07:26,030 S1: to the output of your worst piece, and that's what 127 00:07:26,030 --> 00:07:29,110 S1: you should address first. My friend Joel works at OpenAI, 128 00:07:29,310 --> 00:07:30,910 S1: and he works on a team that tries to figure 129 00:07:30,910 --> 00:07:33,790 S1: out how to make AI benefit defenders as much or 130 00:07:33,790 --> 00:07:37,400 S1: more than attackers, and we do a couple of 3 131 00:07:37,400 --> 00:07:40,270 S1: or 4 hour walks a month. And one of these 132 00:07:40,270 --> 00:07:44,110 S1: walks many, many months ago, he gave me like religion 133 00:07:44,110 --> 00:07:47,470 S1: on this. In his mind, everything comes down to what 134 00:07:47,710 --> 00:07:51,750 S1: Constrains attackers the most right now as well as defenders. 135 00:07:52,110 --> 00:07:56,630 S1: And how is AI going to change or unblock those constraints? 136 00:07:56,630 --> 00:07:59,510 S1: And his quote is attackers aren't constrained by a lack 137 00:07:59,510 --> 00:08:03,110 S1: of access. They're drowning in access. Instead, it's the human 138 00:08:03,110 --> 00:08:08,390 S1: labor currently required to exploit that access. That's the limit. 139 00:08:08,390 --> 00:08:11,710 S1: Remove that and we are effed. And again, that's my 140 00:08:11,710 --> 00:08:14,990 S1: friend Joel Parish. So think about that. We don't have 141 00:08:15,030 --> 00:08:18,030 S1: a target problem and we don't have an access problem. 142 00:08:18,030 --> 00:08:21,030 S1: We are stuck at the exploit phase. Right. And if 143 00:08:21,030 --> 00:08:24,750 S1: you think back to those pipelines and workflows, you can 144 00:08:24,790 --> 00:08:28,550 S1: kind of like put a little red mark around one 145 00:08:28,550 --> 00:08:31,350 S1: of these brains, one of these blue brains, and you 146 00:08:31,350 --> 00:08:34,350 S1: can actually make one of them red or something and say, 147 00:08:34,350 --> 00:08:37,470 S1: this is the one that's really hurting us. Now start 148 00:08:37,470 --> 00:08:40,030 S1: to think about AI. Now start to think about AI 149 00:08:40,070 --> 00:08:45,110 S1: agents and saying, can we spin up 10 or 100 150 00:08:45,150 --> 00:08:48,710 S1: or 1000 or 1 a million agents to help with 151 00:08:48,710 --> 00:08:53,270 S1: this one particular spot, which, based on the theory of constraints, 152 00:08:53,270 --> 00:08:57,670 S1: massively speeds up the entire pipeline because that was the blocker, 153 00:08:57,710 --> 00:09:00,870 S1: that was the constraint. So thinking back to our pipelines 154 00:09:01,190 --> 00:09:03,670 S1: and again, you want to think about what your pipeline 155 00:09:03,670 --> 00:09:07,710 S1: looks like, right. For your productivity, your security workflows. Think 156 00:09:07,710 --> 00:09:12,030 S1: about where they're currently constrained and how your constraints compare 157 00:09:12,030 --> 00:09:16,990 S1: to your attacker constraints. Core idea here is to ask 158 00:09:16,990 --> 00:09:21,150 S1: how AI will affect those constraints for both you and 159 00:09:21,150 --> 00:09:26,910 S1: your adversaries. And once those constraints get unblocked, where does 160 00:09:26,910 --> 00:09:30,790 S1: it get blocked next? Right. And how can we use 161 00:09:30,790 --> 00:09:36,670 S1: agents to consecutively just keep unblocking or even apply the agents? 162 00:09:36,710 --> 00:09:39,069 S1: I mean, kind of breaking away from the concept of 163 00:09:39,429 --> 00:09:42,270 S1: theory of constraints, but you could kind of find all 164 00:09:42,270 --> 00:09:46,700 S1: the blue icons and just say, hey, let's scale the 165 00:09:46,700 --> 00:09:49,940 S1: crap out of this and let's try to increase our 166 00:09:49,940 --> 00:09:53,140 S1: quality scores at the same time. So we were talking 167 00:09:53,140 --> 00:09:58,980 S1: about 124 assessments previously. We're talking about 84% quality level. Well, 168 00:09:58,980 --> 00:10:01,380 S1: if we actually have those metrics we actually have a 169 00:10:01,380 --> 00:10:04,699 S1: workflow like this. We actually have a diagram. Well it 170 00:10:04,700 --> 00:10:08,080 S1: just becomes a numbers game. Okay. Can we do a 171 00:10:08,080 --> 00:10:13,380 S1: thousand assessments instead of 100. Can we take the 84% 172 00:10:13,380 --> 00:10:17,460 S1: quality level to 85% or is it only 79%. But 173 00:10:17,460 --> 00:10:20,380 S1: we're doing a thousand, so it's still worth it, right? 174 00:10:20,420 --> 00:10:22,900 S1: Those are the types of questions people are going to 175 00:10:22,900 --> 00:10:25,020 S1: be asking when they start doing this. So before we 176 00:10:25,020 --> 00:10:28,140 S1: go into the next piece, I want to quickly cover 177 00:10:28,260 --> 00:10:30,580 S1: my definition of agents because we're going to talk about 178 00:10:30,580 --> 00:10:34,620 S1: agents a decent amount. So lots of different definitions out there. 179 00:10:34,620 --> 00:10:38,380 S1: And I think it's good to level set before proceeding. 180 00:10:38,820 --> 00:10:41,500 S1: So I think it's an AI system component that's capable 181 00:10:41,500 --> 00:10:47,059 S1: of autonomously taking multiple steps towards a goal that previously 182 00:10:47,059 --> 00:10:49,660 S1: would have required a human right. And if you break 183 00:10:49,660 --> 00:10:52,980 S1: that down, it's a component, right? It's not all of AI. 184 00:10:53,020 --> 00:10:57,380 S1: It's a piece of AI. It's autonomously pursuing multiple steps. 185 00:10:57,980 --> 00:11:02,500 S1: So autonomously. So it's given a goal and it's autonomously 186 00:11:02,700 --> 00:11:06,860 S1: chasing that goal by taking multiple steps on its own. 187 00:11:06,900 --> 00:11:09,820 S1: That's the autonomous part. That's the goal part. And the 188 00:11:09,820 --> 00:11:12,380 S1: last part is kind of the most important steps that 189 00:11:12,380 --> 00:11:17,339 S1: could only be done by human previously. Right. So this 190 00:11:17,340 --> 00:11:22,740 S1: means not scripting, not automation, not basic programming. Because if 191 00:11:22,740 --> 00:11:25,900 S1: that were the case, it would already be scripted, right? 192 00:11:26,100 --> 00:11:28,980 S1: So one or more of these steps that it's taking 193 00:11:29,020 --> 00:11:32,579 S1: autonomously towards the goal can only have been done by 194 00:11:32,580 --> 00:11:37,620 S1: human previously. And I think my overall favorite definition of 195 00:11:37,620 --> 00:11:42,900 S1: AI is actually technology that does something cognitive that previously 196 00:11:42,900 --> 00:11:45,340 S1: could have only been done by humans. So it's kind 197 00:11:45,340 --> 00:11:47,860 S1: of in that mind frame. The next idea I want 198 00:11:47,900 --> 00:11:49,620 S1: to talk about is a frame of thinking I've been 199 00:11:49,620 --> 00:11:52,340 S1: thinking about for like the last six months or so, 200 00:11:52,380 --> 00:11:56,900 S1: which I call AI state management. So the idea is, 201 00:11:57,179 --> 00:12:01,220 S1: AI's ultimate form, or one of its ultimate forms, is 202 00:12:01,220 --> 00:12:04,819 S1: to collect and understand the current state of a system 203 00:12:04,820 --> 00:12:09,180 S1: to capture and articulate the desired state. You know, we 204 00:12:09,179 --> 00:12:10,940 S1: know what the current one is. Now. We want to 205 00:12:10,940 --> 00:12:14,700 S1: know how we wish it looked, and then use however 206 00:12:14,700 --> 00:12:21,420 S1: many pieces of intelligence or agents, or thinking or reasoning 207 00:12:21,420 --> 00:12:23,620 S1: or whatever you want to call it, which is basically 208 00:12:23,660 --> 00:12:29,140 S1: AI to help us go from the former to the latter. Again, 209 00:12:29,140 --> 00:12:33,380 S1: collect and understand the current state capture and articulate the 210 00:12:33,380 --> 00:12:38,700 S1: desired state, and then use intelligence, automated intelligence AI. I 211 00:12:38,700 --> 00:12:42,059 S1: didn't think that was actually an acronym, but, um, that 212 00:12:42,059 --> 00:12:46,980 S1: was accidental, but. Yeah. Automated intelligence. Artificial intelligence is what 213 00:12:46,980 --> 00:12:50,700 S1: it really means. But you're using all those tools available, 214 00:12:50,700 --> 00:12:56,579 S1: reasoning agents, context, all of that to go from the 215 00:12:56,580 --> 00:12:58,740 S1: state that you're in to the state that you want 216 00:12:58,740 --> 00:13:02,020 S1: to go into, right. The state that you wish you were. 217 00:13:02,620 --> 00:13:08,699 S1: This is an extremely powerful universal use case for AI 218 00:13:08,740 --> 00:13:11,220 S1: as it gets more and more advanced, especially as it 219 00:13:11,220 --> 00:13:15,020 S1: gets larger and larger context. Because even a small company, 220 00:13:15,020 --> 00:13:17,620 S1: you can't fit everything into the context we currently have. 221 00:13:18,300 --> 00:13:20,760 S1: The latest model that just came out has a 10 222 00:13:20,760 --> 00:13:24,860 S1: million token context, but even that is not big enough 223 00:13:24,860 --> 00:13:28,420 S1: to hold, you know, the full state of an IT system, 224 00:13:28,420 --> 00:13:30,820 S1: the full state of the business, the full state of 225 00:13:31,059 --> 00:13:33,940 S1: all employee activity. Like that's a lot of stuff. Plus 226 00:13:33,940 --> 00:13:36,260 S1: you have to update it right every five minutes, every 227 00:13:36,260 --> 00:13:40,140 S1: ten minutes, every 30s every hour. However often are you 228 00:13:40,140 --> 00:13:43,179 S1: going to do. It takes a lot of context. This 229 00:13:43,179 --> 00:13:47,140 S1: is the ultimate idea, the main concept here. Current state. 230 00:13:47,260 --> 00:13:50,700 S1: Desired state. How do you transition? So in order to 231 00:13:50,700 --> 00:13:52,940 S1: do this, you need to have a massive amount of 232 00:13:52,940 --> 00:13:55,699 S1: data as I just talked about. And that state has 233 00:13:55,700 --> 00:14:00,460 S1: to be updated as often as possible, maybe easier for 234 00:14:00,460 --> 00:14:03,780 S1: a smaller company still. Actually not possible for a smaller 235 00:14:03,780 --> 00:14:07,979 S1: company today, but soon it will be. But for larger enterprises, 236 00:14:08,340 --> 00:14:12,220 S1: extremely non-trivial. We're talking about if you wanted a really 237 00:14:12,220 --> 00:14:16,780 S1: high resolution state, I mean, that would have to be terabytes, 238 00:14:16,820 --> 00:14:20,340 S1: at least gigabytes depending how much you summarize. But what 239 00:14:20,340 --> 00:14:22,580 S1: you build is your current state, your desired state, and 240 00:14:22,580 --> 00:14:27,820 S1: then you ask questions continuously as your main form of 241 00:14:27,820 --> 00:14:32,740 S1: managing the program. And you finally get to recommendations from 242 00:14:32,740 --> 00:14:36,900 S1: the system. It's really powerful and it's not really theoretical. 243 00:14:37,180 --> 00:14:41,420 S1: I actually built a system like this in 23. So 244 00:14:42,260 --> 00:14:47,260 S1: the contents of the context are of course everything. This 245 00:14:47,260 --> 00:14:50,260 S1: is everything about the system. But the advantage is you 246 00:14:50,260 --> 00:14:54,260 S1: can feed it your goals, your logs and your projects 247 00:14:54,260 --> 00:14:57,780 S1: and your budget and your team. And what are people 248 00:14:57,780 --> 00:15:00,540 S1: currently working on? What is the current status? All of 249 00:15:00,540 --> 00:15:02,660 S1: this stuff, you know, all those stand up meetings or 250 00:15:02,660 --> 00:15:05,460 S1: all those planning meetings that all kind of goes away? 251 00:15:06,020 --> 00:15:10,020 S1: It's just data inside of this context. And I want 252 00:15:10,020 --> 00:15:11,780 S1: to take a look at how it all fits together. 253 00:15:11,780 --> 00:15:13,980 S1: So like I said, I built a system like this 254 00:15:14,180 --> 00:15:20,420 S1: back in 23, very early after I guess ChatGPT came out. 255 00:15:20,580 --> 00:15:24,580 S1: I started building this system, and it's a working system 256 00:15:24,820 --> 00:15:28,940 S1: that uses this using a fictional company called alma. So 257 00:15:28,940 --> 00:15:32,500 S1: I started off with tons of context about this fictional 258 00:15:32,500 --> 00:15:39,050 S1: company history, mission, goals, projects, risks. Team members budget, all that. 259 00:15:39,050 --> 00:15:42,410 S1: And then I said, okay, what questions do I actually 260 00:15:42,410 --> 00:15:47,090 S1: want to ask about this? Because I do. My background is, uh, 261 00:15:47,130 --> 00:15:51,330 S1: information security. So cyber security. And, uh, I've been doing 262 00:15:51,330 --> 00:15:54,530 S1: a lot of security consulting, a lot of security assessment, 263 00:15:54,570 --> 00:16:00,490 S1: a lot of security program management. And I have essentially 264 00:16:00,490 --> 00:16:04,530 S1: been using this system before I came out, because it 265 00:16:04,530 --> 00:16:08,609 S1: doesn't really require AI. It just requires really good questions 266 00:16:08,610 --> 00:16:11,130 S1: and a really good understanding of the company. And I 267 00:16:11,130 --> 00:16:14,090 S1: was doing that manually without AI. And now with AI, 268 00:16:14,690 --> 00:16:18,210 S1: it's completely ridiculous. So let me show you some of it. 269 00:16:18,250 --> 00:16:21,330 S1: So here's one of the most common questions that we 270 00:16:21,330 --> 00:16:23,730 S1: get when we're managing a program. What is the list 271 00:16:23,730 --> 00:16:26,610 S1: of projects that we're working on? And here's the answer. 272 00:16:26,650 --> 00:16:29,850 S1: Notice the fact that I just gave it an echo. Right. 273 00:16:29,890 --> 00:16:32,210 S1: This is a command line version of the system. But 274 00:16:32,210 --> 00:16:34,210 S1: give me a list of the projects we're working on, 275 00:16:34,210 --> 00:16:38,170 S1: along with a ten word summary. Okay. And now here's 276 00:16:38,170 --> 00:16:40,290 S1: all of our projects. This is the type of thing 277 00:16:40,290 --> 00:16:42,930 S1: that would take quite a while to get an answer on. 278 00:16:43,210 --> 00:16:45,770 S1: If you're a manager or your director or the VP 279 00:16:45,770 --> 00:16:48,290 S1: or whatever, and you're like, hey, what are we currently 280 00:16:48,290 --> 00:16:52,250 S1: working on? Not too many places, unfortunately, have like an 281 00:16:52,250 --> 00:16:54,450 S1: intranet that you can go to and just like get 282 00:16:54,450 --> 00:16:58,210 S1: the current list. Turns out that things old, there's actually 283 00:16:58,210 --> 00:17:00,850 S1: four versions of it. There's competing versions of it because 284 00:17:00,850 --> 00:17:03,210 S1: some other team is doing the same thing. Actually, it 285 00:17:03,210 --> 00:17:05,250 S1: lives in this Google doc. Oh wait, we got rid 286 00:17:05,290 --> 00:17:08,290 S1: of that Google doc. That one's deprecated. Now go check 287 00:17:08,290 --> 00:17:10,850 S1: this new one. Oh, it's also in confluence. Oh did 288 00:17:10,850 --> 00:17:14,890 S1: you check Jira. Like it's a total mess. Here you 289 00:17:14,930 --> 00:17:18,810 S1: are asking the single unified AI context and it gives 290 00:17:18,810 --> 00:17:22,730 S1: you the answer. Here's another common one. What are our 291 00:17:22,730 --> 00:17:27,210 S1: metrics that are related to our projects? How are they related? 292 00:17:27,530 --> 00:17:29,330 S1: This is the type of thing. This is a giant 293 00:17:29,330 --> 00:17:32,410 S1: blue icon. This is mental work that a human on 294 00:17:32,410 --> 00:17:35,449 S1: your team must do to try to figure this out. 295 00:17:35,450 --> 00:17:38,290 S1: This thing just instantly mapped it. Just like that. Instantly 296 00:17:38,290 --> 00:17:42,530 S1: told you we are working towards fixing these metrics or 297 00:17:42,530 --> 00:17:46,290 S1: improving these metrics. And that's why we're doing these projects 298 00:17:46,690 --> 00:17:49,050 S1: like that. Here's another one. We start with the risks, 299 00:17:49,050 --> 00:17:51,369 S1: and we want to see how those risks relate to 300 00:17:51,369 --> 00:17:56,050 S1: our projects. Right. So we have our risk register in here. 301 00:17:56,050 --> 00:17:59,410 S1: And we could see why we're working on particular projects 302 00:17:59,410 --> 00:18:02,250 S1: because of these risks. And we have a mapping for that. 303 00:18:02,250 --> 00:18:04,330 S1: So now we have a breakdown of why we're working 304 00:18:04,330 --> 00:18:06,690 S1: on what. And we can ask questions like are we 305 00:18:06,690 --> 00:18:09,770 S1: focusing on the right things with our efforts? Here I'm 306 00:18:09,770 --> 00:18:14,410 S1: asking about remediating critical vulnerabilities on crown jewel systems. Look 307 00:18:14,410 --> 00:18:17,770 S1: how clean this narrative is, right? Here's our progress on 308 00:18:17,770 --> 00:18:21,369 S1: remediating critical vulnerabilities on crown jewel systems. Cool. This is 309 00:18:21,369 --> 00:18:22,850 S1: the type of thing you can just copy and paste 310 00:18:22,850 --> 00:18:25,409 S1: this and send it on to a leader or whoever 311 00:18:25,410 --> 00:18:27,290 S1: you need to send it on to, assuming they have 312 00:18:27,290 --> 00:18:30,130 S1: the access to see it. And yeah, shows a consistent 313 00:18:30,250 --> 00:18:33,410 S1: reduction in the time taken to remediate critical vulnerabilities on 314 00:18:33,410 --> 00:18:37,889 S1: crown jewel systems, improving from 21 days in October 22nd 315 00:18:38,250 --> 00:18:41,770 S1: to less than six days by March 2024. That's a 316 00:18:41,770 --> 00:18:44,010 S1: cool narrative and you've got the data right there. This 317 00:18:44,010 --> 00:18:47,570 S1: is total critical remediation, and this one we're calling a 318 00:18:47,570 --> 00:18:50,970 S1: pattern to actually output the graphable data. And this is 319 00:18:51,010 --> 00:18:54,290 S1: for something called fabric, which I'll put a link to 320 00:18:54,330 --> 00:18:56,690 S1: this video right here. Should be over my head or something. 321 00:18:56,770 --> 00:19:01,850 S1: So this one says create a TRC graph. So now 322 00:19:01,890 --> 00:19:04,730 S1: look at that. If you recognize that that's actually CSV. 323 00:19:04,930 --> 00:19:07,410 S1: Guess what you could do with CSV. Boom. Now you 324 00:19:07,410 --> 00:19:10,889 S1: have graph data you can add to any tool to 325 00:19:10,930 --> 00:19:13,810 S1: get an instant visualization of progress over time, to put 326 00:19:13,810 --> 00:19:16,810 S1: it in a presentation or whatever that just flew out 327 00:19:16,810 --> 00:19:21,050 S1: of the eye automatically in a couple of seconds, right? 328 00:19:21,090 --> 00:19:23,410 S1: So those are pretty cool. That's like level one. Cool. 329 00:19:23,810 --> 00:19:27,130 S1: Let's move on to level two of pretty cool. And 330 00:19:27,130 --> 00:19:29,169 S1: I can give you a million more examples because you 331 00:19:29,170 --> 00:19:32,210 S1: can literally just ask anything of the system, and as 332 00:19:32,210 --> 00:19:34,449 S1: long as it's somewhere in the context, it's going to 333 00:19:34,450 --> 00:19:37,850 S1: make all the connections itself, right. But those don't truly 334 00:19:37,850 --> 00:19:41,370 S1: show the power of telos file structure, which, uh, again, 335 00:19:41,410 --> 00:19:44,050 S1: that's another project you could check out here, uh, which 336 00:19:44,050 --> 00:19:47,890 S1: I put together a while back. But if you want 337 00:19:47,930 --> 00:19:50,570 S1: to see the actual power of this, level two is 338 00:19:50,570 --> 00:19:53,649 S1: where it starts. So let's look at at a more 339 00:19:53,690 --> 00:19:57,250 S1: advanced example of like a what if question. So I 340 00:19:57,250 --> 00:20:01,370 S1: tell the system that Nadya is leaving, uh, which she isn't, 341 00:20:01,369 --> 00:20:04,609 S1: but I tell it to help me readjust things based 342 00:20:04,609 --> 00:20:08,250 S1: on project priorities. And here's what it gives me. And 343 00:20:08,250 --> 00:20:10,130 S1: keep in mind, the more detail I have in the 344 00:20:10,130 --> 00:20:13,290 S1: team section about skill sets and experience, the better this gets. 345 00:20:13,490 --> 00:20:16,290 S1: But look at this. Nadya is leaving the team and 346 00:20:16,290 --> 00:20:19,770 S1: I need to give her projects to someone and or 347 00:20:19,770 --> 00:20:23,090 S1: get additional help. Use your expertise as a risk professional 348 00:20:23,490 --> 00:20:26,850 S1: to help me reassign her work and or get additional 349 00:20:26,850 --> 00:20:31,649 S1: help keeping in mind our risks and priorities, especially our 350 00:20:31,650 --> 00:20:35,850 S1: critical projects like that's a little bit unnecessary. Like, you 351 00:20:35,850 --> 00:20:38,570 S1: can actually not give it that context and it's still 352 00:20:38,570 --> 00:20:43,210 S1: going to figure it out. Look at this. Reassign this 353 00:20:43,210 --> 00:20:47,450 S1: project to this other person. Assign this one to this 354 00:20:47,450 --> 00:20:50,530 S1: other person. Hire a contractor to do the WAF install. 355 00:20:50,530 --> 00:20:53,609 S1: And it's got reasoning for this. Keep in mind what 356 00:20:53,609 --> 00:20:57,930 S1: we're asking is a complete and absolute cybersecurity expert. Okay. 357 00:20:57,970 --> 00:21:00,810 S1: So it knows the different skill sets. It knows the 358 00:21:00,810 --> 00:21:04,730 S1: different requirements. So it's giving us advice on what to outsource, 359 00:21:04,970 --> 00:21:08,409 S1: what to do later. Just completely defer and what to 360 00:21:08,490 --> 00:21:12,450 S1: prioritize in which person to give it to. That's insanely helpful. 361 00:21:12,690 --> 00:21:15,810 S1: Insanely helpful. So what about adding new stuff to the context? 362 00:21:15,850 --> 00:21:18,770 S1: How do you update the program? Well, because AI is 363 00:21:18,770 --> 00:21:20,810 S1: seeing the whole picture, you could just add it to 364 00:21:20,850 --> 00:21:24,530 S1: the bottom of the context. And if you're using Rag, 365 00:21:24,570 --> 00:21:27,810 S1: you just add an additional entry. If you're using, you know, 366 00:21:27,850 --> 00:21:30,410 S1: a single file or you're using CAG or you're using 367 00:21:30,450 --> 00:21:33,369 S1: whatever you're using, you can simply append to it. You 368 00:21:33,369 --> 00:21:35,209 S1: don't have to go in and clean and keep the 369 00:21:35,210 --> 00:21:40,969 S1: thing perfectly, you know, upkept and maintained, because when it 370 00:21:40,970 --> 00:21:45,250 S1: reads the update, it'll realize that that supersedes the previous thing. 371 00:21:45,330 --> 00:21:47,010 S1: As long as you have like a timestamp, which we 372 00:21:47,010 --> 00:21:50,690 S1: do here. So this is how you can update progress, 373 00:21:50,690 --> 00:21:54,970 S1: for example, on a KPI, you know, July 2024 Criticals 374 00:21:54,970 --> 00:21:57,209 S1: are now being fixed in nine days. So what I 375 00:21:57,210 --> 00:21:59,850 S1: added here are a few lines with new metrics updates, 376 00:22:00,130 --> 00:22:03,370 S1: bottom items here. And guess what? When I run my 377 00:22:03,369 --> 00:22:06,970 S1: previous question, here's the graph that it produces. Look at that. 378 00:22:06,970 --> 00:22:11,210 S1: It added new items to the graph to the CSV output, 379 00:22:11,730 --> 00:22:14,170 S1: because we added them to the context. And the I 380 00:22:14,210 --> 00:22:16,450 S1: just figured out okay, we're going to update the graph. 381 00:22:16,450 --> 00:22:18,410 S1: So now let's see how powerful the system is for 382 00:22:18,410 --> 00:22:22,570 S1: knocking out very time consuming everyday tasks for any security program. 383 00:22:22,609 --> 00:22:25,360 S1: I mean, like so much of security is actually just 384 00:22:25,359 --> 00:22:29,200 S1: doing this stuff, responding to auditors. Here, I'm telling it, 385 00:22:29,200 --> 00:22:32,440 S1: I need a narrative on critical vuln remediation times for 386 00:22:32,440 --> 00:22:37,120 S1: an external auditor. Not only wrote the narrative, but added 387 00:22:37,119 --> 00:22:39,960 S1: in our actual numbers, it just wrote it with all 388 00:22:39,960 --> 00:22:44,000 S1: the numbers already intertwined. This is the type of thing 389 00:22:44,000 --> 00:22:47,800 S1: that can derail somebody in the security program for hours, 390 00:22:48,440 --> 00:22:50,560 S1: or a couple of days to go and find all 391 00:22:50,560 --> 00:22:54,000 S1: this data, see, pull from the different things or whatever. 392 00:22:54,160 --> 00:22:58,280 S1: Like this just takes massive time away from security teams 393 00:22:58,280 --> 00:23:00,679 S1: to be able to do this. So think about how 394 00:23:00,680 --> 00:23:03,880 S1: much time an AI like this actually saves. And here's 395 00:23:03,880 --> 00:23:06,360 S1: one for the executive team. And notice how it changes 396 00:23:06,359 --> 00:23:09,800 S1: the tone according to who you're actually making the report for. 397 00:23:10,200 --> 00:23:12,160 S1: And the more detail you put in the context file 398 00:23:12,160 --> 00:23:17,280 S1: around sensitivities, reporting preferences, or in the reporting pattern you 399 00:23:17,280 --> 00:23:20,159 S1: use for each audience, the better this output is actually 400 00:23:20,160 --> 00:23:22,760 S1: going to be. So you can set these up for 401 00:23:22,760 --> 00:23:27,280 S1: as many groups as you regularly report to, right? Internal team. 402 00:23:27,280 --> 00:23:32,080 S1: The board. Auditors, internal, external, whoever. So those are pretty cool. 403 00:23:32,720 --> 00:23:37,120 S1: Level three. Now let's look at some insanely helpful functions 404 00:23:37,119 --> 00:23:40,280 S1: that will save even more time. So by a show 405 00:23:40,280 --> 00:23:44,520 S1: of hands, who likes security questionnaires? Okay, let the record 406 00:23:44,520 --> 00:23:48,480 S1: show nobody. This is absolutely insane. This is another section 407 00:23:48,520 --> 00:23:51,840 S1: of the tlos file, the context file that I created 408 00:23:51,840 --> 00:23:55,840 S1: for this fictional company. Okay. It's a set of updates 409 00:23:55,840 --> 00:23:59,680 S1: to Alma's IT infrastructure and security controls. And just like 410 00:23:59,680 --> 00:24:02,840 S1: everything else, look how casually these are written. This is 411 00:24:02,840 --> 00:24:07,560 S1: like a logbook migrated from Google Workspace to Office 365, 412 00:24:07,920 --> 00:24:12,120 S1: with MFA enabled for all users. It's not in a schema, 413 00:24:12,119 --> 00:24:14,879 S1: it's not in a format. It's got a basic time 414 00:24:14,920 --> 00:24:18,959 S1: stamp there. Rolled out Sentinel one on 50% of corporate 415 00:24:18,960 --> 00:24:23,560 S1: laptops in August of 2020. That's all you need to say, right? 416 00:24:23,840 --> 00:24:26,399 S1: And you see how I've got a history here. Okay. 417 00:24:26,440 --> 00:24:30,480 S1: All the way back to July 2019. Basically, we're a 418 00:24:30,480 --> 00:24:33,479 S1: soup sandwich. Back then, admin accounts still not required to 419 00:24:33,480 --> 00:24:39,480 S1: use Toofar company laptops. No, MDM everyone is admin. It's it's, 420 00:24:39,640 --> 00:24:44,440 S1: you know, mass chaos pandemonium. And then as the updates 421 00:24:44,440 --> 00:24:47,680 S1: start going down through the months, it starts to get 422 00:24:47,680 --> 00:24:51,080 S1: better and better. And this is just it's just a logbook. 423 00:24:51,080 --> 00:24:53,040 S1: So now the question is we're being asked by a 424 00:24:53,040 --> 00:24:56,919 S1: customer what percentage of endpoints are protected by endpoint protection. 425 00:24:57,160 --> 00:24:59,480 S1: How often are you asked stuff like this in a 426 00:24:59,480 --> 00:25:02,920 S1: security questionnaire all the time. And they'll ask it in 427 00:25:02,920 --> 00:25:05,040 S1: a different way each time. So you can't have like 428 00:25:05,080 --> 00:25:08,760 S1: a static question and static answer if you're in vendor management. 429 00:25:08,760 --> 00:25:11,960 S1: You've been down that road before. So here's the answer 430 00:25:11,960 --> 00:25:13,879 S1: written in a form we can copy and paste directly 431 00:25:13,880 --> 00:25:18,399 S1: to the customer. All came back from alma, the AI system. 432 00:25:18,400 --> 00:25:22,240 S1: Here's another one asking how much our infrastructure is US based. 433 00:25:22,640 --> 00:25:24,600 S1: I asked to give me an answer for both cloud 434 00:25:24,600 --> 00:25:27,760 S1: and on prem, and to give the response in an 435 00:25:27,760 --> 00:25:31,840 S1: email gives me the 100% answer that comes from our 436 00:25:31,840 --> 00:25:36,600 S1: Tlose context file. So we just recently got this where 437 00:25:36,600 --> 00:25:39,120 S1: all of our stuff, all of our cloud environments, all 438 00:25:39,160 --> 00:25:42,640 S1: our different AWS regions, all of our, you know, on 439 00:25:42,640 --> 00:25:46,080 S1: prem stuff, it's all US based. That was an update 440 00:25:46,119 --> 00:25:48,359 S1: that at the bottom of the Telos file, keep in 441 00:25:48,359 --> 00:25:51,000 S1: mind the context file up here could have been the 442 00:25:51,000 --> 00:25:53,760 S1: worst answer because that's the old stuff. But now that 443 00:25:53,760 --> 00:25:57,360 S1: we have that updated, the answer is now 100% and 444 00:25:57,359 --> 00:25:59,120 S1: we have it perfectly in an email, we can just 445 00:25:59,119 --> 00:26:01,840 S1: send that to somebody. And ultimately we're heading towards being 446 00:26:01,840 --> 00:26:05,920 S1: able to ask questions like these. How should we move 447 00:26:05,920 --> 00:26:09,160 S1: from our current to desired state? Going back to that idea, 448 00:26:09,160 --> 00:26:13,280 S1: we talked about what should be in our desired state, right. 449 00:26:13,320 --> 00:26:17,280 S1: That that's a, you know, big brain question. Big brain 450 00:26:17,400 --> 00:26:20,120 S1: question what should be in our desired state that you 451 00:26:20,119 --> 00:26:23,120 S1: actually don't see in there? How about this? How will 452 00:26:23,160 --> 00:26:28,520 S1: attackers exploit our desired state? Right. Natural. Follow on. How 453 00:26:28,520 --> 00:26:31,680 S1: should we change that desired state to make it less exploitable? 454 00:26:31,720 --> 00:26:36,080 S1: All looking at the same unified context. So the insane 455 00:26:36,080 --> 00:26:40,240 S1: thing about this is realizing that most software is kind 456 00:26:40,240 --> 00:26:43,200 S1: of the same in cyber. We're pulling in data from somewhere. 457 00:26:43,200 --> 00:26:45,800 S1: We're trying to pull signal out of that data, and 458 00:26:45,800 --> 00:26:48,119 S1: we try to do something with it. Right. But it's 459 00:26:48,119 --> 00:26:52,480 S1: exactly the same with productivity apps or B2B processing or whatever. 460 00:26:52,760 --> 00:26:54,760 S1: It's all the same. So the core idea here is 461 00:26:54,760 --> 00:26:58,440 S1: that when the AI understands where we are and where 462 00:26:58,440 --> 00:27:01,520 S1: we want to go, it can help us get there. 463 00:27:01,520 --> 00:27:04,280 S1: And that's where this entire program management thing that I 464 00:27:04,280 --> 00:27:07,840 S1: just showed you that I just demoed, that's real AI system, 465 00:27:07,840 --> 00:27:10,160 S1: like it's sitting right here. I can ask you questions. 466 00:27:10,720 --> 00:27:13,560 S1: And obviously my context file for this thing was a 467 00:27:13,560 --> 00:27:17,000 S1: few hundred lines, right? I think it eventually got to 468 00:27:17,040 --> 00:27:20,920 S1: a few thousand lines. But as the AI systems scale right, 469 00:27:20,960 --> 00:27:24,560 S1: we've already got millions of tokens available. This is just 470 00:27:24,560 --> 00:27:26,800 S1: going to get easier and easier to do at larger 471 00:27:26,800 --> 00:27:29,119 S1: and larger scales. So we're going to be able to 472 00:27:29,119 --> 00:27:32,040 S1: do the same kind of stuff even for a company. 473 00:27:32,680 --> 00:27:34,800 S1: So this next idea is kind of a culmination of 474 00:27:34,800 --> 00:27:37,600 S1: all of this in a way to prep for attackers 475 00:27:37,880 --> 00:27:41,239 S1: in a more tactical way. Like what exactly are they 476 00:27:41,280 --> 00:27:44,000 S1: going to launch at us, and how can we get 477 00:27:44,000 --> 00:27:46,600 S1: ready for that? So the whole concept starts with asking 478 00:27:46,600 --> 00:27:50,480 S1: one question what do attackers wish that they could do 479 00:27:50,480 --> 00:27:53,920 S1: to us? What can they do now? What can they 480 00:27:53,920 --> 00:27:56,720 S1: do in like six months from now? And what can 481 00:27:56,760 --> 00:27:59,879 S1: they do in 18 months or 24 months or 36 months? 482 00:28:00,280 --> 00:28:03,520 S1: And obviously, you can't know exactly the answers to those 483 00:28:03,520 --> 00:28:07,240 S1: because you don't know what text exists at that time. 484 00:28:07,480 --> 00:28:09,600 S1: You don't know how advanced the AI is going to be, 485 00:28:09,640 --> 00:28:13,120 S1: how good agents are going to be. Right. But we 486 00:28:13,119 --> 00:28:18,389 S1: can start with a giant list of attacker capabilities, some 487 00:28:18,430 --> 00:28:20,950 S1: of which they have already, some of which they're about 488 00:28:20,950 --> 00:28:24,630 S1: to get, some of which are coming soon and some 489 00:28:24,670 --> 00:28:27,150 S1: of which are distant in the future. But if we 490 00:28:27,150 --> 00:28:30,949 S1: start with what we know, the ideal is with what 491 00:28:30,950 --> 00:28:34,189 S1: would be devastating to us. I'll give you an example 492 00:28:34,190 --> 00:28:39,229 S1: of devastating to us. They can instantly identify all open 493 00:28:39,230 --> 00:28:43,510 S1: services on the entire United States at any given time, 494 00:28:44,230 --> 00:28:49,790 S1: and can launch an attack against all of those services simultaneously. 495 00:28:49,790 --> 00:28:53,750 S1: Any of those attacks that get inside and are successful, 496 00:28:53,750 --> 00:28:58,830 S1: they can pivot within 30s to compromise other systems internally. 497 00:28:59,310 --> 00:29:02,710 S1: They can instantly crawl around and find all the different, uh, 498 00:29:02,710 --> 00:29:06,550 S1: stuff like an advanced human red team. They could figure 499 00:29:06,550 --> 00:29:10,110 S1: out what the business does, uh, what their most sensitive, uh, 500 00:29:10,110 --> 00:29:13,469 S1: talking points are. Their most sensitive, like worst case scenarios 501 00:29:13,470 --> 00:29:17,950 S1: are and it could right then the exploit where and 502 00:29:17,950 --> 00:29:22,750 S1: the ransomware and the ransom notes and the verbiage and everything. 503 00:29:23,230 --> 00:29:26,150 S1: And then it could actually call with AI and start 504 00:29:26,190 --> 00:29:29,430 S1: negotiating the ransom because it already has the stuff. It's 505 00:29:29,430 --> 00:29:31,510 S1: all zipped up. Oh, and by the way, they're already 506 00:29:31,510 --> 00:29:34,430 S1: selling to all the different information brokers at the same time. 507 00:29:34,910 --> 00:29:38,750 S1: And this took them four minutes. Okay. And they could 508 00:29:38,750 --> 00:29:43,110 S1: do that to an entire country. That is science fiction 509 00:29:43,110 --> 00:29:45,830 S1: right now. That is science fiction because we're talking about 510 00:29:45,830 --> 00:29:55,070 S1: millions upon millions of IP addresses, web locations, APIs, websites, 511 00:29:55,230 --> 00:30:01,590 S1: attack surface components, basically. Um, so IPS services, uh. Net 512 00:30:01,630 --> 00:30:04,270 S1: ranges like think of how much work you have to 513 00:30:04,270 --> 00:30:06,870 S1: do to actually go and do that in a short 514 00:30:06,870 --> 00:30:09,550 S1: amount of time. Think of how much infrastructure you would 515 00:30:09,550 --> 00:30:14,270 S1: have to have. So in this list, these capabilities are 516 00:30:14,310 --> 00:30:18,030 S1: further down the list. So the idea of like, okay, 517 00:30:18,070 --> 00:30:21,950 S1: look at a very small startup with ten people and 518 00:30:21,950 --> 00:30:26,150 S1: find their infrastructure, find their subdomains, determine what open services 519 00:30:26,150 --> 00:30:30,550 S1: are available, see if any open services are for sensitive services. 520 00:30:31,150 --> 00:30:35,070 S1: Get back a list of those within ten minutes. Okay, 521 00:30:35,510 --> 00:30:38,990 S1: that might be possible today. That might be possible in 522 00:30:38,990 --> 00:30:41,790 S1: less than ten minutes. Uh, fairly soon. It depends on 523 00:30:41,790 --> 00:30:44,110 S1: the capability of the attacker, but it's a type of 524 00:30:44,110 --> 00:30:47,070 S1: thing that's within the realm of possibility. Further down the 525 00:30:47,070 --> 00:30:50,150 S1: list is doing that for a larger company or doing 526 00:30:50,150 --> 00:30:53,910 S1: it in five minutes instead of ten minutes. So now 527 00:30:53,910 --> 00:30:57,950 S1: what you start to see is the better the tech gets, 528 00:30:58,110 --> 00:31:02,510 S1: the more realistic these things become. You start moving down 529 00:31:02,510 --> 00:31:07,110 S1: the list of capabilities until you start moving towards the 530 00:31:07,110 --> 00:31:10,990 S1: sci fi worst case scenario I gave you in the beginning. 531 00:31:10,990 --> 00:31:15,750 S1: So we basically start with this giant list of attacker capabilities, 532 00:31:16,150 --> 00:31:19,790 S1: some of which they already have and most of which 533 00:31:19,790 --> 00:31:22,510 S1: are not possible yet. For each one we collect a 534 00:31:22,510 --> 00:31:24,910 S1: bunch of information for it, some metadata that will help 535 00:31:24,910 --> 00:31:27,910 S1: us like figure out how it might be used against us, 536 00:31:27,910 --> 00:31:31,190 S1: or how difficult it could be, like cost or talent, 537 00:31:31,190 --> 00:31:34,630 S1: or scalability or constraints or whatever. Because this is what's 538 00:31:34,630 --> 00:31:37,990 S1: going to determine how, when the tech improves, which parts 539 00:31:37,990 --> 00:31:40,070 S1: of these are going to change. And here's what makes 540 00:31:40,070 --> 00:31:42,830 S1: this thing so powerful. When I talk to companies who 541 00:31:42,830 --> 00:31:45,870 S1: are thinking about cybersecurity and AI, the number one question 542 00:31:45,870 --> 00:31:49,750 S1: I hear is what should we build? What should we build? Like, okay, 543 00:31:49,750 --> 00:31:51,709 S1: I hear about agents, so I build agents. What do 544 00:31:51,710 --> 00:31:55,510 S1: I build agents to do? What should they defend? Should 545 00:31:55,510 --> 00:32:00,270 S1: they monitor like they don't know where to start? So 546 00:32:00,270 --> 00:32:02,790 S1: this framework is a really good answer to that. It's 547 00:32:02,790 --> 00:32:06,030 S1: basically saying, here's what's going to be thrown at you 548 00:32:06,030 --> 00:32:09,550 S1: because here's what the attacker wants to do. So what 549 00:32:09,550 --> 00:32:11,790 S1: are you doing to be ready for each of these? 550 00:32:12,630 --> 00:32:16,030 S1: So you basically take this attacker capabilities. That's why it's 551 00:32:16,590 --> 00:32:20,990 S1: Acad right. It's for defense as well. You use this 552 00:32:20,990 --> 00:32:25,590 S1: attacker capabilities map to build your defense right. The same 553 00:32:25,590 --> 00:32:28,790 S1: way that they're using AI to power this list of 554 00:32:28,790 --> 00:32:32,550 S1: capabilities and move down the capabilities list. You could be 555 00:32:32,550 --> 00:32:35,910 S1: doing exactly the same for defense. So what we end 556 00:32:35,910 --> 00:32:39,710 S1: up with is an AI cyber infrastructure that basically looks 557 00:32:39,710 --> 00:32:44,110 S1: like the attackers. We are constantly assessing ourselves. Basically, we 558 00:32:44,110 --> 00:32:49,190 S1: are executing everything on this attacker capabilities list as a 559 00:32:49,190 --> 00:32:53,830 S1: unified engine. It's a continuous kind of like a continuous 560 00:32:53,830 --> 00:32:58,550 S1: red team type situation. Or in the current, you know, parlance, 561 00:32:58,550 --> 00:33:03,630 S1: it's essentially a attack surface management or external attack surface management. 562 00:33:04,110 --> 00:33:08,150 S1: But really that starts to merge with automated pen testing. 563 00:33:08,310 --> 00:33:12,790 S1: Automated red team attack. Surface management. Like all these things 564 00:33:12,790 --> 00:33:17,470 S1: start to merge into this AI powered execution of these 565 00:33:17,470 --> 00:33:21,070 S1: attacker capabilities. And it's just continuous. And the same thing 566 00:33:21,070 --> 00:33:23,710 S1: that the attackers are doing is what the defender has 567 00:33:23,710 --> 00:33:26,750 S1: to be doing to themselves so they can get there first. 568 00:33:26,750 --> 00:33:29,670 S1: As we get this capability up and running, the entire 569 00:33:29,710 --> 00:33:34,950 S1: game becomes two things making sure we're adding new techniques 570 00:33:34,950 --> 00:33:39,070 S1: to look for, right? So we're making this thing smarter. 571 00:33:39,110 --> 00:33:43,030 S1: We're making it more knowledgeable, right. New techniques, new attack 572 00:33:43,030 --> 00:33:48,070 S1: surface new services to watch out for, you know, new 573 00:33:48,070 --> 00:33:51,750 S1: ways of finding information, whatever new threat Intel and making 574 00:33:51,750 --> 00:33:55,430 S1: sure we find our issues. The second one here, making 575 00:33:55,430 --> 00:33:59,469 S1: sure we find our issues and fix them faster than 576 00:33:59,510 --> 00:34:02,190 S1: our attackers. Right. So this is a race. This is 577 00:34:02,190 --> 00:34:06,540 S1: a giant game where they are running the Acad system. 578 00:34:07,020 --> 00:34:12,460 S1: We are running our own Acad system, this automated continuous 579 00:34:12,460 --> 00:34:17,420 S1: red teaming system that knows everything about our company. Right? 580 00:34:17,460 --> 00:34:19,859 S1: Going back to the context thing. They're going to have 581 00:34:19,860 --> 00:34:24,020 S1: this this giant context that knows everything about us too, right? 582 00:34:24,180 --> 00:34:27,500 S1: But they have a disadvantage. They should have a disadvantage 583 00:34:28,060 --> 00:34:32,340 S1: that they don't know all of our internal configs, hopefully. Right. 584 00:34:32,380 --> 00:34:34,580 S1: So we should be able to move faster than them. 585 00:34:34,940 --> 00:34:36,819 S1: But we have to start by building the same thing 586 00:34:36,820 --> 00:34:40,380 S1: that we know they are building. So both attackers and 587 00:34:40,380 --> 00:34:46,140 S1: defenders basically end up building a world model of our company. 588 00:34:46,780 --> 00:34:49,459 S1: What we do, what we care about, what matters to us, 589 00:34:49,460 --> 00:34:52,859 S1: what our weaknesses are, what's most valuable that we have 590 00:34:52,860 --> 00:34:56,500 S1: to different types of attackers, and what we must avoid 591 00:34:56,500 --> 00:34:59,580 S1: at all costs. Then we are looking at the capabilities 592 00:34:59,580 --> 00:35:03,259 S1: map to see what new capabilities are coming online as 593 00:35:03,420 --> 00:35:07,060 S1: a result of the advancing AI, we could say, okay, well, 594 00:35:07,700 --> 00:35:10,700 S1: we don't have to defend against that yet. For example, 595 00:35:10,860 --> 00:35:14,540 S1: we don't have to defend against an attacker that can 596 00:35:14,540 --> 00:35:19,580 S1: see every single change and attack it within five seconds. 597 00:35:20,219 --> 00:35:22,820 S1: We do not have to defend against that right now. 598 00:35:22,980 --> 00:35:26,500 S1: That would cost tens of millions of dollars. We would 599 00:35:26,500 --> 00:35:29,860 S1: have to hire a massive staff. We would have to 600 00:35:29,980 --> 00:35:33,939 S1: massively scale our automation. It would be an investment that 601 00:35:33,940 --> 00:35:36,860 S1: would sink the company, most likely because the company needs 602 00:35:36,860 --> 00:35:40,339 S1: to be building products. They can't spend that much on security. 603 00:35:40,980 --> 00:35:43,540 S1: So that's like a thing we don't have to worry 604 00:35:43,540 --> 00:35:46,100 S1: about yet, because we know the attacker doesn't have it 605 00:35:46,100 --> 00:35:49,580 S1: yet either. That's why this map is so important, because 606 00:35:49,580 --> 00:35:53,220 S1: we could say as the tech starts improving, we could say, 607 00:35:53,219 --> 00:35:56,140 S1: you are here, right? You are here on this level 608 00:35:56,140 --> 00:35:59,219 S1: of like what is possible in the world given the 609 00:35:59,219 --> 00:36:03,260 S1: current state of automation, given the current state of AI, right? 610 00:36:03,300 --> 00:36:06,500 S1: So we're looking at the capabilities map to see the 611 00:36:06,500 --> 00:36:10,140 S1: new capabilities coming online as a result of advancing AI. 612 00:36:10,260 --> 00:36:13,020 S1: And we could say, okay, yeah, this is where we 613 00:36:13,020 --> 00:36:15,739 S1: need to be currently. This is where we need to 614 00:36:15,739 --> 00:36:19,420 S1: be thinking about coming up soon, because we know that's 615 00:36:19,420 --> 00:36:22,700 S1: what our attackers are about to get. So it's not 616 00:36:22,700 --> 00:36:25,300 S1: just what you need to build, but it also helps 617 00:36:25,300 --> 00:36:28,939 S1: you inform you know when to make what investment of 618 00:36:28,940 --> 00:36:32,340 S1: what size in what area. So I'm still talking to 619 00:36:32,580 --> 00:36:35,060 S1: a few people about how much of this I should 620 00:36:35,060 --> 00:36:37,380 S1: actually release, but I'm pretty sure attackers are going to 621 00:36:37,380 --> 00:36:40,660 S1: figure this out anyway. So I think the priority needs 622 00:36:40,660 --> 00:36:46,219 S1: to be on enabling defenders. We've got over 60 capabilities 623 00:36:46,219 --> 00:36:50,420 S1: so far for attackers and around 40 so far for defenders, 624 00:36:50,420 --> 00:36:53,900 S1: because even though the defenders should be using the attack 625 00:36:53,900 --> 00:36:57,620 S1: map as well, there are also defender capabilities. And this 626 00:36:57,620 --> 00:37:00,540 S1: is something Jason Haddox and I talk about a lot. 627 00:37:00,580 --> 00:37:02,660 S1: And I think his idea was to start with a 628 00:37:02,660 --> 00:37:06,420 S1: defender side basically like what is the what are the 629 00:37:06,420 --> 00:37:10,620 S1: core things like automated SoC and uh, you know, incident 630 00:37:10,620 --> 00:37:12,940 S1: management and stuff like that. What are those things that 631 00:37:12,940 --> 00:37:16,819 S1: we could use current AI to just improve? Um, so 632 00:37:16,820 --> 00:37:19,580 S1: I think we're we're thinking about the combination of the 633 00:37:19,580 --> 00:37:23,420 S1: two together. And, uh, yeah, we've got 60 attacker, 40 634 00:37:23,460 --> 00:37:27,980 S1: defender right now. And, uh, looking to release this within, uh, 635 00:37:27,980 --> 00:37:32,380 S1: 30 to 60 days, uh, depending on a few outstanding conversations. 636 00:37:33,060 --> 00:37:35,660 S1: So what I love about it is it can give 637 00:37:35,660 --> 00:37:38,940 S1: us tremendous focus. We can base our efforts on what 638 00:37:38,940 --> 00:37:41,700 S1: we expect to face in the real world. And based 639 00:37:41,700 --> 00:37:44,779 S1: on the combination of what AI is possible and what 640 00:37:44,780 --> 00:37:48,819 S1: the attacker wishes they can execute. So what I love 641 00:37:48,860 --> 00:37:51,980 S1: about this is how it can just give us tremendous 642 00:37:51,980 --> 00:37:55,620 S1: focus as defenders, right? We can base our efforts on 643 00:37:55,620 --> 00:37:58,819 S1: what we can expect to face in the actual world 644 00:37:58,820 --> 00:38:02,860 S1: from our attackers. And based on this combination of what 645 00:38:02,900 --> 00:38:07,020 S1: AI is possible and what the attacker wishes that they 646 00:38:07,020 --> 00:38:09,340 S1: could do to us. So I want to end by 647 00:38:09,340 --> 00:38:11,980 S1: answering the two questions we started with. How does software 648 00:38:11,980 --> 00:38:15,500 S1: and cybersecurity change? When we add AI and specifically agents, 649 00:38:15,500 --> 00:38:21,980 S1: it changes everything. It replaces human intelligence in workflows, right? 650 00:38:22,020 --> 00:38:24,620 S1: And it does so at a scale that addresses the 651 00:38:24,620 --> 00:38:28,860 S1: theory of constraints problem for attackers, making them far more capable. 652 00:38:28,860 --> 00:38:31,219 S1: That's what's going to allow them to move through these 653 00:38:31,219 --> 00:38:36,780 S1: stages of the attacker capabilities map. Right. It is solving 654 00:38:36,780 --> 00:38:40,700 S1: this the various pieces of the theory of constraints. Second 655 00:38:40,700 --> 00:38:44,380 S1: question how should we prioritize our efforts around adding AI 656 00:38:44,420 --> 00:38:49,500 S1: agents to our cybersecurity program? Start by building your AI 657 00:38:49,540 --> 00:38:52,819 S1: state management system. Now I'm calling this like a UCC, 658 00:38:53,420 --> 00:38:57,500 S1: a unified company context. And there's a million companies working 659 00:38:57,500 --> 00:39:00,299 S1: on this, right? Microsoft has their own version. You know, 660 00:39:00,340 --> 00:39:03,860 S1: Databricks is working on theirs. Splunk is probably working on theirs. 661 00:39:03,860 --> 00:39:06,899 S1: I imagine the game is about to become bringing all 662 00:39:06,900 --> 00:39:10,060 S1: company data into a thing that I can see and 663 00:39:10,060 --> 00:39:12,540 S1: hold in its brain all at once. I'm calling it 664 00:39:12,540 --> 00:39:16,140 S1: UCC Unified Company Context. But who knows what Gartner is 665 00:39:16,140 --> 00:39:18,980 S1: going to call it? Like, everyone's just going to like, 666 00:39:19,020 --> 00:39:22,779 S1: figure this out, uh, fairly soon. So you've got to 667 00:39:22,780 --> 00:39:26,500 S1: start building this thing now because your attackers are going 668 00:39:26,500 --> 00:39:30,419 S1: to be building a UCC for you, which all their 669 00:39:30,420 --> 00:39:34,140 S1: attacker tools are going to then use to attack you 670 00:39:34,580 --> 00:39:38,339 S1: on a continuous basis. If you weren't vulnerable this morning 671 00:39:38,340 --> 00:39:41,140 S1: at 9 a.m., maybe you will be at noon. It's 672 00:39:41,140 --> 00:39:44,899 S1: going to check again when something they learn that you 673 00:39:44,900 --> 00:39:47,540 S1: just acquired a new company, you have a new attack surface. Oh, 674 00:39:47,580 --> 00:39:50,779 S1: maybe they'll now attack you now because of that. Maybe 675 00:39:50,780 --> 00:39:53,940 S1: they learn things through your bug bounty. Maybe. Whatever it is, 676 00:39:53,940 --> 00:39:57,730 S1: the context and the situation on the ground continues to change. 677 00:39:58,010 --> 00:40:01,490 S1: So they're going to have this world model of you. 678 00:40:01,969 --> 00:40:04,649 S1: You need to have a better one. That's the trick 679 00:40:04,650 --> 00:40:07,650 S1: of this. And the second thing you want to be 680 00:40:07,650 --> 00:40:10,689 S1: doing with this context that you have is figuring out 681 00:40:10,690 --> 00:40:13,530 S1: what your desired state is and what your current state is. 682 00:40:13,570 --> 00:40:16,810 S1: And this is going to allow you to say things like, 683 00:40:16,810 --> 00:40:22,489 S1: what should I fix first? Vulnerability prioritization. I've been in 684 00:40:22,489 --> 00:40:27,489 S1: the vulnerability management space forever, and it's so frustrating to 685 00:40:27,690 --> 00:40:30,489 S1: try to figure out how to prioritize a vulnerability. And 686 00:40:30,489 --> 00:40:34,330 S1: people are like trying to put that information inside the 687 00:40:34,330 --> 00:40:38,009 S1: vulnerability itself with like a CVE rating or something. You 688 00:40:38,010 --> 00:40:41,209 S1: can't get the prioritization from the vulnerability itself. It has 689 00:40:41,210 --> 00:40:47,050 S1: to come from the context of where it's affecting something, right? 690 00:40:47,090 --> 00:40:50,850 S1: The company itself. So this UCC is going to be 691 00:40:50,850 --> 00:40:54,969 S1: the context for doing vulnerability management So you're talking about 692 00:40:54,969 --> 00:40:57,969 S1: current state to desired state. You're going to say, I 693 00:40:58,010 --> 00:41:02,009 S1: don't want any critical vulnerabilities in my crown jewel systems. 694 00:41:02,410 --> 00:41:06,850 S1: That's one of my items in my desired state. Now 695 00:41:07,010 --> 00:41:09,290 S1: when you ask the question, what can I do to 696 00:41:09,330 --> 00:41:12,850 S1: make that real? It will then say, well, based on 697 00:41:12,850 --> 00:41:15,810 S1: the fact that you have these vulnerabilities and these things 698 00:41:15,810 --> 00:41:18,290 S1: in your risk register, and the fact that we know 699 00:41:18,290 --> 00:41:21,330 S1: all these dev teams and we know what dev teams 700 00:41:21,330 --> 00:41:24,169 S1: are using, what GitHub repos, and we know what code 701 00:41:24,170 --> 00:41:27,850 S1: changes go in and we know who owns what applications. 702 00:41:28,010 --> 00:41:29,650 S1: This is the holy grail. This is the thing that 703 00:41:29,930 --> 00:41:32,810 S1: management has never had which you will have due to 704 00:41:32,850 --> 00:41:36,330 S1: unified company context. You'll then be able to find the 705 00:41:36,330 --> 00:41:39,930 S1: actual vulnerability that can be fixed by the actual developer 706 00:41:40,250 --> 00:41:42,410 S1: in the actual code repo, and you can submit your 707 00:41:42,410 --> 00:41:45,450 S1: own PR and just have them approve the PR right. 708 00:41:45,489 --> 00:41:47,049 S1: This is the type of thing that you could do 709 00:41:47,050 --> 00:41:52,770 S1: with this unified company context. So the next piece is 710 00:41:52,770 --> 00:41:57,330 S1: start working on your set of defender capabilities. Based on. 711 00:41:57,370 --> 00:42:01,570 S1: Now you have your own context. You understand the attacker 712 00:42:01,570 --> 00:42:03,850 S1: context that they're going to be moving through as this 713 00:42:03,850 --> 00:42:09,610 S1: tech tree improves. And now you can start improving your system. 714 00:42:09,610 --> 00:42:13,530 S1: You can start assessing yourself continuously using these same exact 715 00:42:13,530 --> 00:42:19,610 S1: techniques and using your own internal UCC. And if you 716 00:42:19,610 --> 00:42:23,130 S1: do that, I think you're going to be in extraordinary shape. 717 00:42:23,130 --> 00:42:25,250 S1: That's what I want to share today, and we'll see 718 00:42:25,250 --> 00:42:29,129 S1: you in the next one. Unsupervised learning is produced on 719 00:42:29,130 --> 00:42:33,530 S1: Hindenburg Pro using an SM seven B microphone. A video 720 00:42:33,530 --> 00:42:36,450 S1: version of the podcast is available on the Unsupervised Learning 721 00:42:36,450 --> 00:42:39,609 S1: YouTube channel, and the text version with full links and 722 00:42:39,610 --> 00:42:44,890 S1: notes is available at Daniel Missler newsletter. We'll see you 723 00:42:44,890 --> 00:42:45,490 S1: next time.