S1: Unsupervised Learning is a podcast about trends and ideas in cybersecurity,
S1: national security, AI, technology and society, and how best to
S1: upgrade ourselves to be ready for what's coming. All right,
S1: welcome to Unsupervised learning. I'm here with Burrell Taylor, head
S1: of Minaya, and it's great to see you.
S2: It's great to be here. What a pleasure.
S1: Awesome. So before we get into the IO stuff, uh,
S1: it looks like you have a pretty interesting background. And
S1: I just want to, like, get a walkthrough of that
S1: real quick. Like, uh, what gets you excited about tech? Like,
S1: what have you been doing in tech all these years? And, uh,
S1: just like to hear about you.
S2: And, wow, it's a it's a very long and short time. Um, basically,
S2: since I'm 12, I'm programming, um, I'm into tech. I
S2: also like I wasn't too into game development and I
S2: also thought game development. And then I went into slightly
S2: into cyber of it, like hacking games into getting more
S2: like a, you know, getting money in the game or,
S2: you know, points or something.
S1: Yeah. Yeah. Higher. Higher scores.
S2: Exactly. And it's so fun. And yeah, especially in the
S2: early days, it was so easy.
S1: Yeah. Because everything everything was local, right? Uh, all the resources,
S1: everything was stored locally. So you could just edit it
S1: and it would appear on the server side. Yeah.
S2: Yeah, I remember all the all the Hexa, all the Hexa. Yes.
S2: All the hex that they tried to find the right
S2: place with the, with the scores to, to change. And
S2: it's also fun. Um.
S1: Yeah. You imagine the tools that we have now. It
S1: would be so much easier. But I.
S2: Know it's.
S1: Crazy. Yeah, yeah, yeah. Um.
S2: And then, like, you know, since then, I'm, I'm moved
S2: into more network security operating systems. I also continue into
S2: the Army. And so I have like a lot of
S2: experience in Army stuff like cryptography, networks, research. Um, then
S2: I was the first engineer in augmented reality startup. Um,
S2: and afterwards I built my own company and we did
S2: prioritization for, uh, for cloud native alerts, like for container,
S2: container image scanning. Uh, and then we got acquired and
S2: into mend, which is the same company I'm in now. Um, yeah.
S2: And it's now basically it's called Mend Container. Today it's
S2: the container reachability. And now also I moved into mend AI,
S2: which is a new product, and we founded, uh, to
S2: do AI security.
S1: Very, very cool. What do you think? It's like the
S1: common thread going through all of this. Like the main thing, uh,
S1: driving curiosity.
S2: Um, wow. Amazing question. Um, I think I always was really, uh,
S2: really excited from new, uh, new industry changing technology. It
S2: was always the technology that, uh, that, that, like, was
S2: enabling something and, and make it so exciting, uh, especially
S2: in the startups world where you have these giant companies
S2: and one day or like in a process, pretty quick process, um,
S2: they're all disrupted. There's tons of new things, uh, you
S2: can build. And yeah, it's a challenge it to challenge
S2: the Giants. So it's amazing.
S1: Yeah. So, um, are you mostly interested in the network
S1: security stuff? Obviously you're doing AI stuff now. Everyone's doing
S1: AI stuff. Um, but like network security, application security. Uh, like,
S1: what is like your center of mass? Like your favorite thing.
S2: So for sure, 100% application security. It's so exciting. But basically,
S2: at the end of the day, it's all about developers
S2: and the vulnerabilities, um, like you have in your code. Um,
S2: so my focus is now application security and AI security
S2: for inside the application. And that's like our special take. Um,
S2: we look just on application, we try to, you know,
S2: there's so many issues there. Um, also on the AI
S2: components there. So.
S1: Yeah, what do you see is the biggest problems right
S1: now in Appsec? Um, obviously a big part of that
S1: is going to be AI because there's so many AI applications.
S1: But would you say AI is the biggest sort of
S1: application security thing happening right now?
S2: I thought like we looked on many issues and when
S2: we founded the Atom Security and it was clear the
S2: biggest issue is prioritization and the the like the size
S2: of the backlog of vulnerabilities of critical vulnerabilities you have. Yeah.
S2: And then now with all the reachability technologies that allows
S2: you to to like reduce the noise, basically remove the
S2: false positives and the, the trend of platforms and aspm
S2: allows you to take all the findings in one place
S2: and prioritize them smartly. And like we see the biggest
S2: issues now like a what's introduced in AI with AI.
S2: And then there's like two different things. There's the I
S2: the employees are using in order to be more productive
S2: so that it's reduced some risks. And and what we
S2: see as like the biggest risk. And because we like, uh,
S2: we see how the world is changing and how our
S2: customers are moving like towards the, um, in the products. Um,
S2: is the AI components inside applications and especially the AI agents, um,
S2: inside these applications that are in production.
S1: Okay. So when you say AI components, do you mean like, uh, libraries?
S1: Like what other pieces other than the agents do you
S1: mean by AI components?
S2: So it's a great question because like in the beginning, um,
S2: when when I like did simple data science before before,
S2: you know, ChatGPT and llms, um, there wasn't many components.
S2: You know, you had data set and then you build
S2: with it some some machine learning model. And and now
S2: it's the amount of components is exploding because you have the, um,
S2: you know, the data layer and you have data set
S2: and then you have also data is like changing because
S2: you have data for training. But if you take already
S2: a model which is another component and then fine tuned it,
S2: or just doing alignment. So you have many types of data.
S2: You also have many types of models. You have models
S2: you're using for training, models you're using for uh, for, uh,
S2: for like for fine tuning and models are using as is. Yeah.
S2: So that's more components. But we also have all the
S2: components on the code layer. And we have the system
S2: prompt and the agent Ancient tools and the third party
S2: tools like MCP servers. Yep. And the actual agents that
S2: can speak with many of these tools. And we have
S2: like many agents we're already seeing, we already see adoption
S2: of like multiple multiple agent frameworks.
S1: Yeah. And maybe the APIs themselves, which they're not actually
S1: AI components, but those agents will be calling back to
S1: traditional APIs as well. Yeah. So that's a pretty good
S1: list of the components. You're going to mention something else
S1: like another another component.
S2: And I'm definitely someone just asked me today like what's
S2: the difference between like old APIs, third party APIs and
S2: MCP servers or agent tools? And basically like you, you
S2: would you would think there's no difference. And I would argue,
S2: but like, tell me what? Maybe it's also a question
S2: for you. Tell them what you think. I argue the
S2: main difference is the way you're using them and not
S2: the tool is basically the same tool. The way you
S2: use them is that it's more close and more connected
S2: to your data, your critical data.
S1: Yeah, yeah, I would say another big difference is that, um,
S1: usually when you build an API from scratch, you have
S1: a skilled developer who is going through very systematic process
S1: to define exactly what methods are possible. They still make mistakes, obviously,
S1: because we have API security problems. However, at least it's
S1: being manually done right. Whereas with um, I feel like
S1: with MCP servers, the problem is you could have a,
S1: you know, data and you can have APIs on the
S1: back end. You spin up this NTP server and it
S1: just kind of goes and collects all that functionality and
S1: presents it's its own new APIs that could be used
S1: by the ancient tools. So I feel like you could
S1: create more functionality without knowledge. And that that's kind of
S1: the issue, because you might be surprised by actually what
S1: can happen through that NCP server. So I think, yeah,
S1: I think a lot of stuff is being stood up
S1: with NCP servers, and the people hosting it don't actually
S1: know all of its capabilities.
S2: It's a great point. Also, in a way, you can't
S2: know all the capabilities because part of the like, the
S2: best thing about this AI revolution is that the interface,
S2: but also the worst, is that the interface is fuzzy.
S2: The input and the output is fuzzy. It means.
S1: Yeah, it's 100%.
S2: You can't define exactly the signature of the function with
S2: all the variables. That's only integer. You can't put string there, right?
S2: So now you can afraid only from integer overflow maybe.
S2: And and and now the interface is so fuzzy. It's text,
S2: it's PDF it's voice. It's PDF with image that has
S2: some something inside it. You have so many options. So
S2: the text is huge.
S1: That's right. And then there's also the issue of like
S1: the fuzziness of if you're actually interacting with an agent
S1: on the receiving side that is front ending but separate
S1: from the MCP server. If you're talking to an agent
S1: and it has the ability to use the tools, you
S1: might be able to confuse it or trick it into
S1: using the APIs it has available in unsafe ways. Right. Um.
S2: That's a great point.
S1: Yeah. And it might it might respond back and say no.
S1: And you ask in a different way and it still
S1: gives you results.
S2: And by the way, we've seen in the wild and many,
S2: many patterns, malicious patterns we've seen with open source libraries,
S2: like in the beginning of a, you know, the concept
S2: of SCA and open source security and with models and
S2: like a concept, like a typosquatting when you're trying to
S2: do phishing to humans. Yeah. So humans are close, you know,
S2: the wrong open source library because they change something. They
S2: added a dash. So same with models. We've seen it.
S1: Yeah. For like for like npm packages, stuff like that.
S1: Package managers.
S2: Exactly. So we've seen it also with models in the
S2: evolution of us of our product. And now we're seeing
S2: it with both with agent tools and libraries. So cursor
S2: uh you can very easily precursor uh to use a
S2: typosquatting package. Let's call it this way. And same for
S2: NTP servers. You have malicious FTP servers.
S1: Right? That's a that's a great point. And that could
S1: just be a man in the middle that just passes
S1: on requests. Right?
S2: Exactly. It's totally it's look 100% legit.
S1: Yeah. But I'm saying when you submit to that malicious one,
S1: it could be still submitting to the the real one
S1: and returning you real results. But in the meantime, gathering
S1: data or doing whatever.
S2: I guess that's the worst, because when you have a
S2: Bitcoin miner, I guess like the let's call it malicious
S2: actor knows that it's going to be caught in the
S2: next few months, and you're trying to make the best
S2: out of this month. And this silent, silent man in
S2: the middle type of malicious packages, malicious models and malicious servers,
S2: that's probably the worst, especially when it's third party. so
S2: you don't need even to open source your server.
S1: Yeah, MCP is going to need some serious security help
S1: very quickly. It's because everyone's just running full speed, installing
S1: as many of these things as they can. And yeah,
S1: it's it's a good point. It's it's a really big
S1: mess right now. Um, what what other things? Uh, we
S1: got malicious MCP servers. Um, we've got agents that can
S1: be tricked. I always talk about just like the, um,
S1: the agents having too many tools available. Uh, because oftentimes
S1: I think when if the business is pushing, like, we
S1: must have I, we must have I. Oh, and by
S1: the way, we must have an agent. They just give
S1: the agent access to too many tools. And, um, the
S1: guardrail infrastructure isn't really there yet. I don't know if, uh,
S1: you guys have something around this, but, um, Um, like, uh,
S1: I use bedrock a lot, and bedrock has, um, uh,
S1: some pretty cool guardrails stuff built in, but we're running
S1: way faster than the guardrails are being laid down. So
S1: I just feel like there's a there's a mismatch between
S1: the amount of power that agents have and the amount of, um,
S1: access and power that they have.
S2: I think this asymmetry is was right for every new
S2: category in security. That's true. I think AI security is
S2: probably one of the quickest categories to to catch up.
S2: If you look like on on games, for example, game
S2: security on operating systems, um, in the early days or
S2: networks in the early days, we still have protocols like
S2: DHCP and ARP or DNS. It's so easy, so much
S2: trust into into other, uh, other people in the early days.
S1: Yeah.
S2: Categories like a car's security. Um. Oh, yeah. Security. Very.
S2: It's amazing how quickly they adopt security practices even before
S2: the log4j2, uh, happened. Of the industry?
S1: Yeah, it's a good point. Yeah. ARP is always the
S1: one that trips me out the most. It's like. It's like, um.
S1: You don't even have to ask a question. You could
S1: just receive answers like, oh, by the way, um, here
S1: is the Mac address that you're supposed to talk to.
S1: And your host is like, thank you very much. I
S1: will update that table immediately. It's just.
S2: It's amazing.
S1: And the fact that it all still works. Yeah. Interesting.
S1: So I think I agree with you. I mean, we're
S1: basically running with scissors with I, um, it's funny because
S1: I've been in the space for so long, like 90
S1: since 99. So we, uh, and I started mostly in
S1: network security. And as you move to web security, you
S1: have to relearn all the network security issues, right? We
S1: learned that those lessons for ten years, 15 years, we
S1: forget them when we move to web security, we forget
S1: them again. You go to mobile security, you forget them
S1: again a little bit when you go to cloud security.
S1: And now with AI but yeah, maybe. Well, okay, here's
S1: the question. Why is AI security picking up so fast
S1: compared to the other ones? Why is the delay shorter?
S2: Um, I think it's I think if you like, if
S2: you think about it, um, everyone understand all the products
S2: are going to have AI. And because of this fuzzy
S2: interface and that you want to you want it to
S2: be connected into your most important data sources. And I
S2: think everyone understands kind of a game of of who's
S2: going to be the first lock forward in a way.
S2: Maybe everyone just understand it's a matter of time and
S2: you don't want. It's just you don't want it to
S2: be you. The first lock for Jay.
S1: That makes.
S2: Sense. I understand I understand all the companies, like all
S2: my customers that like trying to push hard for to
S2: develop with AI, to develop AI into their products. Because
S2: if they won't do it, all the competitors will do it.
S2: I mean, it's a huge advantage. Also us as like
S2: a security vendor, we need to we're using AI to
S2: solve problems in our products to to understand code and
S2: to suggest code. Remediations. And it's something that you know,
S2: or you're going to be the first to do it
S2: and you have this advantage, or you'll be the last
S2: to do it. And you Like you lose the game.
S1: Yeah, that could be it. The the fact that it
S1: just feels so big. People just have a natural fear
S1: of it. Whereas maybe it was slower with the previous revolutions.
S1: How do you see the distinction between security of AI
S1: versus AI security.
S2: Security of AI? Basically, with every new every category in security.
S2: When you say security, it usually means securing X, right? Like, yeah,
S2: a cloud security. It's securing the cloud. SaaS security is
S2: securing SaaS somehow with AI security. Uh, it's it's still
S2: confusing because I guess it's in the early days, uh,
S2: I think we should align that AI security is securing
S2: AI and not like it's not, uh, securing the the
S2: output of AI or something like that. Um, but a
S2: time will tell, I guess.
S1: Yeah, yeah. I wonder if they start to merge. I
S1: think the reason, maybe one of the reasons that it
S1: started out being separate was this whole concept of, um,
S1: just the model, uh, remember, uh, data poisoning. It's not
S1: talked about as much as, like, um, in 22 or 23,
S1: but it was like, what is the, um, can the
S1: data be poisoned that, uh, you know, the eyes are
S1: being trained on? Right. So it was like, I don't know,
S1: there's just not nearly as much focus on that anymore.
S1: And now it's more about it. I would agree with you.
S1: I think it's actually merging more now.
S2: Yeah. Also. Yeah. Also like, uh, in the beginning everyone
S2: spoke about like, AI driven security. Um, and it's kind
S2: of funny because, uh, anomaly detection, it's basically like using
S2: AI in order to find anomalies. Um, and so many
S2: of the categories are already using heavily, heavily based on AI.
S2: But if we look in the future, there's not going
S2: to be any vendor that's not using AI. That's right.
S2: Also in application security, like your promise at the end
S2: is suggesting how to harden your code. And of course
S2: you're going to use AI.
S1: That's right. That's right. We don't have any database companies
S1: because um, what we do that that only make databases
S1: or whatever. But every company is a database company. Every
S1: company is an Excel company. Like. Yeah, it's just, uh,
S1: it's just built in, um, so so what what do
S1: you think about the current solutions? Uh, what do you
S1: think about current solutions in terms of, like, what are
S1: the current, like, appsec vendors, like the ones that have
S1: been around for, you know, 15 years or whatever? How
S1: are their solutions like By solving these problems that we've
S1: been talking about.
S2: And the really simple answer we just don't. And like
S2: a classic abstract, abstract solutions, doing a really good job
S2: in detecting some patterns in your code of vulnerabilities, like
S2: CW is at the end of the day, inherently is
S2: a pattern, and not it's not a technique to solve it.
S2: To find the CW, the way we think about it
S2: is about finding patterns and CVS, which today is maybe
S2: a bit sad day for CVS. And let's say there's
S2: many other security advisories. GA the others, you know, the
S2: GitHub security advisory, the Ruby Security advisory. So all of them, um,
S2: it's something that it's in your libraries. Um, so the
S2: way you think about it is only libraries and code issues.
S2: But the thing about AI security is that first, to
S2: even to detect these components, it's not enough to look
S2: in libraries. And libraries can give you a hint. And
S2: actually it's something it's a great thing we do. We
S2: only using the libraries tell you the hints. It it.
S2: I remember the day when we found you can do it.
S2: You can just just take the libraries even before you put,
S2: you know, some heavy scanners on all the places. Just
S2: take the libraries and extract all the hints you can
S2: about the usage of AI. And that was like a great, uh,
S2: you know, uh, Aurora for us. Um, and, uh, and
S2: that's maybe the first step, basically. How do you find models?
S2: Models is something um, or you find it the art
S2: of the model artifact, which can be either the repository
S2: in the container in some S3 bucket or spatial, uh,
S2: you know, models, repositories. So that's a new place for models.
S2: But if it's, you know, third party, if we're using
S2: some inference providers, which from my experience and most customers
S2: starting from using inference providers, most companies, they start with
S2: some prototype and they will use the easiest, you know, OpenAI, Azure,
S2: OpenAI bedrock. Yeah. And they can so so that's something
S2: you can't find anywhere. Not in the container. Not the
S2: it's not an artifact. And the only place is in
S2: the code. And none of our current solutions is like
S2: is positioned us to find it. And so, so like, uh,
S2: you need a new, new solution. And you I think
S2: the AI security companies are going to be in your
S2: products are going to be something like totally new, and
S2: that is going to be merged into the current DevSecOps
S2: and Appsec workflows.
S1: Yeah. The way I think. Think about it. Or I
S1: always used to think about it because my background is
S1: largely web security, um, which we would have a major
S1: distinction between dynamic and static security. Right. So, like, I
S1: was at 4 to 5 for a very long time,
S1: and there were the static people over there, and we
S1: were the dynamic people. I feel like the AI piece
S1: definitely is on the dynamic side. Um, it has to
S1: be right. So rather than just like web testing and
S1: API testing, it's got to be more comprehensive. Um, so
S1: let's just jump right into.
S2: We, you know, we started with, uh, with the static
S2: part of AI because we found that the biggest issue
S2: is threat modeling just to discover what you don't know
S2: you have. And we found that, uh, that's we found
S2: that it's not just a big issue. It's like huge
S2: issue because most companies just don't know what they have
S2: on the order of ten. But then we because of that,
S2: we moved into the dynamic section. And when we're now
S2: like trying to offer both because at the end of
S2: the day I a model, there's a bunch of numbers
S2: and like code you can understand. You just can't understand it.
S2: And the only way is through through a conversation, through
S2: like dynamic and simulating attacks, through pentesting basically.
S1: Yeah. Yeah. Absolutely. So. So let's think about that. So, um,
S1: just talk me through like how your solutions are set up.
S1: Like what is the the basic tagline for it like, um,
S1: is it, uh, is it asset discovery? Are you discovering
S1: like the the structure of the application? Are you enumerating
S1: like controls? Roles. Are you testing controls? What exactly is
S1: it that the suite does?
S2: Um, so it's it's kind of both. Um, we're starting from, uh,
S2: statically scanning all the assets, all the AI resources, components,
S2: you can call it in different names, and to find
S2: all the AI you have. And usually at this stage
S2: we find that usually we start like, you know. Like
S2: average company will say we have some AI components, some
S2: AI driven applications. Then where we do the initial scan,
S2: we found it's more than that by a factor of ten.
S2: And and that's every time exciting to see like and
S2: you know it's it's you in the security. It's really
S2: easy to say that it's bad. But actually it's pretty
S2: amazing how the industry is trying to push harder to
S2: push forward. So so even without the security knowing everyone,
S2: just trying to use it, trying to use it, trying
S2: to make a smarter applications and more valuable products. Um,
S2: so so we detect it. And then the question is, uh,
S2: if I have dopesick, uh, it's a drug. Maybe it's
S2: bad because rag connected to data, you know, you don't
S2: want to leak this data, and you may be concerned
S2: about dopesick unless you're in China, which is a very
S2: it's the opposite. You probably want to use dopesick. And
S2: so you just want to know what you have. Then
S2: you want to detect all the risks you have in
S2: these components. And which is kind of a composition analysis, um,
S2: when you think about it in deeply, not only libraries
S2: in SCA. Um. And that's amazing. You find, uh, legal risk.
S2: You find security risk for all these components. It can
S2: be models, uh, you know, uh, agents and speed to
S2: MCP servers, agent tools.
S1: And how are you getting all the components? Are they
S1: providing them to you, or are you getting them dynamically? Like,
S1: how are you getting these from the customer?
S2: Um, so we started with taking the hints from the
S2: open source libraries, and then we used these hints to
S2: look into the code and find, uh, all the components
S2: in code. Um, and if we have, uh, if you're
S2: using self-hosted models or like open source models, uh, we
S2: looked for artifacts. So we'll find a file that we, that,
S2: you know, from the magic of the file, from the
S2: headers of the file, we know it's a model. And
S2: then we have some fingerprinting, uh, technique to, to match
S2: it into the actual file. It's a inherit from. So
S2: if you took a file, fine. Tuned it, you know,
S2: took something from hugging face. Finding it and using it
S2: will tell you about this file and that it's related
S2: to hugging face model and hopefully not malicious one.
S1: Oh, interesting. I mean, that's that's an offering by itself, right?
S1: Just like asset discovery.
S2: Of course, you'll be surprised by how much, um, you know,
S2: there's a the proportion between what you know and what
S2: you don't know, uh, is, is large.
S1: It's pretty.
S2: Amazing.
S1: Yeah. Whenever, whenever I do security assessments, I usually have
S1: a really large visual that I'm building out throughout the week.
S1: And honestly, it's just laying out what the application does.
S1: It just lays out what the functionality is, where the
S1: data is flowing. And as I'm interviewing more and more
S1: developers and more and more people in the company, I
S1: just bring them in and show them this thing and
S1: they're like, oh yeah, everyone starts taking pictures, they start taking,
S1: they're like, oh yeah. Yeah. So because they don't have
S1: any documentation that's actually this good. And it turns out
S1: if you just see it, if you just visualize it,
S1: you're like, it's obvious to everyone who walks in the
S1: room that this is a problem and, you know, when
S1: it's not visualized, um, or it's not explicitly laid out. Yeah.
S1: You just miss the stuff. Okay. So so you have
S1: the list of components.
S2: By the way. It's a, I like like all the
S2: trend of showing topology.
S1: Yes.
S2: What you have. Yeah I love this trend. Um, I
S2: think there's a really large debate in the security industry
S2: because people say at the end of the day, it's
S2: not it's not showing the data. You need tables. But
S2: there's something about visualizations that like pass the give you
S2: the value shows it in a different way. And it's funny.
S1: I would just show an arrow and I would like
S1: color code the arrows like according I would have the
S1: data classification for the company on the board. And if
S1: it's one of the like last two data classification like, um,
S1: sensitive secret, whatever their classification is, then I would just
S1: have all the connecting dots or the connecting arrows be red.
S1: They're like, why are these red? Because, um, you know,
S1: Sarah over here or John over here said that that
S1: type of data classification is included in this data. And
S1: suddenly after you talk to 20 different people, the whole
S1: board is red, right? The whole board's red. And they're like, okay,
S1: I didn't realize the problem was this bad. Yeah. And
S1: then those documents end up being used as the official
S1: document for the application going forward. So I feel like
S1: this is this is absolutely needed, especially for I when
S1: I talk to people about AI implementations, I'm just like,
S1: Show me exactly where the agent is in this workflow.
S1: Show me exactly which APIs it has access to. And
S1: as they start writing it down, they're like, oh, well,
S1: I think I see the problem. I haven't even said
S1: anything yet.
S2: Yes. It's amazing. You know, I love threat modeling. I love, like,
S2: doing it, uh, with customers. And you started from it
S2: so much.
S1: Yeah, absolutely.
S2: And at the end of the day, uh, like, it's
S2: not enough because you need to detect the issues. Um,
S2: so there's malicious models, which is, like, surprisingly, surprisingly, surprisingly, like, uh, getting, uh,
S2: evolving a category. And because in open source models, there
S2: is a you think a model is only like a
S2: metric of numbers, but actually it has also some serialized code.
S2: It's like many of the many types of models are
S2: like pickle files. Yeah. Not necessarily. It can be a
S2: family of, uh, of types of people files, uh, which
S2: is a code that is serialized into some, like, uh,
S2: some opcodes.
S1: It's always it's always the parsers. It's always the parsers.
S2: Yes. Yeah. And it's going to, you know, pull code.
S2: It's theoretically it can pull another binary, uh, from remote
S2: and run it. Yeah. So it can be super malicious.
S1: Yeah.
S2: It's also like a known risk, you know, like other
S2: classic vulnerabilities. Um, only this week I saw a company that, uh,
S2: scanned all the papers, like, all the academic papers, and
S2: try to extract the existing attacks and to map, you know,
S2: attacks to models. Hmm. I think we still it's still not,
S2: you know, we're still not there at the end of
S2: the game. It's not like, uh, see that you have
S2: so many security advisories. I think we're getting there. And
S2: we're for sure more mature as an industry.
S1: Yeah. Okay. So let's say someone what is an ideal
S1: customer look like for you, like in terms of they
S1: come to you and they say, I have this problem
S1: or this problem or this problem and you say, okay, perfect.
S1: That's exactly what we do. What would those problems look
S1: like and what would you tell them? The solution is.
S2: Um, there's two types of companies and I see all.
S2: It's a company that's concerned about AI, but they don't
S2: know what to do. And so that's kind of the
S2: free threat modeling stage. That's that's a stage where the
S2: first thing you need to do is discover everything. You
S2: have all the agent models. Um, usually a company will say,
S2: we have a policy that says we're only using, you know,
S2: Azure AI. And then we'll find many, like a hugging
S2: face model lipstick and other service providers. And and that's
S2: the first stage. And once you we hope to do
S2: the threat modeling, we see how the companies start like
S2: thinking and getting more advanced. And the other type of
S2: companies are the a bit more sophisticated ones where they
S2: already have some like 2 to 3, let's say, or
S2: a few like major AI driven products. They know about them.
S2: They already like a threat model them. So the discovery
S2: will still help them a lot, but the more advanced
S2: they know, they have some, let's say uh, tax uh application,
S2: uh reviewer, automatic reviewer. Um, that like is very smart,
S2: but it's very risky because it's a PDF and something
S2: can happen. Usually they wouldn't know exactly what the risks.
S2: They wouldn't know that what they need is, you know,
S2: some something that will simulate attacks of sending PDF with
S2: an image inside the image. It will have some prompt injection,
S2: but they will know they will. Heard about the OWASp
S2: top ten for applications, which is, by the way, great
S2: a list of threats and, you know, great awareness.
S1: Yeah, that makes sense.
S2: And then we like tell them, hey, you have here
S2: potential a malicious model. Check it. You have here IPsec.
S2: And that's the findings from the dynamic scanning of the
S2: you know, all the attack simulations. Okay. And then we
S2: need to fine tune how to have to do what's
S2: the right attacks. What's the most important attacks to check.
S1: Nice. Okay. And then what what is the product like.
S1: What is the product suite do like what are the
S1: the pieces of functionality.
S2: And so it's basically what we spoke about. And it
S2: starts with discovery of all the components. Then it moves
S2: to the risk of each component individually. And once you
S2: know about all the components you need somehow to connect
S2: them into a behavioral risk and to to understand, to
S2: contextualize all the risk of the entire components together. And
S2: because you can't have a system prompt, you can't have
S2: a model. And you can have some, you know, rug
S2: that connects them to a database. But unless you understand
S2: you have this database, this system from twist, this model,
S2: that's the only way to understand that you have context leakage.
S2: And so for that we have these behavioral risk rating
S2: automatic rating attack simulations and drop everything. We're doing a
S2: mitigation what we call governance and mitigations and where we
S2: where you can create policies to prevent let's say if
S2: it's Rug and Leipzig block it, this facial combination. And
S2: in the near future we're going to release the ability
S2: to what we call guiderails, which is not guardrails. It's guiderails.
S1: Interesting.
S2: The concept of mitigating putting like a the mitigations into
S2: the code.
S1: Not like.
S2: A.
S1: Firewall. Like dynamic. Dynamic detections as it's coming in.
S2: No. So actually not dynamic. And the point is that
S2: integrated into the development process. And the developers will get
S2: suggestions for mitigations to the code. For example, add to
S2: the system prompt this and that in order to block
S2: this redeeming findings in order to mitigate from these findings.
S2: For example, a format your output of LLM into more
S2: like a less fuzzy format because you don't need a
S2: fuzzy output. So why, uh, like why open your attack
S2: surface and.
S1: So many.
S2: Small things that everyone has. Everyone got these issues and
S2: you have agent, maybe the same agent should not access
S2: the data and run code afterwards, because if someone will
S2: like prompt injection the agent, you can run code that
S2: access the data. And so you know these simple. There's
S2: so many simple steps no one knows.
S1: Yeah, that makes sense. Okay. So it's it's the initial
S1: assessment of, like, what the attack surface is. Um, just
S1: discovering the assets. Then there's the dynamic assessment, and then
S1: there's mitigation.
S2: Exactly.
S1: Perfect. Um. All right. What do you think is happening next?
S1: What are you worried about happening coming up? Uh, trends
S1: or risks that you see coming up soon?
S2: Um, I think we it seems like we have a
S2: every new like it seems like every, every every old
S2: category in security and need to find a way to adapt.
S2: And because, like speaking with customers and we see that
S2: identities that is like, you know, kind of an existing, uh,
S2: category with tons of issues now has a new aspect
S2: in the AI driven applications because you want to manage,
S2: you want to make sure one user can't access a
S2: different user data. And so, you know, it's old problem
S2: with a new suit, let's call it. And so.
S1: Yeah.
S2: We see so many so many, so many nutrients. And
S2: exactly like you said before, so many new same problems
S2: and new problems which basically the same. So that's one
S2: thing I think, which we need to remember it that
S2: all the new things are basically old.
S1: Yeah. Yeah. I like you're talking about um, yeah. I
S1: think the distinction between what an agent is doing and
S1: making sure it's not acting on behalf of an actual human. Right. Uh, making. Yeah,
S1: non-human identity versus human identities and making those distinctions, having
S1: separate policies, separate policies for those. I think that's going
S1: to be important.
S2: Yes. And, you know, like the same concept. You want
S2: multi-tenant also for agent. You want permissions also for agent and.
S1: Separation of duties. Like all like you said, all the
S1: old stuff we have to re-implement relearning the lessons from,
S1: you know, 25 years.
S2: Exactly. And but I think like if you, if you
S2: look on the new things classes and you see that
S2: multiple agents of course, but multiple agent frameworks, agents orchestration
S2: and create more issues because it's way more complicated to
S2: understand how it works and the communication between agents. Open
S2: the make the exploit way more complicated. It's way more complicated,
S2: but like way stronger. A very similar to to what
S2: you have when you try to exploit memory corruptions and
S2: that you try to jump between a different places in
S2: the program in order to finally run the code. And
S2: so it's very similar to here. And if the code
S2: is if you have more attack surface, when you have
S2: the code is larger and you have more places and
S2: more gadgets that you can more places you can jump
S2: between each other until you run the code. And here
S2: you have many agents. So you can you need just
S2: to find the agent, you know, to send the right
S2: prompt into to run the code, I think.
S1: I think it's worse, like you said, I think it's
S1: worse with multi-agent because that's more opportunities for tricking. Like
S1: you might have a smart one. Um, so a prompt
S1: injection might not detonate with the first one you're talking to,
S1: but it might pass it along to a dumber one
S1: behind it. And that one it will detonate on. Yeah. Interesting. Um.
S2: Also, if you think about it, when you have more
S2: components that speaking with each other, it's hard to track.
S2: What's the purpose of each one. So you're going to
S2: have these, you know, uh, let's call it a small
S2: drift in, in the communication between them. Uh, there's one
S2: that should do X, the one should do Y, but
S2: they will have somewhere like some very small interaction that
S2: will probably be exploitable.
S1: Yeah. Because for each one.
S2: Individually is totally okay. It looks fine.
S1: Yeah. So for the CISOs that are listening, like, what's
S1: the one piece of advice like what's the one thing
S1: you would say to someone who's implementing AI or trying
S1: to secure it?
S2: Um, I would say the, the easiest thing to do is,
S2: at least from my experience, is to try to create
S2: really hard and forcing policies. And that prevents everything and
S2: blocks everything. And that's really easy. Um, but I think
S2: the right way to go is to create integrated workflows
S2: like any other security issues, and that will help to enable, um,
S2: enable developers and enable the products to be really good
S2: and valuable in the AI driven, um, and more and
S2: less disable, um, and it's right for everything. And I
S2: think it's especially for AI and I see it firsthand, uh,
S2: maybe a bit too much, unfortunately.
S1: Yeah, that makes sense. And where can people learn more
S1: about the company?
S2: Um, so Mend-ooyo is an apps company? Um, traditional apps company.
S2: One of the first SCA vendors that grew into, you know, SAS,
S2: SCA container scan container image scanning. Like I spoke before
S2: about the ability technology that got acquired for and now
S2: also AI security. And we love what we do. We
S2: have a lot of customers. Um, and uh, we're I
S2: think we're the first AI security vendor in the market
S2: doing it. Shiftleft. And so we're really proud about it
S2: and hoping to make awareness in the industry about it.
S1: Awesome. Well, it was great chatting with you and, uh,
S1: look forward to chatting again.
S2: Lovely. It was my pleasure.
S1: All right. Take care.
S2: Thank you.
S1: Unsupervised learning is produced on Hindenburg Pro using an SM
S1: seven B microphone. A video version of the podcast is
S1: available on the Unsupervised Learning YouTube channel, and the text
S1: version with full links and notes is available at Daniel
S1: Miessler newsletter. We'll see you next time.