Speaker 1: Quick question, do you know about the Xzu Till's back
Speaker 1: door hack? What backdoor?
Speaker 2: Wait?
Speaker 1: Wait wait wait wit what the xu tills back door?
Speaker 1: I have no idea what you're talking about.
Speaker 3: I don't know what that is.
Speaker 1: This is something almost nobody's heard of, but in the
Speaker 1: spring of twenty twenty four, we narrowly avoided a complete
Speaker 1: technological disaster.
Speaker 3: So you've you've never heard of this though? Nope?
Speaker 2: Yeah, I just searched it up on Wikipedia and it
Speaker 2: seems way too nne.
Speaker 3: To read about.
Speaker 1: These aren't just random people. These are other journalists people
Speaker 1: in general who keep up with the news.
Speaker 4: Okay, I was like, wait, what did I miss?
Speaker 3: And I feel bad, But I guess maybe I'm not
Speaker 3: the only one. What journalists who doesn't know what this
Speaker 3: is about?
Speaker 1: Even they didn't really know about what could have been
Speaker 1: the biggest hack in the history of the Internet.
Speaker 2: If this had not been caught, then this would have
Speaker 2: been a skeleton key that would have allowed these attackers
Speaker 2: to break into tens of millions of incredibly important servers
Speaker 2: around the world. We probably would have had airlines not working,
Speaker 2: trading halted ATM's, not working, banks not working, people not
Speaker 2: able to get their money, you'd have a huge loss
Speaker 2: of credibility of technology in people's lives.
Speaker 1: Alex Stamos is a cybersecurity expert. Specifically, he's the chief
Speaker 1: Information Security Officer or CISO at a cybersecurity company called
Speaker 1: Sentinel One, and he's the former CIECO at Facebook. He's
Speaker 1: also a lecturer in the computer science department at Stanford
Speaker 1: and this attempted hack is something that is still keeping
Speaker 1: him up at night.
Speaker 2: It's fallen out of popular discussion, but among people in
Speaker 2: security we're still talking about it. It uncovered a real
Speaker 2: fundamental weakness that terrifies lots of people who have responsibility
Speaker 2: in this area.
Speaker 1: And what scares them most, and this should scare us too,
Speaker 1: is that this was caught by complete chance.
Speaker 2: We just got lucky, right like one dude got really
Speaker 2: bored and noticed a tiny little change in the speed
Speaker 2: of one program executing and pulled the thread. And on
Speaker 2: the end of this thread was a humongous, ticking time bomb.
Speaker 2: It was one dude, and he should never have to
Speaker 2: buy a beer for himself ever again underus freuend I'm raising.
Speaker 4: A toast to you right now. This is just water,
Speaker 4: but I wish it was more.
Speaker 1: Kaleidoscope and iHeart podcast is killed switch com dexterous.
Speaker 2: Amentarians.
Speaker 1: If you've never heard about this, that's no reason to
Speaker 1: feel bad. But if it hadn't been caught, it absolutely
Speaker 1: would have affected you. What kind of activity were talking
Speaker 1: about here.
Speaker 2: Well, we really don't know, and because we don't know
Speaker 2: who the attackers are, we don't know whether that would
Speaker 2: have been used for really quiet surveillance. It could have
Speaker 2: been used for national security intelligence gathering purposes. It could
Speaker 2: have been used for a humongous heist of hundreds of
Speaker 2: millions or billions of dollars of cryptocurrency, or it could
Speaker 2: have been used as part of a massive cyber attack
Speaker 2: to shut down millions of can and cause massive disruptions.
Speaker 1: One of the main reasons that this potential attack isn't
Speaker 1: talked about much is because the details are kind of technical. Well,
Speaker 1: some of the details are a lot of this stuff
Speaker 1: is really just basic human behavior. It's stuff that you
Speaker 1: or I could do if we really wanted, and it
Speaker 1: shows us that sometimes the best hacks are the simplest ones.
Speaker 1: Let me break it down for you. In late March
Speaker 1: of twenty twenty four, Andres Freund who's an engineer at Microsoft,
Speaker 1: was sitting at his desk doing his job when he
Speaker 1: discovered a malicious piece of code in this little known
Speaker 1: tool called xdu tilS. This code created a method that
Speaker 1: would allow hackers to access a.
Speaker 3: Lot of different computers.
Speaker 1: Maybe right now you're thinking, okay, so why is this
Speaker 1: the problem for me? I mean, I don't use xdu tilS,
Speaker 1: so they couldn't get on my computer. And yeah, maybe
Speaker 1: you've never heard of XCU tills. Actually I hadn't either,
Speaker 1: and I did what most people do when they don't
Speaker 1: understand something about a computer, call an expert. But it
Speaker 1: turns out that this really well respected expert he found
Speaker 1: out about XU tills when I did.
Speaker 2: Yeah, so I personally had not heard of xdu tilS
Speaker 2: before this.
Speaker 1: Even you really, yeah, I had definitely not heard of
Speaker 1: XU tills. I figured you would have hearing that you
Speaker 1: had not heard of it before all this happened. Frankly,
Speaker 1: that's a little bit more scary to me. Now, So
Speaker 1: why did this backdoor into a program that no one
Speaker 1: seems to know about matter so much? And what is
Speaker 1: XU tills.
Speaker 2: This is the brilliance of what these attackers did XCU
Speaker 2: tilS is an ingredient to an ingredient to an ingredient
Speaker 2: to something really important. So the thing that they wanted
Speaker 2: to have a backdoor into is a really important program
Speaker 2: called open ssh.
Speaker 4: So this is something that every tech he has heard of.
Speaker 1: All right, but what if you're not a techie. So
Speaker 1: in order to understand the XU tills haack, we do
Speaker 1: need to back up and understand something that xcutil is
Speaker 1: used in this thing called open ssh.
Speaker 2: This is the program that the majority of Unix like systems,
Speaker 2: especially Linux, also Max and some other operating systems, allow
Speaker 2: you to access them remotely over the internet.
Speaker 1: I'll get to the open later, but SSH stands for secure, show,
Speaker 1: and let's just focus on secure right now. If you
Speaker 1: think of the difference between posting a tweet online and
Speaker 1: dming someone, you're actually kind of halfway there. Open Ssh
Speaker 1: allows you to communicate with a remote computer just like
Speaker 1: you were sitting there right in front of it. So
Speaker 1: even though you're far away, if you want to send
Speaker 1: a message, or install programs or delete files, you know
Speaker 1: that the connection is safe and that nobody else can
Speaker 1: see what you're doing or tamper with that connection.
Speaker 2: So you know, when you see like people in the
Speaker 2: matrix typing really fast, see a lot of text, right
Speaker 2: if somebody is doing that remotely, it's probably over open ssh.
Speaker 1: You might think this doesn't matter for you because you
Speaker 1: don't use open ssh, but you do because that's what
Speaker 1: you use to connect to systems running Linux. Around the world.
Speaker 2: Linux has become the standard operating system for the cloud.
Speaker 2: So when you talk to Google, you're talking to a
Speaker 2: Linux system. When you talk to Facebook, you're talking to
Speaker 2: a Linux system. When you talk to Apple, you're probably talking.
Speaker 4: To a Linux system.
Speaker 2: Right now, the system that we're talking to each other
Speaker 2: with almost certainly is running Linux. So the vast majority
Speaker 2: of systems you talk to in the cloud are running Linux.
Speaker 1: Linux is used for Apple's iCloud, for social media sites
Speaker 1: like Facebook, Instagram, for YouTube, for Twitter, It's used for
Speaker 1: the New York Stock Exchange. Gamers use it when they
Speaker 1: run Steam or they play games online, and the list
Speaker 1: goes on. The vast majority of the Internet runs on
Speaker 1: Linux and open ssh. Make sure that it's you logging
Speaker 1: in and not somebody else.
Speaker 2: When you log in and you get your mail, the
Speaker 2: server that holds your mail has a stage on it
Speaker 2: the server that holds your social media SSSH, the servers
Speaker 2: that have your banking information at ssh. It's the door
Speaker 2: by which you get into these systems.
Speaker 1: So open ssh is incredibly important to the Internet and
Speaker 1: all the cloud systems that we rely on, and because
Speaker 1: of that, it has a lot of eyes on it.
Speaker 1: Trying to hack open Ssh directly would pretty much be impossible.
Speaker 1: Someone would catch you pretty quick.
Speaker 2: People pay a lot of attention to it. A lot
Speaker 2: of people run their code scanners on it, a lot
Speaker 2: of people look for bugs in it, and so it
Speaker 2: has been a while since open ssh has had itself
Speaker 2: a humongous security flaw in it. If you just join
Speaker 2: the open ssh project and said, hey, I'm a new
Speaker 2: guy that nobody ever knew, here's my code, everybody would
Speaker 2: be super suspicious, right m h. And whoever these bad
Speaker 2: guys are, they know that. So what they did was
Speaker 2: they looked at open ssh, and they looked at its
Speaker 2: dependency graph, what we call They looked at all the
Speaker 2: stuff that goes into open ssh, and what they saw
Speaker 2: was open Ssh depends on other things.
Speaker 1: This is where xzu tills comes in. Xu tills is
Speaker 1: one of the things that open Ssh depends on what
Speaker 1: does xzu tills actually do.
Speaker 2: It's a compression library, so it's just a library that
Speaker 2: is used to make data that comes in smaller so
Speaker 2: that if you're moving like a big file back and forth,
Speaker 2: it can fit down a smaller pipe. Right, you might
Speaker 2: be talking to a server on a satellite link, you
Speaker 2: might be talking over a modem. Right, you might be
Speaker 2: talking over a cell phone, and so you want your
Speaker 2: big file to fit into a smaller pipe.
Speaker 1: If you've ever used the zip file on your computer,
Speaker 1: you get the general idea. Smaller files can be transferred faster,
Speaker 1: which is important when you're dealing with so much data
Speaker 1: flowing back and forth. Xdu tilS allows open Ssh to
Speaker 1: be both safe and fast, but that's the trick. By
Speaker 1: inserting a back door into xu tills, the hackers created
Speaker 1: a way to access anything being transmitted via open ss.
Speaker 1: That meant they could not only read supposedly secure messages,
Speaker 1: but remotely run code on any server that uses open ssh.
Speaker 1: And since basically the entire Internet uses this thing, once
Speaker 1: you're in there, you can do anything you want.
Speaker 2: You could have used it for a bunch of very
Speaker 2: quiet surgical attacks over a multi year period, or you
Speaker 2: could have done one humongous big bane where you knock
Speaker 2: out a huge chunk of the Internet all at once.
Speaker 1: But how did hackers get access to xdutails in the
Speaker 1: first place. Well, remember when I promised to tell you
Speaker 1: about the open in open Ssh. Open Ssh and also
Speaker 1: Linux are open source programs. This means that anyone can
Speaker 1: look at the source code because it's open and it's
Speaker 1: posted publicly. The idea is that if everyone works together
Speaker 1: on the code, it'll be better and the publical benefit.
Speaker 1: And so anyone's free to look at the code, to
Speaker 1: learn from the code, or even to remix it for
Speaker 1: their own use. And even if you have no interest
Speaker 1: in all that nerd stuff, you still use versions of
Speaker 1: open source code every day on basically all of your
Speaker 1: devices when.
Speaker 2: You're running open source software, which people don't understand. Basically
Speaker 2: everybody is right. So what kind of phone do you have?
Speaker 2: Do you have an iPhone or Android?
Speaker 3: I actually have an Android?
Speaker 4: Yeah, okay, so Android.
Speaker 2: A humongous chunk of that code is open source, right right,
Speaker 2: And that is code that is maintained by volunteers that
Speaker 2: you have no idea who those people are. Google has
Speaker 2: no idea who those people are. Right, Google collects all
Speaker 2: this code from around the internet, they package it all
Speaker 2: up and then they put it on a phone, or
Speaker 2: they send it to Samson, and Samson puts on the phone.
Speaker 1: And before we get any further, iPhone people, this applies
Speaker 1: to you too. Your iPhone uses a lot of open
Speaker 1: source code also. And don't get me wrong, this is
Speaker 1: not a bad thing.
Speaker 2: It's great because it's free and it makes the phone cheaper,
Speaker 2: and it's cool that we all get to contribute. But
Speaker 2: the flip side is is that, yes, OpenSSH itself gets
Speaker 2: lots of love. The Linux kernel gets lots of love, right,
Speaker 2: But something like XCU tills, which is this tiny little
Speaker 2: component over here, does not get lots of enough. And
Speaker 2: xd details at the time was maintained by one person.
Speaker 2: That one dude was then manipulated to giving up control
Speaker 2: of it, and the person he gave up control of
Speaker 2: it too, turned out to be a totally fake persona
Speaker 2: to not exist.
Speaker 1: This is where we get to the human part of
Speaker 1: the story. The one guy who was maintaining xeu tills,
Speaker 1: his name was lost to Culin. He'd been maintaining xeu
Speaker 1: tills since two thousand and nine, and he was the
Speaker 1: sole maintainer for the project. He wasn't being paid for it.
Speaker 1: He was a volunteer. That's usually how open source projects go.
Speaker 1: In twenty twenty two, LASTA. Collins started to get a
Speaker 1: lot of requests to make updates to the code. Throughout
Speaker 1: the year, multiple accounts, seemingly out of nowhere, started complaining
Speaker 1: that Colin wasn't working fast enough and implying that if
Speaker 1: he wasn't interested in doing this anymore, maybe he wasn't
Speaker 1: the guy for the job and the pressure was getting
Speaker 1: to him.
Speaker 3: In June of.
Speaker 1: Twenty twenty two, Colin wrote in a public note, quote,
Speaker 1: I haven't lost interest, but my ability to care has
Speaker 1: been fairly limited, mostly due to long term mental health issues,
Speaker 1: but also due to some other things. He also went
Speaker 1: on to remind people that quote, It's also good to
Speaker 1: keep in mind that this is an unpaid hobby project. Thankfully,
Speaker 1: right about that time, a new programmer had come into help.
Speaker 1: This new person's name was Gia Tan. Colin seemed a
Speaker 1: little relieved that finally someone wasn't just complaining but helping.
Speaker 1: In that same note from June, he wrote that he'd
Speaker 1: been working a bit with Gatan on exeu utils to
Speaker 1: address all of those complaints, and he said about gia quote,
Speaker 1: perhaps he will have a bigger role in the future.
Speaker 1: We'll see. Over the course of a few years, gia
Speaker 1: Tan really started to gain lesa Collins trust. Gia Tan
Speaker 1: was the ideal contributor. He didn't just help when he
Speaker 1: was asked to, but he would offer to take on
Speaker 1: more work, and by twenty twenty four, Colin had made
Speaker 1: gia Tan a co maintainer on the project, which allowed
Speaker 1: him to add code without needing approval.
Speaker 4: This is a human attack, right.
Speaker 2: It all happened in the open, but the way they
Speaker 2: did it was they created these fake personas where one
Speaker 2: guy super friendly and one guy's a jerk, and the
Speaker 2: jerk basically is abusing the person who's maintaining the software
Speaker 2: and saying, oh, I need this change, I need this change.
Speaker 4: You're so slow. Why are you so slow?
Speaker 2: And remember this guy's not getting paid right like, and
Speaker 2: so eventually basically bully this guy to say, oh, I'm
Speaker 2: tired of doing this, I don't want to do it anymore.
Speaker 2: And then the nice guy's like, oh, well you know,
Speaker 2: I'll do it for you. I'll take over man, let
Speaker 2: me take this burden.
Speaker 1: For you, right, very convenient, right, And.
Speaker 2: This took several years, and so this shows you kind
Speaker 2: of the long play. They're willing to spend months and
Speaker 2: months and months and in fact years building these personas,
Speaker 2: because like, look, if you just created an account and
Speaker 2: you're like, hey, i've got code, take it, that wouldn't
Speaker 2: work right with these people. Figured out is that you
Speaker 2: have to create these personas. They have to seem real.
Speaker 2: You have to make posts, you have to contribute legit stuff.
Speaker 2: You've got to create.
Speaker 4: Kind of a history, build a relationship, you have to
Speaker 4: build a relationship.
Speaker 2: And so the guy who maintains it gives it up
Speaker 2: of like, oh, thank you so much for taking this
Speaker 2: burden from me, because look at these jerks.
Speaker 4: Now.
Speaker 2: Of course he doesn't know that the jerks works for
Speaker 2: the same team, or maybe you're even the same person
Speaker 2: as the nice guy, right, And then he hands it
Speaker 2: over to this nice guy who's a friend of his,
Speaker 2: and then the friend takes it over and then does
Speaker 2: a bunch of legitimate stuff, and then in the middle
Speaker 2: of all that legitimate stuff inserts a very very subtle backdoor.
Speaker 1: I've seen this back door talked about using the phrase
Speaker 1: sophisticated that it was very sophisticated. Yes, in some ways
Speaker 1: it sounds sophisticated, but in some ways it sounds like
Speaker 1: it kind of wasn't because a lot of it just
Speaker 1: revolved around getting somebody to give them some access.
Speaker 2: The code was sophisticated, The method of getting in there
Speaker 2: was very human. It was bugging a guy until he
Speaker 2: gave up control. Yes, right, just being a nuisance, Just
Speaker 2: being a nuisance. So who was behind those fake personas
Speaker 2: We don't know for sure, but Alex has a theory
Speaker 2: that's after the break over the course of years, the
Speaker 2: one guy maintaining this very important tool called XEU tills
Speaker 2: Lasa Colin was being bullied and manipulated online to give
Speaker 2: a persona called Gia Tan a lead role in handling
Speaker 2: the code. But who is gia Tan. Everybody's been asking
Speaker 2: this question of like who did this, Who's behind this?
Speaker 2: Most of the names have kind of an Asian origin, right,
Speaker 2: So there's accounts like Jagar Kumar. The key one is
Speaker 2: Gia Tan, which is like, could be Chinese, could be Korean.
Speaker 2: Most of the either the names or the technical indicators
Speaker 2: point to right. So the time zones that this person
Speaker 2: was working into are kind of the East Asian time zone,
Speaker 2: so it's like Beijing or Korea. The names are Asian.
Speaker 2: Everything points to Asia, which makes a lot of people
Speaker 2: think it's Russia actually because it's just too perfect, right,
Speaker 2: because it's just like it's somebody spent three years doing
Speaker 2: all this work, and then you're like, like, let's say
Speaker 2: you're Chinese. Are you going to use like a Chinese
Speaker 2: name as your fake name? Are you going to spend
Speaker 2: three years but then work in your normal time zone?
Speaker 2: And the generally the only actor who has shown this
Speaker 2: level of patients who's been willing to spend three years
Speaker 2: working on a back door like this. The only people
Speaker 2: who have ever done that is either the United States
Speaker 2: or the SVR. So Russia, okay, yeah, are really the
Speaker 2: only groups where you've seen people spend years kind of
Speaker 2: doing this kind of work. And a lot of people
Speaker 2: don't think you would be the US doing something like this,
Speaker 2: that they would never mess with something this important because
Speaker 2: also the thing the Russians are like was blame other
Speaker 2: people right again, because we never got to the point
Speaker 2: of again used. Usually attribution is done after something's used,
Speaker 2: and so it's a lot easier to figure out because
Speaker 2: then you can ask quebono right, who benefits.
Speaker 4: But like all of these indicators.
Speaker 2: Pointing specifically to kind of China or Korea makes you
Speaker 2: think it's just a little too obvious.
Speaker 1: A major theory in cybersecurity circles is that Giatan isn't
Speaker 1: one person. It's potentially multiple people, but likely Russian hackers
Speaker 1: working for the SVR, which is Russia's foreign intelligence service,
Speaker 1: and that they tried to cover their tracks even if
Speaker 1: it wasn't consistent.
Speaker 4: The guys who.
Speaker 2: Worked for the professionals will change their time zones specifically
Speaker 2: around who either what allows them to avoid detection or
Speaker 2: specifically around whatever they're doing for attribution.
Speaker 1: Well, there were some times that the time zones actually
Speaker 1: pointed to an Eastern European time zone or time or
Speaker 1: another time zone, right.
Speaker 2: Yeah, I mean there's there is a little mixed right,
Speaker 2: So somebody could be working from the eastern side of Russia,
Speaker 2: or they could be waking up early in Moscow Saint
Speaker 2: Petersburg and then they slipped right.
Speaker 1: In other words, they might have just slipped up and
Speaker 1: forgot to change their time zones. Because remember this happened
Speaker 1: over the course of years. Maybe somebody had an off
Speaker 1: day and forgot to change the computer settings. But Alex
Speaker 1: has another reason for suspecting Russia over China.
Speaker 2: Chinese hackers, for the most part, work very rigorous hours.
Speaker 2: You can almost always tell when Chinese hackers are working
Speaker 2: because they work office hours. They work eight to five,
Speaker 2: eight to six. Really, it's like very regular, yeah, okay,
Speaker 2: Whereas it's much harder to do time zone stuff based
Speaker 2: for the Russians because they they will work whatever hours
Speaker 2: they need to work. You know that that scene in
Speaker 2: like one of the Born movies where it's like the
Speaker 2: club scene. I always think about this with the Russia.
Speaker 2: There's like a club scene in Russia, and it's like
Speaker 2: you think it's the middle of the night and he
Speaker 2: walks out it's like ten am or something. Right, It's like,
Speaker 2: that's what I think about with Russian hackers, whereas like
Speaker 2: for China, it's amazing because it's like, oh, six pm
Speaker 2: in Beijing.
Speaker 4: You know, it's like, you know, everybody goes home.
Speaker 3: The hacking stops.
Speaker 4: Yeah, or like Chinese.
Speaker 2: Chinese New Year Lunar New Year, everybody goes home, go
Speaker 2: sees their parents in the village or whatever, like hacking stops.
Speaker 4: It's amazing.
Speaker 1: And in the case of Exit the UTILS looking at
Speaker 1: the timing when this ga Tan was submitting code, there's
Speaker 1: a bunch of submissions during Lunar New Year, but during
Speaker 1: big Eastern European holidays like Christmas crickets. But that leaves
Speaker 1: a question, what's the motive. Why would the Western SVR
Speaker 1: want to do this.
Speaker 4: So open everybody uses.
Speaker 2: That's why this is so powerful is you don't have
Speaker 2: to have a specific target in mind, which is why
Speaker 2: you'd also spend three years doing it. Because let's say
Speaker 2: you're at the SVR, you know, no matter what war
Speaker 2: you're involved with, no matter what target you're going after,
Speaker 2: opens station is gonna be useful. So this is probably
Speaker 2: a team of the SVR who they don't know what's
Speaker 2: gonna be used for. They're just they know they're gonna
Speaker 2: get a medal.
Speaker 1: You'll be able to use this at some point, Yeah,
Speaker 1: who knows for what?
Speaker 2: And the US does the same thing, right, Like, there's
Speaker 2: people whose job it is to get the capability, and
Speaker 2: it's other guy's job who understand the geopolitics, who understand
Speaker 2: the intelligence.
Speaker 3: To use it.
Speaker 1: But Thankfully last spring, Andres Freud, the Microsoft engineer, was
Speaker 1: able to discover the back door. But this was all
Speaker 1: by chance. He wasn't looking for it.
Speaker 2: He works on a database called Postgress, so he doesn't
Speaker 2: work on xdutils. He works on Postgress is a big
Speaker 2: open source database program that Microsoft uses in their Azure cloud.
Speaker 2: So I'm guessing that's why Microsoft pays him. And in
Speaker 2: the next version of Debian, so a popular Linux distribution,
Speaker 2: Postgress was running a little bit slower, just tiny, tiny,
Speaker 2: a little bit tiny, tiny, little bit.
Speaker 1: Right, so tiny, like how much slower?
Speaker 2: Like in one specific circumstance, it's taking a couple of
Speaker 2: milliseconds longer to do something right, So like a millisecond, yeah,
Speaker 2: but like if you're a database guy, that's a lot, right,
Speaker 2: And so he is super looking into what is going on,
Speaker 2: and he realizes, oh, it's not actually Postgress that's doing this,
Speaker 2: it's open as hish, And so he could have stopped
Speaker 2: there because he could have been like, oh, well, it's
Speaker 2: not my problem, right, it's not my thing, and then
Speaker 2: maybe nobody would have looked at it, right, Like you
Speaker 2: could see an open source is that people pass problems
Speaker 2: to each other all the time, right, So it is
Speaker 2: this is like I think a normal like a normal person,
Speaker 2: even open source developer, would have been like, oh okay,
Speaker 2: I looked at this, it's not me. I'm gonna let
Speaker 2: it go. But he did not let it go. He
Speaker 2: ended up digging into okay, well what changed in open
Speaker 2: as the sage and then he looks into open a
Speaker 2: sh and sees this code and so what the What
Speaker 2: the attackers that did is they created what's called a
Speaker 2: no bus back door nobody butt us.
Speaker 1: No bus or nobody butt us is a way of
Speaker 1: creating a backdoor into something where nobody but us or you,
Speaker 1: the hackers have the key.
Speaker 2: They wanted, skeleton key that only they can use. But
Speaker 2: no bus back doors nobody but us back doors ar
Speaker 2: are actually hard to sneak in because they're pretty like
Speaker 2: obviously sketchy.
Speaker 1: So instead of doing everything all at once, they delivered
Speaker 1: multiple patches in multiple different places, little things here and
Speaker 1: there that wouldn't raise suspicion if you looked at one
Speaker 1: or two or three of them, but layered on top
Speaker 1: of each other, they created a key that only they
Speaker 1: could use.
Speaker 2: And so because they did all this stuff to kind
Speaker 2: of obfuscate it and make it super secret. They actually
Speaker 2: created the performance impact that unders saw and then went
Speaker 2: way out of his way to pull and then he
Speaker 2: posts in a public post, guys, this is super sketchy, right, like,
Speaker 2: look at this code. There's no good argument for what's going.
Speaker 1: On here, right, So, I mean I kind of have
Speaker 1: to wonder about what the implications for this are. I
Speaker 1: mean this clearly it almost worked. Do you think there's
Speaker 1: hackers out there saying okay, yeah, yeah, let me change my.
Speaker 3: Approach and maybe this is the way to do it.
Speaker 4: Yeah.
Speaker 2: I mean what I'm afraid of is we haven't found
Speaker 2: any other ones like this. So what I thought would
Speaker 2: happen is at the time, I'm like, oh man, we'll
Speaker 2: have one or two more of these because everybody started
Speaker 2: looking and then nobody else found any other ones.
Speaker 4: Which terrifies me.
Speaker 3: You think there's more like this out there?
Speaker 4: I think it's quite possible it's more like this.
Speaker 2: Yeah, Like, if anybody has an idea, two or three
Speaker 2: other people have had the idea, right, So I can't
Speaker 2: imagine these are the only people who are like, oh,
Speaker 2: I'm gonna go bully some maintainer of one of the
Speaker 2: five thousand libraries on Linux to go take it over
Speaker 2: or submit a patch. I can't imagine there aren't other
Speaker 2: ones now, are they in OpenSS H or are they
Speaker 2: something much more subtle?
Speaker 4: I don't know.
Speaker 2: I mean, this would have been both in kind of
Speaker 2: one of the worst possible places, and it would have
Speaker 2: been a skeleton key that only this attacker could have used,
Speaker 2: which is like kind of the worst case scenario. It's
Speaker 2: also the hardest level of difficulty, right, these people picked
Speaker 2: the hardest level. Said, if you want to do something
Speaker 2: much simpler is you go after a much lesser used
Speaker 2: service that's specifically at the target that you're going after.
Speaker 2: If you're going after a specific target and you're like, oh,
Speaker 2: they use this specific this one specific service that's much
Speaker 2: less popular, that doesn't have all these eyeballs on it,
Speaker 2: then you don't have to be as tricky.
Speaker 1: There haven't been any like this in OpenSSH, but there
Speaker 1: have been other attempts that the Open Source Security Foundation
Speaker 1: and the Open JavaScript Foundation have found that use similar
Speaker 1: social tactics. One project received emails from accounts asking to
Speaker 1: be designated as project maintainers despite having little prior involvement,
Speaker 1: and two other projects saw very similar suspicious patterns. This
Speaker 1: kind of social engineering is really effective because you don't
Speaker 1: have to manipulate code. You just manipulate the person who
Speaker 1: has their hands on the code. And it's only going
Speaker 1: to get easier to do and harder to detect.
Speaker 2: Now we're at the point where with AI, like you
Speaker 2: could be fake now and I have no idea if
Speaker 2: you really exist or vice versa.
Speaker 1: Wait are you are you suggesting that doing something like
Speaker 1: this might be a little bit easier because somebody could
Speaker 1: fake that they actually exist. Oh yeah, with a phone
Speaker 1: conversation or a video conversation.
Speaker 2: Oh yeah, we're already seeing that from the ransomware actors.
Speaker 2: It's easy for phone, right, So you're already seeing them
Speaker 2: fake people's voices. So people are getting phone calls from
Speaker 2: like their CEO. The CEO goes on CNBC for two minutes,
Speaker 2: they get their voice from CNBC, they plug it into
Speaker 2: a AI voice library, and then you call and like, hey,
Speaker 2: it's Bob, I need you do a million dollar transfer. Right,
Speaker 2: So that kind of stuff, and now you see real
Speaker 2: time video too. It's not perfect, but it's getting there.
Speaker 3: Yeah.
Speaker 2: The trick, by the way, if this happens to any
Speaker 2: of your listeners. The trick is you can ask people
Speaker 2: to move, touch things in the background, do three sixty
Speaker 2: on the head. It's harder for them to do ears
Speaker 2: forever reason, but they'll get there, right, So, like if
Speaker 2: I asked you to take your glasses off, it'd be
Speaker 2: very hard for the model, Like take your glasses off.
Speaker 1: By the way, hold on, for those of y'all listening
Speaker 1: at home, I took my glasses off here, just double
Speaker 1: check it.
Speaker 2: Oh, you kind of frozen me when you did that,
Speaker 2: So that's sketch man, it's sketchyf as my students say. Sorry,
Speaker 2: they keep me on my Sanford soons.
Speaker 1: But you know, in the future, though, it is going
Speaker 1: to be easier to spoof people's personalities, yeah, and stuff
Speaker 1: like that. So these things that you're suggesting right now
Speaker 1: they work now, are they going to work in a year?
Speaker 3: So?
Speaker 2: I mean, the good thing about this is open source
Speaker 2: developers have become much more paranoid, right, So people have
Speaker 2: become much more paranoid about new people. And there's a
Speaker 2: downside of that, right that if you're trying to get
Speaker 2: into open source, it's harder. There have become projects where
Speaker 2: it's like, okay, great, let's meet up in person. If
Speaker 2: somebody's willing only to communicate with you an email, then
Speaker 2: you have to be kind of sketched out. Now, there
Speaker 2: have been some changes since this. I think people have
Speaker 2: been more paranoid. There's been a bunch of work On
Speaker 2: the flip side of AI is that traditional code scanning
Speaker 2: tools PREI code scanning tools are not extremely good at
Speaker 2: detecting this kind of malicious code. But there is some
Speaker 2: hope that some of the newer AI based code scanning
Speaker 2: tools could could do this kind of stuff at scale.
Speaker 2: The flip side is is AI is really good at
Speaker 2: writing code, So you know, do you not have to
Speaker 2: be SVR level anymore to be able to write a backdoor?
Speaker 4: That's good, That's probably true as.
Speaker 1: Well, it's open source too much of a risk in
Speaker 1: the age of AI, and can we protect ourselves from
Speaker 1: another hack like this?
Speaker 3: That's after the break.
Speaker 1: So this and I want to get back into kind
Speaker 1: of the play by play here, but a lot of
Speaker 1: this hinges on open source. So and I think one
Speaker 1: of the really kind of concerning things about this entire
Speaker 1: thing that happened or almost happened is the fact that
Speaker 1: it basically happened in broad daylight. Yes, and it happened
Speaker 1: because this is open source. The thing about open source,
Speaker 1: I think, is when you start to explain it to
Speaker 1: somebody who's never heard of it. Are you familiar with
Speaker 1: the galaxy brain meme?
Speaker 4: Yeah?
Speaker 3: Do you know what I'm talking about? Yeah?
Speaker 1: So I feel like this is like that galaxy brain meme,
Speaker 1: where at the very top, when you tell somebody to
Speaker 1: open source, the response is, this is a terrible idea.
Speaker 1: Everybody can see the code. And then you get a
Speaker 1: little bit further down it's, oh, this is a great idea.
Speaker 1: Everybody can see the code, and then they hear about
Speaker 1: xutails when we get down to the bottom, and it's
Speaker 1: a terrible idea. Everybody can see the code. What's the
Speaker 1: true galaxy brain take on this for open source?
Speaker 2: I mean, people go back and forth. So one of
Speaker 2: the ideas is that if you can see all the code,
Speaker 2: you can see all the bugs.
Speaker 3: Right.
Speaker 2: Is the idea that because it's open source, that it
Speaker 2: should be more secure than closed source because you could
Speaker 2: see the flaws. I don't think that has empirically turned
Speaker 2: out to be true, right, And so I think what
Speaker 2: I would say is I'm a big proponent of open source.
Speaker 2: I think it's great. I think it has a humongous
Speaker 2: economic benefit to the world. The truth is is the
Speaker 2: entire kind of cloud competing revolution we're all living through
Speaker 2: only exists because of open source software. So that's an
Speaker 2: incredible thing. That's a wonderful thing. Open source is great
Speaker 2: from an economic perspective, it is great from an innovation perspective.
Speaker 2: We should not pretend that it magically solves trust and
Speaker 2: security problems. And if you're a company that's relied upon
Speaker 2: open source, you have a ethical and moral obligation to
Speaker 2: deal with the security aspects of it, and it contribute back.
Speaker 2: And I do think that is something that's gone lost,
Speaker 2: is that people have just kind of assumed somebody else's
Speaker 2: dealing with it, and everybody assume somebody else is doing
Speaker 2: the security work, and that turns out not to be true.
Speaker 1: You know, I think that really gets the core of
Speaker 1: what a lot of this is. Because if somebody sees
Speaker 1: XU tills there was a potential security flaw in that, Okay,
Speaker 1: well I don't care about that.
Speaker 3: What's that?
Speaker 1: Oh, well, you know it's involved with open as a stage. Well,
Speaker 1: I don't use that either. I don't have that app
Speaker 1: on my phone. I don't know what you're talking about.
Speaker 1: And in this weird way, I feel like the more
Speaker 1: and more technology actually starts to become just magic, that
Speaker 1: things just work. Yeah, we are less and less actually
Speaker 1: tech literate. All the stuff that was science fiction even
Speaker 1: ten years ago, two years ago, frankly is it's just
Speaker 1: normal now.
Speaker 4: Yeah.
Speaker 1: And so we're able to do so much with technology
Speaker 1: just regular people things we just do with our phone
Speaker 1: every day, that we've become really removed from the technology itself,
Speaker 1: and so less and less of us, fewer and fewer
Speaker 1: was actually know how to use a computer. Yeah, and
Speaker 1: so this feels totally removed from us. This is like, oh,
Speaker 1: this is some weird nerds shit. I'd like, I don't
Speaker 1: use that nerd program. Doesn't affect me.
Speaker 2: Yeah, No, you're totally right. I mean, I tell my
Speaker 2: Stanford students. Security is one of the best fields to
Speaker 2: get into professionally because it's the only part of computers
Speaker 2: it gets worse every year. Everything else magically gets better. Man,
Speaker 2: So you could find yourself in any other field being
Speaker 2: made irrelevant, But if you get into security, you have
Speaker 2: job security for life because every year.
Speaker 4: I've been in it. It's gone worse.
Speaker 2: And one of the reasons is because you say it's nerdship.
Speaker 2: But even the nerds we get the normal median nerd
Speaker 2: gets further and further away from the truth, the reality
Speaker 2: of what's going on on computers. So when I learned
Speaker 2: how to program, I learned assembly language, right, I learned
Speaker 2: how to write like the lowest level languages. And then
Speaker 2: you know, they stopped teaching assembly language unless you took
Speaker 2: special classes, and you learn, like in Python, right, like
Speaker 2: a very high level language that you don't even you know,
Speaker 2: you don't learn how to like do memory management.
Speaker 3: Right, I mean Python pithon.
Speaker 1: And it's even just to break this down like Python
Speaker 1: for a casual person, you can look at it. You
Speaker 1: can kind of tell what's going on. It basically looks
Speaker 1: like English. Yeah, assembly is letters and numbers.
Speaker 2: Right, right, But the nice thing about assembly is it's
Speaker 2: the truth of the matter.
Speaker 3: Right.
Speaker 2: It has a one to one matching to what the
Speaker 2: processor itself is doing. And from a security perspective, if
Speaker 2: you look at it, is the reality of what a
Speaker 2: security flaw is is seen in the assembly.
Speaker 3: Right.
Speaker 2: In Python, you get further, you get abstracted away, you
Speaker 2: get further from the reality of what's actually going on
Speaker 2: on the computer. Now what you see it's incredibly powerful,
Speaker 2: it's incredibly cool, and so I'm gonna I'm not gonna
Speaker 2: crap on it because I think it's an incredibly good
Speaker 2: thing for people. But you look at like Claude three
Speaker 2: point seven code. You know, this new Claude model, and
Speaker 2: you see people on Twitter who don't know anything about
Speaker 2: computers and they're able to program now because they can
Speaker 2: go into there and they could say, build me software
Speaker 2: that does X. And that is going to be terrible
Speaker 2: for security. It's super cool for people's economic opportunities because
Speaker 2: any bigap you can become a program right now. But man,
Speaker 2: are people in security gonna love it because now you
Speaker 2: don't need to know anything about how computers work and
Speaker 2: you're just gonna ask the AI system to build it
Speaker 2: for you.
Speaker 4: And I see it with my students.
Speaker 2: Stanford students like one of the top computer science programs
Speaker 2: in the world, and you can graduate and not actually
Speaker 2: really understand how operating systems work. I apologize to Sandford
Speaker 2: Computer Science department, right, but really like you can have
Speaker 2: a totally productive career in Silicon Valley and not really
Speaker 2: understand what's going on three or four layers down. In fact,
Speaker 2: it's better for you not to write. It's better because
Speaker 2: you're at the high level where you're much more productive.
Speaker 2: You're much more productive having the AI do the work
Speaker 2: for you. You're much productive having get hub Copilot help
Speaker 2: you rewrite stuff. You're much more productive using all the
Speaker 2: cloud intermediation layers. And so that's one of the reasons
Speaker 2: why security gets worse every single year is that we
Speaker 2: add these layers of abstraction that makes things easier for people.
Speaker 2: And AI is the ultimate abstraction layer, because now you
Speaker 2: can talk to computers and plain English and have them
Speaker 2: do incredibly complex things.
Speaker 1: The thing about this whole story, I mean, I'm thinking
Speaker 1: about you know, we're in a time right now where
Speaker 1: anything bad happens or almost happens. Netflix documentary, Hulu documentary,
Speaker 1: it's a true crime podcast. At some point, I don't
Speaker 1: see that happening with this. This is something that, as
Speaker 1: you were saying, was almost it truly could have been catastrophic. Yeah,
Speaker 1: but it's also kind of boring.
Speaker 2: It's money, Well, you don't think I could get I
Speaker 2: could sell ten episodes to Netflix on this.
Speaker 1: If you can hire me as a producer, I'd love
Speaker 1: to help. But you see what I'm saying, it takes
Speaker 1: a while to even explain what the heck we're talking about. Yeah,
Speaker 1: And I think that comes back to some of this
Speaker 1: in the same way that this vulnerability was introduced via
Speaker 1: social engineering. A lot of this is social I mean
Speaker 1: a lot of your work you probably think about this.
Speaker 1: How do you get people to care about something like this?
Speaker 2: I mean, so that's that's a challenge. That's one of
Speaker 2: the biggest challenges. If you're like a chief information security officer,
Speaker 2: one of your big jobs is getting the rest of
Speaker 2: the company to care about security. Ciso's we have a
Speaker 2: reputation of being the people who say no all the time.
Speaker 2: So I was the CISO of Facebook and I once
Speaker 2: walked into a meeting with a bunch of other vps
Speaker 2: and somebody literally said like, oh shit, some of this
Speaker 2: year like hey guys, I.
Speaker 4: Can hear you. I can hear you, and like, no, no,
Speaker 4: it's not you.
Speaker 2: It's just like whenever you come like it's just because you're
Speaker 2: telling us, like there's a coup in Turkey or something terrible,
Speaker 2: Like because I was just the bare or bad news, right.
Speaker 2: But this is what's a real challenge for my colleagues,
Speaker 2: and it's a real challenge for us as a society.
Speaker 2: People don't want to think that the systems that they
Speaker 2: rely upon are fragile, and I think that's like a
Speaker 2: real problem.
Speaker 3: What do we learn from this? What is it?
Speaker 1: Let me just say because I don't I personally don't
Speaker 1: think just being out and talking to people, if I
Speaker 1: was trying to if I try to tell somebody, hey, yeah, man,
Speaker 1: what do you think about the xdutails thing?
Speaker 3: Have you?
Speaker 4: You know, hey buddy, what's up?
Speaker 3: Yeah? Has that Has it changed anything about how? Yeah?
Speaker 1: Has that changed anything about how you go about your life?
Speaker 1: People can tell me no. So I got to ask
Speaker 1: somebody who's actually closer to this. Has this changed how
Speaker 1: you approach things? Has this changed how the industry approaches things?
Speaker 1: Has this changed how? I mean the theory that you're
Speaker 1: putting out is that this is a state actor? Has
Speaker 1: this changed how national security is being looked at?
Speaker 2: So for companies that know what they're doing, it has
Speaker 2: changed that they approach open source. For a handful of
Speaker 2: really big you know, like the Googles, the Metas, the Amazons,
Speaker 2: the Microsoft's, the really big tech companies that do a
Speaker 2: lot of open source work. They are looking more carefully
Speaker 2: at open source for security companies to do this work.
Speaker 2: We're investing in software and AI that can do this
Speaker 2: work for us. But it has not changed anything massively. Right,
Speaker 2: We're still running Linux, We're still all pulling in fifty
Speaker 2: thousand packages. We have these humongous dependency graphs. The truth is,
Speaker 2: you can't just pivot all these things, right. It has
Speaker 2: made us more concerned about these problems. When you talk
Speaker 2: to CISOs, my colleagues and I, we're all more concerned.
Speaker 2: But we can't magically pivot off of the infrastructure we
Speaker 2: have built over a decade. I do not think we've
Speaker 2: dealt with the fact that if you get on the
Speaker 2: subway in the morning and you look around, most of
Speaker 2: the people on that train in their pocket. Ex Utils
Speaker 2: is in their pocket. Every single person in there, hundreds
Speaker 2: of copies of their social Security number is sitting on
Speaker 2: servers that would have been backdoored by this attack. That's
Speaker 2: how you can think of it, right, So that's how close.
Speaker 3: We can man.
Speaker 1: So just some closing thoughts here. Again, the reason that
Speaker 1: most people don't know about what was almost the biggest
Speaker 1: hack in the history of the Internet is because this
Speaker 1: is really hard to describe to a non technical audience.
Speaker 1: I mean, when you say XU tills or Linux or
Speaker 1: open ssh, people's eyes just rolling the back of their heads.
Speaker 3: But we can't.
Speaker 1: Allow tech literacy to be a barrier to understanding how
Speaker 1: the world works and the truth is, even beyond all
Speaker 1: the tech jargon, a lot of these things are very
Speaker 1: human and they're not so hard to understand. And so
Speaker 1: that's one of the things that we're really trying to
Speaker 1: do here on kill Switch as we keep doing these episodes,
Speaker 1: is to open it up so that more people are
Speaker 1: able to feel like they're part of the conversations that
Speaker 1: affect all of us. And that is it for this one,
Speaker 1: for real. Thank y'all so much for listening to kill Switch.
Speaker 1: You can hit us up at kill Switch at Kaleidoscope
Speaker 1: dot NYC if you've got any thoughts or if there's
Speaker 1: anything you want us to cover in the future, and
Speaker 1: you can get me at dex Digi that's the d
Speaker 1: e x d ig I on Instagram or blue Sky
Speaker 1: if that's more your thing, and if you like the episode.
Speaker 1: You know, take that phone out of the pocket and
Speaker 1: leave us a review. It helps people find the show,
Speaker 1: which in turn helps us keep doing our thing. And
Speaker 1: this thing is hosted by me Dexter Thomas. It's produced
Speaker 1: by Sena Ozaki, Daryl luck Potts and Kate Osborne. Our
Speaker 1: theme song is by Kyle Murdoch, who also makes the
Speaker 1: show from Kaleidoscope. Our executive producers are Ozma Lashin, mangesh
Speaker 1: Hot Togodur, and Kate Osborne. From iHeart our executive producers
Speaker 1: are Katrina Norville and Nikki E.
Speaker 3: Tour. That's it for this time.
Speaker 1: Catch on the next one.