xz Exploit Is WILD - Must See Bash Part

Recorded live on twitch, GET IN
Article
gynvael.coldwind.pl/?lang=en&...
Guest
/ lowlevellearning
/ lowlevellearning
/ lowleveltweets
My Stream
/ theprimeagen
Best Way To Support Me
Become a backend engineer. Its my favorite site
boot.dev/?promo=PRIMEYT
This is also the best way to support me is to support yourself becoming a better backend engineer.
MY MAIN YT CHANNEL: Has well edited engineering videos
/ theprimeagen
Discord
/ discord
Have something for me to read or react to?: / theprimeagenreact
Kinesis Advantage 360: bit.ly/Prime-Kinesis
Hey I am sponsored by Turso, an edge database. I think they are pretty neet. Give them a try for free and if you want you can get a decent amount off (the free tier is the best (better than planetscale or any other))
turso.tech/deeznuts

[@JannePaalijarvi]

My heart breaks when hearing all this new info about Lasse Collin. His companions betrayed him and now he is there alone, unable to trust anyone, battling mental issues and still trying to clean up one of the biggest security fiascos of the decade :(

[@frozenlettuce653]

I do suspect that some of the pressure that he received was part of a coordinate attack

[@markhaus]

@@frozenlettuce653very likely and hopefully this spurs more active efforts in taming the community around maintership. Not just allowing giant companies to freeride on over worked underpaid contributors. Same for people who use these projects on smaller scales but abuse and exploit the work of maintainers. It’s literally a security risk now if there was a coordinated social engineering attack on this poor maintainer of you allow these projects to go on with shoe string support

[@LookRainy]

@@frozenlettuce653that’s very likely. Most of those accounts were just disposable accounts created with the sole purpose of posting the nasty comments to pressure the original maintainer

[@nefrace]

​@@frozenlettuce653 of course it is. The moment I saw these messages about progress and stuff I thought about coordinated attack

[@astronemir]

They probably fucked with him behind the scenes on alt accounts etc. If they go this deep not beyond messing with his socials and so on, to cultivate him as a vulnerable target

[@JmbFountain]

What's the lesson here? Don't get between a DB engineer and performance.

[@local9]

Trust me, don't

[@pedrolopez8057]

Having been a DB Engineer and having been blamed many times for poor performance when it actually it was some half-bright code monkey we get very thorough and very cranky.

[@vitalis]

The real lesson, working on open source projects doesn’t pay

[@lucaxtshotting2378]

@@vitalis the real leson is use winrar

[@isaacalves6846]

Gladly TempleOS is doing just fine.

[@MuhammadYusuf-nz5nj]

So you know that from FIRESHIP 🔥

[@orbatos]

Everybody is fine, only an idiot would be pulling from git to build a library for server deployment and most servers are on 5.4.x.

[@Rugg-qk4pl]

@@MuhammadYusuf-nz5nj Or we know it from.. TempleOS

[@gwentarinokripperinolkjdsf683]

Unrelated, but temple OS is actually a lot cooler than a people think and it is basically the perfect OS for learning about hardware, as it's complete lack of permissions and it's unique paging setup makes working with hardware very easy, also the fact you have the holyC REPL, you can experiment without friction. Seriously go take a look at zeal OS (a modern port) and start messing around with it

[@DarkerCry]

@@orbatos God bless

[@arbyyyyh]

Hot take: Everyone has been saying that this is proving the short comings of OSS, I think the opposite. If some idiot got themself a job at MS and did something like this, you would see that PS-Remote or perhaps RDP takes an extra half a second and say "what did MS do now??" and move on with your day. The beauty of OSS, the ability for someone to look at the code did what it needed to do: Someone who had nothing to do with the project was able to look at the code and sound the alarm.

[@random_bit]

Preach brother, this is a strength, not a weakness

[@NetscapeSato]

exactly. you cant peek behind the curtain with proprietary OS'es. With linux and OSS, anyone can view or modify code. Thats the whole reason this was even found in the first place.

[@craigslist6988]

💯💯💯💯 people failing to see how good this is. How would MS even be able to tell if someone put in a backdoor in anything closed source, it would be impossible to spot ten seconds of performance lag on anything from MS, let alone half a second.. and noticing one more backdoor among the dozens of 'telemetry' backdoors they already put in there? Nah.

[@evancombs5159]

The irony here is that this exploit was discovered by an MS employee.

[@random_bit]

@@evancombs5159 sometimes you gotta work for the devil to get the bag

[@weakspirit_]

the scariest part is the social engineering did on Lasse. this person was manipulated for YEARS and the team (probably) behind it saw the opportunity and exploited it. exploiting Lasse's mental health, trust and desire to pass on the torch. this is actually evil

[@themodfather9382]

You guys are awfully quick to clear this guy's name, it's sad when people get falsely accused, but that's life.

[@SpiDey1500]

They may also caused the mental health problems…

[@privateagent]

Maybe you guys forgot about the covid psyop. Everyone is prone to be manipulated.

[@connorskudlarek8598]

@@SpiDey1500 my god, didn't even think about it but they could totally have had accounts sending him hate comments over the xz utils not getting enough updates, which caused him to want to find someone else to take it over.

[@squirlmy]

​@@themodfather9382 absolutely no evidence Lasse Collin did anything wrong, while I see Andres Freund being congratulated with quips like "lifetime free drinks", which makes treatment of Collin even more stark contrast. No it's not "life" to throw wild accusations. It's just more prejudice against "mental health issues" being not real, so he must be suspect. Really uncool to suggest this.

[@aaaaanh]

Freund isn’t even a security engineer (disclaimer at the end of the post on openwall). Man is just that big of a gigachad.

[@Julzaa]

He is now I guess

[@homeape.]

That's what freunds are for 🎶

[@aaaaanh]

Freundlich neighboorhood engineer 🫡

[@1337kaas]

Database engineers are from a different breed man

[@Ellefsen97]

«Freund» is also German for «Friend», which is very fitting here

[@st0ox]

It took me over an hour to realize that this wasn't an April fools.

[@kuertoes]

This whole situation just feels like a movie. The fact that this is real is insane. And I can't decide what's more impressive - developing this backdoor or finding the backdoor ... this just shows me how little I actually know. I feel vulnerable ... just let me cry...

[@magicmulder]

It feels like someone held a genius coder hostage to develop the backdoor and then the criminal half-assed the distribution of the backdoor (using accounts that were just created to push for inclusion, come on! That's like composing Beethoven's Tenth and then playing it on a glass bottle.).

[@sultanhanga]

And they're guessing there is more backdoors

[@the_real_ch3]

There's an episode of The Sopranos where the FBI spends the entire ep putting a bug into a desk lamp and then planting the lamp in Tony's basement where he talks business with his associates. They only capture a single conversation, of Tony talking to a plumber about his water heater, before the whole scheme is undone by Meadow grabbing the lamp and taking it to her dorm room. Feels like a good metaphor for this guy's exploit getting caught so quickly.

[@superscatboy]

Or that time the CIA spent millions putting a microphone and radio transmitter into a cat, deployed the cat, and it immediately ran into a road and got hit by a car.

[@benjaminhon86]

Saved by some random engineer benchmarking postgres which 99.9% of SE engineers won't even have time to do :D

[@themodfather9382]

ssh is very widely used so yeah people will benchmark commonly used tools, it was a weird mistake

[@peachezprogramming]

I don't know how developers are so smart that they can find this shit. Blows my mind

[@zelllers]

yeah so what hasn't been found yet that's out there right now. Don't trust the computers!

[@blackjackjester]

This is not doing good things for my imposter syndrome

[@allsunday1485]

Wait until you hear about mathematicians

[@smnomad9276]

@@allsunday1485 what the hell does mathematicians have to do with any of this

[@00jknight]

I imagine the guy discovering this was just saying "Wtf is this" the entire time as he unravelled the shit storm.

[@havokgames8297]

You and lowlevellearning have really good energy together. Great video. More collabs please.

[@awesomedavid2012]

I think this backdoor ultimately is going to do more good than harm, because now people are on the lookout for backdoors in tests and similar wild exploits.

[@complexity5545]

You dang right about that. My last 3 days have been auditing and catching up on dependencies and scraping the git projects' comments. So many eyes are on it now.

[@EwanMarshall]

The pushing might be because there are 2 other things happening that are each likely to kill the attack chain. 1. openssh was already working on their own method for calling systemd-notify without linking it (upto now they did not link it due to them being very careful on dependency checking). Debian, Fedora and OpenSuse were patching sshd to do this linking. This is how liblzma got linked to openssh at all, this wouldn't be done anymore. 2. systemd is looking at better isolating and reducing their dependencies, especially for more critical parts of the system themselves and liblzma is looking to be dropped as a dependency. Given these things, this backdoor may have been on a sudden clock where it is get it in next release or it is likely to be several years of setup for nothing.

[@chilversc]

I was also thinking once in the wild they're on a time limit before someone notices it so as soon as the malicious code was merged they need to infect as many machines as possible before that happens.

[@EwanMarshall]

@@chilverscthat is always a risk, there is a chance that the moment this backdoor is actually first used in earnest it would set off some intrustion detection of why are we suddenly getting an ssh connectiong from some foreign country where we don't have any offices or something.

[@sfulibarri]

Yea this makes a lot of sense especially given how ham fisted the push was compared to the slow preparation.

[@robertjif6337]

I got hit with skill issue every line of the article

[@kzalesak4]

Really? Where?

[@DuRoehre90210]

28:25 The obvious reason for the rush is probably a branch+ticket+PR in systemd repo to switch the library loading to runtime and be optional, and this looks almost ready. Just imagine, the hard work of many years to be flushed down the sink.

[@fernandobalieiro]

Imagine all the potencial back doors we still dont know about

[@ThePlayerOfGames]

The way that open source projects are worked on; at least you can find them in open source code. This compromise began when the attacker did a Social Engineering attack to get onto the project and add these commits In closed source code you'll never know you were compromised

Imagine! I remember that years ago the curl author shared a tweet about some guy saying something like "Thanks to curl (codebase) I can enjoy all my CVEs bounties"...

[@youtubelisk]

No. It seems pointless.

[@demolazer]

No way this is the first time. It's too sophisticated and well thought out to be an opportunistic attempt.

[@simonschneider5913]

@@ThePlayerOfGames exactly. i think this being found and explained the way it is now is actually great for open source, isnt it?

[@SimpleTubeKK]

NSA must be pissed right about now, months of planning gone to waste

[@joseoncrack]

Wherever this comes from, be certain that this is just one attempt among a bunch.

[@autohmae]

The scary thing is: if this is a state actor, that means this is just 1 of their attempts and they have multiple irons in the fire. Because they know some attempts will fail.

[@spicybaguette7706]

Years, even. IIRC the social engineering attack started somewhere in 2021/2022

[@craigslist6988]

tbh the US has better SE. And wouldn't have to "bug fix" their exploit lol

[@emptydata-xf7ps]

This isn’t months. This is years of build up. “Jia Tan” had been contributing to xz for 2 years before being granted commit access. The length and sophistication of this exploit all but points to a state actor/team. All interrupted by some over zealous solo developer trying to speed up his application.

[@mhdmuzaffar-vr5mb]

Where is the 13% accurate guy who was going to solve Open Source Issues, weren't he supposed to take our jobs??

[@ChuckNorris-lf6vo]

Yeah the LLM can translate this to English no problem so what are these guys analyzing here? Just wasting their time.

[@NeverTrust298]

​@@ChuckNorris-lf6vo yeah I just asked chatgpt about how to fix the current state of the Open Source Community and yeah I totally agree these guys are wasting their time AI

[@ChuckNorris-lf6vo]

@NeverTrust298 I can't comment without seeing your prompt and the llm output and opensource community is too broad maybe you mean Linux kernel and core components ? Or are you trolling ?

[@bugzpodder]

if you were given 20 issues from 20 different repos and asked to address them in a week, realistically speaking you might get maybe 20%-25% done in that amount of time if you are that good. And that's probably the best you can ever do, but then imagine the 13% accurate guy can one day do 50% or more with an upgraded model

[@edgars9581]

​@@bugzpodderBut then you waste 87% of the maintainer's time. Because they can only tell if a contribution is bad when someone looks into it deeply. It isn't better for open source if (even *if* the model is 50% accurate) half of the PRs maintainers need to go through are plausible looking but actually don't work

[@blackjackjester]

I'm an industry veteran of 15 years. I understand some of these words.

[@chupasaurus]

Things that run COBOL don't have native xz libraries. /s

[@JeremyAndersonBoise]

@@chupasaurusCOBOL, lol wut? 15 years was not as long ago as you surmise. Learn to history. Probably the guy was writing JQuery 15 years ago.

[@chupasaurus]

@@JeremyAndersonBoise /s stands for SARCASM.

[@Ellefsen97]

My speculation is that the person is not Chinese. The information that the name had mixes of Mandarin and Cantonese makes it sound more likely that it’s a non-Chinese person attempting to create a Chinese identity. I think it’s a very clever ploy to leave digital breadcrumbs that align with people’s existing beliefs. People want it to be a big grand Chinese cyberattack, so by intentionally choosing a Chinese sounding username people will immediately jump to that conclusion. We obviously can’t rule out the potential of it being a state sponsored cyberattack and perhaps even a CCP coordinated attack. But I think it’s important to be aware of our existing confirmation bias

[@magicmulder]

I think they picked the name as another test of how easy it would be to sneak in malicious stuff. People in the US would be very suspicious of Chinese contributors, so an actual attempt to be as sneaky as possible would probably use a French or Swedish name. Look at the first vulnerability in 2021, replaced a secure function with an insecure one while having an apparent Chinese name, and it got through. That was part of the test.

[@Ellefsen97]

@@magicmulder I like this theory, it makes a lot of sense. It’s like the Nigerian Prince emails where they are sending obvious scams to filter out people that are smart enough to recognize the scam

[@Ellefsen97]

@@magicmulder Makes a lot of sense. This theory is similar to how email scammers purposefully make the scam more obvious to filter out the people smart enough to avoid getting scammed. So this could be a similar thing where they tested the waters before commiting the time needed to create the backdoor

[@SoapSoapCrayon]

Having extremely complicated bash scripts that modify files during the build step is kind of wild in 2024. I'm not sure why you'd even set a build system up like this, seems like hell to work with, let alone audit for security.

[@autohmae]

Let's be very clear: he already was the active maintainer of the project, if he wasn't the code would probably not have been accepted. This was only possible because he played the long game. Assuming it's even a he, not a she or a group, etc.

[@mk72v2oq]

It's called GNU Autotools and there are still huge amount of projects using it. Migrating to something else like Meson or CMake takes a long time.

[@grzegorzdomagala9929]

You can do it in makefiles too. And if the project uses scons (python based build tool) it's even easier.

[@mike200017]

For me, the big problem that this has exposed is the vulnerability inherent to the OSS / Linux / GNU building and packaging systems. It's an arcane mess of Makefiles, Bash scripts, ad hoc patches, and tar-ballz inside tar-ballz. It's long overdue for some security to be built into all that, like properly sandboxing builds versus tests, and having verifiable steps. In this particular exploit, it looks like a crazy mess of bash magic, but ultimately it's scary because of how easy it was.

[@nb6175]

it exposes a psychological weakness in test code really. It's mind tormentingly bureaucratic and boring and our mind just defaults to "looks good to me" ... "ah yes, random shit... that looks like good random shit to me" and "tests PASSED".

[@dorianxonic]

If anything, this proves why OSS is good and why it keeps improving.

[@avarise5607]

GNU package installer when? Gotta admit, build and package process is whack, each dev does their own random shit

[@jamesnewman9547]

The nice thing about standards are that there are so many to choose from. Good luck getting all of those projects to adhere to a single system. Maybe creating bug reports and pressuring them would help?

[@simonschneider5913]

@@dorianxonic i, too, think that OSS is simply the least bad option out there, and thats demonstrated perfectly with this story now.

[@fpsmeter]

All major security agencies should be after the perpetrator(s). The caliber is HUGE. If those guys aren't caught and the whole thing is silenced then it must have been state sponsored.

[@canberkbaci]

Attackers didnt have much time left, as xz dependency was about to be removed/lazy loaded from libsystemd, breaking the backdoor. Might be the reason why they pushed for it.

[@darkstudios001]

smart

[@notapplicable7292]

Almost definitely a nation state. Lots of carefully crafted obfuscation & social engineering. I think over all this is a very strong argument for reducing our reliance on shit build systems

[@evancombs5159]

I think it is unlikely to be an individual, but it could be any large nefarious organization not just a nation state.

[@magicmulder]

@@evancombs5159 To me it almost feels like an org that had one genius coder and then total doofuses trying to actually get the code published.

[@complexity5545]

We're not going to narrow it down until we do heuristics on the accounts. Right now the bad guys are scrubbing and even scrubbing sticks out like a sore thumb. If it is a state or gov't sponsored effort, then a deal will be made behind closed doors.

[@jesusmgw]

"I'm receiving 16$ a week from my patrons, my goal is 20$ a week". Open source culture right there.

[@alexnoman1498]

It's someone's personal blog, no? Making $2k+ a year from a blog sounds very reasonable.

[@complexity5545]

@@alexnoman1498Especially if you're in a non-western country.

[@thegrumpydeveloper]

“Reproduce the binary via the source code.” Npm just a giant binary basically at this point. Needs to be compiled by an independent source.

[@jonathancrowder3424]

*laptop bag with stickers all over it lid opens* How do you do fellow open source maintainers?

[@pianochess1882]

19:26 "fork yourself" lol. new insult dropped

[@magicmulder]

I've been saying "fork" and "shirt" ever since watching The Good Place.

[@wietvergiet]

Not really a new thing. I saw someone walking around in a "Go fork yourself" tshirt before.

[@Volvith]

This opens up a whole new world of attack vectors. Even just the proliferation of this one aside, we have no way of knowing just how broad the compromise is. Scary shit.

[@danielschmider5069]

not really, first of all stop building half the operating system off of tarballs which arent peer-reviewed, OR actually inspect and scrutinize what's in them, especially some big "testfile_good-trust-me" binary which is loaded during the build process, for absolutely no reason at all.

[@theApeShow]

Exploit discovered because some guy on the internet didn't like the noise his fans were making. Head cannon.

[@xiangfred214]

Potential State Actor behind this attack

[@orbatos]

Clearly, it's flat out espionage

[@roymarshall_]

PSA about a PSA

[@kiwikemist]

LMAO is it Russia Gate 2.0 for you libs

[@orbatos]

@@kiwikemist Apparently you don't know what Russia did then or what "lib" even refers to. But no, this is espionage by an organized group and it is targeted. we don't know yet who made it, but the list isn't that long. Learn what words mean and try again.

[@kiwikemist]

@@orbatos lmao this is funny like the new Havana syndrome hysteria.

[@StrengthOfADragon13]

Suspending the original maintainer with appropriate explanation could be net positive regardless of if he was intentionally involved. Sometimes a forced break from things is good (also might keep him from getting hate mail while things are hot)

[@maxparker4808]

Compression algorithms do a lot of data deduplication so a real test file will have duplicated data to prove that the algorithm actually works. Files with high entropy don’t benefit much from compression. Just noting this as it would be expected for test files on a compression library to have that kind of repeated/duplicated data. If I were an attacker I might theorise that adding data to a test file would be less conspicuous if the added data compressed effectively 🧐

[@snowSecurityneeded]

Flip is my favorite editor.

[@xthebumpx]

Neovim is my favorite editor. But Flip is cool too.

[@rogergalindo7318]

prime and flip W right there

[@wetfloo]

this is my favorite comment

[@asjsjsienxjsks673]

It’s because it’s open source that we’ve discovered this. Had it been hidden, we would’ve never known about it

[@pluto8404]

exactly, theres gaurenteed to be russian and chinese spies in every major US tech company. No doubt about that. Who knows what sort of damage they are doing. Netflix probably has a spy as well, probably a streamer too, as a disguise.

[@alpacamax3404]

Had it been hidden the backdoor probably wouldn't be introduced in the first place lol. This xz situation definitely complicates stuff.

[@gileee]

​@@alpacamax3404 Not true. If he had a job for some Microsoft team he could have slipped the same code through.

[@lucasjames8281]

@@gileeeDave Plumber had made at least 1 video about this, there are significant QC checks in place at Microsoft. You’d need multiple people on the inside, which is possible. But much much much easier in open source

[@mortvald]

@@lucasjames8281 the thing is microsoft can spare like what a couple thousand engineers? open source you have a much bigger population, it doesn't matter what the backdoor is, it'll get found out. good luck finding this kind of issues with just a few hundred to thousands. Then there is the elephant in the room, what of microsoft approved backdoors? they're not your friends either

[@scooter4196]

Where was Devin when we needed him?!!!?

[@michaelb4727]

How can you be sure it's not Devin?

[@julians.2597]

​@@michaelb4727 the backdoor works

[@mllenessmarie]

?

[@bionic_batman]

too busy inserting print statements into some random python scripts

[@andreasgkizis2135]

after watching for 56 minutes i was already at " i am too stupid for this " however hearing the priameagean say it made me LoL

[@fulconandroadcone9488]

low level learning is lock picking lawyer of software, they would a neat team

[@Tristan-mr3pk]

I love prime’s content recently I’m just so friggin happy! ❤

[@canadiannomad2330]

Would isolating the build and test environments (ie via containers) limit this class of attack? Might take longer to build, but if the test suites can't touch the binary that is going out, then the injection should be impossible, no?

[@nuclearmedicineman6270]

The sudden rush to get it done after taking 3 years to set it up sounds a lot like management interference, like there's a boss demanding results.

[@NoidoDev]

Some other comment pointed out that some lazyloading dependency of XZ in systemd would soon have been removed, and this is most likely what they were interested in.

[@magicmulder]

It sounds like the coder selling his backdoor and the buyer being incompetent in getting it out.

[@complexity5545]

@@NoidoDevI read that too from some BSD guys. This is so big that even the BSD security nut guys are on it (and the gov't).

[@user-qr4jf4tv2x]

Exploiter: i would have gotten away with it if it weren't for those meddling Microsoft guys

[@ark_knight]

*Exploiter - I would have gotten away with it if it weren't for those meddling friend guy

[@AngeredZeus336]

If this happened inside of a large proprietary C/C++ code base, for example a foreign independent contractor with a fake identity at Microsoft or Riot Games was compromised and committed a malicious tar ball, most of the country would be compromised and almost no one would have the ability to find the issue. I don't think businesses are immune from attacks as sophisticated as this. At least with open source we have a chance to find the backdoors.

[@digicyc]

This has been happening for some time. There was a case where a group at a university tried to sneak in a backdoor into the Linux Kernel and got dang close before someone found it and Linus then went back and pulled ALL code that came from that University and banned them from any and all commits going forward. It was much more complex than this condition, but interesting it wasn't as popular because it wasn't on the twitters.

[@Nullzero98]

0:44 welcome to Costco I love you

[@jameshinds2510]

Question: would the backdoor still be relevant if SSH is disabled? Most linux Desktop users do not have SSH enabled, so this would mean the target was entirely servers.

[@xB-yg2iw]

Yeah it was backdooring the OpenSSH server process, if you aren't running that you are good

[@AndreKatov]

This episode of yours is so damn interesting! I'm really enjoying it. Thanks

[@paxdriver]

I'm so glad you two tag teamed this bad boy. What a delicious bro AF gigchad exploit lol I absolutely love this, it's a work of art.

[@bokunochannel84207]

imagine someone inject a crypto mining code into you CI pipeline.

[@Max24871]

This seems so obvious once you hear it, I'm surprised it hasn't happened yet/wasn't reported widely.

[@Papageno123]

12:40 this case may or may not be state affiliated. but it's extremely obvious there are state actors who are intended to deploy subtle bugs into widely used software.

[@samiralibabic]

The best demonstration of human intelligence and creativity I've ever seen.

[@mariobrito427]

Great analysis, thanks for going through it! This is truly scary stuff! it really makes you think how much stuff is out there actually compromising open source software that we're not aware of... 😢 Consider the following: this was only caught because of increased delay introduced by the exploit code. Now, what would have happened if whatever actors who cooked up this mess added a simple delayed activation logic? The exploit would be everywhere and likely no one would have been the wiser Scary scary shit

[@lobaxx]

Just an FYI: Lasse is pronounced ”Las-eh”, not ”Las”

[@c0ldfury]

The guy that discovered the backdoor and "got suspicious" needs approx. $100 million deposited in his account and be bought a beer.

[@The1RandomFool]

There must be more compromised packages.

[@ShankingDisaster]

LLL got me hip to the importance of C, i friggin love the latest Prime collabs!!!!!!!!!

[@koltonkulis4763]

Because of the number of heads in this command, I've been calling this The Hydra.

[@RYOkEkEN]

amazon,twitch,google ,youtube gotta retro actively pay open source creators going back to at least 95

[@victordvickie]

man jblow really predicted these

[@smnomad9276]

what did he say?

[@victordvickie]

@@smnomad9276 kzbin.info/www/bejne/jXjInIp8rcp3bMUsi=zBJA4Sc6Dyk_mwyO

[@zanilen98]

​@@smnomad9276kzbin.info/www/bejne/r6G9an2sitaUl9Esi=jpgqQSxR1oHxHhAd

[@lucasjames8281]

He described exactly this sort of thing happening. That there’s thousand of nation state threat actors who’s role is to do stuff just like this

[@autohmae]

I watched his video, some of his assessments in his video are definitely wrong, but I think it all comes down to: you are accepting code from random person on the Internet. That code possibly needs more attention than some of that in your own company. But notice how with the XZ hack they played the long game, to become the new main contributor. NSA does the same when they infiltrate a company. The difference is basically 0.

[@johanngambolputty5351]

There was something about moving to zstd from xz in this video, but looking at what the xz package is required by on my system, zstd is one of them (along with rustup and the base package)... Kinda goes back to being able to scarily run arbitrary code at build time in stuff like build(dot)rs (which I remember Jon Gjengset talking about), I guess being more readable/auditable than some of the arcane build systems is one step, but yeah, some sandboxing, like even having all the features (like network or filesystem access), but having to turn them on one by one as needed, and having to justify turning them on to maintainers... because otherwise it all comes back to trusting upstream. I mean given that they set the scene for themselves, by patching the fuzzing library and what not, could still potentially be bypassed, but the more steps a bad actor would have to go through (so long as it doesn't add much more steps to normal users), the less likely.

[@ViolentOrchid]

I would guess the 5 checks for Linux has something to do with finding where you are in the memory. You land somewhere in the checks, go until you find the last open/close square brackets, and then you know where you are. You could probably find the checks for Linux being passed to the OS for evaluation.

[@matthewrease2376]

Some binary can't be reproduced with code, like image files for example. Those were never compiled, they were just created. But even an image could have code in it.

[@magicmulder]

Wasn't there a recent attack on the boot process using a replaced boot loader graphic?

[@hemmper]

Shouldnt `binary_blob | manipulation | eval` be a red flag that could maybe be scanned for somewhat automagically? Not sure if I understand everything here though.

[@green.rabite]

This was wild that they found it!

[@beerat4405]

41:03 One note on chinese name things: many groups do speak multiple dialects of chinese, particularly in areas like Malaysia or Singapore (where the Tan last name would be used in Hokkien communities) or other areas with large dispersed chinese populations. My fiancée’s family, for example, primarily speak Mandarin and pronounce their chinese names in Mandarin, but use the Hokkien anglicization of their surname. So, while it’s a good to look at, it’s not necessarily indicative that the Jia Cheong Tan name is fake.

[@Maxible]

Read KenThompson's "Reflection on Trusting Trust" next 😁

[@jeffwells641]

"The Three Body Problem" is the best sci-fi book I've read in years, hands down. Also, I refused to get the sequels because the first book freaked me out so much, and I know things don't actually get serious until books 2 and 3.

[@magicmulder]

Read them, it's worth it.

[@gima123123]

Honestly, seeing how much effort was put into this makes me think the guy who did is simply a madman. Like lots of steps could be skipped with same effect. But guy wanted to prove a point and flex his genius on everyone

[@magicmulder]

Also zero preparation for the actual push to get it included in distributions. 3 years of backdoor preparation and then they use two freshly created accounts to push distributors? Sounds incongruous to me.

[@TreeLuvBurdpu]

This is a very good review. I'm glad you guys are both getting paid and making a profit to do this valuable work.

[@Filakhtov]

This hack makes my production code look poor with all of its robustness and future proofing 🤣

[@RobRoss]

I think it was Richard Stallman who warned us about this kind of thing the 1960s!. It’s one of the things that is supposed to make Open Source software more secure than proprietary software. But the price is eternal vigilance.

[@maksymiliank5135]

On the other hand, If something like that happened in a proprietary codebase, nobody would even notice because they wouldn't have access to the source code

[@complexity5545]

And stop using blobs. I hope it makes true open source instead of binaries sometimes (for drivers (cough cough Nvidia and broadcom)).

[@remrevo3944]

Could the "I Know About the XZ Backdoor" blog article please be linked too?

[@wanking9040]

Everyone shitting on obfuscated binary files but no one has mentioned the use 'eval'? eval should be an immediate red flag in any language

[@magicmulder]

But eval was hidden in the binary blob that was compressed and obfuscated. It could not have been found by a scanner.

[@wanking9040]

​@@magicmulder You're wrong. I just re-watched the video to double check and the there's an eval in the .m4 file that starts things off, before we touch any blobs. Timestamp 46:48

[@tune6000]

Why

[@nightvision32768]

​@@wanking9040I think he was mentioning eval at 55:16, but still

[@xB-yg2iw]

​@wanking9040 True, but this .m4 file never hit any version control, only the two test files it is carving the payloads from. The .m4 was only included in the malicious release tarballs.

[@Nil-js4bf]

The ingenuity of humans is amazing and sometimes scary. We did manage to harness the power of the atom in nuclear bombs decades ago after all.

[@nb6175]

This is so obscure I'm getting paranoid about the guy who even found the bug in the first place... my brain is like, "oh HOW CONVENIENT, you just simply stumbled on that!?" but then just has nothing to put after that. ... like maybe this was a compromised APT that was already under observation and "discovering" the backdoor was just a parallel construction--a way to expose it without exposing that they have a peep-hole into the APT's activities.

[@IronicHavoc]

Dude chill

[@nb6175]

​@@IronicHavocIt's OK bro. I've learned to type real quietly so they can't hear my keystrokes through the matrix. It's everyone else I'm worried about. TRUST NO ONE.

[@lowhat]

Exactly. If the NSA discovered it, they would find a misdirected way to disclose it.

[@craigslist6988]

if the NSA wanted to expose it they wouldn't need to find some guy to claim he found it.. they could do the most Chad git move ever and make a new anonymous account and submit a PR on the repo titled "Fix backdoor introduced by.." That'd be the most epic commit ever..

[@xB-yg2iw]

Meds now

[@Papageno123]

8:45 He said he noticed it because of high CPU usage, not because the slowdown

[@ineverknowdoyou]

i thought he said he discovered it because of the slowdown, and was suspicious of the CPU usage after. though i admit i didnt go back to rewatch that part

[@MikeBurton89]

Tom would have caught it without the need to experience a random slowdown.

[@Firstname_]

Tan Jia Cheong is a pretty legit name in Singapore

[@stxnw]

The way he types in his PR is also oddly Singaporean..

[@arcuscerebellumus8797]

I get the "no comments in my code" policy, but whenever I see expressions like this -> (49:10) -> it kinda starts falling apart for me... I think in such a case it really starts being nearly crucial to comment. Not even necessarily "what" you're doing, but more importantly "WHY"!

[@jimhrelb2135]

1:02:35 these guys vibe so hard LLL can keep up the tempo by telling a story out of no where. Keep up the good work :D

[@WaseemAshraf]

Being this backdoor so much complex, I highly doubt it is being implemented for the first time. From start to end, everything seems well crafted and maybe improved on the possible previous iterations.

[@schwingedeshaehers]

but it is to specific to xz that it is to easy i think

[@peteredmonds1712]

Seeing a lot of commentary on this issue pointing out how catastrophic this *could* have ended if it weren't for Andres' diligence. While that is of course true, the takeaway from this cannot be the story of how one very knowledgeable and detail oriented man saved the world. The discovery of a sophisticated, catastrophic RCE like this *necessarily* requires unlikely circumstances. If the attack was not discovered through these unlikely circumstances, we would never know how sophisticated and catastrophic the attack is. Conversely, if the attack was not sophisticated, it would not require unlikely circumstances to discover. Therefore, it is expected that catastrophic and sophisticated attacks will be discovered through unlikely circumstances. This is something like the anthropic principle for cybersecurity. The real takeaway here is that the more effective and catastrophic an attack is, the more unlikely you are to discover it.

[@stxnw]

“gaslit by the whole community” it’s literally just one guy bro..

[@mis4vr877]

Open source community in general with all open source projects

[@IronicHavoc]

IIRC there were like coordinated sock puppets trying to get PRs pushed through.

[@IronicHavoc]

Yeah they were referring to the other accounts that are now suspected of being sock puppets

[@stxnw]

@@IronicHavoc even if they were individuals themselves, it would still be like 3 at most lol, nothing to cry over. ive seen women get more hate under instagram comments.

[@xdxalexxdx]

I think that adding the 5 lines on the changes was to make the pad the file length so the exploit can work

[@aes0p895]

This is why I leave my repos on private most of the time. I've never been totally sold on open source; I have been on the bad end of unreasonable expectations too many times.

[@khalilzakariazemmoura8995]

I don't think this is related to Open source specifically. This could happen even in commercial software. Nothing in the source, everything is split between the tests and the build system!

[@autohmae]

Agreed, I do think maybe the git repo with the normal code should be separate from the repo with the test-code. And both should not be run in the same environment. Only the result of the build (without test-cases) should be packaged.

[@khalilzakariazemmoura8995]

​@@autohmae Totally agree since the source code is not compromised

[@autohmae]

@@khalilzakariazemmoura8995 the scary part is, is the real issue: the active maintainer was the compromise. I really hope Linux distributions and package maintainers take a couple of lessons out of this. They are the most important barrier after code review by the people directly involved in the project itself.

[@canardcoincoin1582]

It is really MOSSAD-ish.

[@ksk31337]

reminds me of obfusacted PHP malware from 5-10 years ago somehow, just the looks of the payload/malware-snippets "de" obfuscated

[@BudgiePanic]

This video flew by, didn't even feel like an hour+

[@Exilum]

Others might get scared by this. I, on the other hand, am getting reassured a bit. There's bound to be backdoors, the fact people are finding some, means there's one less backdoor to worry about.

[@bertram-raven]

From WarGames: D1: "You're telling him about our backdoors!" D2: "Backdoors are not secrets!" D1: "Yeah! But you're giving away all our best stuff!"

[@Ellefsen97]

The main scary part for me is that it was barely found and that it was found accidentally. I’m very glad that it did get found, since this will likely lead to developers scanning through tons of OSS libs

[@JordanShurmer]

plot twist - the person who found the backdoor is the person who implemented it. He's just after a bonus

[@thecodereviewer977]

This is a very similar approach to NodeJS event-stream backdoor. It's just better obfuscated, hidden in files meant to be garbage for tests. event-stream was more obvious because it had the encrypted payload but no legitimate use for that blob. It was triggered in the build (similar to test) and injected the payload only on a specific target project.

[@Griffin519x]

This was a great stream. I missed the last 20 minutes or so because of work

[@josefkaras7519]

is it possible that the jia guy is also innocent? if all the malicious code was pushed by hansen, then it is possible that jia just didnt look at the request at all and just pushed it in? or he looked at the code and because it was so well obfuscated, he didnt notice? dont know all the details, just trying not to jump on the guy, since i am not fully convinced yet.

[@Oshroth]

The problem with that assessment is that Jia has been pushing suspicious code for years, as they mention in the video, the first thing the Jia github account did was push a change to a library replacing a safe_printf function call with a unsafe version. And Jia only continued to push more suspicious code, basically sprinkling it in over 2 years. So it is highly likely that this was a planned attack

[@josefkaras7519]

@@Oshroth oh, thx for the comment, i must have missed that

[@darkquaesar2460]

it's the NSA

[@autohmae]

I don't know the country, but state-actor is an option.

[@darkquaesar2460]

@@autohmae It's literally the NSA, it's not even the first time they infiltrated Linux to push intentional backdoors to the Linux OS.

[@4lpha0ne]

Wondering, when this story (incl. the attackers) will end up in some movie or TV show, like in good old movie days (e.g. "23" about a West German Telekom hacker, who got in trouble with Soviet KGB).

[@alexfedorov1160]

I believe GH repo was blocked so that automatic build systems don't pull tars from there. Despite that Lasse Collin stated that GH repo is unaffected, who knows?

[@OdyseeEnjoyer]

xz -V returns 5.4.1. I love Debian Stable

[@abbe9641]

Do not ask a potentially malicious software what version it is, use your package manager to do so, use common sense on the internet.

[@ark_knight]

`sudo apt list --installed | grep xz` That's what the above guy meant to say. Run this instead of asking the malicious program its version. Because that means you are executing the program, potentially running it.

[@OdyseeEnjoyer]

@@abbe9641 I do only that command because I know 5.6.1 is only available on testing and sid and I was just curious to know how old my version was. Stable doesn't get updates that fast