Log4j Lookups in Depth // Log4Shell CVE-2021-44228 - Part 2

[@rjhornsby]

I’ve always liked your sort of casual handwritten visual style, and it especially shines here. I can more or less read Java, but there’s so much code in just this one tiny - almost incidental - library, it’s nuts. The way you approach highlighting, focus in on small code blocks, and your red/white callouts make it super easy to follow your narration. So, thanks! and Merry Christmas.

[@Fardeus]

true.....

[@jeffvalore5010]

your "secure design" section hits the nail on the head. They made the classic mistake of trusting user input, and never considered "how could this be abused?". JNDI aside, it shouldn't have taken a whole lot of thought to consider that anyone who has access to read the logs but not access the server or code directly, can just log all the server env vars (DB connection strings? AWS creds?) and potentially steal information. IMO this entire substitution "feature" of log4j never should have existed. Even if it was "secure" this is still a lot of bloat for features that a very small subset of log4j users would ever utilize.

[@theyreMineralsMarie]

Exactly, this is a classic separation of responsibilities problem. No logging framework needs to also have this level of environment lookup functionality.

[@Lukas12220]

Or disable the feature by default

[@mskiptr]

Yep, this is a job for plain variables and lambdas and not a separate string evaluation engine

[@TheCardil]

Log4j did nothing wrong, apart from having to much features. The culprit here are developers who pass user controlled data to be logged. That should never be done. Those values belong only to the internal database, and shouldn't be leaked to logs. If devs wouldn't log user data, it would be safe to get logs from production. Often that's not possible as they are littered with user owned data.

[@hb-man]

@@TheCardil Logging uncontrollable user data is what logging is about: "Could not login user 'attackerstring', wrong password." You cannot expect the developer to escape for any special chars to prevent code injection, just because the logging engine has surprising features.

[@monognome9202]

As a developer, this vulnerability caused so much chaos throughout the weekend in webapps using Java. But you explaining it makes it seem so simple. Amazing video 🙌🏽🔥

[@jamess1787]

I'm still astounded at how much is still written in Java. It's wild.

[@monognome9202]

A whole lot man. It blows my mind everyday.

[@MuhammadBinZafar1]

I'm glad that you have 380+ videos. I never want to run out of your videos. I've been listening since 2018🎈. Always enjoyed every single moment with you. Each video is unique, knowledgeable, perfect 🎉. I wish the best for you!

[@spicybaguette7706]

I'm curious whether anyone ever has needed the _recursive_ lookups... It's a logging library! Not a string substitution one! Why is this even enabled by default in the first place?

[@lgfrbcsgo]

Because someone somewhere thought it would be a good idea to be able to change the log format on the fly without a redeploy?

[@ritamdey8427]

This was my reaction when I read about this feature! The lookup and complex formatting should've been a add-on library rather than the main one, which should've stuck with logging strings and simple formatting like %t etc.

[@karigucio]

for real, never would have thought a logging library somewhere in the code checks for infinite recursion

[@desmond-hawkins]

The point is to avoid crashes due to the recursion limit being reached, not to explicitly log recursively. It's common to log the details of some objects with toString(), and sometimes people will log objects that have toString methods that are more complex than they should be. To be clear, none of this is about an ideal case, but to prevent a _worse_ case to an already sub-par situation. If you log an object with toString which causes it to dump a whole bunch of information to try to be helpful, and this information is calling methods… you can easily get into one of these situations. Imagine a Circle class, with a center of type Point and a radius of type float. You could have an overzealous toString do something like: return "Circle(center="+center+", radius="+radius+", area="+this.area(); - now if your call to area() does something like log.debug("computing Circle area for " + this); - then you end up blowing up the stack. Purely by accident.

[@EER0000]

Great video, nice to see an actual dive into the code and not just more talking heads. I wish it was easy to keep the codebases clean and simple, but so far any project that I worked on with a team devolved in a mess of dependencies. The hardest part about keeping things simple, is to convince stakeholders that it's worth spending hours to implement (e.g.) centralized logging yourself when there's a library like this already available. They'd rather see me working on a shiny new button than on stuff that nobody sees in the backend.

[@passerby4507]

But you can't just write everything in-house. How do you decide what gives?

[@EER0000]

@@passerby4507 when I was a kid I would be making everything myself using just notepad, because that was the way to flex as a dev. With time and experience you learn which kinds of things are worth doing yourself, and which ones are not. I would never write a pdf converter for example, but I have written a basic SOAP parsing library using standard xml functions.

[@AFchump78]

“I have no clue about Java or maven with dependencies” i felt that in my soul. I went back to 3 deserialization issues after the log4j learning curves to reattempt that exploit after the Java and java development learning curve that log4j caused. Beautiful and tiring at the same time.

[@vaisakh_km]

😂i trained a complete year in Java and I made many projects in my training period,. But after getting job I never touched it... Now when I tried to make a hlowld , I even forgot to put public static void main in class.... ....😑 All the hard work just vanished innu head... Now I wondering how I did build big spring projects that time...

[@santi5655]

7:40 totally agree, it's crazy how complex a logging library can be for no reason at all, simplicity is also good for keeping the code maintainable and easy to understand

[@T0m1s]

It's a mindset. Java developers seem to want to split their code into as many classes as possible, and this behavior seems to persist even when they switch languages. OOP is cancer.

[@comradepeter87]

5:18 What was the point of having so many classes and factories and functions if the actual functionality is just shoved into one function?

[@jackbendtsen8002]

Absolutely, simple code means all of the code that your program could run, not just the code you wrote. I feel like not enough people truly understand this. Thank you for your insights beyond the topic at hand.

[@novelhawk]

4:50 Isn't it pretty much unoptimizable? How are you going to find a substring without iterating the string?

[@lassipulkkinen273]

I was wondering the same thing, although from when I was last using Java (years ago) I think there was an Iterator thing that was supposed to have a small performance advantage over c-style for loops by eliminating bounds checking or something... Also, maybe String.indexOf would be even faster if it has an optimized (i.e. non-JIT-ed) native implementation.

[@krzysztofdziembaa4982]

There are string searching algorithms, which may be faster than "brute force", but (according to an email in Newlib's mailing list) for short strings (which are quite popular) the naive algorithm is the fastest. Other algorithms (may) require preprocessing and/or memory to allow faster searches and whether this pays off depends on the processed data. Two-Way algorithm is one of such algorithms and is used in e.g. glibc.

[@lassipulkkinen273]

@@krzysztofdziembaa4982 I knew about those, but I don't think they are useful if the searched-for string is one character. While "${" is technically longer than that, the code *is* optimized to only look for "{" after it's found a "$", and "$" is not a character that's going to appear often in the input, apart from lookups of course. That said, the parsing could just be moved to an initialization stage if the "lookups in format parameter values" misfeature was removed, so... (maybe this is already being done; I don't actually know.)

[@ThePC007]

You do it using a finite state machine. Granted, the substring is so short that it likely doesn't matter much anyway.

[@tunasalmon1243]

The best gift for christmas for an aspiring developer. Thank you so much

[@cdhagen]

This library is a prime example of over-engineering! That's why I don't 100% agree with people who claim that it's just „because a few overworked guys are thanklessly maintaining this thing for free“. Yes, maybe, but who forces them to build crazy features like this into what *should* be a relatively straightforward logging library?

[@lal12]

No I don't agree. Yes the jndi stuff and java indirections are examples of overengineering and insecure practices. However the bug in specific is basically a classic injection bug, which should be a common pattern how to handle this.

[@cdhagen]

@@lal12 True, unless you are overwhelmed by your own complexity. ;-) Also, maybe this library would have more maintainers if it weren't so complex. The fact that this "classic injection bug" has gone unnoticed for so many years speaks volumes.

[@lal12]

@@cdhagen well I don't disagree. And Log4j probably has issues with documentation and usage patterns. That being said it is kind of unfair to blame the developers of Log4j instead of the developers of the individual applications.

[@DanKaschel]

They're not slaves to you or anyone. They're allowed to add features. Even ones with bugs. They define what it "should" be, not you.

[@cdhagen]

@@DanKaschel Of course, I can't bar them from adding some more of these golden "features" where a logging library makes random calls over the internet depending on what magic string is passend in. Keep up the good work, I guess! 🤡

[@klotzklotz1]

I think another interesting aspect of this vulnerability is that, if your organization used "defense in depth" and configured firewalls to block outgoing LDAP connections, exploitation of the vulnerability does not easily work. Unfortunately many people do not restrict outgoing network connections of their servers though :-(

[@berndeckenfels]

Yes egress filtering, not for “outgoing ldap” but “outgoing anything” since the attack allows arbitrary tcp ports.

[@klotzklotz1]

@@berndeckenfels ok, if the target port of the outgoing TCP connection is not necessary the LDAP port I stand corrected :-)

[@isitanemail]

I was waiting for something that could explain in depth the full vulnerability,log4j and jndi. These videos helped me a lot in understanding. I would request to release such content more and probably a little quicker to help clear these things. Some other explanation caused a lot of confusion

[@einsjannis]

I really enjoyed your videos about log4j I really happy that some one added something new and I actually wad able to learn sth new

[@superblaubeere27]

Some people take it a bit too far with the OOP. Splitting code in three-liner functions and douzens of classes does not really help the readability of the code haha

[@vert3x560]

ppl just don't understand OOP. They make 3 liner functions, and then some monstrosity 500 lines of code and for-ifs. To be honest. Clean Code clearly says: Keep It Simple. So I can imagine rewriting log4j into some clean representation with almost all features in like no more than 10 classes. One class one feature.

[@DanKaschel]

Good OOP would have prevented this issue. There never should have been multiple places where the nolookup needed to be checked for whether to evaluate a lookup.

[@mateowang6570]

Very very interesting from a developer perspective. Thank you for this!

[@anttijokipii]

Thanks for you content. It is really good. I have worked as programmer/architect/CEO on multiple decades, but have background from hacking. It is interesting that security and development is actually so separated.

[@ko-Daegu]

this is the most complicated tut on the internet about this.. thanks

[@bluejimmy168]

At 13:15, he said keep format string static to fix the problem. What does he mean by that? If a string is static then we dont need user input? If we need user input then it is not static? If I ask a user for input and want to output it back to the user then the string not static anymore? Thanks.

[@ninjanape]

thanks for these two videos - very informative, very well presented :) !

[@hblaub]

Really nice Java library. I think we should load custom code per socket from somewhere, too. It's going to be totally safe and sane.

[@comradepeter87]

No wonder it took so much time to detect this vuln. Most people who just wanted to explore the library were probably intimidated away by how bloat it is. Actual function that does the job has 5-10 functions of bloat over it. Can't believe so much CPU time has been wasted on just Java OOP things throughout the computing history.

[@telnobynoyator_6183]

not as much cpu time as you might think because java optimizes that away later in the vm, it's more of a readability issue

[@luis_musik]

@@telnobynoyator_6183 i think you're overestimating how much the compiler can actually optimize away. most oop features (especially stuff like virtual functions etc) are not zero cost.

[@telnobynoyator_6183]

@@luis_musik The java compiler doesn't optimize those out, but the JVM almost surely does when translating the java bytecode into binary.

[@DelusionalLogic]

@@telnobynoyator_6183 "surely"

[@telnobynoyator_6183]

@@DelusionalLogic In a sure way

[@caujka]

Lovely explanation! Absolutely marvellous! Thanks!

[@danbondarenko7894]

Thank you for the great explanation! I especially liked that you went step-by-step through the vulnerable code -- this helps me grasp what it might feel like to write vulnerable code. If you're looking for ideas of what to cover next, then I'd propose explaining what happens within JNDI lookups: when does remote code get executed? what would be a legitimate use case of this remote-code execution part of JNDI? Thanks again.

[@nekuskus5218]

Thanks for the great explanations! Merry Christmas!

[@rydmerlin]

Can you elaborate on how you get to remote execution though?

[@stanleycoffey]

maybe go watch the black hat talk

[@jasontiscione1741]

It happens through object deserialization which uses the remote class loader. The log4j programmers did not actually intend for object deserialization to take place, but it can be triggered. The log4j code creates an InitialContext and passes the JNDI name to lookup(), expecting a value that it will immediately cast to String to be logged. But inside lookup(), Java notices the crafted array with the serialized Evil object stream in the attacker's JNDI response and decides it can return an Evil object to log4j instead of a String, so it loads the Evil class and then proceeds to run the Evil static initializer. This crap was designed for the 90s Internet when Java meant applets.

[@kawcco]

This library is hilariously bloated. Who knows how many other vulnerabilities are hiding in the Log4J Jungle?

[@TenderBug]

What terrifies me is how many Apache libraries out there. Even more so other opensource libraries with big community driven projects. Log4J ! Serving Lookups Since 2013

[@kawcco]

@@TenderBug Agreed. We need to extensive code review and fuzzing of the entire Apache family so that we can cover our bases.

[@jhbonarius]

Have you read many comparible libraries? In my experience al mature libraries are like that. It's both a result from feature creep and common design patterns. The thing is: everybody's now complaining and nitpicking on this one, as something was found. ( a captain hindsight moment. )

[@kawcco]

@@jhbonarius The fact that they're all like this *is* bad! I'm not defending **anyone** who writes code like this!

[@jhbonarius]

@@kawcco is it your responsibility to defend them? Anyhow, code usually doesn't start out "bad", but 'evolves' into it over time. Call it "code rot". Especially in understaffed projects this can happen, and log4j was understaffed and open source. This is by far not the worst code I've seen. And considering the popularity of log4j, it wasn't disfunctional.

[@anacierdem]

Having the developer pass the contents instead of parsing various things is a much better approach even from a software development viewpoint here. It is inversion of control, which effectively reduces inner complexity.

[@ltxr9973]

Very good comment about hidden complexity. It's rampant in Java.

[@desmond-hawkins]

Regarding the question at 7:14 about using ${env:HOME} vs log("... {}", System.getenv("HOME")): there's more to it than this trivial use case. While the example given here is of course less risky and still just as easy to read, expressions are also expanded from other places where you're really writing config, not code. For example, when logging structured messages (often to Splunk or similar) you often want not just the log message but also the hostname, the process name, the environment, maybe the cluster and namespace if you use Kubernetes, etc. You can configure Log4j to include such values as named fields in your structured log events, with config like . You're not writing code here, but config: this goes in log4j2.xml. What's the alternative if we block all such expansions? You'd likely have to write your own Layout class and implement each field in code. You could also have your own (non-expanding) config file to map environment variables to event fields, but then (1) now it's a lot more code to write and (2) it's still not as flexible as Log4j since you can't do things like having a field named "cluster-namespace" with the value "${env:K8S_CLUSTER}-${env:K8S_NAMESPACE}". So yes, expression evaluation *is* useful in the sense that you can have advanced logging configuration without code, which covers a vast majority of use cases. Adding any kind of network call to this system is insane, but it is certainly possible for this feature to be flexible and useful without going overboard.

[@TheNovakon]

Why every somebody needs lookups in realtime? We are processing logs on different node - all lookups can stay unresolved and eventually resolved later during processing - because realtime processing slows down a lot

[@desmond-hawkins]

@@TheNovakon Being able to query logs in Splunk or Logstash a few seconds after an event happens is *essential,* and working on any system without this ability is much more difficult. What do you do if you have to wait a while for the batch logs to be delivered? What happens during an outage if you don't have access to logs to understand what's going on?

[@siimk13]

Not to mention that any code change requires a roll out/deploy to production environment whereas with log4j you could just change the log config file and have log4j reload it. Easy as pie.

[@desmond-hawkins]

@@siimk13 Good point, and changing code also means extra build time, in almost all cases. It's certainly needed for Java since you have to compile it, but these days even interpreted languages have a "build" time if only to create a docker image or any other kind of package.

[@TheNovakon]

Any example please!!!

[@NareshKommuri]

Thank you for sharing, really interesting perspective. Awesome in-depth analysis in few minutes. Cheers!! 👏

[@berndeckenfels]

I remember the time when I was reviving the design concepts DJB used in qmail. One was a very careful logging concept. It was going in depth into timestamps (TAI logging has no problems with timezones and leap times) as well es escaping user provided data. It is just important that log messages can’t escape the log format (line feed and separator evasion). Once you are that thorough with design you would not 8ntroduce such lookups and expansion patterns, I always missed such a escaping system and structured handling of log message arguments in the typical java frameworks.

[@protiumx]

Great video! I don't understand how companies do not audit their dependencies before adding them to their stacks. I hope it's also a lesson for the big companies that make use of open source libraries to start funding open source projects that add value to their products/business.

[@blizzy78]

That's very easy to answer: It takes a lot of time - and therefore money - to audit something. Especially when you're not intimately familiar with the inner workings. That's time and money companies are not willing to spend because it doesn't directly produce earnings.

[@jhbonarius]

Blizzy1978 is completely on the point. Imagine you're a small software development company. In order to get a contract, you have a small margin. You have to deliver within a few months. There's no time to develop every part and framework yourself, let alone test and mature it. Also: doing everything yourself is risky, because you often don't have enough experience with everything. I.e. we don't reinvent the wheel and use libraries everybody is using: if everybody's using them, they're probably mature is the assumption. There's no time and money to audit anything usually. Money doesn't grow on trees and it's a competitive market.

[@Dropje86]

Man, you have no idea how many arguments I’ve had with developers about levels of abstraction. The DRY principle has been taken way out of its original intent and people now just add layers of indirection for the sake of it. So often things can be kept much simpler by repeating yourself.

[@deviousmethod1310]

thank you for another amazing work!

[@danthe1st]

+1 for mentioning insecure deserialization of untrusted data.

[@berndeckenfels]

7:35 I agree that env: lookup for the sake of easier logging is not a good security design, but I think the idea was using lookup in patterns or config declaratively. Looking up messages is just stupid,

[@mrkretschmer]

what if I replace the lookup function directly with return null ? =) fixed ?

[@Klatuu82]

and another very good video. thank you for your work.

[@TheBiggreenpig]

7:35 Yeah, when I learned about the JNDI and environment variable possibilities of log4j, I was wondering why not simply let the programmer retrieve those and pass them as a string. Log4j definitely does more than it should. In no code we ever used any lookup, but we had them enabled unknowingly.

[@Ostap1974]

@13:45 -- was my first thinking as well... Why would anyone ever in any language write a text fromatiing function that has a formatting handler outside of the format string.

[@supermonkeyqwerty]

7:25 As a CTF'er by hobby and a developer by day job, I don't think this is a good idea from a library developer perspective in the first place. Whenever you provide 2+ different ways of doing the same thing, you're making code less consistent and harder to refactor in the future. Imagine that at some point in the future, you or your team decides to move off of environment variables and want to use a configuration file instead. You might think of searching for all usages of `System.getenv` in the project and replacing them, but that would leave this `${env:...}` logger config thing incorrectly untouched. Of course, it's only a cost, and that might be outweighed by a payoff. But I don't think that's the case for Log4j's feature, since the added convenience is tiny, developers might already be used to doing it the 2nd way as the standard, and it doesn't generalize. (What if you wanted to use the same string message in the logger and someplace else?) This happens much too often, because it's so easy to lose the forest for the trees. There've been times when I finally stepped back, and then realize that something was completely unnecessary.

[@lal12]

I think people are kind of looking at this issue the wrong way. Yes I agree the log4j is bloated and jndi is a terrible idea. However at the end of the day this is just a classic example of injection, where the user input was passed into the format string instead of as an argument (to use printf terms here).

[@supermonkeyqwerty]

@@lal12 Honestly, I haven't look at Log4j too closely myself, but does it even have an analogue to printf's `%s`? From its recursive substitute shown off here, I'm not convinced that such an option wouldn't just have the same vulnerability as using the string directly. Either way, whether understanding the entirety of a library's API and possible configuration is the _responsibility_ of a developer or not, it's not something that currently happens or will change anytime soon. I've got no clue what we'll do to prevent something like this in the future.

[@tambling3961]

Holy hell this library is bloated like there's no tomorrow 🤯 Besides the security issues I imagine that it must slow down your code by a significant amount... dafuq

[@alessandro.rossini]

Freaking 2013 maan! It's wild!

[@john33john33]

one thing i learnt from many security issues. Try NOT to write your own parser famous browsers, frameworks screwed up and i ain't outsmart those engineers

[@FUTUREPES]

Explanatory 💝

[@AniltonNeto]

G.O.A.T very well explained :) LOF

[@tlouik]

respect to everyone who watches this on christmas :O

[@firetf]

While i agree with the secure design philosophy, one big reason to have a logger internal formatter is performance. You often have tons of log calls to log levels that are not printed, and then the whole lookup and string formatting can be omitted, while if you do it manually, you would first look up all the things and build the formatted string, just to throw away all that work if it's a log call to debug level but you're on error level.

[@olafbuitelaar]

I Also think there is another aspect in this. Which is configurability. Like the environment variable case. It might not be included in the source. But in some production instance its useful to log this information, it can simply be added to the config. Instead of adjusting the code.

[@amyshaw893]

this thing about using less complex stuff for security reminds me of how i approach coding games. Recently I was working on something in a team, and I was like "ok, we need to play some sounds. we can use a simple sound manager and shove it through unity's inbuild sound system, done." But then this designer comes over and wants to use this huge complicated library that allows you to do mixing on the fly and spatial sound and allows the designer to make complex soundscapes without having to do any code and all this crap and I'm just like "...or we could just play some sounds, and you use audacity to make the sounds, and our git repo doesnt explode from all the new files, increasing clone and pull times"

[@BatteryProductions]

effin awesome man!

[@roboknight]

Unfortunately, you won't be able to use an XML file, as Log4j does, to enable configurable logging if they just use in-line environment variable look-ups. However, the real question is, do they really need this kind of flexibility in a logger? Maybe or maybe not. Maybe they could have parsed the file way before the library goes and does anything with it to help secure it. But it is difficult to get simpler when you want flexibility. Hard coding does help secure some things, but can also introduce other bugs (buffer over-flows come to mind), simpler or not.

[@gubgubbubbub]

Awesome, informative video; thank you!

[@icenberg5908]

Amazing. Thank you.

[@ZintomV1]

2:19 - the reason why most Java applications are way slower than their C++ equivalent (I'd argue even C# equivalent)... Not because of the JIT, but because of needless object creation and abstraction, if you just need a simple string, leave it as such! No need to wrap a class around it!

[@omri9325]

My theory about that is that most of the java ecosystem was created at a time were design patterns where THE THING, and everyone jumped in adding patterns without even knowing when to add them, people were forced to add them just to not look idiots in front of their peers. When a peer adds something with many patterns/abstractions. you don't doubt that it's too much abstraction, it might be that you are "too stupid" to understand and you let it slide. Languages that evolved in modern times don't include all those AbstractFactoryManagerImpl shit, they add abstractions only when they are needed.

[@-morrow]

object creation is very cheap in java due to it's memory allocation strategies and is mostly faster than C++ new object creation. besides, for almost all java applications the kinds of speed improvements you get from using C++ instead of java or optimising object creation are dwarfed by the actual bottlenecks (round-trips, db queries, disk seeks, time complex algorithms, etc.)

[@jhbonarius]

@@-morrow you're comparing apples with oranges. Sure maybe the default "object creation" of Java is faster, but C++ is not an object oriented language like Java is. You have much more flexibility. And if you write a custom allocator and pre-allocate, C++ will likely be faster. But: you have to do everything yourself. It's a lower level language.

[@jhbonarius]

Their equivalent? What applications are you talking about? C++ and Java play different roles in the programming ecosystem. They each have their strengths and weaknesses. You wouldn't expect applications to be written twice in each of them, as there's no purpose to that. Instead you would expect larger applications to have parts written in either and some interop between them.

[@omri9325]

​@@-morrow Bottlenecks like the huge amount of extra memory needed by JVM applications. The classloading system in java is painfully slow on program start-times. The only thing java can do faster than C++ is on optimizing things it learns to predict from the runtime. C/C++ can technically optimize everything better, but with a lot more effort to hand craft them.

[@C10udburst]

wouldn't it make more sense to make the lookups into a different library for example? why is this a part of a logger??

[@LeandroCoutinho]

"I'm a prophet now!" kkk Great job!!!

[@shemmo]

8:40 thumb up for this

[@sikkavilla3996]

linus, thank you for providing an in depth analysis to this vulnerability. Very much appreciated!

[@SimGunther]

12:20 Proof that he's a time traveler 😎

[@tehwinsam3522]

Just want to wish everyone merry christmas !

[@krzysztoflewandowski8262]

CVE-2017-5645 was kinda first on the same

[@cescfabregas4377]

Great video!

[@lijnk]

The main reason the library has all this indirection and events is so it's thread-safe, easy to write new appenders (not everything uses just files and stdout), and so it's unit testable (good dev practices mean the code should be tested). If they didn't have that, then all the log messages would be jumbled up in the same line, it would be impossible to add new appenders like logstash or slack, and whenever a new release is made, the maintainers wouldn't be able to guarantee the project functionally works. With that being said, some eyebrows should have been raised when lookups were performed in the log message itself.

[@omri9325]

You don't need to defend the bad choices that were made there, the code is still shit.

[@ZintomV1]

@@omri9325 lmao

[@lijnk]

@@omri9325 The overall structure of any decent logging library is to take log events and pass them to appenders that write them somewhere, which the library does. Of course there's bad code within it, the noLookups section proves this, but saying the entire thing is garbage is about as asinine as saying security is only there to hinder developers. If you believe the whole thing is shit, you're welcome to contribute cleanups as it's open source.

[@omri9325]

​@@lijnk "saying the entire thing is garbage is about as asinine as saying security is only there to hinder developers" there is no logical progression towards that statement. I'm not going to to touch any of this shitty code, I'll rather contribute to projects that are healthier and have a future. Let that shit die.

[@TimLF]

I agree log4j is pointless bloat. Spring (86% market share) and Tomcat (61% market share) are not using log4j, netty is using log4j.

[@jamess1787]

1) Design for simplicity. 2) get it working 3) revisit project 6 months later realizing you forgot target #1. *Ends up with 1000's of lines for some basic function*

[@timelessnesses]

*f a c t*

[@TheNovakon]

13:51 - THIS IS THE CORE OF THE PROBLEM! The whole format vulnerability is easy to understand but this is treachery!

[@Ms.Robot.]

This is nice. ❤️

[@cybersecurity3523]

Good job bro 👍👍👍

[@brunoais]

9:17: It may seem weird but I've been screaming about this kind of use of log4j since 2017, I think. However, I haven't seen any of your stuff until 2019. I didn't know log4j had jndi, btw. I stopped using it back in 2016/7, IIRC 12:51: This vulnerability with databases is very very prevalent! I never had such issue If I knew log4j had jndi, I would have known how dangerous the lib is much sooner. Thankfully, someone found it. Still, I wonder how many systems were attacked due to that.

[@borisbo94]

7:40 this is actually how you SHOULD write software. From a software engineering perspective. This is the KISS (keep it simple stupid) principle. This library is over complicated for no reason… But take this aside… you should never parse strings to code.. same as you shouldn’t use exec/eval etc.

[@lal12]

I mean it depends how you mean it, but quite often it is necessary.

[@LiamDennehy]

@7:39 "Complexity is the enemy of security"

[@ruaidhridevery1458]

Devs get so much convenience and utility from _eval everywhere_ that preaching the simplicity of being more static is a really hard sell unfortunately. For this style of all-things-to-everyone library, it is nearly impossible for the devs not to embed interpreters.

[@codewithprabesh010]

Good Explanation. It's really helpful👍👌

[@amarmohmed1648]

I have the same passion that you had when I was young, and more than that, I saw THE last video and I did not understand anything, so I saw your first video, which is the intro video Please, would you teach me the name of that passion?

[@passerby4507]

Classifying log4j string substitution as syntax sugar isn't exactly fair. It allows for recursive lookups, e.g. have an environment variable store what's supposed to be looked up. Now that may or may not be what you want, but it's not the same thing.

[@TheNovakon]

Please any use-case to have this; ${env:WHATEVER} and put into env variable WHATEVER=${jndi:ldap:...}. ??? Why I need this? However even content of enviroment variable should not be trusted...

[@voidedname]

I'm a software engineer, and I hate most magic libraries. ORMs are my biggest gripes, log4j is another, spring boot is a huge issue... Magic is always bad, it exponentially increases the mental complexity, all the magic that isn't clear from the code has to be kept in mind. But sadly, in the java world at least, you're more or less forced into using them. There are ways to make the code prettier, I won't get into it. But I absolutely hate invisible side effects. If I want my code to cause side effects then I damn better actually tell it to do that side effect. The reason it's so popular is because it (wrongly) seems like you can write code faster and more efficiently. That is true when prototyping and in the initial phase of a project, but the hidden complexity quickly swallows it after just weeks where you spend an ungodly amount of time and mental effort to maintain and develop further features...

[@marcoantonio7648]

too much time to run Maven oh yeah, I felt that

[@obsidianskin9502]

good stuff

[@drakey6617]

10:47 in my opinion is not always correct. They first arguments can actually also be in registers and not on stack. Stack is used when the first registers are full. At least this is for ARM

[@dzaima]

"here's how it looks like with a simplified stack in memory"; 32-bit x86 passes everything on the stack, for example

[@drakey6617]

@@dzaima Yeah, true. I suspected that this is true for x86; that is why I said "not always" and "for ARM"

[@rastabong420]

@14:59 am I watching an old Godzilla film? 😁

[@jhbonarius]

Hahaha, all these endless (and typical Java) abstraction layers... and then you finally come to the parser, and that's just a much too long method written in a C-style way.

[@Torterra_ghahhyhiHd]

8:37 i think developer like to introduce this kind of complexity for 2 main reasons 1 he want to be needed in the job he is hoping to add new capabilities more power to the program the program like the art of the artist have some subjectivity and is never finish always want to ad some extra features or hidden potential that only the creator can exploit more.

[@altairbueno5637]

“It took me soo long to learn how to use maven with dependencies” Literally every java developer ever

[@ekfliu]

at least it's not gradle.

[@altairbueno5637]

@@ekfliu its Definetly not Cargo

[@mapp0v0]

You have reminded me, as someone who hasn't programmed in years why I feared java. Of course this bug was discovered earlier. Didn't you see the reaction on the Chinese. I am sure the Five-eyes are equally pissed off.

[@TheMAZZTer]

This huge bug is caused by a desire to have string interpolation in Java, which doesn't have it. The problem is string interpolation is done by the compiler, meaning runtime strings (especially user input) can't trigger interpolation. String.Format looks like Java coders' best bet, but again, keep the format string static! But you won't have issues with recursive format specifiers (I would hope)! Example of string interpolation in other languages: JavaScript: `This is a string ${expression + goes + here}` C#: $"This is a string {expression + goes + here}" Looks very similar to how Log4j does it, but in both cases having ${} or {} in any of the variables won't trigger a string interpolation, so it's safe.

[@jasontiscione1741]

The bug was 10% caused by string interpolation and 90% by Java's eagerness to whip out a remote class loader and start executing mysterious crap without being asked, which seemed like a good idea in the 90s.

[@passerby4507]

@@jasontiscione1741 Funny thing, python pickle is the exact same.

[@kawcco]

@@passerby4507 I rarely see `pickle` in real-world code at all, and if it is it's usually for efficiently storing instances of Python's top-level data-types (strings, numbers, sequences, dicts) locally. Sending Java objects over the wire seems to be scarily common, however. I believe this is because Java's standard library lacks a convenient way to send data over the wire without touching XML. Us Pythonistas have `json` and a much easier to use XML lib, while Java users have *multiple*, painful XML libraries that must all be used together and no way to parse JSON without bringing yet another external dependency into the fold.

[@ZintomV1]

4:50 - May be part of the cause of why Minecraft runs so slow, I imagine there's a significant performance uptick if a server enables noLookup and this whole "loop over each char" thing is skipped.

[@wouter11234]

Looping over each character of a string causes nearly zero performance overhead, unless you are getting tens of thousands of logged characters per second, in which case you'd be having other problems

[@stewartzayat7526]

Using java causes massive performance overhead.

[@NewtonMD]

Hey security researchers, you missed the *"rediculous"* example

[@mskiptr]

Tbh, format strings really are an anti-pattern. They conflate two different functionalities and make it less obvious what should happen where. They can be made safe (vide printf in Idris) but in such case it's an absolutely pointless complexity. We should just stick to clear and unambiguous abstractions. If we want to log some string literal, we should be able to just do so right away (without any need for escaping or lookups). If we want to log a value of some variable it should work the exact same way. And if we want logs to be parametrized with the future environment, then we should either use lambdas (Environment -> String), or some specialized data structure and not a string-but-with-an-overloaded-meaning!

[@fabricio5p]

Honestly this code feels like they intentionally wanted to go fancy with the architecture and end up with a classic freaking mess

[@aatif._.alamyt]

It's not about the code being a ridiculous example. It's actually about drive and power

[@kasuha]

That thing with the check for "nolookups" present in one branch and missing in the other displays serious problem with open source software: Neither the implementer nor the reviewer were familiar enough with the method to notice the problem. In many cases, people just don't have competences assigned and they just nudge the product at random places wherever they see it fit while missing in-depth knowledge of the part they're messing with. What's worse, there are already cases of actual vulnerabilities being intentionally pushed into open source projects disguised as fixes. Sometimes they're obvious but there are also really well prepared ones which take a while to be recognized and removed. And who knows how many are there that nobody but the author knows about.

[@sodiboo]

7:39-8:48 Yeah, the readObject makes your code simpler, which is great for readability, but this isn't the complexity that's good for secure software, and the JSON makes your *what actually happens* simpler, which is less readable because you write more complex code, but overall your PROGRAM as a whole is way less complex and (not necessarily, but because of the context i'll argue it's) more secure

[@Jankoekepannekoek]

You make it seem that every Java codebase uses lots of indirection and long class and method names, but that's really not the case. This is really an "enterprise software"-issue.

[@kawcco]

And guess who uses enterprise software? Seemingly everyone. Also, nonsense like this is rampant in the Java standard library; have you ever attempted to parse XML before?

[@AdamFJH]

Long method and class names are good for readability.

[@omri9325]

Most of them do because of the inherent culture of abstractions in the java ecosystem

[@kawcco]

@@AdamFJH To what extent? I could make each of my method names a paragraph which describes every action taken. Is that too long? The real issue however is that there is far too much indirection. Why do I need to make a factory to make a builder to make a thing which then makes yet another thing which finally returns the thing I actually want?

[@kawcco]

@@omri9325 Which is the fault of object-oriented programming.

[@thesenuts4472]

As a hacker sorry developers no love it is what it is.

[@dummypg6129]

When people make feature-rich apps ending up with tons of vulnerabilitu.

[@kallocainsynthemesc4172]

I'm surprised that log4j is written so unclean. Never bothered to look deep into it.

[@godmodeon666]

Why is Java so incredibly disgusting. Not the language itself, but the general code style in the culture of the programmers. There is so much indirection and over-abstraction, it is disgusting. That is a library to log things. Why is it so grotesquely overcomplicated. It reminds me of that Fizz Buzz Enterprise Edition meme. It's so disgusting to look at Java code 99% of the time.

[@ptechlead]

Hang on, let's not beat around the bush: Java itself is disgusting (Language and VMs), which may have been okay if its users were infinitely wise and result-driven, but they aren't, to say the least. Java is the king of verbosity to help its users always look busy and productive. Java advocates and forces the use of the ugliest programming style, OOP in its ugliest form. Sadly this is what many new programmers are indoctrinated with through and right after collogue, perpetuating this madness. The only arguments for Java would have been security and type safety, but this vulnerability, and the mindset behind the existence and usage of the libraries that enable it shows you that this ship has sailed and sunk a long time ago.