It's Really Just That Bad

[@carltongannett]

Worst news ever: the guy who made this benchmark impressed the csuite with the squiggly balls and promoted him to be your boss and supervise you rewriting your codebase from C to Fortran

[@christopherdessources]

And bans all strings longer than 100 characters

[@Digitalgems9000]

haha fr

[@random6033]

pls tell me this is a joke

[@regretamine.drifts]

jesus christ man i didn't come to this video for a nightmare

[@DFPercush]

accurate

[@alexeyku8926]

And then ChatGPT will learn on this data and will recommend you Fortran as fastest language ever

[@NoUselessTech]

I mean, it sounds like it’s on par with C when all was said and done. So is it wrong? No. But I definitely don’t see a Fortran resurgence in the works.

[@isaacdruin]

Even worse, the Fortran code used for the benchmark may have been GPT's doing in the beginning (the string limit of 100 coming from some generic snippet of code GPT was trained on at some point). So now not only is this code producing the wrong results, GPT may train on it again and insert the 100 character limit next time some programmer asks GPT to write an algorithm for them. Recycling garbage to make more garbage.

[@nectismanta8080]

@@isaacdruin Imagine GPT write valid Fortran code 😂😂😂

[@NoX-512]

@@NoUselessTech Then you didn’t understand one of the points of this video. The result of these types of benchmarks can not be generalized. All it shows is that C and Fortran are, probably, comparable when comparing strings.

[@infinitivez]

@@isaacdruin As someone who's been in fortran and pascal for years, I can say this is such a rookie mistake. So much so, that I had the very same assumption: it has to be AI jibberish. There's just no way someone would preallocate a buffer like this, without using a sensible limit (say 65535?) or why they didn't just allocate it using a pointer to begin with? I'm just utterly confused how anything but AI could be involved.

[@klasus2344]

Zig only requires 75% of the text that Rust requires. Zig is faster to type and faster to say than Rust. Though way behind C, Rust comes out way ahead of Zig, alphabetically.

[@NabekenProG87]

Those are the important metrics I use to choose which language I will use each week

[@Reydriel]

lmaooo

[@hanifarroisimukhlis5989]

That's funny af. We should bake every algoritm into the compiler as single-letter shortcut. 🤔

[@pokefreak2112]

Based benchmark Really weird no-one uses the A language anymore when it's clearly the best

[@npc-drew]

and unsafe by default

[@lanfeust06]

"100 is one of the largest number known to man" Chat never disappoint

[@bitti1975]

Anything behind 3 is just "large".

[@GreedoShot]

The human eye can't even see beyond 24

[@lanfeust06]

The number of likes is too large, I cannot comprehend

[@Capero10]

It used to be, but numbers got hit by inflation :(

[@markolson8569]

The Fortran code had the right idea; you gotta wow them with your blazing speed and then tell them that comparison past the first 100 chars of the string is only available in the premium version

[@FraserChapman]

Fortran++

[@johnsmith1953x]

@@FraserChapman Fortran2023. Blows away ALL languages. Easy.

[@suchislife801]

What kind of savage posts this benchmark online. Shenanigans....

[@my_online_logs]

fortan is free but there are compilers that free and paid

[@tylermfdurden]

Casey's energy of borderline livid but not to the level of screaming, just to the level of "I need to explain every detail of why I'm angry with you" is how I feel at work trying to explain things to people.

[@sciencedaemon]

He sounds dumb exaggerating this stupid issue. It's like he has never been confronted with real problems.

[@happyducky9872]

​@@sciencedaemon he was pretty specific about why it was an issue, and that it was egregious enough to the point that a basic double check corrected the issues. Specific intended use cases + low required rigor. Its wholly irrelevant to whatever you seem to think its about.

[@Muskar2]

@@sciencedaemon This channel is entertainment in nature. Casey did pretty serious stuff and invented world class algorithms and patterns. IMGUI and "Killing the Walk Monster" (Walk Manifold) are examples. And I'd argue "Designing and Evaluating Reusable Components" is up there too.

[@sciencedaemon]

@@Muskar2 And? Does that mean commentary can't point things out, because you perceive entertainment? Please use a better strawman argument next time so your lack of a real argument isn't so obvious. Would you also like to censor all opinions you disagree with?

[@sciencedaemon]

@@happyducky9872 meh, it could have all been summarized. There was no need for an hour over this screw up. This stuff is par for the course when you are investigating ideas in programming. There is nothing novel in it, no insight.

[@schalkbower1194]

The funniest thing, I asked chatGPT to "create a levenshtein benchmark in fortran"... ChatGPT also had the 100 len limit. So it looks like the fortran code was largely a copy and paste job.

[@AlexandreJasmin]

This or that particular program was in its training set.

[@ProfRoxas]

First rule of IT and CS: it depends

[@g3rzin]

Actually that's the second rule. The first one is turn it off and on again!

[@hanifarroisimukhlis5989]

Erm actually it's the first rule of science reporting 🤓

[@Kane0123]

Actually the first rule is graphs don’t lie.

[@niklas6047]

The first rule is it's the DNS' fault.

[@PavitraGolchha]

it depends ... on my machine

[@ChipKragt]

Went to the repo, they've closed all the issues that point out how bad the benchmarks are. They've also fixed the Fortran issue, but haven't updated any of the sites that are showing the visuals. Smh. (this isn't just because I'm a Rubbish and hate to see it so low down)

[@FlorianZier]

Yeah, looks like "it is what it is now" is more or less the default answer. Also, if the issues adress a specific problem, people shall open a PR with code changes. As far as I'm aware a pull request can still have a linked issue to provide discussion and more insight.

[@iangirvan8141]

@@ChipKragt I like to think of this as a tell. I'm guessing the controversial result was actually a feature.

[@fernando-loula]

It is unbelievably strange that they, why keep the graphs after the corrections, when they know it is wrong?

[@advertslaxxor]

When I see a "graph" like that, I treat it like my unit tests. The tests are passing because the tests are wrong, not because the code is working :D

[@Titere05]

Hahah green unit tests mean whatever you're testing here is OK, but I wonder what it is you're really testing

[@advertslaxxor]

@@Titere05 We're testing "return true", it is passing. We do good work!

[@tordjarv3802]

Never trust a unit test you havn't seen fail

[@hanifarroisimukhlis5989]

You should learn fuzzing lol. It'll find you some arcane test data that'll crash your program faster than you can say "oh shit".

[@SergTTL]

Casey: I'm mad because I have to do this for the second time. Me: Really looking forward for the third episode now LOL.

[@versacebroccoli7238]

This is definitely AI written Fortran code. A human who actually knew what that 100 meant would have known better. But AI just knows that usually 100 chars is long enough for most strings. I can't count how many times ChatGPT has given me char buffers of size 100.

[@thekwoka4707]

You'd except it to be more like ....255 or something...

[@ILsupereroe67]

Don't attribute to AI what can easily be explained by incompetence

[@versacebroccoli7238]

​@@ILsupereroe67the incompetence was not proofing the code that AI obviously wrote.

[@rmidifferent8906]

​@@ILsupereroe67 The AI is incompetent so there is pretty big intersection there

[@gamechannel1271]

Chatgpt just regurgitates the text that is found in 1 or more articles that exist on the internet already. So some popular site is using 100 character strings in its code examples.

[@matthartley6537]

I think people are missing out on another feature of fortran on display here. Not only is it faster, but it also stores arbitrarily long strings in just 100 bytes! Thats amazing!

[@jamesclark2663]

I have a compression algorithm that can store anything in zero bytes. You can't get anything back out of it but man if you need to store a lot of data it sure saves space!

[@Digitalgems9000]

@@jamesclark2663 haha

[@somebody-anonymous]

​@@jamesclark2663 /dev/null is web scale

[@matthartley6537]

@jamesclark2663 if you need to get data out, you can just use /dev/random, and you might get it back!

[@Muskar2]

@@jamesclark2663 Wow, so you're the author of the infamous free() compression algorithm. You should totally make a ball diagram and compare it to other compression algorithms like H.246, DCT and GC.Collect()

[@JustinDejong]

Guy "Been around since the 70s. Primagen "1957 Guy "1960s sorry" Prime in his head "I want this guy to come back again so we're going to move on"

[@LtdJorge]

Exactly 😂

[@JeremyAndersonBoise]

😂

[@UnidimensionalPropheticCatgirl]

I mean FORTRAN wasn’t really “production-ready” until 66 ( I am sure there were some early adopters having fun with earlier versions but that wasn’t the FORTRAN lot of people knew), but Casey probably just misspoke.

[@wolfgangsanyer3544]

I convinced myself he said 1967 lol

[@Houshalter]

It's an off by one error, happens all the time.

[@doanamo]

I cant get over how much this speed visualisation with bouncing balls sucks.

[@samsak1239]

Love me some bouncing ball sucking

[@brady1123]

The Rust code similarly performs two heap allocations per distance calculation rather than using two pre-allocated arrays.

[@LtdJorge]

They should have put the text in variables, which don't even need allocations, since those are embedded in the binary in the text section.

[@thesenamesaretaken]

​@@LtdJorgeyeah I'd have thought a little script to paste the text into the source code for each language would have made more sense because then there's neither command line parsing nor file io, but I'm not a software engineer, so maybe there's an obvious reason why that's stupid and I'm just dumb.

[@electric26]

​@@thesenamesaretakenwith Rust you can `include_str!("input.txt")`, which does exactly what you want but without scripts

[@hanifarroisimukhlis5989]

@@electric26 But then LLVM _might_ over-optimize that, so the result is invalid.

[@electric26]

@@hanifarroisimukhlis5989 let input = include_str!("input.txt"); let input = std::hint::black_box(input);

[@Iantrypsk]

I had the same reaction as prime of "why the fuck Fortran optimized for this shit what is the trick". Casey was like hell no thats bullshit and investigated the bullshit, respect

[@vasiliigulevich9202]

@Iantrypsk latest version has Pascal in the first place. It leads with significant interval. We can expect same bug there.

[@dumbpenguin900]

When the CS/Devs set up pseudo scientific experiments and then publishes without checking their work..its pretty wild. Stop making business majors look smart. Casey is goated.

[@AloisMahdal]

To be fair, there's one thing they dd get right: the colors of the balls really match the colors of the logos.

[@rmidifferent8906]

This is the kind of thing that should make the authors of the benchmark so ashamed that they should never write any articles again. There was nothing stopping them from writing the outcome of every comparison to the file to at least check whether their implementations are doing the same thing - even if they delete it later to make the benchmark more focused. They did not do it though. They made a mistake so foolish that they should lose all of their credibility as anyone worth listening. Because you don't know whether the next time they will not compare the outcomes of different data sets "by mistake" - which is precisely what they did now

[@dead-claudia]

a better benchmark would be "how fast can you find the distance between two randomly generated 1m byte strings". the algorithm would require 400 million int compares for that, and you'd need to allocate at minimum a buffer of 10mb (which exceeds per-core L2 cache on every processor). it'd also be interesting to see a gpu implementation of it.

[@cynodont7391]

My Fortran is a bit rusty but if I remember well, the language does not allow arrays of strings of different length. This is why the benchmark had to do a lot of manipulations to create a array of long strings padded with spaces and later use trim_len to ignore the padding. However, there is a simple trick to solve that problem: Put the allocatable string into a structure and then create an array of struct. I do not have a fortran compiler to check the syntax but that should be something like that type MyString character(:), allocatable :: str end type type(MyString), allocatable :: args(:) ... allocate( args(1:num_args) ) do i=1, num_args call get_command_argument(i, length=arg_len) allocate( args(i)%str(arg_len) ) call get_command_argument(i, args(i)%str ) enddo

[@Exilum]

I really started this video thinking "it's a bad benchmark again, who cares", then I got to the part where it became clear the results themselves were wrong and damn now that's real shit

[@hanifarroisimukhlis5989]

21:25 Visual Studio: _Bonjour_ EDIT: And that is why differential fuzzing is important. You never know your "optimized" version is always correct compared to naive one.

[@hinzster]

I believe Visual Studio grabbing all those file associations and making you wait for it to start, just to close it down because in 99% of the cases you didn't want it to, is the single reason Windows is *horrible* for development. Use anything, even a potato will be better, but don't use Windows.

[@hanifarroisimukhlis5989]

@@hinzster Remember, when you type `py ...` on command line it'll automatically try to install *Microsoft Store version* of Python. Not even PATH can override it, you have to specifically remove it in some obscure settings.

[@in2vd]

In the zig version they pre-allocate two arrays for the distance calculation (which seems smart). BUT they also allocate an array just to figure out what the length of the two pre-allocated arrays should be (which is just the length of the longest input string).

[@huvarda]

The plot twist of the fortran code trimming the arguments to 100 characters was hilarious 33:50

[@noomade]

why spoil it then...

[@huvarda]

@@noomade because a lot of people probably dont have time to sit down and watch a 50 minute video? do you think someones experience of watching this video is going to be ruined because i provided a point to jump to to see an interesting part? idk man. this isnt the sixth sense its a programming youtube video

[@noomade]

@@huvarda You are a hero! LOL

[@DFPercush]

I facepalmed so hard. And yes it's long, but the suspense was very entertaining.

[@ashutoshsamantaray6596]

@@huvarda I watched it before going through comments. Was frustrating. Like man just tell me and I will move on with my life

[@pavel45568]

After about 10 minutes in I just realised that they are talking to each other. Until that moment I thought he is commenting a recorded stream)

[@XxZeldaxXXxLinkxX]

Same, I thought he was just having some god tier timing with his comments and extremely coincidentally relevant remarks from casey

[@antoniogarest7516]

Lol

[@robertvandrovec]

I looked at the Zig implementation, it uses Arena around Page Allocator! And that its not the only mistake there . . .

[@Caesim9]

Like, what???

[@NoX-512]

Probably written by chad gipidy.

[@robertvandrovec]

@@Caesim9 Few unnecessary comparisons, overall it HEAP allocates, even args. If you want to keep the HEAP allocations dont use page_allocator for this. In Zig its so easy to use STACK, especially since you have known input size. Also the algorithm could be improved. Overall the LLVM Languages should be around the same speed. This comparison is not correct since they are not doing the same thing, WHEN THEY CAN BE 😆.

[@hanifarroisimukhlis5989]

But how would you allocate an unknown length of string? In stack?

[@NoX-512]

@@hanifarroisimukhlis5989 As far as I remember, the C implementation use argv[] directly, thus avoids extra mem alloc.

[@perguto]

The name is the CASEYGEN! 🔥🔥🔥

[@ninomojo]

The Primatori!

[@ahmetyazc7969]

"maybe a 100 means like... give it a 100%" LMAO

[@guidogiuntoli4184]

Lesson: don't trust anybody, as Torvals said, “Talk is cheap. Show me the code.”

[@mage3690]

Also an ffmpeg tweet: "Talk is cheap. Send patches."

[@AnonUser-b6y]

Part of the Fortran speed isn’t just truncation, by having a static length (100) for each string the allocation is faster and potentially it’s even one giant string blob in memory for better preloading. The issue is this is not a like for like comparison because the other languages use dynamic string lengths.

[@dealloc]

It is even shown that you won't even need heap allocations as per the C version. So why all the other languages use any form of allocator is beyond me. But this is exactly what Casey speaks about is the problem.

[@johnsmith1953x]

Its also basic information theory. Restricted chars to 100 allows for massive optimization.

[@roccociccone597]

I bet that there are countless NPCs on LinkedIN sharing that initial benchmark... xD... I would pay Casey to teach me low level concepts about programming... I love these in-depth dives into programming man.

[@CompiledGabriel]

I'm pretty sure he actually has an optimization course lol

[@IgorStojkovic]

His course is named Computer, Enhance. Just Google it.

[@SimonBuchanNz]

"NPCs" ugh

[@code-dredd]

No, I want more grumpy Casey Muratori code reviewer. Angry code review is one of my spirit animals.

[@miketerry4309]

In CS 1250 years and years ago, I wrote a Pascal program to compare the speed of six different sorting algorithms. The first thing I did was to make sure that the timing was just surrounding the actual sort algorithm, whichever one of the six it was the second thing I did was to make sure that I wrote the output from the sort to a disc file so I could then do a gift from the original to Other the output from one algorithm to the output of another algorithm, making sure that each algorithm was functioning correctly. If the author of this blog post had done that one simple little thing they would’ve realized the four train program was broken prime and the other gentlemen are 100% correct do not take any benchmarks at face value not unless they’re an industry standard benchmark.

[@thetomato2]

damn this guy was programming in the dark ages

[@ChaotikmindSrc]

@@thetomato2 I did pascal too XD Lot of fun at home on my crappy 286

[@maighstir3003]

@@ChaotikmindSrc Reading quickly, it could be interpreted as "1250 years ago" rather than "many years ago, in Computer Science 1250".

[@ChaotikmindSrc]

@@maighstir3003 Well, programmign on a 286 with msdos was maybe not the dark ages, close enough !!!

[@_vicary]

laziness and incompetence is the most infuriating thing in the industry

[@nakatash1977]

Don't forget arrogance

[@non_complete]

@@nakatash1977 the unholy trinity. Lazy arrogant incompent.

[@GackFinder]

Dude made the algo in Java and then asked ChatGPT to generate slop for all other languages, and now ChatGPT will be fed it's own slop.

[@Zzzooooppp]

8:30 it’s not really n^4 it’s more like n^2*m^2 where m is average string length and n is number of strings, no?

[@lubiefeniksy]

It is but really Casey was doing a mental shortcut there (to find the computation intensive part) more than actual analysis. It is O(n^2*m^2) where n is the number of strings and m is the maximum length of the string.

[@0xero7]

KZbin is STRUGGLING with the compression 😂

[@borkware]

For whatever reason it decided to drop me to 480p, which is chonky pixels. Switching to a higher res makes it easier to read the text.

[@AnzenKodo]

you should have made Community Notes on original post

[@zzzzzzz8473]

absolutely , legit spreading misinformation

[@lasershark1237]

Oh Jesus Christ, from 20 mins onwards... the revelations... oh God.

[@TheEVEInspiration]

"This benchmark written by experts for experts" has me smiling.

[@chilversc]

Perhaps someone should submit a PR to correct that as it's not really useful for experts, and it wasn't written by experts. Maybe change it say "This benchmark was written".

[@CharlesVanNoland]

The fact that the benchmark is using the bouncing-dot visualization is all the evidence you need that the benchmark is made by a goober that doesn't know what they're doing - a regular #SillyclownValley Bro

[@tordjarv3802]

If someone say "Programming language X is faster than language Y" they are probably not very good at either X or Y.

[@ashutoshsamantaray6596]

I feel like that still stands up to mark if you compare C to Python.

[@toby9999]

In the case of languages within the same class, I'd agree, e g. C# vs Java. But that claim wouldn't apply generally.

[@Takyodor2]

No, natively compiled languages > JIT/interpreted languages in terms of speed, just how it is

[@dead-claudia]

@@Takyodor2yes...mostly. there are exceptions. java, when using a server/desktop jvm, is in the same performance class as c for many tasks. c# is similar, and there are full aot compilers for both that bring both (especially c#) to near full parity. luajit is also famously able to auto-vectorize even through jit inlining and can also inline temporary closure calls, so for some numerical tasks, lua running in luajit is in that same performance class for many tasks. also, julia's jit is single-stage, and compiles immediately on first call. between that, their use of llvm, their extensive use of specialization, and type stability post-specialization, they're in the same performance class as c and fortran, just with slower function calls (but not returns) and slow global accesses in general (due to bad design imho). julia also supports multithreading, and there's a well-known library that lets you run julia transparently on gpus.

[@chepossofare]

The fun thing is that Muratori in Italian means "Builders".

[@matias-eduardo]

That’s actually awesome haha

[@birthdayzrock1426]

bricklayers

[@MikkoRantalainen]

9:52 The colors are used to express that a single line of code can be split over multiple locations in assembly. For example, the for() statement in C code is split on lines 95,96,114,115,116 and all these lines have identical color to make it easier to notice this. The colors appear purely random so green doesn't mean something else than red.

[@janwalewski1997]

Prime I really love your videos with Casey. I always have such a good time watching them. Sadly I don't have the opportunity to watch your stream so thanks for reposting these gems on YT.

[@oblivion_2852]

Just for fun I want to do a "Will it SIMD" benchmark suite where I try to simd an algorithm automatically and manually in each language. I'd like to understand which languages have good data oriented design principles and performance potentials. Since most languages use llvm now I just want to know what frontends are easy/elegant to write simd. I've seen rust simd and it looks horrifying

[@oblivion_2852]

Might be an interesting challenge to make a flat hashmap. Which includes some pretty complex math, memory management, paging, indexing, searching, insertion etc. And after its finished you'd have a bunch of potentially faster implementations in each language

[@ashnealpowell7747]

Whoa! This is exactly the sort of thing in exploring in Rust right now, are you able to provide examples of ugly Rust SIMD stuff?

[@stefanalecu9532]

What if you don't have intrinsics available? Do you lower down to assembly?

[@oblivion_2852]

@@ashnealpowell7747 prime read an hour article on rust simd for a static S-tree for dna lookup. It had simd code examples in rust... They didn't look readable

[@oblivion_2852]

@@stefanalecu9532 I'd first be looking to see if the compiler can even produce simd output. If you didn't know even Java now has simd performance if you hot path a code block enough and the runtime can optimise it on the fly. I noticed when doing >100k operations java would simd the code

[@DJ-3maj]

21:19 This little maneuver is gonna cost us 51 years

[@Digitalgems9000]

LOOL

[@Digitalgems9000]

bro i gotta make a meme video out of that

[@BudgiePanic]

I need 1GB of RAM cooper

[@Digitalgems9000]

@ we are locked!

[@Mglunafh]

VS: got 'em!

[@combatcorgiofficial]

>fortran is esoteric webdev moment

[@obkf-too]

I am loving this, keep up the energy Casey.

[@smc4229]

This just goes to show about how micro benchmarks can hurt as much as they can help..this was a fantastic discussion

[@SeanDaley-m3c]

Honestly, I think the worst thing here is that it looks like they fixed the code but didn’t readjust the “graph” that they show? It still seems the same as the video but the length stuff has been adjusted.

[@iangirvan8141]

I'm sure they got some serious increase in traffic on their "graph" page that they are enjoying.

[@mousedits]

Crazy, Fortran was my first language in 2017 at university for mechanical engineering, some old ass professor that is addicted to rocket science was still teaching us that. Honestly, such a impossible language to learn as your first one, was something that kept me away from programming in general for years :(

[@rmidifferent8906]

The best part is they seem to have this exact issue with more than one language. They have an issue about FreePascal about truncating the input...

[@Ackermann5]

Love the way Casey dissects everything. Really interesting stuff!! Bring him to the stream more, please!

[@1337cookie]

Plot twist; The project is just data poisoning for LLMs.

[@Kim-e4g4w]

I hope Casey one day writes a book on C and his take on tips and tricks for making games in C.

[@matias-eduardo]

I think he could write a book on Software Architecture and it’d be an instant classic. He’s quite good at it.

[@maxoumimaro]

Hope to see Casey more often ❤ I love a good "let s look at the code". I would love it if Prime and Casey would each show a dive into some codebases, like "let s look at some aspect of the rendering of this game"

[@consciousmi4842]

I really enjoyed the frustration this guy showed and enjoyed watching the whole episode.

[@Tony-dp1rl]

Just as a side note, several of those C methods using pointers were missing "restrict" keyword use, which in many C compilers would have made them much faster in my experience.

[@senyai]

Calling `strlen` n times instead of (2 * n^2) times would also help.

[@Henrik_Holst]

not entirely sure when the compiler can see the source since it then can defer automatically that the pointers are restricted. restrict is mostly there when you call functions in libraries where the compiler cannot analyze the code.

[@tapwater424]

@@Henrik_Holst Maybe GCC with O3 is smart enough to optimize it, but generally functions have to be static for certain optimizations to happen automatically since it guarantees it's not a library function

[@handle-h5m]

@@tapwater424 There's only 1 source file and it's the main unit, functions within are as good as static, and the pointers are almost certainly local variables which is just another reason it wouldn't matter.

[@my_online_logs]

rust achieve what restrict in c provides by default, so the compiler can optimize rust code better than c code because in reality c/c++ programmers skill are shit. its why rust is the king 😎😎

[@araa5184]

Lmao that moment of regret when you accidently open MSVS and it loads 😂

[@javierolazaran7227]

I love it when Casey comes over. I don't even care I have to watch the stream/video using shades. BTW, Fortran is used in several supercomputers that run physics simulations for research.

[@scheimong]

16:38 to skip all the yapping 27:38 to skip more yapping 33:45 to skip to where the Fortran impl fucked up 38:28 to skip to the proper results

[@zbyniew]

Zoom zoom zoomer

[@HairyPixels]

thanks bro

[@noahtah1511]

idk im here for the yapping

[@scheimong]

If you want to watch the whole thing, go for it. I'm not going to tell you what to do. But if you got 5 minutes, here's everything you need.

[@mirata9]

Yeah jfc I’m at 5 min. where is the content

[@james-s-smith]

29:23 "No one saw that." -Casey Muratori, 2025

[@ike__]

Looking into Fortran like that is like being a historian! I can image people will be doing the same thing in a few decades with the code we write today

[@xdmich6018]

Ai already uses parts of old broken code made by newbies

[@sjfsr]

When I first saw what you were doing, the concerns and questions you came up with, is exactly what I was thinking. I questioned: what is the code? is it optimized? and this will just mislead too many.

[@berbold]

Visual Studio desperate to be part of the conversation 😅

[@MacSvensson]

🤣

[@ShaneFagan]

There is also another key thing about the benchmark that no one seems to have called out which is "would anyone in this language who cares about performance do it this way?". I checked out this in Python for instance and was able to get it at least into the same ballpark as C, that wasn't me trying to prove Python is as fast as C but there is a big difference between basic language implementations and taking a step or two into it further.

[@karapuzo1]

This benchmark is clickbait and the fact they are talking about it for so long means it works.

[@danser_theplayer01]

I understand none of these languages, except maybe the 2 or 3 of windows cmd commands he wrote. But I can't stop watching. I NEED to find this 2ms difference, where ist it?! This is too exciting.

[@1337cookie]

I love when Casey is upset.

[@JeremyAndersonBoise]

Angry Muratori is the best. 🎉

[@eliasepg]

really loved this video! Really informative, well explained, it's just cool.

[@foreignconta]

My eyes are bleeding from the compression of that screen sharing even at 4K KZbin resolution. Edit: And now they are putting pascal at the top, and guess what? FreePascal defines String as a synonym for ShortString, which cannot be longer than 255 bytes 🤷

[@Monoffel]

While several people categorize the main issue being that the Fortran sample does less work than the other algorithms, it's not even objectively wrong. Like Casey said, the issue is more so just with the benchmark itself. Because the question of what this benchmark is trying to assess is just void. If the min distance is 7 for very large inputs, then clipping at 100 could be legitimate optimization. If you are in a production setting, where on a good day, you find a min length of 15, then clipping at 100 could be a good optimization. So the main issue is still that microbenchmarks are useless, at least in the context of figuring out the "best language". (The only uses for microbenchmarks are maybe stuff like figuring out minor compilation details of one particular language, or testing different instruction sets and hardware differences). There's a lot of things you can do better here, like ensuring each language sample follows the same number of algorithms steps in the same order, or validating outputs, or testing correctness, or timing only the critical section of code. But none of these would solve the fundamental issue, which is microbenchmarks not being suitable to answer the question of "best" or "fastest" language.

[@KevinInPhoenix]

There are lies, damn lies, and benchmarks.

[@earx23]

you beat me to it.

[@ArturdeSousaRocha]

This should be a must-watch at CS courses.

[@BudgiePanic]

Reminds me of a video I saw a while back about how Big(O) program analysis doesn't really matter if you know your problem size is constant ahead of time, isn't great that our FORTRAN program "knows" it only has to process strings that have a maximum length of 100 while everyone else has to handle variable lengths

[@thekwoka4707]

the long strings don't really matter for the comparing against strings under 100 in the algo, since once it hits the end of one string, it just adds the count of the remaining characters in the other. since it can only "add" as a change. But it does matter for comparing 2 long strings.

[@000TheMatheus000]

why does the video looks like 360p even in 4k? i cant see anything 😭

[@JG-nm9zk]

Looked at the Lua code. disgraceful. Edit: On my machine their LuaJIT runs 55.8ms and with 2 small changes I got it down to 22.9ms. LuaJIT is Faster than Zig!

[@JG-nm9zk]

Their "optimization" made it slower They had -- Pre-compute string indices for faster access local s1_bytes = {string.byte(s1,1, -1)} local s2_bytes = {string.byte(s2,1, -1)} ... s1_bytes[i] == s2_bytes[j] But if you just do s1:byte(i) == s2:byte(j) boom more than 2x faster

[@skaruts]

lol it's even faster if you let it run for longer. LuaJIT takes a little bit to gather enough information about your code to be able to accelerate it properly. So the benchmarks are always slower at the start. That's why I always run my benchmarks in waves, so I can ignore the first wave or two. I can't imagine how crappy the zig code must be...

[@bartosz0.]

Hearing: "I don't know but I'm excited" by Prime gives me wings.

[@chimeces]

Stellar work and story telling by Casey 👏

[@flyingsteaks]

For me, the most hilarious part actually was finding out that tweet and reading the replies and quotes, some supposedly experts trying to defend this bullcrap is gold lmao

[@andrewdunbar828]

A collaborative platform for such benchmarks which includes both testing and contributions to both implementations and tests would be cool. I'd be surprised is they don't already exist.

[@ungrim97]

Primes point about peoples inability to verify the code because the language space is so large now, is one of the reasons why I have such an issue with devs being expected to use so many different languages. They end up being a jack of all and master of none. I would much rather someone skilled at using a small few languages well, then being able to use a plethora of languages at a basic level. They end up missing all the nuances and gotchas in each language and end up producing bad code as a result

[@DarthJane]

I checked the C# version, it randomly uses Math.Min for the comparison, which isn't inlined. Just copying the method and putting an MethodImplAttribute on there with AggressiveInlining and the code runs easily 3% faster. However that is offset by C# actually having an early termination check if the strings are sequence equal or have a zero length, which the C version doesn't have.

[@spicybaguette7706]

I just love Casey roasting poor benchmarks

[@TheRadischen]

the odin creator ginger Bill, also complaines about these benchmarks all the time

[@jsalsman]

As someone who is currently trying to parallelize Needleman-Wunsch matching on CUDA, I would like to say, AAARGHHH!

[@pocpic]

I'm only interested in this stuff at a hobby level, but my first question would be "sure, but why?" followed by "okay, but does that matter in practice?". EDIT: wow, I didn't expect the whole thing to be this bad.

[@hydrobolix3365]

JS over C#, PHP under Scala/Node, Someone doesn't deserve strings

[@carltongannett]

For most use cases just writing good code that does it’s work in the most efficient way to reduce time and/or memory complexity is way more impactful than what language you use.

[@ped7g]

There I was, spotting at 6:38 two `strlen` calls and thinking maybe Fortran is using Pascal-like string with length precomputed, waiting for another 20+minutes for the story to unfold in direction I did hope it will not go. So it was just novice level incompetence, nothing at least slightly more tricky. Dear benchmark author, thank you, my trust in humanity went again lower. Son, I am disappoint.

[@KangoV]

This was a cool watch. Thanks. On the JVM we use Java Microbenchmark Harness (JMH) which handles building, running and analysing nano/micro/milli/macro benchmarks. Do other languages have something similar?

[@UnidimensionalPropheticCatgirl]

You would use HPCs from perf (or some library that helps with them) in C++, don’t know how well this is supported in other languages.

[@kursatkuyumcu]

when someone is going to do this benchmark on bogo sort

[@el_quba]

Well, out of curiosity I looked at the Rust version, because couldn't believe it's doing so badly and I found the rows being allocated on heap so of course it will be slower than C.

[@thedeemon]

Their length is only known at run time, so stack allocation becomes not as simple and straightforward as the naive approach here.

[@vytah]

@@thedeemon the levenshtein function could take two more &mut Vec parameters for those two buffers, and grow them if needed. Or you could even preallocate them in main based on the max size of the arguments.

[@thedeemon]

@@vytah yes, you could for this particular benchmark, but that's not how typical functions are implemented in general. Usually you need a function that can take two strings and return a result, its inner needs for internal temp buffers shall not be a part of the API.

[@g0t4]

fixed length strings in fortran?

[@Veptis]

Looking at the code, especially with these comments... Might it be language model generated/translated? I wonder if this 100 limit was made up by GPT in the end.

[@LtdJorge]

Likely

[@stefanalecu9532]

I've generated Fortran code with GPT before and it always, always likes to choose 100 as an upper bound, even when (len=:) (aka unbounded) strings and arrays were an option.

[@1337cookie]

@26:04 It'd be a shame if the output of the program was limited by the speed of the terminal...

[@LtdJorge]

There's no slow down, because they are passing the data as arguments to the binary, so at startup, all the data is there already. But of course it is completely stupid.

[@_FFFFFF_]

There is still a slow down. Please see where the string is printed to terminal in the Fortran version.

[@Henrik_Holst]

@@_FFFFFF_ he removed that at 36:24 so that debug output was never part of the timing

[@1337cookie]

@@LtdJorge Casey made a reference terminal because the windows terminal was slow, in that project somewhere he references how the output speed of the terminal can limit the speed of a program. twas but a joke.

[@Chimperly]

Yo prime, ive seen a few videos of this guy and he seems really cool, but i dont actually know who he is. In the future do you think you could add the guest name in the description? its Casey something, and he writes in C, and obviously really smart.

[@GaneshPalraj1991]

It's casey muratori

[@UnidimensionalPropheticCatgirl]

Casey muratori, as far as I know his history is microsoft -> gas powered, working on 3d tech -> RGT (which later turned into Epic Games), working on video codecs and 3d animation SDK for their game engine tech -> brief stay at Thekla -> retirement, writing about performance and low level stuff on his blog and making the handmade hero video series about game engine development.

[@Muskar2]

@@UnidimensionalPropheticCatgirl Interesting that you interpret it as his retirement. I know he's working on at least one secret project that has been a big undertaking and risk, and their website still suggests it needs funding. He is no longer explicit about details, so I'll refrain from providing those too - but it's basically a game genre innovation. I have assumed that's what he's working on whenever he isn't doing other things to get funding for it.

[@UnidimensionalPropheticCatgirl]

@@Muskar2 I honestly don’t know much about Caseys current projects outside of hearing him talk about revamping the website. But I guess it depends on what you consider “retirement”, I would say casey falls into that category at this point but also can see why someone might argue he doesn’t.

[@Muskar2]

​@@UnidimensionalPropheticCatgirl I mean, I don't have insight into his everyday life, nor care to have, so you may well be right. I was just curious that you interpreted it that way. He mentioned the game project several times on HH and used to mention it on the website and his SoMe. Who knows if it was abandoned or if he is pseudo-retired.