Linus Torvalds schools Lennart Poettering on the importance of users

Additionally Alan Cox weights in on how not to do changes, using GNOME 3 as an example.
This is an excerpt from ELCE 2011 kernel development panel with Linus Torvalds, Alan Cox, Thomas Gleixner, Paul McKenney, and moderated by Lennart Poettering.
free-electrons....

[@unfa00]

Huge thanks to whoever took the time to caption this. It'd be really hard to follow along at times without that.

[@felipec]

You are welcome. It took me a long time to write the captions (and understand what they were saying), but I think it was worth it.

[@foobar-9k]

@@felipec Another watcher here thanking you for your efforts! Much appreciated!!!

[@Filaxsan]

@@felipec Definitely yes, thanks bro!

[@testingstuff6111]

Only if it were someone capable to take the audio and make some open source magic on it to make it better ...

[@unfa00]

@@testingstuff6111 Unfortunately reverb removal is a blind spot in the libre audio space - the best you can do is try multiband gating or spectral denoising, but with such drastic changes in input levels between speakers I don't think it'd do any good.

[@julkiewicz]

That is the single most important reason why Linus succeeded leading his project. Without that mindset he would be nowhere. Too many engineers think about abstract ideas, making code nicer, better architected, but never stop to think about the users. Your software no matter how brilliant, is completely useless without someone actually needing it and using it.

[@anthonyobryan3485]

I absolutely agree. I have users of my software at work that have become very specialized in how they use it. Even in places where I think the user interface could be done better, I leave it as-is. Once users have gotten accustomed to how the GUI works, they will revolt if it's changed; even if that change is ultimately for the better.

[@Ianuarius]

@@anthonyobryan3485 My buddy figured out that you just make the changes anyway and when the users revolt, you change most of it back, and you'll get points for doing that. Meanwhile, the stuff you didn't change back is still in the software.

[@ThatPianoNoob]

you kinda need both.. only thinking about the users will make you end up with unmaintainable code.

[@Aomicplane]

Makes me fear what'll happen to all of Linux if Linus were to pass away... Let's hope that won't happen for a very long time

[@Gazeld]

"but never stop to think about the users." Did you start thinking before writing this sentence? ;)

[@udirt]

"So my first question: is what you do hypocritical?" Such a nice person, would we not all love someone like that in our life?

[@-dash]

tbf it provoked an interesting response from the panel. He was an effective villain

[@samuellourenco9526]

Oh yes, I would kick him out right away. Such humble arrogance and lack of finesse.

[@wysoft]

@@samuellourenco9526 you act like this is a new thing with software engineers. it's why it didn't provoke that sort of response - everyone in that room has worked with Poettering's type before.

[@redemptusrenatus5336]

@@wysoft I'm sure they all dislike the Poettering types as well. I know I do.

[@phantomhck]

I'm glad that Linus always fights for the users.

[@azimuth4850]

I like how he called them hypocritical, questioned whether it was realistic to prioritize old behaviors remaining functional, and then proceeded to complain about all the broken behavior that affected him personally.

[@telotawa]

and then moved to microsoft!

[@NeverTrust298]

@@telotawa such a hypocrite!!!

[@knghtbrd]

Sadly, Linus explains why this is important and Lennart didn't and I think still doesn't understand why that's important.

[@anonapache]

on the other hand i think it's valuable for a project of this size, to have people who question the status quo. if things make sense, they will be explained thoroughly and inspire also others to think about the questioned aspects.

[@eternal864]

I can't believe how idiotic his stance was in this talk. But as other commenter mentioned, I'm glad that it was well explained by the panelists.

[@TAP7a]

Alan always looks like he's ready to recite the entirety of the System V source code from memory, or like he's prepared to identify bugs in your executable files by opening them in a text editor

[@adamradiv]

I bet you a hundred trillion dollars he can

[@lonewolf198016]

What a huge difference between group of seasoned maintainers concerned about user experience and a guy who shouod have never been placed as head of init system project.

[@aaronvaldes3104]

Completely Agree, and the bozo now left to work for Microsoft.

[@rid9]

This dude make Linus look like a diplomat.

[@Schcarraffone]

He is german as far as i know...

[@karaloop9544]

He is a massive twat and systemd should have been handed over to a sane person without god complex and actual users in his list of priorities.

[@leonbishop7404]

​@@Schcarraffonefinnish

[@ToveriJuri]

@@leonbishop7404 He's meant Poettering

[@leonbishop7404]

@@ToveriJuri yeah you're probably right here

[@taint_misbehavin]

I gained some new respect for Linus and his crew from this video. Articulate defense of the concept of making sure that the user experience doesn't break, while simultaneously introducing new things.

[@felipec]

It's not rocket science. It takes more work, but the users of the software are worth it.

[@clarkd1955]

Linus is 100% correct about breaking the interface. Much software isn’t useable not because of a lack of features but from making older code unusable without changes and much debugging. Back in 80’s dBase totally broke with dBase 3 when it went to dBase 4 and lost it all. Just one of many examples (Python 2-3).

[@davidolsen1222]

Look it just took 12 years or so to kill Python 2. And some of the stuff they killed needed to die. But, yeah, anybody whose written 2/3 compat code can tell you that really EOL was very helpful and thank goodness it finally had enough stuff to push it along.

[@cleverclover7]

and then you have m1 and nvidia driver changes that break everything but people are willing to jump through near inifinite hoops to get running because of the production upside.

[@akapbhan]

@@davidolsen1222 I worked 4 months converting python2 project to python3 project. It was possibly the hardest work I've done as legacy project had 100k+ lines of code. We had a huge regression which helped in identifying and fixing the issue but it was hard.

[@diegofrancisco425]

(one of the many many reasons why I ended up coming back to Windows...) Still to this day, there still isn't one system-wide audio equalizer app for Linux, there was one but it used Python2 extensions to function and the repo was not maintained, the code was not updated to accomodate newer dependencies... one of the many examples of the linux daily nightmare

[@deviantsemicolon618]

​@@diegofrancisco425Pulse effects and easy effects exist. Do those not do equalization?

[@bigmikeobama5314]

a huge thank you to the entire linux community for creating and maintaining all of the wonderful amazing linux things that i get to enjoy on a daily basis, and a huge thanks for doing it with love, in a way that preserves my freedom to choose and my freedom from being spied on. you are doing God's work.

[@uFlock]

Poettering just provided the best arguments against ever using anything he ever touched or involved with - PulseAudio, SystemD or Avahi.

[@paulie-g]

All of them hot garbage and top of the list of things that broke and required me to fiddle with them for no good reason.

[@udirt]

Iirc ConsoleKit and some of the HAL/DBus also go to his trashpile. His legacy is in the 30s of projects.

[@ktwomountain]

@@paulie-g Poettering moved to Microsoft!🤣

[@deineroehre]

@@ktwomountain I thought he was working for them a long time ago since Windows 11 looks like an old Linux from 2009 with Aqua desktop Theme...

[@MichaelBehrnsMiller]

As an application dev, this is the most I've ever felt on the same page as Linus.

[@polic72andDrD3ath]

In my work, I've slowly developed an API for myself that my programs all share. Let me tell you, keeping code backwards compatible for your "users" becomes _far_ more attractive when you are that user itself. I've learned the hard way that redesigning things to be cleaner should never break old systems.

[@samuellourenco9526]

Oh yes. It is complicated, but absolutely necessary.

[@JonMartinYXD]

Dogfooding. I'm not a developer, I'm a sysadmin. When I worked in a computing science department at a large university, supporting the undergrad labs, I ran the same thing on my desktop that we deployed to the labs. If you make something, you have to be willing to use it yourself. Reminds me of way back in the day when Microsoft used some kind of *nix for their ftp servers. They didn't trust Windows NT Server for the job.

[@dinobotpwnz]

Amen Linus. Writing this from an Arch box running openRC. More people on the mailing list should've listened to you.

[@henrikholst7490]

The Universe is not beautiful enough for me to dare to believe that was said with intent of being humoring. 😂🥇

[@sorvex9]

Ok dinobotpwnz, I am writing this from google chrome running on Windows 10

[@killingtime9283]

So Pottering is first criticizing the kernel-devs mantra of "not breaking userspace" and later moves on to complain about the times his own project have been affected by breaking changes. Alright. Linus have talked about this in several interviews and have said that some breakages are unfortunately unavoidable, but they always strive towards not breaking things. Which sounds like a good approach.

[@felipec]

Ultimately that's all you can do: try your best not to break user experience, which is the exact *opposite* of what Lennart and his gang does.

[@mrkitty777]

Lennart went to the dark side and works for Microsoft now. 🥵

[@foobar-9k]

@@mrkitty777 It might be hell for him then, as Microsoft always was far better at maintaining compatibility with old binaries than any other OS (I've run my own Win32 binaries on Windows versions that came 20 years later from when I compiled the original ".exe"s), Edit: I meant "maintaining", not mainlining. Auto-correct failed me again 😀

[@mrkitty777]

@@foobar-9k Microsoft did introduce incompatible bugs for competition though, a lot even. Tiny sneeky bugs, like libre office going kaput on office 365.

[@SaHaRaSquad]

@@mrkitty777 To be fair not even MS Word can correctly handle O365 documents and vice versa. This whole thing is just broken, you can't even be sure you'll get the same result on the exact same hardware & software version.

[@mghemke]

Summarized succinctly, this is the epitome of "don't blame the user". Thank you Linus, and Alan. Because of this, you have a huge and loyal user-base. I'm pretty sure that 2023 is the year of linux on the desktop!

[@SlCKB0Y-sb1kg]

People have been hoping for the “year of Linux on the desktop” for at least the 25 or so years I have been using it…

[@pedrogorilla483]

It’s 2024 now. Sorry to say, it wasn’t 2023. I think whatever year it is, the change towards that will be gradual and very slow.

[@Nachtgrabb]

This talk was very important and now i understand the political an human side of the changes regarding systemd.

[@alexxx4434]

Could you elaborate?

[@derpoblizist9076]

@@alexxx4434 in short, poettering doesn’t care as much about backwards compatibility and the „linux way of doing things“

[@emblemi6345]

For Lennart's original question, there is a reson to use old userspace on new kernel. Suppose your systems run fine as is and don't have resources/need to upgrade and then you need to scale. This is done by adding hardware and it is harder to buy older hardware. Old kernel might not have drivers for new hardwares. This is similar to philosophy behind Ubuntu's hwe kernel on lts.

[@bloepje]

I use linux as a bootloader on a nas device. This was a very old kernel, and I always loaded the new kernel from an MD device on disk using kexec, and then rebooted that new kernel. The old kernel barely fit on the flash, but contained the drivers for the sata. After an upgrade I was greeted with a systemd message that I should upgrade my kernel first before booting the system. I am no longer able to load the new kernel from disk, because userspace is blocking access to disk. I honestly don't know how init got replaced by systemd. And there are so many places where I can't just upgrade a kernel because I have honestly no idea if it will boot after that.

[@emblemi6345]

@@bloepje for your usecase systemd is not for preboot userspace. You need a minimal and static (not upgraded unless security issue or such) init that accesses the MD device and kexec. Much like libreboot.

[@emblemi6345]

@jshowa o linux almost religiously avoid abi stability. For example you need to compile kmods when you upgrade the kernel. This to some extent forces modules to be partially open source.

[@talkysassis]

@jshowa o OK. Now convince Adobe to make Photoshop open source.

[@lesto12321]

​@@talkysassis First convince me why they would need to be kernel modules. On the syscall ABI, IIRC Linux is stable (while windows is not)

[@duracell80]

Phased approach and feature flags. Users first. Always. Without users there is nothing.

[@vincei4252]

Ah, the mole that was planted years back has made it back home. Embrace and extinguish. The funniest thing is I just came from a video entitled " 'Rockstar' developers are the WORST developers" I think that title describes Poettering perfectly.

[@pelic9608]

Brushing up on your -- Continuous Delivery -- skills, huh? 😄

[@flynn3649]

@@pelic9608 His comment was rejected by QC and now it's own it's way to git revert hell.

[@georgeparkins777]

There's no doubt that Linus has got to be hell to work with, but when I listen to him speak I don't think I'd trust anyone else so much to maintain the kernel. There is no doubt of his intelligence and integrity. Also there's no question that the interviewer is the one being a hypocrite. Sometimes he says it's stupid to maintain compatibility (when he can't understand why you would want it) but when it affects his project suddenly he's mad that they didn't maintain compatibility and he has a list of ten increasingly esoteric items.

[@defnlife1683]

It’s Lennart Poettering lmao

[@kvegh]

This is refreshing, puts things into perspective. Thank you for recording and sharing.

[@OOD2021]

He's perhaps an expert in his field, but what an an arrogant , impatient and argumentive guy this interviewer.

[@gregoriodia]

The dude who literally had dead threats for how he broke Linux userspace compatibility for NO good reason tries to scold a legend like Linus. Ridiculous.

[@web3wizard381]

not sure how this delusional narcissist ever got to have such foothold

[@hansdampf2284]

Systemd is a very good reason to make things a little different.

[@NBGTFO]

He's a legend alright... a legendary prick.

[@root8272]

I actually like systemd, but find his public approach to this as incredibly obnoxious. He's not just talking to Linus either, but Alan Cox, and some other absolute graybeards.

[@gg-gn3re]

he's always been a POS

[@liamh1621]

The whole "isn't the philosophy of not breaking things a bit backwards when things evolve so fast" thing is wild to me. It's exactly because things change quickly that not breaking things is so important. Anyone who's used a rolling release distro knows that if you don't update for two or three weeks, then install a new program with all it's newer dependencies, things are likely to break. If the kernel did the same thing, Linux would be nigh unusable. Idk if Poettering gets that the old and new stuff can work together. He asks if they just want things to stay frozen in time, as if stuff like containers didn't come straight from new kernel features. I guess his way of adding it would have been to just delete /bin and say "welp, tough luck"

[@MrYossarianuk]

Poettering now works at Microsoft.. Maybe he was a mole after all..

[@baumstamp5989]

he is not a mole. $$$ talks

[@MrYossarianuk]

@@baumstamp5989 To some people. I remember Eric S. Raymond turning down Microsoft in fairly spectacular fashion

[@McZsh]

He will be doing Linux work at one of the largest Linux operators. We need to stop being hypocritical here. Every dollar that wents into Linux as an ecosystem, that widens its audience is a dollar for good.

[@MrYossarianuk]

@@McZsh Very good point (my comment was a joke.)

[@unidorsalicosahedron7416]

@@McZsh It's not as simple as _how much_ you're getting paid, it's about _who or what_ is paying you, and Microsoft is hardly friendly to FOSS, I'm afraid. It's actually infamous for its at times shady software, and slimy business tactics. There are many examples, but take a look at EEE: en.m.wikipedia.org/wiki/Embrace,_extend,_and_extinguish

[@Suesses-Einhorn]

The open source cummunity is helping the world a lot, now im in a position where i would not use windows, even if i would get paid for it. Im so proud of them creating such nice things. This should be the norm for everyone. No need for bloody wars and cyberwar (also with stupid comments).

[@arnavgrover4631]

Ah yes cummunity

[@nonenothingnull]

The FREE SOFTWARE community

[@mrkitty777]

Lennart Poetering now works for Microsoft. He is excommunicated. Maybe he will systemd Windows 🙃

[@nonenothingnull]

@@mrkitty777 or he will microsoft the non free distros

[@mrkitty777]

@@nonenothingnull Microsoft owns a lot of Ubuntu already so Ubuntu is lost.

[@KangJangkrik]

"Software breaks when version changes" I'm looking at you, Android

[@JoaoPedro-ki7ct]

OMG, they even got Steve Jobs there

[@stevenbjerke2825]

I was thinking the same thing

[@knaz7468]

He would never have been so silent ... or without a black turtleneck

[@grev.]

if you look a lot of linus's famous lkml blowups, most of them are in response to developers breaking user space. even senior developers, and developers in this video. honestly without linus the entire project would be in shambles with how frequently devs try to downplay the severity of user space breakage.

[@cjcox]

I still play UT2004, in my case on openSUSE Leap 15.4, using Nvidia. Still works, even audio. Sure, there's a bit of work, but the flexibility that allows really old stuff continue to work... amazing.

[@anthonyobryan3485]

Interestingly, I sometimes also play ut2004. When the hard drive it was originally install on died, I copied it from a backup (copied as-is) to the new hard drive. That was back in 2007, and it's still on that same hard drive which is still in my primary computer which has gone through many hardware and operating system upgrades since then. The most work I have to do to play ut2004 is to click on the desktop icon. It works perfectly.

[@doltBmB]

ECE came with a native linux build right?

[@supercellex4D]

This is why Windows is successful

[@RobBCactive]

The problem with Poettering is rather than fixing his stuff to run things that rational sysadmins had established, he'd decide that on a modern desktop that you shouldn't do X, that it was unnecessary, so systemd(8) would issue messages about stuff being broken. When OpenSuSE switched to systemd(8) there had to be an intervention to avoid rejection as installations had long supported separating out /usr. It was found that their restriction could be solved by simply mounting /usr after / on boot as part of the initial system, where there were seperate filesystems and the more agreeable guy Kay Sievers (IIRC) made sure systemd(8) stopped whining.

[@BusinessWolf1]

I have no fucking clue what I just read, but I hope to understand it one day

[@RobBCactive]

@@BusinessWolf1 the video lecture "the tragedy of systemd" IIRC might explain the controversy to you. For me detail cannot be helped, specifics are required to explain just one example how Pottering wouldn't cooperate so gained a very bad reputation with knowledgeable sysadmins not just net hate.

[@antdah]

Wow. That sounds absolutely insane. As someone from the BSD world; I'm used to having /usr/home, /usr/jail, /usr/local, /usr/share, /var and more on separate volumes. The beauty of UNIX is that not only should you be able to have them on separate volumes, but on separate hosts.

[@RobBCactive]

@@antdah exactly .. and that's why OpenSuSE made it a requirement and Kay Sievers helped. Poettering seems to be one of those narrow minded developers who thinks he knows best about everything and digs his heels in.

[@samuellourenco9526]

To note, systemd has an issue with Michochip USB devices, when the default Microchip udev rules are used. The devices simply hang some time without being recognized. That happens if you install MPLAB, or at least some version of it. Basically, udev should not hand when it doesn't find a program that it has to run for that device upon enumeration. It should pass execution on another thread and continue on its merry way.

[@n1k0n_]

"I fight for the user" - Tron

[@scdecade]

Linus's defense of interfaces is magnificent! He's a prince among men

[@cleverclover7]

This is a really important concept for makers. Whenever I add a feature I try to get as much honest user feedback as I can right away. Testing can only cover so much and doesn't reflect the user expeirence, burden of change. Many times there are things I think are objectively definite improvements but I revert or roll out in a more staged approach because of user feedback. I know that if people don't use my stuff there is no point in making it.

[@lendall00]

The captions are a life saver, thx op

[@kevinsimmons5481]

The way that Lennart is asking the question could not be more loaded with opinions on the answer. If you want an answer, you ask a question, you don't put your assumptions into it.

[@steventhebrockwell]

Lol, nobody watching this should be surprised that Lennartware is basically a competitive sport in breaking user workloads over and over, for his amusement.

[@complexity5545]

A perfect addition to M$. I don't like the way his brain works.

[@SR-ti6jj]

This is like when Anakin was rejected by the Jedi Council

[@jessepollard7132]

and both anakin and Pottering were wrong.

[@joshuaclayton6940]

The Linux Kernel is about the ONLY thing in the I can drop in a much newer one with every expectation it will just work.

[@helloitismetomato]

You can say it's bad that Windows has application-specific legacy fixes that it maintains permanently so things like Sim City continue to work (or rather old MS Office or whatever-actually Sim City no longer works today because it's 16-bit code), but you can't deny that Windows has the most amazing backwards compatibility. Everything just works, and continues to work, without so much as a grumble. I can run software from the 90s on my Windows 11 for ARM64 install and you don't even notice the emulation layer. Personally I think that's wonderful and it's well worth the extra complexity.

[@talkysassis]

The main problem with their approach is that they don't even try to integrate the legacy code on the new system (and that's why Windows is so bad on performance, it can't use new tech on most of it's base process). They could make very little changes to the old libs to make them static as a last patch to keep thing working well.

[@paulie-g]

This is not actually true. "Everything" does not "just work" and does not "continue to work". Your set of anecdotes is not data. In particular, nothing that touches the GPU is safe. Linux has issues in this regard, but it's not the kernel's fault - most of the breakage is in glibc and other shared libraries. If the original build didn't supply its own versions like Windows software does, there is no reason to expect equivalent results.

[@DonAlcohol]

on the other hand one could state that microsoft undoubtfully has many excellent programmers , maybe even the better ones in the industry, yet if they were put to task to actually come up with new and innovative better systems and safer integrations, and actually test their releases instead of relying on metrics and dumps from their users after the fact , while at the same time keep small dev teams to support past OS releases on a part time basis (like a win nt4 and a win2000 team wich would cover prolly 90% of old outdated software that is mission critical out there ) they could have had rocksolid os with much better public response to them

[@talkysassis]

@@DonAlcohol That's what IBM does with Z/OS. This is the reason Z/OS is the best to use on really critical systems like Banks.

[@paulie-g]

@@DonAlcohol No. No one decent goes to Microsoft except at the top end. Their top end marquee hiring is what produces the impression they are on the whole 'above average'. It's a PR effect, not a real world one. There is absolutely nothing wrong with crash dumps or traces. This is, in fact, standard in all products sold with enterprise support. All the commercial Unices recent enough had that. An enterprise which is having repeated crashes or had a high-value interruption wants to be able to send the info needed to debug and have the issue diagnosed and fixed/explained. This is nothing to do with whether releases are tested or not. Most crashes are driver-related and you can't QA all the combinations. Microsoft does indeed QA drivers under its driver certification scheme, but most consumer end-users happily install unsigned ones.

[@No-mq5lw]

Seems like Poettering here seems like he's intentionally or not trying to pursue his own vendetta here. His questions feel like they don't understand the why and just point out the is, almost like a weapon against LT. And bringing up systemd felt like the pinnacle of this vendetta.

[@ToumalRakesh]

Poettering really doesn't get it. Just like when he thought programs should be killed when a user logs off. Or that systemctl should reverse the verb/noun order of commands. Or when he thinks that systemd isn't monolithic because "it consists of individual programs". Or when he decided to use dbus for systemd. I'm all for renewing things but it's akin to designing a new hammer. There's a reason why hammers look the way they do, and you have the deepest possible understanding of hammers and their use before you can hope to offer a completely different design that's actually better. Something like that is really difficult to improve. There's very few parts in Systemd that didn't fail to do that. Take timesyncd for example, which doesn't even slew the time and instead just *sets* it, and if you're lucky kernel slewing will fix the worst. I had a project with ESA where time consistency on linux machines was important and timesyncd made the results unusable. Replacing it with NTP (itself a piece of software with plenty of weird design choices) fixed the problem because, lo and behold, it has matured and has been used in scientific setups for decades. My only hope is that over time, systemd can evolve beyond the preschool stage. Though some decisions are pretty much set in stone, and were made simply because the people making them didn't bother to look why hammers are made the way they are before deciding they would invent a better hammer, one with two handles for lefties and righties, and advertise it as the new best way to drive screws into wood.

[@paulie-g]

I suspect not. Poettering is still lead, and even if he got run over by a bus no one of sound mind is going to work on that sh-tswamp of a project.

[@baganatube]

Until this day, I'm still fixing stupid problems caused by systemd from time to time. I honestly don't understand those systemd fanboys.

[@diegofrancisco425]

please some distro gets us back to the original untainted 2015/2016 linux experience... how it felt being an Owner of a system

[@EvzenEmanuel]

systemd is the worst thing that happened to the Linux in the past decade or so. It is such anti-pattern and anti-KISS bloatware...

[@Cowboydjrobot]

My biggest complaint with every panel I’ve ever watched is how poorly reporters, moderators and everyone ASKS questions. Don’t ask three long leading questions in a row! Provide some background and ask a question that fits into a single sentence then shut up and let the people answer.

[@diegofrancisco425]

you're asking for humans to have common sense, and apparently that does not exist anymore

[@diegofrancisco425]

as insufferable some of these old dinosaur older generation can be (and trust me, I hate most people from that 50/60's generation and what they became after they became rich) I think I hate even more this new soy pedantic generation that pushes conflict and having the last word in an argument over logic and reason.... we are doomed

[@edhalferty]

Lennart is like the devil whispering in Linus' ear: "Go ahead, break userspace. You know you want to. You know it'll feel good. You have no idea what heights Linux can soar to... you just need to let go of your principles. Come on, everyone else did it. *gets angry* You're just being self-serving! People don't like you! You should just break compatibility already! Join with Systemd and together we'll take on Windows and Apple, and who cares about those old Linux users and their crappy OLD software..."

[@HUEHUEUHEPony]

fuck I hate him sooo much

[@christopherjackson2157]

Is this guy playing devil's advocate or asking a serious question? His example is version numbering? It kinda sounds like the question he actually wants to ask is: "what if MY mistakes are YOUR fault?" Sometimes the dumbest questions provoke the most interesting answers I guess lol. I think Linus is making a distinction between "breaking userspace" and "breaking user experience" here that the question is trying to gloss over. At 2min 41sec.

[@gregoriodia]

Pottering talking about worrying about user experience is the biggest hypocrisy ever, there is NO need for systemd to break userspace compatibility or not making binary logs optional not forced... yet he did that. I'd like to remind everyone that he is the behind all the systemd decisions that turned out into decade of init systems wars... and all we needed is that self-centered egoist with overgrown ego not to force everyone to just cater to his design.

[@skilletpan5674]

Just guide people towards devuan for a systemd free fork of debian.

[@HrHaakon]

But don't you know, if you moved away from Upstart, you could get compatibility with GNOME! Remember that horseshit?

[@skilletpan5674]

@@HrHaakon who in their right mind would want to use a window manager that states they want to hide/disable configuration settings from their users? I think it was gnome 2 that had that in their docs. I haven't used a gnome distro/window mamager in ages. About 10 years. Kde and xfce are generally better options.

[@Ilikewater-andice]

its a weirdly german trait to be like that

[@RobertHancock1]

Binary logs forced? Whose propaganda have you been reading? So many constant misleading tropes get brought up over and over on these topics, usually originating from people who have no idea what they're talking about in the first place.

[@astronemir]

Modern python ecosystem community devs needs to hear this answer. Thank god for 1 year depreciations as a courtesy by some but others just hack away things all the time. A famous plotting library (not matplotlib) for data science comes to mind.

[@miraculixxs]

Is that systemd-broke-init-and-fu*ed-up-everyone Leonnard-Poettring? Wtf

[@EmilNicolaiePerhinschi]

Poettering, the author of avahi, the DNS somethingorother that for some reason gets installed by default and makes your DNS resolution slower :(

[@zvxcvxcz]

That shit was taking my network down like clockwork once every 24 hours. In the bug report the devs try to claim it isn't Avahi doing it, but that is the very last message in the syslog before the network stops working and disabling the service made the problem go away. I don't believe they ever were able to figure out the problem.

[@HUEHUEUHEPony]

@@zvxcvxcz made by soy programmer

[@microcolonel]

Wouldn't say this was ‘schooling’. Lennart had a good question, Linus and the panel had a good set of answers.

[@EvzenEmanuel]

It definitely was ‘schooling’. Lennart do not follow this principle and it shows. So it definitely was a new thing to learn for this abomination of a developer in the Linux world.

[@microcolonel]

@@EvzenEmanuel So abominable, he makes three massively successful pieces of software that every major distro willingly adopts?

[@EvzenEmanuel]

@@microcolonel there are not many admins that adore systemd's stability, reliability and compatibility on their production servers...

[@microcolonel]

@@EvzenEmanuel I'm one of the many who do. systemd has never caused an issue in production after being configured according to the manual in any system I have overseen. Brand new components may be less than ready, but nobody's forcing you to adopt those.

[@EvzenEmanuel]

@@microcolonel well, my experience is quite the opposite. There's probably reason many of us, administrators don't like Lennart's software. Pulseaudio is also not known for superb stability and reliability. Same goes for avahi. Anyway power to you if your experience is good so far.

[@MikkoRantalainen]

Linux kernel not breaking backwards compability is the reason why you can upgrade the kernel and have pretty good probability of everything still working after reboot. Consider every piece of news where "security update" broke a Windows computer. That happens way too often. Linus makes extra effort to not break other programs even for major updates where huge parts of the kernel have been rewritten. That said, Linux kernel has intentionally broken stuff and e.g. Linux kernel nowadays requires minimum of 8 MB of RAM (that's megabytes, not gigabytes). It used to work just fine with only 4 MB RAM.

[@paulmichaelfreedman8334]

Agreed. Microsoft breaks the user experience a lot, Windows 11 being a prime example. Things are slightly different but that's enough to have you searching for where the hell everything is .

[@manticore4952]

I can run software and games 20 years old on Windows, OS updates don't break my apps either, hardware also works predictably. Linux as a Desktop OS is unpredictable, unstable and a complete mess.

[@paulmichaelfreedman8334]

@@manticore4952 I haven't had any problems with Linux in that respect. I have more problems with windows update refusing due to an error. That said I do use windows most, convenience I suppose.

[@zvxcvxcz]

@@manticore4952 Barely 20 years old... I would know as I have late 90s games that don't run whatsoever on modern windows. LMAO. Sorry, I have no stability issues on Linux, nor predictability, etc... Only time I did was when I briefly ran some bleeding edge Arch, because no shit, bleeding edge software does that. AoE II, original disk version actually works well on Linux these days... does not run so hot on modern Windows, you need to go get the updated definitive edition for that to work in Windows well.

[@zvxcvxcz]

@@paulmichaelfreedman8334 Windows update manager is hell... keeps installing a bad stripped down super basic graphics driver that only does like 640x480 so I keep having to manually reinstall the proper good working one that does 1080p no problem and it takes a bit of a hacky work around to get it to take the install. Windows has shit support for old hardware. On top of that, for several months it kept trying to install an update that would fail halfway through and totally freeze the system, and then on reboot it would try to do it again after about 1-5 minutes. And because Windows update stopped allowing you to blacklist an update ages ago... and only lets you put it off about a month... was hell.

[@ilovesamosa]

Linus: "no project is more important than the users of the project". Apple: what? Microsoft: what? Amazon: what? Google: I'm working on that.

[@pseu8664]

Switched to a mac. Have no issues with the base kernel. But systemd, gnome etc. is (interestingly) exactly what Poettering purports to hate. They move fast, they frequently break things and they change their API. The entire desktop is always in flux, unlike the kernel and applications used for server-tasks.

[@marloelefant7500]

Didn't he said that in a world where these things are moving fast, sticking to old versions is anachronistic? He didn't mentioned that he specifically hates that.

[@OrbitalCookie]

Macos might look fine at first, but you will find that Apple will regularly break old apis and expect devs to recompile and release new app versions. And it's not open source where you can see what's going on.

[@sandsack123]

So funny to see the systemd guy complain about breaking compatibility.

[@OrbitalCookie]

Seems like he is having fun by pointing out minuscule mistakes of other people, even calling them hypocrites. He's like that co-worker who wants to appear good by pointing out flaws of others. "Look how smart I am, I notice things!".

[@jongeduard]

Amazing talk, valuable recording, with both great questions and great answers. Great software development comes with a lot of considerations. Compatibility is absolutely preferred, but for example security as well, this combination comes with trade-offs. But apparantly a lot of things have been done so well that Linux is still the most popular kernel in the world. Open source is a fantastic thing, it's about freedom and cooperation.

[@AmirHosseinHonardust]

Breaking Changes As Progress... Seems like an Apple-charger mentality.

[@diegofrancisco425]

there has to be a movement of distro creators out there to create a distro without systemd and without pulseaudio for that original 2016 experience... it was such a slick and efficient system back then

[@stewie3128]

Many distros don't use systemd. On distrowatch, there's a search option of "not systemd" for your preferred init system - that's how badly many people want to avoid it. Void, Devuan, MX, antiX, etc. Some of those distros' primary purpose is to run Debian or Arch or whatever without systemd.

[@weignerg]

8:00 How does he not yet understand. If it ain't broke don't fix it. New features cool.. but don't you dare break what works. You have to juggle existing and future "features" so that the don't conflict.

[@hinzster]

Seems like Paul Eggert (or whoever forced him to do it, I just used "git blame" :P) needs to listen to Linus. We, as users of GNU grep (and by extension egrep/fgrep) don't care that and why it was decided in 2009 that egrep/fgrep are deprecated or, as you call it, obsolete. There are tons of software that now start spewing your deprecation message, and it's not funny. It's so not funny that the Debian project has decided to patch the original GNU source so the deprecation warning disappears, and I have confidence that they will backport the functionality once the GNU programmers decide to actually remove it. It may be a simple patch, a simple change, a simple modification of behaviour, but with widespread use comes the danger of breaking userspace, and GNU grep is apparently in this phase where they do just that right now, and it's just stupid.

[@vancedajcg8724]

A frontend dev trying to educate a backend dev how to write a kernel. LP is cringe.

[@jbs.]

Poettering has broken linux so much. Clearly an inside job.

[@rabbirt]

He's just Bill Gate's estranged son.

[@samuellourenco9526]

After all, without users, there wouldn't be a need for computers. A program should respect the users instead of deciding for them. Lennart chose another path. Also, systemd and pulseaudio often break the userspace. It is not rare to have an issue with both, being systemd not allowing devices to enumerate properly, or applying the wrong driver to them, or simply messing them up, or pulseaudio messing up my audio settings completely. So, Lennart is the hypocritical one. He has two major projects that were shoved into the Linux ecosystem, and both of them don't respect the user.

[@NibsNiven]

Is Poettering the weasel in the green shirt who accused Torvalds of being hypocritical, naive and self-serving? Torvalds is 100% the opposite of all that. Imagine calling the guy who created a Unix clone and gave it away to the world for free, "self-serving".

[@karaloop9544]

Most likely a massive case of psychological projection.

[@urbaniv]

Man sich an interesting panel and then there is no official version with good audio... Huge thx to the uploader to work in the captions

[@deineroehre]

7:38 DID YOU COPY THAT? Users just want to get their work done, there is really no need for the latest fancy things. This should be hanging in every programmer's office. Every other productive industry uses 30-40 year old machines with no problems, they are expected to work this long and that spare parts are available. Why don't car manufacturers, software developers (OK, Tesla is both) and all other tech companies get this simple fact? The user just and simply wants to use the products, there is in 99% of the cases no need for a change. The last 6 Updates of windows 10 didn't improve anything important, they just annoyed the user with printer problems and the try to get the user persuaded to get a microsoft account or use onedrive, which isn't necessary if you already use a professional solution like Nextcloud. Even systemd behaves strange at some times on some of my servers with no log what is wrong ( in the opinion of systemd).

[@martinvlcek5332]

imagine if every time you got into your car, you had to be worried that the designer decided to reprogram the pedals and your gas was now your break, or your clutch, and all your buttons on the panel also switched up, all because some dbag decided it was more optimal, even tho you got used to it and it was how you like (maybe you even reconfigured it yourself). it would drive you insane. software engineering has far too much arrogance, and the user is often spat on. this is why Apple was hugely successful under Jobs, because he cared ultimately about the end-user, as it should be. when we drive cars, we don't consider finer details like how much gas actually goes into the engine, and most of us don't tune it - we just want a working car. that is precisely what most people want from their computer, for it to just work. nothing pisses me off more than when code stops working, because someone decided to change some library, to make it .0001m/s more efficient, and now i have to spend 12 hours figuring out what went wrong. breaking changes are immensely disrespectful, except for very important changes (like security). people like Poettering think the world revolves around them, but the rest of us don't need to spend all the hours of our lives fixing things that didn't need to be broken.

[@Idiomatick]

I remember spending like 3 months arguing with the devs to not release gnome 3. I gave up and switched to Mint right before gnome 3 was released and killed Ubuntu and a half dozen other distros. And all progress that Linux had made becoming mainstream.

[@diegofrancisco425]

the change from 2016 to 2017.... what a nightmare that was... I still remember, suddenly commands that I known and used daily would not work anymore, applications becoming broken, slower, heavier, bloated... laggy... lag on linux, what an absurd... I was very happy with a frozen (no updates) ISO of Linux 2016 in a USB stick... used that one up until 2021 and had an absolutely marvellous time enjoying the best of what linux had to offer... when it was time to update for new hardware, I gave up on adapting to systemd and migrated back to Windows... for my own sanity sake, I'm very glad I made that switch, mental health is important

[@nikitasavelyev9171]

10:00 That expression

[@unfa00]

I think Linus said "oops" at that moment :D

[@KarthikS30712]

Poettering is no doubt very right in asking those questions but Linus and Alan's point about backward compatibility is the ground where "improvement" is tested.

[@deth3021]

The point is that in order to be able to innovate quickly, you need a stable platform. Otherwise you spent a majority of your time, porting your code to new interfaces.

[@complexity5545]

Its 2023 and I'm trying to fix some systemd stuff or incompatibility with my current audio and video setup. I typed in I hate lennart poettering. I remember when this guy got the job back in the 2000s and screwed up the audio stack. This video came up. I don't hate the guy, but his works have plagued me. Avahi also screwed me many times. I hate systemd directory hierarchy and not being able to figured out which scripts are called (without reading 10 years worth of debug notes). Shutdown waits 2 minutes now, but at least I figure that out by telling linux to shut down immediately. For him to sit there and act like he advocates for the user is absolute bat $h1t crazy. But he's gone now (maybe (to the land of Bill and Melinda Gates)). Meanwhile people hate on linus. Linus is the greatest of the bunch (and then Alan); those are some men with a good head on their shoulders.

[@xgf122]

funny gnome 3 was first thing that I thought in terms of breaking user experience, sry never used to 3 and 4, using MATE instead and later Cinnamon on my main workstation

[@felipec]

Yeah, it's funny how they said it was a temporary fuss because "a few people don't like change". A decade later people are still using GNOME 3 as a case study of what *not* to do to your users. GNOME 3 is the New Coke of open source software.

[@helloitismetomato]

@@felipec I've heard this tons and tons of times in the past: you can't criticize our changes, because "people just hate change" and in a little while you are guaranteed to "see the light" and realize how wrong you were. Whenever you hear that line, you know you're talking to someone who believes they can't be wrong.

[@CRCinAU]

@@helloitismetomato This is the same reason why Firefox keeps moving into obscurity and losing market share.

[@alterego157]

Gnome has exceptionally poor software development practices. Every update breaks tons of stuff. Gnome devs absolutely don't give a flying f about user and dev experience.

[@jessepollard7132]

@@helloitismetomato and you know they are idiots.

[@patriciaverso]

I mean, taking into consideration how arrogantly put the question was, Linus didn't really "school" anyone. He was actually pretty tamed in my opinion.

[@alexparker7791]

10:37 [inaudible] is RCU

[@felipec]

Ahh, yes. Updated. Thanks.

[@unfa00]

Crowdsourced lip reading :D

[@cefre00]

Listening to this it just came to me that LP is like a mathematician: focusing on the elegance of the solution, not really caring of its usefulness. The guys are the engineers: will it work? Will anybody us it? Yes, OK then we will do it that way, so people will use it. :) I like engineers more...

[@laughingvampire7555]

1. because versions shouldn't be based on arbitrary numbers. 2. software making even with all the testing is made carelessly

[@ToniCorvera]

... and Poettering learnt nothing

[@karaloop9544]

Wouldn't be surprised in the least if he was a plant to sow discontent all along.

[@HUEHUEUHEPony]

@@karaloop9544 he was paid by russia to make western programmers less efficient

[@RodrigoCastroHdz]

The moderator calling "hypocritical " to the panelists ... Its a joke itself

[@farrellraafi1301]

This actually happens with the 100+ versions of chrome and firefox... It broke the internet

[@desktorp]

Poettering really is a contemptible individual. The amount of arrogance oozing out of him is almost visible to the naked eye. Torvalds stating that he feels "really, really bad" when userspace is occasionally broken is the exact opposite of Poettering, who almost seems to thrive upon breaking other people's stuff. Pulse's old slogan, "The Software That Currently Breaks Your Audio" says it all. His strategy is to write something that he knows will cause problems and then expects others to work around the problems caused by the forced adoption of his crappy software. Of course I won't even get started on systemd. I despise him.

[@seriouscat2231]

I came here to see people hate him and I'm not disappointed. Actually it gets better and better the more I read the comments.

[@michawhite7613]

@@seriouscat2231 You have to be a very weird kind of person to hope for more hatred

[@seriouscat2231]

@@michawhite7613, no. I am just afraid that people will give up and lose hope. I wish there was an alternative init system or more modular system layer that could compete with it and that could be adopted by at least half the major distros. To me, systemd is a dictatorship of mediocrity. So the less people like it and Poettering, the more hope there is that something better will come to replace it.

[@michawhite7613]

@@seriouscat2231 We don't need to bully people to get better software. It's not worth it.

[@seriouscat2231]

@@michawhite7613, I think you are intentionally misunderstanding what I have said so that you can feel better about yourself. It sounds creepy when you presume to be able to say what I need or don't need. None of your business. You have your views, I have my views. There is no 'we'.

[@LubosMudrak]

Yeah, developing a GUI apps for Linux distributions is sometimes like building a castle on shifting sands...

[@jessepollard7132]

Which is one reason to reject Gnome.

[@jessepollard7132]

MOstly because GNOME keeps changing in incompatible ways.

[@HrHaakon]

Java + use system look and feel is pretty nice, not gonna lie.

[@erikkeever3504]

It's not just GUI toolkits, though they're the most visible offenders. Practically all of the APIs underlying almost every package/library in Linux are an ever shifting desert sand pattern. The problem is SO bad that it's one of the biggest reasons to containerize, because it's better to flush cheap GB down the drain if you can bring your own .so's with you than deal with the shifting sands. In HPC we've just given up and created Spack to simply modularize everything and automatically import the exact versions of all libraries that code X needs. On the backend it means compiling X twenty different ways, and 20x the storage for packages, but again that's cheap... The benefits of containers without having to involve a hypervisor. Meanwhile in Windows, for all the crap we give them: "sure, put that 25 year old Win95 game CD in and I'll run it."

[@talkysassis]

@@erikkeever3504 You guys can just shift your work to flatpak and stick with a base runtime. It will do the work for you.

[@sergiocoder]

Does it not bother kernel developers that third-party kernel modules constantly run into compilation issues with new kernel versions because they changed some API? VirtualBox and graphics drivers come to mind. It would be nice to have backwards compatibility at module level I think... As a user, I hate dealing with this.

[@steventhebrockwell]

Kernel modules are by definition not part of the user space ABI, and are in most cases also GPL violations. It probably bothers them, but kernel modules create their own ABI and API. You can't maintain what you can't even define in the first place. In fact two third party modules can be mutually exclusive.

[@RobertHancock1]

They literally don't care. As far as they are concerned those third party modules should not exist. I am generally inclined to agree.

[@joshallen128]

​@@steventhebrockwell I thought they were using the syscall linking exception

[@steventhebrockwell]

@@joshallen128 Not sure what you mean. Third party kernel modules don't interface with the kernel using syscalls. Otherwise they wouldn't be kernel modules.

[@joshallen128]

@@steventhebrockwell I was thinking they were since they operate in user space or firmware outside the kernel area

[@Voltra_]

Now that's software development

[@krantideep77]

What do I do if I accidentally clicked x and there is just a fruit on the screen

[@diegofrancisco425]

I see two distinct generations of humans portrayed in the video: the Old generation, while not perfect and they can become quite selfish when they reach positions of power, the old guard can at least communicate and negotiate... and the New generation, which comes across as these whiny pedantic insufferable beings that wants to drive a point across by force or being "shocking" just for the sake of it... both kinda suck honestly, but this new generation of socially inept kids out there is turning everything into a bureaucratic centralized nightmare to live under... and the tendency is that trend going forward if we simply sit and watch they gain territory

[@matte2160]

Man, Maddog’s lost a LOT of weight.

[@GamingBlake2002]

Linus is so lucky, he got to sit right next to Steve Jobs!

[@MsDuketown]

Funny thing about versioning is that it took the Linux kernel ages to arrive at 4.9, but it sprinted to 6.x in no-time. But how important is that if you can name it how you want?

[@diegofrancisco425]

8:17 GREAT point by Linus... it's NOT mutually exclusive... the other guy (once again) is thinking about mutual exclusivity because his brain is obsessed about centralization, centralize everything, obsession

[@powerfulaura5166]

Lennart Poettering is anti-user.

[@deineroehre]

08:15 a good user interface does not need a mouse for most tasks, there are several customers who still use a DOS program for their company because they can work much faster than switching between mouse and keyboard all the time. Fancy looks of programs don't matter in a productive environment, there is work to get done and not really interest to look at these programs.

[@LaPingvino]

About the last question on specific fixes, that is the biggest part of GPU drivers xD

[@allrandomvids6129]

Man how you research that beginner as very fit with all of your voice pronunciation and video tutorial so hot....

[@laughingvampire7555]

In this moment Linus knew how Andrew Tannenbaum felt.