Global IT chaos - CrowdStrike boss warns return to normal will take time | BBC News

The boss of the cybersecurity company CrowdStrike has said he is “deeply sorry” for the global IT chaos caused by a flawed software update to Microsoft’s Windows operating system which was provided by his company.
George Kurtz, the CEO of CrowdStrike, warned that it may take some time for the fix which has been implemented to work and for services to return to normal.
Millions of people around the world have been affected, after transport networks, businesses, healthcare services, TV networks and payroll systems were among those hit by the biggest global computer outage ever.
Computers failed to start-up after the flawed update and technology experts have said that every affected PC may need to be individually provided with a software fix.
Reeta Chakrabarti presents BBC News at Ten reporting by Zoe Kleinman, Emma Vardy, Katy Austin and Hugh Pym.
Subscribe here: bit.ly/1rbfUog
For more news, analysis and features visit: www.bbc.com/news
#BBCNews

[@zvndmvn]

A moment of silence for all the receptionists who are getting yelled at today ✊

[@Eidolonian]

They get yelled at everyday in this modern “I need it now” world.

[@TheBestCat1290]

Yeah. Sucks they are being paid so low and have to deal with karens because this company screwed up

[@WhatSoEverThingsAreLovely]

I am one of those receptionists & people yesterday called me "rude" & wanted to make a complaint bcuz "I refused to give them a doctors appointment!" 🙄

[@vickywitton1008]

The receptionists at my surgery had one HELL of a day! They got yelled at a lot!

[@vickywitton1008]

❤

[@nerdy-zeig7774]

Reboot in safe mode and manual intervention... For millions of devices? Yeah good luck with that

[@christopher9727]

Do you know Jesus Christ can set you free from sins and save you from hell today Jesus Christ is the only hope in this world no other gods will lead you to heaven There is no security or hope with out Jesus Christ in this world come and repent of all sins today Today is the day of salvation come to the loving savior Today repent and do not go to hell Come to Jesus Christ today Jesus Christ is only way to heaven Repent and follow him today seek his heart Jesus Christ can fill the emptiness he can fill the void Heaven and hell is real cone to the loving savior today Today is the day of salvation tomorrow might be to late come to the loving savior today Romans 6.23 For the wages of sin is death; but the gift of God is eternal life through Jesus Christ our Lord. John 3:16-21 16 For God so loved the world, that he gave his only begotten Son, that whosoever believeth in him should not perish, but have everlasting life. 17 For God sent not his Son into the world to condemn the world; but that the world through him might be saved. 18 He that believeth on him is not condemned: but he that believeth not is condemned already, because he hath not believed in the name of the only begotten Son of God. 19 And this is the condemnation, that light is come into the world, and men loved darkness rather than light, because their deeds were evil. 20 For every one that doeth evil hateth the light, neither cometh to the light, lest his deeds should be reproved. 21 But he that doeth truth cometh to the light, that his deeds may be made manifest, that they are wrought in God. Mark 1.15 15 And saying, The time is fulfilled, and the kingdom of God is at hand: repent ye, and believe the gospel. 2 Peter 3:9 The Lord is not slack concerning his promise, as some men count slackness; but is longsuffering to us-ward, not willing that any should perish, but that all should come to repentance. Hebrews 11:6 6 But without faith it is impossible to please him: for he that cometh to God must believe that he is, and that he is a rewarder of them that diligently seek him. Jesus

[@outlawJosieFox]

Spare a thought for the IT department? Nope. What about all the individual users without access to a tech department?

[@jacobfoster6773]

​@@outlawJosieFoxIndividual users wouldn't be a crowdstrike customer.

[@BehindWorld]

​@@outlawJosieFox individual users aren't use a top tier cyber security product like crowdstrike falcon on their machines.

[@hiranom20]

​@@outlawJosieFoxIndividual users? Have you checked your laptop today?

[@EPiche09]

Glad to know we are so fragile.

[@user-sd9sn7wf3o]

This is why I still prefer paper money transactions, always handy to have cash around

[@fionastevenson4366]

What IT outage? There was no chaos, life went on as normal. Of course I'm just a normal person going about her daily life.

[@imconfused1237]

Except, we’re not so fragile. This was a few hours worth of disruption. Then back to normal. It actually shows just how resilient we are.

[@garrymcfadden4105]

@@fionastevenson4366well said

[@A_O_Leary]

They used and Excel sheet to track the CoVid in the UK at the start of the pandemic. This is many things but surprising is not one.

[@jjstarrprod]

Sounds like the Y2K bug, only 24 years late.

[@DarkCreed]

Nostradamus been sleeping on the job

[@skootergirl22]

No it sounds like south park

[@Zxlok]

@@skootergirl22I highly doubt you even existed during y2k

[@skootergirl22]

@@Zxlok I was born in 88 and I remember the 2000s panic of the millennium bug

[@BondJFK]

​@@skootergirl22No trailer park

[@adroitspartan7907]

airlines right now: "You can Check-in any time you like but you can never leave."

[@HuePhong-mz3qh]

😂😂😂

[@Arsenic71]

Welcome to the Hotel California. You, sir, won the comment section, my respect!!

[@adroitspartan7907]

@@Arsenic71 - ach ja, Thanks!

[@adroitspartan7907]

@@Ann_T_Social - if you say so 🙂

[@Jmcinally94]

Moment of silence for all the IT folks working flat out this weekend, travelling from office to office just to boot in safe mode and install updates in person.

[@Robert29011]

Praise to God Almighty!!! I’m favoured financially with Bitcoin ETFs,Thank you buddy.$63,700 biweekly profit regardless of how bad it gets on the economy

[@adiratnadunia]

Huge! Been trying to trade on my own for a while now, but it isn’t going well. few weeks ago I lost about $70,000 in the trade. Can you please at least advise me on what to do?

[@Robert29011]

Well, I picked the challenge to put my finances in order. Then I invested in cryptocurrency,stocks,through the assistance of my discretionary fund manager,

[@Robert29011]

James Werden

[@dylantoney-eq5jb]

Such a genuine personality!! He is really a good investment advisor. I was privileged to attend some of his seminars. That’s how I start my crypto investment.

[@ManNghiBui-jf5gf]

Wow, that’s very nice. Please how can i be able to reach out to your broker. My income is in a mess please

[@mrhay2u]

Terminator rise of the machines started the same way 😢

[@fpvDRE]

true skynets aware...

[@G.A.C_Preserve]

@@fpvDRE yes.

[@torganya]

Source?

[@EvgeneXI]

No it didn’t… Skynet was an artificial neural network-based conscious group mind and artificial general superintelligence system for a start. This is a system for receptionists that possesses no intelligence whatsoever. It also doesn’t control the military. It also can’t become self aware so we won’t have to try and turn it off which won’t cause it to attack us with nukes to preserve itself. The irony is that you’re probably a bot yourself. Almost all of a KZbin is robots at this point.

[@skootergirl22]

And evil villens exist

[@Littlelamb2023]

Somebody at crowdstrike was sick of bosses getting big bonuses. Asked for a pay rise, was told no, so.....

[@kamkarter8821]

No company should have this power

[@kaleidoscopeon]

Agree.

[@TheRealDuckofDeath]

The main issue is, Windows 10 and 11 are sort of designed to prevent this kind of power for antivirus software. The problem is, they all started creating aggressive workarounds to regain all the power they used to have, pretty much like a rootkit malware. When that breaks, it breaks everything as it is now the core of Windows. I suspect these companies will get even less access come Windows 12. Other nightmare scenarios is, if the third party antimalware stops responding, Windows is often left completely without any protection. I suspect CrowdStrike's relatively fast growth from nothing to being "everywhere" in little over a decade is partly because everybody got the impression their software wasn't as reckless with all of this, as the old Antiviru-zillas Symantec and McAffee.

[@vegas7023]

​@NOT.MI5.MI6The problem is not electric, it is the over reliance in computers and software.

[@phillhatton4492]

Tell that to Microsoft

[@TheBestCat1290]

Exactly. We need to go back to the olden days where stuff like when media companies weren't owned by a couple of big companies like Disney, Warner brothers, comcast etc and when thousands of companies owned the media

[@middleground5084]

What I don’t understand is having worked in IT for over 35 years initially as an analyst programmer then as financial systems consultant, any system upgrades ALWAYS HAD A BACKOUT PLAN which in a nutshell rolled back the system to where it was before the upgrade 🤔

[@newwineskin5494]

Just shows you how this is deliberate.And every summer they have issues with travelling at Airports.something they are doing. Anyways cash is king.Paper too.

[@skootergirl22]

​@newwineskin5494 Yes, let's just use paper and pen like the olden days

[@newwineskin5494]

@@skootergirl22 😂lol both.Like what's all this new system at the gp surgery for online only?People are sheep and they are just following all this changes blindly.

[@madface669]

Proper networks have WSUS server. Just shows how many poor designed networks out there

[@georgeeeles3024]

The CrowdStrike software loads on pre-startup which is usually before any backup system that a company may have runs. If there’s a startup exception, the system automatically halts the rest of the startup and reboots. In this case, the system then loops itself with the glitch and causes the blue screen. Long story short, any backup system then requires the user to start their device in safe mode which is basically “isolation mode” which can’t be done automatically. It is sad, but these companies basically couldn’t do anything to prevent it. They also can’t opt out of CrowdStrike updates since the software updates regularly to patch exploits/vulnerabilities.

[@AJB2K3]

No compensation? What a cop out, the companies should have made sure to to have a backup, They didn't so they ruined the customers. People deserve compensation.

[@MartinMcMartin]

Software update launched worldwide without testing, who do they think we are? Software update my ass.

[@Tubeman777]

Exactly their audience must be all females (just like the all BBC presenters are in this video) who would believe it pure ignorance.

[@stuartmorgan9327]

It definitely was an update. A cyber attack gains nothing from turning machines off. Why it failed testing is the question we want the answer to. Crowdstrike shouldn’t be having QA issues like that.

[@DanielMcAteer]

​@@stuartmorgan9327 A cyber attack gains nothing from turning machines off? There are plenty of ways to benefit from that. Short crowd strike stock. Be a competitor. Etc.

[@Thebt7]

Try keeping your dumb conspiracies to yourself Martin.

[@josephwren-qc4bw]

"without testing"......like the covid vaccine.

[@jonathanboyle6548]

A good reason to refuse digital currencies pushed by our government.

[@denisemurphy3915]

These IT companies are never required to pay compensation for their failures of systems we had no say in. Now with A.I. taking over the jobs effectively now done by humans, the future looks fragile. "Reverting to pen and paper" = always have a Plan B and C, and always carry some cash.

[@the42nd]

I worked for a large security vendor that was breached and they do indeed pay through insurance in most cases.

[@turdFerguson565]

Cash is always a good plan I will say though AI is more a tool atm than replacing jobs.

[@MarekKolenda-cr8vr]

Wait what digital currency coming and computer systems go down

[@kh_trendy]

You can't hold a whole company responsible for stuff like this. This stuff happens all the time in software development. Sometimes it's as simple as you not having a particular hardware configuration. You enjoy KZbin for free, and it's entirely possible a small code change could bring the entire platform down. If that was a huge financial fine every time it happens, no one would be able to afford the software we've come to rely on day to day.

[@denisemurphy3915]

@@kh_trendy Complete lack of engineering oversight and procedural governance. Who let's an update go out automatically without thorough debugging-CrowdStrike. This should be the end of this amateur outfit, watch their share price collapse. Microsoft also needs to bring in greater checks of the work of third parties, a system they have continually automated to get rid of paid human staff.

[@samhartford8677]

Ridiculous. How come they did not have proper testing at Crowdstrike. Properly named company, though.

[@mattsmith1210]

this is why cash is king where many retailers are going card only, if we cant pay we cant take the item away

[@peterbat8433]

Just a prime example of just how vulnerable we are relining om tecnology

[@apl175]

I think the amazing part is the sheer number of systems - airport kiosks, cash point machines, grocery store POS, petrol pump systems, NHS, etc. that run the same protection software that my office laptop runs.

[@pll3827]

It was the best available software for the job apparently, until it wasn't.

[@postminchoppa]

Also why tf are they connected let alone auto updating

[@wabriones276]

@@postminchoppaprobably becausen they’re all running on azure vm’s or vm’s. Which by default….auto update.

[@yellowtomato]

The cloud... I knew there was rain in that cloud..😂

[@BreatheTheMatrix]

It was the best software to use and no one expected this to happen for sure

[@joesummer]

It was inevitable due to over reliance on tech for everything

[@awesomegmg956]

See, the reason for not having all these tech for one day, will cause so much damage, if you do not rely on tech, then basically it means you are going to experience it every single day.

[@baruchben-david4196]

Reliance on tech is the future, whether we like it or not. The real problem is poor QA. I would think a simple test on a test system would have shown the problem. Evidently, they didn't even do that. I can't imagine how that could happen.

[@LennyHenrysDorsalCavity]

Your over reliance on tech, made you point out the obvious in a KZbin comment section… Well done Captain Hindsight.

[@skootergirl22]

Says someone using a device that does pretty much everything

[@BondJFK]

Ok boomer

[@anandsharma7430]

One important lesson to learn from this mess should be that cost-cutting to please the shareholders should not take precedence over IT systems and teams who ensure business continuity. Of course, we won't actually learn it, because corporate profits aka "shareholder rights" trump all other laws, rights and duties.

[@stuartmorgan9327]

But that’s not the lesson. Crowdstrike is probably about the most expensive XDR solution you can get. So companies being affected by this have not been underfunding IT. Crowdstrike might invest more money in their QA team but that’s about it. The issue is the only way to avoid these sorts of issues is to duplicate all of your systems but using completely different hardware and software so that you’re 100% completely different. That includes internet and networking as well. Which is completely impossible.

[@anandsharma7430]

​@@stuartmorgan9327 What I was referring to was that due to compliance and profit target requirements, corporations resort to the cheapest security solution - which is a one-size-fits-all cloud-delivered product with 24x7 updates directly into the kernel, which replaces a security team of salaried qualified personnel dedicated to security. One qualified security person's salary is much higher than a Crowdstrike subscription. There are so many millions of businesses of all sizes that do not run Crowdstrike and do not get hacked everyday in the absence of a similar product. I'm not against automation either. Macs have better kernel security. Crowdstrike has been messing up their Linux updates similarly, but there is at least an option in Linux to run Crowdstrike in a lower privilege mode. Qualified DevSecOps personnel have setups which allow PXE boot as an alternative, which is how one guy brought back 1100 servers in 30 min yesterday. The point is again, if your management believes that a cloud solution can replace your security team or IT team, they will do it, and later when the cloud product fails, you don't have people to fix the mess. Now imagine if CS's CDN were compromised and the payload was not a boot-loop inducing null pointer deference due to poor data validation, but an actual payload from some state sponsored actor.

[@zackeze3478]

Great ad for the case against full digital currency

[@carfo]

The “fix” is more accurately instructions on how to physically go in front of each computer and spend 1-2 minutes in order get it to boot into windows. Physically in front of it being the key phrase. The logistics is a nightmare. It’s not a “fix”; I’m tired of media describing it as such.

[@im.empimp]

It is a "fix" in that it does undo the damage, but it is't an automated patch that can just be sent directly to the machines. Many years ago, I worked in an emergency services job (think 911 responder) and it always amazed me that community members would randomly bring us food, particularly on holidays. For those folks working in IT, particularly at really, really large organizations, their communities (e.g. their co-workers in other departments) need to bring them some good food and beverage to help them to keep going through an undoubtedly long, tedious weekend, as they manually fix each machine!

[@joeking88]

Yes, saying they’ve deployed a fix isn’t half the story. Being in IT that is going to be a nightmare to have to go around to every computer and do that, especially on a Friday aswell when we’re advised you should never roll out a update on a Friday(or the early hours) cause of this scenario

[@timturner7772]

@@im.empimp you a bit hungry? have a byte!

[@im.empimp]

@@timturner7772 🤣 Since you asked, I'd 🧡quite a few bytes of some 🍕. 😋

[@keegankit2863]

Especially now after pandemic, my company has so many outsourced so many manpower in few neighboring countries. Imagine having to fix that. Good luck.

[@user-zs5ed5dz7u]

We have become too dependent on technology for everyday transactions and essetials.

[@stuartburns8657]

Including our ability to spell ;)

[@victoriaappiah2525]

​@@stuartburns8657😂😂😂😂

[@newwineskin5494]

We have not, we are been forced.

[@skootergirl22]

Dude you're using a device that does everything

[@newwineskin5494]

@@skootergirl22 it doesn't do everything. It mostly spies on us.

[@ada3065]

Wake up call for being so dependent on computers

[@infinitebeing1119]

Wakeup call to include secondary redundancy. An alternative operating system rather only depending on Windows.

[@feiyang2561]

Hear Hear!

[@joaquinben4458]

I am going to an island.

[@fallenhero4550]

And what's the alternative genius?

[@kirk5030]

The world coped for years without computers, genius?​@@fallenhero4550

[@SamuelVasilev]

5:50 Thankfully Pen and Paper do not receive faulty updates.

[@enadegheeghaghe6369]

Try keeping up with millions of data points with pen and paper. LOL

[@StikiFing4z]

WEF Klaus Shwab talked about all this during Covid. Now its happening, just like they said it would.

[@mediterraneandiet2483]

So many sheep believe whatever they’re told to believe by their corrupt governments and media.

[@shipit9937]

What a coincidence..!

[@aspman8965]

Pelosi shorted cyber security stock the other week. Similar to that company shorting trump stock last week. Just coincidences lol

[@onatics]

It seems planned. Crowd strike, making massive crowds having a strike.. Interesting.

[@ernstlemm9379]

Software industry should be held accountable for foulty products. Stop the endless disclaimers

[@emilyfloyd6939]

In an era of people not taking responsibility for their actions, it is refreshing to see a leader stand up and take full responsibility and be transparent about it. 👍👏

[@gtavgamer670]

What kind of name Crowdstrike is that ? Strike means attack or some kind of that, crowd means crowd of people

[@mediterraneandiet2483]

Doesn’t sound benign does it? Trojan horse?

[@skootergirl22]

Sounds like a my little pony name

[@tacticalpause3767]

Get to go home early today, thx crowdstrike

[@madraven07]

Globalization, baby.

[@raddastronaut]

Maybe cloudstrike will test their updates before they push them now. Especially, when it’s a sys file for Kernel loading pointers. I’ve seen/been part of a team pushing corrupt files get pushed into service before, but only separate exes, just the standalone failed, when I showed my supervisor it didn’t work. But this, this is nuts! There should be quite a few people responsible for this. The dev(s) who wrote it. The dev manager, maybe the person who pushed it. QA and QA management for not testing it or if they were not scheduled to, clearly needs to be resolved. Truthfully, working for a very large software company myself, this kind of thing could happen quite frequently, and the world is lucky it hasn’t. There are a lot of irresponsible and arrogant people in IT, and it shows.

[@lol-wy8yx]

Crowdstrike!! YOU HAD ONE JOB!! So meta for a company that ensures security to obviously be so negligent with QA and testing. That was your only job was to make sure data was secured and safe. Somebody’s never gonna have a job in the tech field now. Probably a whole team of somebody’s. Reminds me of a Mr. Robot episode.

[@Mrshoujo]

This is why you should have complete control about how & when updates are applied & not let them happen automatically.

[@DC-wq6hx]

Antivirus updates are important. Out of date software leads to much greater risks. This is an extremely rare incident.

[@N1ckZ]

@@DC-wq6hx antiviruses are scams. They only slow down your computer. The standard Windows Defender is all you need

[@enadegheeghaghe6369]

This kind of kernel level Cyber security software usually updates automatically. If it were a manual update and you forgot to apply it or failed to apply it on time, your computer could be hacked

[@SherryDyck]

as a medical transcriptionist that works from home, I understand the value of our work that we provide to physicians. The system crash happened last evening when I was working. Without us typing those reports, doctors don't have the patient history they need. The work is vital and now we will be playing catch up. While tight security is understandable in all these major industries, it is incredible to see what can happen in an instant.

[@robert-73]

How did it happen? Gross Incompetence by CrowdStrike.

[@user-tr1zj]

CrowdStroke

[@matthew8153]

Don’t forget Microsoft. They also didn’t do any testing.

[@MoonLiteNite]

I like how Even this report blames Microsoft. As if they forced users to use CS.

[@PatHand-og9yd]

It is definitely not clear which company is responsible. However, as some cloud developers have posted, the airlines and banks et al ARE responsible too, for their lack of QA and/or contingency plans.

[@hereandnow3534]

Crowdsrroke has root access to the systems they are installed on.

[@movieklump]

Did they try turning it off and on again?

[@user-rn3bk8bw2d]

lol

[@awesomegmg956]

Yeah, 15 times out of

[@veer5827]

😂😂😂

[@antaeusguy]

It's not just CrowdStrike.... the entire world runs on Microsoft, SAP, Oracle... with intricate web of integration with multiple external APIs, third party softwares... and there is no way out. Nobody can replace those systems with another one, because there is no competitors, they are the sole monopoly, cost of switching is very high, and too complex to change.... our lives is so dependent on IT, yes it brings a lot of convenience to our lives, new ways to earn a living, built a business, but don't forget occasionally outage could happen.

[@yanan3681]

There is a way out, its happening right now. Digital global economy incoming.

[@kjpcgaming9296]

"you don't need to change your passwords" (?) CHANGE YOUR PASSWORDS. As soon as someone in Media says that you need to change your passwords.

[@Thebt7]

Lol 😂. So let me get this right. If you think your computer has been compromised how are you updating your passwords. If bad actors have access to your computer wouldn't they just have your updated password.

[@PoorChoiceOfWords1224]

BBC - you are the first news organization that mentioned the safe mode reboot. They also have to disable crowdstrike from within safe mode. Good luck!

[@l.baughman1445]

When businesses and governments started touting digital digital blah blah blah... Some of us were saying "not so fast....what do you do when?..." and we were laughed at. This is part of the growing list of hacks, data breaches, and inability to function.

[@skootergirl22]

It's funny that my boomer mum didn't understand what's going on

[@debbiegilmour6171]

It's not so much that as it is an overreliance on largely one software system from America, that being Microsoft.

[@poro167]

​@@debbiegilmour6171Microsoft isnt the issue here and there arent any alternatives anyways.

[@debbiegilmour6171]

@@poro167 There being no alternatives is very much an issue.

[@BondJFK]

If you hate digital then why are you here. Go back to your cave

[@Mark3ABE]

In 1872, Samuel Butler published “Erewhon”. It describes a fictional society discovered in a remote part of the South Island of New Zealand. At first, it appears as a Utopia. Much of the hard work is carried out by machines. However, eventually, the people become reliant upon the machines and the machines start to take over. It was very prophetic, for 1872, but, even then, the world no longer relied on letters delivered through the post - the electric telegraph had taken over for much important communication. In factories, skilled craftsmen had been replaced by machines, operated not by workers treated as people - they were referred to as “hands” since the workers only useful purpose was to use their hands to set the machinery in motion and keep it working. Even then, the dehumanisation of society had begun.

[@Fab666.]

This isn’t the fault of the mentioned IT company. It’s the fault of every single bank, airline, hospital, government agency etc etc that all put the back burner of their company/organisation in the hands of other ppl that can switch u off so very easily

[@googleuser4207]

Yes executives force this spyware on you under the pretense of "Compliance". For them it is a chekbox in excel sheet that is finished and their yearly bonus secured for "excellent performance"

[@brazendesigns]

Are they going to be sued out of existence?

[@fallenhero4550]

Of course not

[@missprimproper1022]

At my local Tescos, both ATM's have been out of service for over a week. It's the only place I can get cash within a 10 mile radius of my home as all the banks have shut! May have to drive 20 miles over the border to another small town to see if they have any working ATM's there. What are people going to do when the government bring in 15 minute cities??

[@brunoragghianti6963]

And the modern world relies on this one IT source.... that's stupid

[@Astr0b0y8]

Small bug that cost how much in economic loss around the world? Probably billions in productivity loss, so many people affected

[@Jazzynupe1911]

@3:29 To be clear it was not a Microsoft update. It was Cloudstrike that updated outside of Microsoft. They are not in Windows Update.

[@hereandnow3534]

Correct. They also have root access. It is spy software that views and logs what users are doing and reports back to crowdstrike. They have cost lives with this update.

[@believiny0urself]

The company’s name is Crowdstrike. FYI

[@FourDollaRacing]

@@hereandnow3534 Yes, and third-party security software relies heavily on Windows Firewall and other Microsoft software...

[@mosesrocco6614]

That's why personal windows users weren't affected, I suppose.

[@gtavgamer670]

@@mosesrocco6614 Just like is its name, only crowd pc or public pc that got the strike from CrowdStrike

[@henryairconcepts2999]

Imagine if this happens on your electric car. All of a sudden your car stops in the middle of motorway

[@RMX7777]

It's really not a matter of electric cars, but rather modern cars. All new cars being produced are controlled by computers. I've always loved electric vehicles since long before the Tesla motor company was a thing, there isn't anything inherently wrong with them. You could have an electric car without computer controls or internet connections. The issue is our modern reliance on control systems.

[@PikaPluff]

Lol petroleum cars still have electronics - a lot of them.

[@mohammadrizwaan1890]

@@PikaPluff The relatively older ones don't generally receive OTA software updates that control safety critical parts of the car. They are just the same control feedback models programmed on the same microcontrollers across all the cars.

[@challengingshout]

Nope. Stop lying boomer 😂

[@henryairconcepts2999]

@@mohammadrizwaan1890 that's true. Any car has software that can be midified wirelessly is prone to hackers

[@Mark3ABE]

Surely, a GP Surgery could have a local computer system, not connected to the internet, which would not, therefore, ever require any “security” updates. The doctors and nurses would update the patient records directly. The appointments diary would be on this local computer. Running alongside the local computer system would be the national NHS system. So, in practice, if a patient attends to be seen about a rash, for example, the doctor would look at the rash first - he does not need a computer to diagnose it, he uses his training and expertise. Then, of course, having diagnosed the rash, the doctor does need to know the patient’s full medical history, as some drugs might not be suitable for some patients. If the patient has been with the Surgery for, say, thirty years, the local computer should provide a reliable medical history. The doctor could ask the patient whether he had been to hospital recently, or seen anyone else outside the Surgery. If the doctor was still in doubt and the NHS computer was down, he could always wait before prescribing treatment. However, it is scandalous that many Surgeries simply used the IT failure as an excuse to slam their doors shut and have a day off. For example, in the case I mention, the doctor might be able to look at the rash, confirm that it is nothing to worry about and send the patient home. This complete reliance upon IT - almost a mindless slavery to IT, is disheartening.

[@enadegheeghaghe6369]

What if some patients travelled to another part of the country and fell ill? How would the hospitals there access the patients medical records without Internet access? How would you monitor or track epidemics? Also how would the NHS know how much of a particular drug has been used and what to replace if every single hospital or local surgery was running its own closed system? Also if the computers are never connected to the Internet, would their Operating systems, antivirus and other software not be hopelessly out of date? What happens if someone then plugs in a memory stick that has a nasty virus in it? All your computers could be wrecked and all the information lost because you never backed up the data to a central database because "no Internet"

[@Mark3ABE]

@@enadegheeghaghe6369 All valid points. However, we managed, fifty years ago. The patient’s file was sent by the old GP to the new GP. Information was obtained by letter, or in an emergency, by telephone or telegram.

[@roverdover4449]

You'd think they would test the update before rolling it out.

[@im.empimp]

💯 - This is exactly what I've been thinking all day long.

[@TheWebstaff]

Oh bless you. That costs money.

[@vancedkirukan]

Rolling out an update on a Friday is a big no no

[@MrWillyMrBrightside]

@@im.empimp It's literally impossible to test a billion-wide PC update launch, it cannot be done

[@juz882010]

@@MrWillyMrBrightside LOL you can send your software to MS and they will test it for you... for free... they just dont like MS.

[@andrewelliott4436]

Cash ALWAYS works.

[@karenshanley487]

Only if it’s accepted by person or enterprise you’re transacting with!

[@andrewelliott4436]

@@karenshanley487 You're right. And that's why we should all try to use cash right now - whilst we still can.

[@imconfused1237]

@@andrewelliott4436Nah you’re alright. I’m glad I don’t have to walk round with paper in my wallet and coins in my pocket.

[@andrewelliott4436]

@@imconfused1237 Have you been mugged?

[@imconfused1237]

@@andrewelliott4436 I have yeah, back in 2010. Took my wallet, which had all my cards and money in it. Instant financial loss and a faff cancelling everything. Mug me of my phone these days? No problem: block it and claim on insurance. Minor inconvenience.

[@Felled-angel]

In hospital a few days ago for blood test they said they have had another cyber attack today so results are delayed i said "how often dose this happen?" The nurse said every few weeks. This was in Guy's and St Thomas hospital in London

[@garylovell6017]

Does not dose.

[@sams3015]

@@garylovell6017I don’t think anyone cares, it’s clearly a typo

[@Jack_Gibby]

I think that's just the NHS not a global IT chaos.

[@fargoth391]

@@bawilson999 "insecurely attached" lil bro what are you even talking about?

[@fargoth391]

@@bawilson999 "Special private connectivity to 3rd party systems like Crowdstrike." yes... that's how this entire fiasco happened...

[@rjlhee3781]

There should be a linux and browser-based alternative to the programs that these big companies use.

[@jrlx86]

Crowdstrike had a similar incident in April with their Linux version, causing a kernel panic. It just went largely unnoticed in the media.

[@Ghengiskhansmum]

Chrome OS ?

[@skp5725]

Soon Microsoft will will find it's way to replace CrowdStrike with it's competitor.

[@imconfused1237]

@@skp5725 Actually, Microsoft has its own product (Sentinel) for end point security.

[@_Michal_Michal_]

A lack of checking before the update was release, could have essily been sabotage

[@jJustPlayingNZ]

The real culprits are all the management of these affected places for falling for the Sales blurb and purchasing the software in the first place. These organizations often let their accountants and senior management make these decisions based on cost and popularity instead of technical excellence. It is the same with backup software. Often people never test their backup software until they have a problem and then they find out that there is a problem with the backup data. All these issues are well known to the technical people in the organizations but these people are underpaid and undervalued compared to Management people. There is no substitute for excellent technical staff in computers. Believe me on this, I am 80 years old and have 60 years experience working in the industry at all levels.

[@talkinghat88]

“It will take some time …” Thanks for the info

[@MoonLiteNite]

3:30 Microsoft is a victim just like everyone else. Microsoft doesn’t own CS. They didn’t let this “slip through”

[@jirensan828]

Did the CEO explain how this serious bug got past their QA?

[@pic101]

This the only sensible question I’ve seen asked anywhere in the last 24 hours, including in the news. It is the right question to ask, when everybody else is running around with their hair on fire, pulling out all the corny cliches and shouting about chaos. I’m a cloud developer who spent all day working with colleagues around the globe and nobody mentioned it. Granted I wasn’t flying, but how come the planes haven’t invested in hybrid cloud redundancy? It would pay for itself. We need to ask better questions, as this post hints at, about software quality and better code release practices and improved safeguards.

[@vancedkirukan]

​@@pic101cloud developer here too...one of the main important rules we love to follow is to not deploy anything on a Friday....also this feels like totally untested too

[@samgraham6355]

Read a post on Y combinator that suggests they might have bypassed QA.

[@wabriones276]

@@vancedkirukanthis is the golden run in software development.

[@NosyFella]

Probably cutting corners for financial reasons like always

[@eggnogsaber]

"How did it slip through Microsoft's safety nets?" Crowdstrike's safety nets you mean. Microsoft didn't push this update. This shows severe negligence on Crowdstrike's part for not having a staggered rollout of updates

[@roberttalada5196]

Why is Microsoft / Windows being blamed by so many?

[@CRMcGee2]

Due to this only affecting Windows OS machines.

[@notjustforhackers4252]

Microsoft have to digitally sign software for their systems. Without Microsoft approval the software wouldn't be able to execute. They approved the update.

[@CRMcGee2]

@@notjustforhackers4252 Microsoft does not authenticate software updates for non-Microsoft products. Digital signatures serve only to verify the source of the software, not the quality of the code. They indicate the author or issuing party of the software.

[@roberttalada5196]

@@notjustforhackers4252no, applications can download non executable data for their own updating and patching without being certified by windows. Also, this could have happened on any OS, it just happened to be windows.

[@RMX7777]

Ignorance. Everyone is an expert in everything, or at least they beleive they are.

[@keithdesouza8859]

"It takes a while to get a fix , since our customersrs are still testing our software update". Either Crowdstrike did not do proper testing or they got hacked and their update got infected with rogue code that they failed to detect. Either way it's a terrible look for a cyber security firm selling endpoint protection.

[@garylovell6017]

Customers.

[@Annathroy]

I spoke to a guy that does cloud solutions for a big company, he said it's likely Russians but he isn't sure. He advised me to take all of my money out of the ATM

[@juz882010]

@@Annathroy LOL its not the Russians... these companies dont do QA anymore... thats why they cheaper than trend/mcafee. Just people paying for what they got.

[@take2762]

​@russian panic botAnnathroy

[@Annathroy]

@@take2762 bruh

[@vintagelady1]

Why I don't sign up for automatic updates, ever. I translate that phrase to mean "so we can know about your business that is none of our business.

[@isag.7468]

I don't even update unless it prevents my computer from running properly. Still using Windows 7!

[@imconfused1237]

@@isag.7468So you’re using an operating system which is no longer supported and is actively targeted by hackers. I mean what could possibly go wrong.

[@enadegheeghaghe6369]

Crowdstrike Falcon software update is always automatic. That's the whole point of it. It updates quickly and quietly in the background so it can respond very quickly to new and emerging threats.

[@PatHand-og9yd]

Banking, schmanking! I went to my Starbucks today and they were DOWN! 😠 Seriously, their systems were down, but the staff were calm and giving away free tea and coffee!!! Now, THAT is how you build customer loyalty!!

[@Alexia-rk6rv]

Liar

[@envynoir]

Sounds like the ramblings of a schizo

[@kingkong81icloud]

Starbucks 😂, never had one am not paying a fiver for a cup of coffee

[@tychodragon]

You actually beleive its possible for someone to just screw up on their day on the job oops! and theres an outage of this scale? I call BS this was a cyber attack 100%, no way a error of this magnitude happens just like that, by accident

[@josephjordell7923]

It's funny because if this was a cyber-attack "This is not a cyber-attack" would be the first thing that they would say

[@skootergirl22]

Those damn furries

[@vitalyl1327]

Dimwit muct aren't you, gammon?

[@arsenalfan7511]

I lost money from my bank account. I noticed it when my salary was paid, but the money wasn't being credited on the account. Yet the transaction has been completed. I call my bank menu times about it. And they fail to see my problem. They saying the money is there... Well now I have my explanation. So sad. I was so stress about it

[@giggler-w5n]

They took "Testing in production" to another level

[@NUKEzx10r]

There goes your info.

[@m.moonsie]

They deployed on a Friday. What could go wrong??

[@oO-_-_-_-Oo]

All. By. Design. The only question I have is, who did it?

[@thehrschannel4784]

Well I've been to an antiques fair today in York and everyone was using cash for their purchases and business carried on as usual with zero reliance on internet banking... Perhaps it's time we all stopped been so reliant on cards and used proper cash...

[@KontaktoMan]

In Russia, all the systems are up and running💪 One should not rely on a single company.

[@ivito514]

It's amazing to see how those so called "big companies" don't have a contingency plan and no backup systems. At my previous job, if an update would cause cahos, we would be able to revert back to the previous version of the system switching from system A to system B. Those so called victims of crowd strike are victims of their own incompetence by not having a backup plan.

[@imconfused1237]

Oh right. So “System B” must be incredibly vulnerable then, given it’s never updated. What’s that, you updated it on 19/07? Whoops. Poor planning on your part.

[@stuartmorgan9327]

It’s not about no backup, going to backups has issues itself. Firstly what is your RPO depending on that depends on how much data you lose, is it better to lose that data or fix the issue and lose no data? Secondly, you rollback the system, you then also have to prevent the update from taking place again. This is security software by its very nature it’s designed not to allow people to turn it off easily. Also if you turn it off you’ll be exposing the organisation to other risks. Do you accept that risk or wait for crowdstrike to issue a fix knowing that their engineers will all be fully committed to solving this issue. A lot of armchair experts who think they know how things should be whilst not understanding the realities of IT.

[@launchpad53]

​@@stuartmorgan9327finally, someone with sense commented.

[@ivito514]

@@stuartmorgan9327 A backup plan has nothing to do with an actual software backup. If the hardware of 10 computers stop working, do you have replacement hardware in the closet? If the building is on fire, do you have another location? that's a backup plan in case of a disaster.

[@ivito514]

@@imconfused1237 Irrelevant, system B contains the previous version, I said revert to the previous version, not go back in 1980. I guess you have no clue how it works.

[@hansoonteo1353]

Should sue Crowdstrike for compensation

[@emydonavan6254]

Oh, the lawyers are already prepping the suit filing.

[@aaronwestley3239]

​@@user-ro7ps1rr3pexcept that the customers of crowdstrike is not the "you" as a person, but the companies they service. Who actually have the power to sue them to oblivion. That's very obvious if you actually have a a corporate job. The IT department installs software on your company provided PC and you do your job. If a software they had installed broke then it's not your problem. It's the company's problem.

[@NeatWolf]

Pretty sure their Terms of Use already cover this scenario. And whoever used it, took their chances since, apparently, it only (publicly?) happened once so far since 2011 (unverified). They lost 10% on their stock prices and that's already a lot. It had to happen one day. Whoever used the service with 0 supervision should be sued.

[@jtr549]

I'll tell you why it happened, they couldn't be bothered releasing the update to test machines before rolling it out to the public and as a result it broke.

[@GH-oi2jf]

It likely was tested, but the particular vulnerability was not covered in the test environment. Most people don't know how complicated software can be.

[@x_isaka]

They should check it in every version even with virtual os

[@jtr549]

@@GH-oi2jf This affected all PCs running it. Not just a few. Usually I'd agree but I am a software engineer and I know how many assumptions are made right before new releases.

[@mowcowbell]

What is CrowdStrike's motto? "Live on the edge, test in production"?

[@JohnLe-x4u]

This is worse than Y2K

[@agbook2007]

Indeed, it is. One damn company and their incomptence.

[@skootergirl22]

This was thought to happen

[@KKarama1]

my introverted ass that works from home watching the world burn

[@Aaaa-1znt]

Eh? A lot of people who work from home were affected too.

[@skootergirl22]

On paper and pen right? Pigeon mail?

[@Aaaa-1znt]

@@skootergirl22by work from home, maybe they meant twiddling their thumbs and waiting for their UC to come in

[@Thebt7]

Right haha. I woke up was told by my partner that there was a global IT outage. Had mini panic attack that it was a cyber attack. Realized it didn't affect me and continued on with my day. Seeing all the incompetent people with their conspiracies has been entertaining though.

[@enadegheeghaghe6369]

​@@Aaaa-1zntonly those using Crowdstrike falcon software

[@TomboyArmpitSniffer]

Have they tried turning it off and on?

[@nyc1087]

Best comment 😂😂😂

[@meritoriousguy9367]

WE SHOULD ALWAYS DRAW SOME POSITIVE LESSONS FROM SUCH INCIDENTS. Like Technology will always be there , but should always keep basic manual alternatives available. Keeping important numbers, passwords, PIN numbers, email ID & password etc manually written in file.

[@JelMain]

There was something wrong before that - the Bank of England reported issues with CHAPS 24 hours earlier.

[@Ash-nx5rx]

This happened to my private windows pc 3 days ago.....took hours of work to get it to reboot to windows and a lot of patience...

[@Aethid]

How did it slip past Microsoft? It didn't. Microsoft aren't responsible for this.

[@roachtoasties]

Third party companies. Microsoft doesn't check the code for every update done by everyone. If a popular gaming software company creates an update for their "Super Duper Game" and it crashes everyone's PC because of their incompetent programmers, is Microsoft responsible?

[@Aethid]

@@roachtoasties That's what I said.

[@roop1958]

Dont they test the updates in isolated environment before implementing the update?

[@torquetheprisoner]

crowd tech guy:ok the update is ready buuuut it knocks out the os system for a while should we still go ahead with it crowdstrike:yes i wana see the world burn hell good job on the linux and mac debacle aswell crowd tech guy: why thank you😅

[@MSThalamus-gj9oi]

"How did this slip through Microsoft's safety nets?" I wish to make it very clear the Microsoft had NOTHING to do with this. CrowdStrike uses Windows as a platform. The dereferenced a null pointer in a kernel module. ANY operating system would have crashed in this exact same way had CrowdStrike used them as their platform instead, yes even mac OS, yes even Linux. I'm no Microsoft fanboy. I run Linux (BTW and all that). But it's beyond unfair to blame Microsoft or Windows for today's fiasco. It's fully, wholly, and entirely CrowdStrike's fault. They have no business writing software running in kernel mode if they are making mistakes like these.

[@marcuslew523]

Houston we have a problem..

[@agbook2007]

“Lock the doors.”

[@agukagospel]

"Our first deployment of AI generated code was partially successful"

[@purplesprigs]

Crowdstrike - the Boeing of EDR software.

[@cag19549]

"Finally came forward"? They came forward right away. Their CEO has been all over the networks this morning. Mostly to assure people we weren't under a terrorist attack.

[@isag.7468]

Mostly to reassure shareholders who might lose quite a lot of money

[@mclark87nz]

@@isag.7468 Yes, I noticed on Crowdstrike website when looking for updates which there was none there was the word 'Why Crowdstrike", that's going to be an interesting line for upcoming RFP's :)

[@roachtoasties]

Maybe CrowdStrike thinks this is the way to improve their security software. If you push an update that brings down every computer, not even the most experienced hackers can get into a computer network. With every computer being bricked, computers are super-secure from hacking.

[@purplehaze2342]

Like it is not by design 😂

[@saftehbludloineleown1617]

"I'm sorry we're going to have to cancel your appointment. But at least we have a reason this time."

[@Stopreadingmyusername23]

Always wondered what it would be like to live this.. now I know 😂

[@naureen4justice991]

Crowdstrike messed up on the busiest day for flights 🤔

[@franciscouderq1100]

Very calm and clear diction from the 1st lady introducing this page. Striking difference with the US where yelling and speedy speeches are the norm

[@crouchingspider2001]

Crowdstrike living up to their name.

[@randyearles1634]

no excuse for this incompetence!

[@kk-pm4xi]

What's bad is that a security company leaked information about what operating systems are used at airports in a certain country.

[@TarasShyn]

Windows is used on nearly all commercial computers, with extremely rare exceptions, so this information is not secret by any mean. You won’t see major system ran on Linux, except some servers, and definitely not on Mac.

[@bnika33]

I suspect that some of the less important computers (e.g., food-cart ordering, even reservations) can/will use commercial operating systems like Windows. However critical systems like safety, navigation, communications, etc. will almost certainly use either a proprietary OS or one that is a heavily customized version of an industrial OS (e.g., one from VXWorks). For example, the Stuxnet virus was made to target the Siemens software running Iran's (assumed) nuclear plants. I know Linux is use for the entertainment system for several airlines.

[@zaper2904]

@@TarasShyn "Some servers" yeah just a cool 80% of them.

[@Aethid]

Which out of all 2 options... and virtually all computers used in enterprise contexts like this are running Windows.

[@blackxfiied_]

bruh it could be windows or linux you dont need a security company to disclose this

[@user-er1tr7ie5h]

"You may never have heard of Crowdstike before " - hard to forgetThey were involved in the Russia Gate scandal