Why was Facebook down for five hours?

[@sykskysyk]

Facebook going down was a small price to pay for getting this incredibly informative video from you, thanks Ben!!

[@froller]

Let's bring down some other large network in some weird way to get next video! ...just kidding.

[@lilstubthumb]

I would pay for Facebook to be down.

[@froller]

You forgot to add "forever". :)

[@lwizzit]

It’s a win/win

[@francesco45]

Not for Mark ahaha

[@xXRedTheDragonXx]

"I'm connected to a router on AT&Ts network that I happen to have access to" As a former NOC tech for an ISP this was absolutely hilarious to hear you say this so casually. They don't just give anyone console access to random routers!

[@yves.dantas]

I think he still works as a nw engineer. If im not wrong he even worked for juniper

[@dddd6606]

This is better explanation of DNS than the one I had at university, thank you.

[@dm3on]

That's because he is self thought, not sure if Ben has high school diploma.

[@Ripsaw460]

@@dm3on "self thought"

[@Ripsaw460]

@The Poison Donut not only that but for even a person that isn't very knowledgeable in this field, he makes it very interesting to watch and all his videos are very easy to follow and i almost never find myself lost trying to figure out what is going on, ben is the best.

[@Nick-lx4fo]

@@Ripsaw460 oH nO, a slight spelling mistake that totally obfuscates the meaning of his message, what will I do?

[@Ripsaw460]

@@Nick-lx4fo you dont have to do anything, just thought it was funny.

[@tecno2053]

Im a network engineer for a medium size ISP (primarily enteprise fiber). This has become my goto video to explain to people what I do. I work with BGP and the like daily.

[@penguin1714]

I'm pretty sure Ben Eater is actually a physical manifestation of knowledge.

[@12crenshaw]

He not only knows everything, he also have time to live, breathe and share his knowledge

[@charstringetje]

All you need is a default route and the ability to read, and you can acquire knowledge too. Exciting times.

[@youkofoxy]

That sounds like he is a named version of eaters from Digimon cuber sleuth. Eaters are only interested in collecting and sharing information across it's hive mind.

[@martinkuliza]

Ben Eater is an A.I. this physical form that we see is a computer manifestation of his real self, Ben Eater is like the M.C.P. on Tron Part 1 LOL

[@cyber_8778]

@Penguin Hey I remember you! You commented on my video once :)

[@laykefindley6604]

Watching one person setup global router configurations is seriously invoking wizardry levels of mastery. I am truly humbled to watch a work of fine art like this and gives me goosebumps thinking how much power one can wield with the right knowledge and access to modern society.

[@autohmae]

Now multiply this times 70 000 or so, because that is the number of AS-numbers in use in the world, thus close to the number of independent Internet networks who connect with everyone else.

[@askthisscientician6040]

That had to be the best explanation of this I've seen. Not dumbed down, no hand-waving explanations, but carefully and effectively explained with a live demonstration to boot. Thank you.

[@jdatlas4668]

To quote a random Tweet: "it's hard to even explain to a layperson how badly you have to fuck up to get BGP and DNS both as part of the same problem. Like, you need multiple diagrams."

[@nickwallette6201]

LOL -- that was my exact reaction when I heard about this. Like, what happened to break DNS and BGP, that took 6 hours to recover from? That is one COLOSSAL screw-up. With such huge stakes, too. It did make me suspicious what _really_ happened.

[@johnm2012]

It took them 5 hours and 50 minutes to find someone with a physical key to let them into the building.

[@NavidIsANoob]

@@johnm2012 That's false. It DID take them that long to get someone with the necessary protocol expertise to fly in and fix the mess. It's silly to assume that no one on site would have a key to the building. It's reasonable to assume that a corporation like Facebook understaffs their infrastructure and doesn't retain experts to maintain that infrastructure.

[@zacker150]

On site, they use badge readers to access the building. Those badge readers ran on Facebook's network.

[@NavidIsANoob]

@@zacker150 Again, this is false. Why would a badge lock system run on Facebook's global network, as opposed to an internal, segmented network? It makes no sense.

[@critical_always]

Nobody can explain like Ben does. Everything is always at exactly the right pace and detail. I kinda feel smarter learning from this man.

[@---nw9qu]

One of the best explanations of dns, routing, and BGP that I have seen.

[@memoryvice398]

The sound of Ben's keyboard is totally ASMR for a guy like me. Brings back lovely memories of my first PC back in the 90's.

[@NickiRusin]

yeah the clackity clack clack just sounds super comfy

[@javabeanz8549]

Goes back further for me, at the keyboards for an IMSAI and a Polymorphics in the late 1970's, and the college SOROC terminals attached to a PR1ME mini in the mid 1980's.

[@izzurzuhri]

what keyboard/switch he use?

[@lordkell1986]

@@izzurzuhri probably built it himself from breadboards and electrical switches. Each key is the size of a foot and he jumps on them

[@kaitlyn__L]

@@izzurzuhri sounds like clicky Alps to me, might be wrong though, a few other switches “ring” the same way

[@riz94107]

I've been an internet professional since 1994, and i knew all this already, but your explanations are SO GOOD that i watched it just for fun. Great job!

[@L0j1k]

Yea I spent years banging away on dn42 but this is BY FAR the best and most succint and insight-dense "class" on DNS and BGP I've ever seen.

[@TheRukisama]

He does give excellent explanations of things, he lays it out better than I've ever been able to a non-IT person.

[@mattboardman2148]

The subnet mask description was so succinct that I'm using it internally when explaining networking to both our techs and clients.

[@Mine8maniak]

not much changed since 90s, i mean the protocols. right?

[@EdGriebel]

Same. I took a grad networking systems course, Ben’s explanation of BGP is *so* much better than the course, I wish I had this then

[@casualyoutuber6239]

I just enrolled in a CCNA/CCNP course being offered at my University and yes I am as vanilla as they come as long as Networking is concerned but this video was so crystal clear that I may have just revised effectively about 16 credit hours in a single 30minute video!

[@FunnelCakeRyan]

"A router on AT&T's network that I happen to have access to" WHAT A FLEX :D

[@esepecesito]

He works there...

[@jirensan828]

Not really a flex. That server is public and available for anyone to use to validate their BGP routes

[@TomStorey96]

There's a few of them spread around, usually referred to as a "looking glass".

[@tiporari]

Lol I like the "my address has changed by now" disclaimer. Don't poke me please. You are just poking my mouth breathing neighbor.

[@ratbag359]

@@tiporari It's bad enough with bots scanning ports and trying to dump payloads into the open ports they find yet alone the comment section of youtube seeing what holes or services are avalible.

[@thewelder3538]

You are one of the most truly educating people on KZbin. Unlike so many others that are just so superficial with their details, mainly because they don't really understand the subject matter, you still down and give the real information. Now I understand DNS and how it works and it was just nice to see it properly explained. Would have been a bit nicer to have had an explanation about the DNS protocol, but this video is still an amazing explanation. Great work.

[@gnsoares_]

I just had a class about this today and I understood it so much better in this video. Thank you so much for your work Ben

[@realdragon]

It says a lot about me when I still don't understand it

[@SteveJones172pilot]

That's kind of what I was going to say - He boiled down probably 2 or 3 days of a CCNP class into 20 minutes or so.. Of course the "real" class would have confused you with all the other options you could put on those bgp routing statements, but it does make me think about maybe this video should be a prerequisite for diving into those sections of a network course. It's really good to see the basics working first!

[@To-mos]

It's a shame the education system is failing people, I didn't start truly learning until I dropped out of college. It's bizarre how much they charge for schools considering the word is Greek for "free time".

[@einarabelc5]

That's called applying knowledge. Next time, explain it to someone else. Try the Feynman method.

[@indeepjable]

Isnt The Current Educational System Still For Factory Jobs?

[@nebula_wolf3132]

I must say, you're one of the best teachers out there, not only is your explanation simple and easy to understand, but it also lends itself to allowing people to learn on their own, rather than shoving information into our faces (thanks school). Whatever you do keep teaching, you're amazing.

[@infinitenex8165]

Man, networking is so convoluted to me. Its a miracle that the global internet works as well as it does on a daily basis.

[@GummieI]

The number 1 reason it works so well really is the redundancy. Notice how multiple time through this video he keeps talking about dozens if not hundreds of this and that. And that is what makes things work so well, everything is connected in hundreds of ways, so if a route or 2 goes down for whatever reason, who cares, there are still hundreds of routes left to take instead.

[@LMB222]

There are further tools that make automatization much easier. You set up a few things in the beginning, and then sit back and relax.

[@jdpruente]

I tell people that the entire internet only works with gentleman's agreements and handshakes. And that's when I'm being nice about phrasing it.

[@edenrose2374]

@@GummieI except its rarely the case with preferential routing. ISP's sign contracts with third parties to reduce overhead and operating costs by having them operate long-distance runs. - This gives these single-points of failure a Scapegoat if it fails. US and UK are the worst offenders of this, literally blocking or banning competitors networks at hops that would route through their network rather than around it. This means that regardless of speed or outage state, a singular ISP can ensure the bulk of their traffic goes through and is on their and partner networks. This gets worse when we start talking undersea cables. Exit Nodes (backbone) connections to externally connected networks are entirely priority based. When it comes to routing overseas, these contractors get priority. Why? rather than operating their backbone at a flat-cost for a nation, and telling the Goverment to cover the cost of operation so all may benefit... They instead choose to charge rate-based, with tiered flat based contracts available to ISP's. - This means that, your provider has to both pick a rate limit(and quality[jitter] limit) for its clients and limits routing to the connecting country to ONLY this backbone. | Sometimes they have failover's to competitors backbone lines, but since its rate-based there is caps client and SEVERE data-rate penalties. Amazon is the worst offender of this, offering basement priced backbone lines that are PAINFULLY slow(except to /some/ of their services) and this results in much of the worlds internet being measurably slower as a result... Australia has the worlds slowest internet exclusively due to no one hosting services inside australia (draconian encryption laws), and relying exclusively on Amazon and Telstra's dedicated Undersea cables to route outside of Australia(to the UK, the longest possible run). Since nearly everything is hosted in the US or germany. This means reaching these places is usually several hundred hops. - Turning the Fibre network that makes up Australia's rather robust internet infrastructure into a cesspool of garbage slow performance.

[@vintyprod]

@@edenrose2374 this is actually super interesting. thanks for sharing.

[@metalpunk]

I watched this when it was a fresher video, but I'm taking an introductory networking course this semester. While we haven't covered BGP yet, we _have_ talked about DNS, and so this actually made sense this time. Fascinating stuff, even for someone who doesn't really use Facebook

[@BobHolowenko]

Hi Ben. I am a network engineer for an ISP and this was a VERY well done video that talked about the "house of cards" effect of the outage. Great work!

[@bobbyb42]

I'm studying for the Network+ right now, and this video is the best explanation on these topics I have found anywhere. I could go on and on about how great your videos are, but mainly I just wanted you to know that I appreciate you sharing your knowledge with us and putting in the effort to make it easy to understand and fun to watch.

[@lightdark00]

At the end should have been a cutaway to a team of breadboard computers all networked together, that made all this possible. 😊

[@TNTsundar]

Cutaway scene like from the movie “The Prestige” with an eerie music that shows a lot of those breadboards, ICs, wires and then Ben using the monitor and keyboard connected to it.

[@BertGrink]

Maybe that´ll be the topic of an upcoming video?

[@K-o-R]

@@TNTsundar You mean the breadboards that generate the AI known as Ben Eater? 🙃

[@Aighthandle]

The actual servers aren’t that different, just a lot of modularized sheet metal chassises in big housing units with so, so many hard drives

[@mjsvitek]

Next video: "I built 30 breadboard computers so I could make my own internet... The Eaternet."

[@crimsonfart]

I must've watched a dozen videos about the Facebook outage, but somehow only Ben is able to explain it not only in a way that I can understand, but it actually seems simple, even though it absolutely isn't. This is the best tech channel on KZbin, no contest.

[@thewhitefalcon8539]

Except he didn't explain it, he talked about what the servers were doing and how the rest of the internet would know they were down and then said "oh and the reason was they typed something wrong"

[@stephaniecurry45]

I mean complexity is built on understand and learning the simple stuff. This still isn’t like rocket science or anything lol

[@caleballen4721]

You know a person is a good teacher if they can communicate clearly about DNS

[@jonathansteward1]

Ben this is an amazing video, as a network engineer in the industry I always like to listen along to these kind of videos to refresh my knowledge and remind myself of ways in which to explain various content and this was perfect! I was really thinking of making a video on the topic but this does the topic justice and I don't think I could add anything else to the topic! Keep up the great work!

[@meeDamian]

You are undeniably one of the greatest content creators on this platform.

[@therealchayd]

I've never managed to get my head around BGP until now (despite working with network engineers whose explanations just become white noise). Massive thanks!

[@_Stin_]

Now, THIS is how you explain things!! Foreword, introduction, background, examples and real-world demonstration... The best explanation for how the DNS architecture works (and how the rest of the Internet works lol)

[@Will_of_Iron]

Subscribing to your channel is one of the best decisions I've ever made in my life. I didn't even know anything about these and you explained it so smoothly. I guess I'm blessed to have a teacher like you. Keep up the good work!

[@ndupontnet]

That outage just happened seconds after the update of my local Pi-Hole DNS instances, that was quite a head-scratching experience. Thanks a lot for those explanations.

[@zacharybearden9950]

Awesome visuals. As a fellow route/switch guy - I can't pass up a video with BGP in the title. About 7 years ago when I was starting my journey in networking, I watched nearly all of your network videos. The fundamental concepts that series covered set me on the right path.

[@ahensley]

Wow! That seems like a lot of work to set up the demo you did, but then again I don't happen to have access to a router connected to AT&T's live network 😏

[@bitterlemonboy]

How does he even have access to that?

[@SirLightfire]

@@bitterlemonboy my guess is that his full time job is a network engineer at AT&T?

[@Jeff-ss6qt]

Maybe he rents a server there or installed his own? He has his own website that likely gets a lot of traffic.

[@SirLightfire]

@@Jeff-ss6qt but that wouldn't give him access to a top level route table for an AT&T router To even be able to login to something like that is a huge deal

[@scott3489]

AT&T and several other ISPs host public route servers and/or a BGP looking glass where you can view their BGP routes. Their around to assist with troubleshooting multi-carrier internet routing issues. Maybe that’s what he’s connected to? But yeah, the demo he built is incredible.

[@andrewohanian5132]

I am in awe at how many views a video, that is in large part a Juniper router configuration demo, has. This has got to be the most viewed video that details Juniper router BGP configuration. Amazingly well done.

[@SubspaceEmber]

I actually had to do DNS config for my website a few days ago, but wasn’t really sure what I was looking at, but watching this made that make a lot of sense. You can actually almost see how all this grew from just connecting a few computers to each other!

[@emeraldthunder]

Everthings so much more understandable when you explain it.

[@GeoffreyThornton-TheWinGuru]

Ben, I just wanted to say how much I enjoy watching your videos. I've learned breadboarding techniques, logic design, microcode and processor design, and hardware bit error detection and correction (among others). Then, when you create a video that is my wheelhouse, you do so in such a concise and informative fashion that I'd be hard pressed to do the same. You cover the fundamentals of Internet name resolution and route exchange in 30 minutes in what would normally be an intense week to month long seminar. I'm impressed with the level of detail you provide, along with the time it took to create a lab environment to demonstrate the operation of BGP routing. I had not noticed the Facebook outage myself, but at the start of your video, I suspected it might have been a result of BGP route dampening due to network interface flapping, but that would have been for the entire AS and not just specific subnets. Your insight and use of BGP looking glass servers to demonstrate how specific routes were missing is remarkable. I don't have any particular insight as to how this occurred either, but perhaps Facebook is using an internal route reflector or confederation to handle routing Anycast traffic for DNS and ICMP. I don't have any experience with Facebook's network infrastructure, but this outage should hopefully help Facebook and other networks design and maintain more resilient networks. Thanks again for all your time and effort in creating these videos. You have always been a favorite content creator of mine, and this latest video just adds to my adoration. I could tell you were running JunOS, but I'm curious if you were using GNS3 or some other simulator. In any case, kudos for your time and devotion to create such an informative video!

[@hariranormal5584]

isnt the commit command a thing only on junos?

[@GeoffreyThornton-TheWinGuru]

@@hariranormal5584 It's a thing for JunOS and Palo Alto firewalls (PanOS). Juniper devices also let you do a confirmed commit where if you lock yourself out by doing something dumb, it will automatically rollback to the previous configuration. Cisco devices typically commit changes immediately and so can cause issues if commands are entered incorrectly or in the wrong order. The only recourse then is to reboot the device and let the "startup-config" replace the running config. That's why you typically see commands like "copy running-config startup-config" or old school "write mem" to finalize a configuration.

[@hatecrewblaze]

'An AT&T router I happen to have access to' like it's the most casual thing to have hahaha (coming from someone working in tech support at one of germanies biggest ISPs) love the way you present and ur humor :)

[@aelolul]

Thank you for this deep dive! I really appreciate it. I'm a computer professional, but my knowledge of networking is limited to the ip command, modifying /etc/network/interfaces, and cursing at systemd. I hadn't even heard of BGP before this kerfuffle. I watched a few other videos on the topic since the outage, but this one is far and away the best I've see. I feel like I actually understand what's going on. Thank you again.

[@12-343]

Same. Best I can do is IWD and copy paste commands

[@xtdycxtfuv9353]

@@12-343 same LOL. Networking is really hard :(

[@mahanlamee]

You have no idea how good it feels to see a video from you uploaded less than a day ago

[@F4LDT-Alain]

Fantastic. I came for some more details on the outage than given in Facebook's statement and I got the best BGP tutorial I've ever seen. I'm more of a system guy myself, so local routing and DNS don't have much mystery left for me. But internet routing and especially BGP has always been kind of black magic to me. Thanks to you, I've got a grab of all this and the whole picture of that outage makes sense now.😁 Great video. Your simulated networks (and routers, I guess, apart from the AT&T one?) are impressive.

[@TheUnofficialMaker]

I'm just gonna keep it as Black Magic!

[@mattiviljanen8109]

I had to come back to comment. It's unbelievable how well you explained BGP. We touched the subject at school and I couldn't get it. Now I did. Not that I'll ever need it, for reasons, but I did get it. Thanks!

[@AndreaTerenziani]

everybody gangsta till Ben pulls out the heckin AT&T router

[@MxSherwood]

@Maxime St-Louis , Ben has ssh access to a router on the AT&T network, and he blurred out its IP address in the video. That is not public.

[@henrychoo4361]

@Maxime St-Louis do u have access to an AT&T router? Not every Tom, Dick and Harry has it

[@HaraldSangvik]

@@MxSherwood Wasn't that just his home router?

[@krmr]

@@HaraldSangvik No, he specifically said he's connected to an AT&T router happens to have access to and blurred out the hostname.

[@thewhitefalcon8539]

@@HaraldSangvik No, it's part of the actual for real AT&T network, the one that may suddenly stop working and cost millions of dollars if you type a wrong command

[@tsraikage]

you know the video will be as informative as it possibly can when the tutor is building a GPU on a breadboard on his free time

[@junkmail4613]

Ben, I'm no scholar at this, but again (as I say 'bout most everything you do) you'vedone a superb job of detailing intricate(unfamiliar) connections in a clear and lucid manner. Thanks!

[@qpSubZeroqp]

Agreed, thank you!

[@nicholaswest9146]

Your way of explaining things is incredible! I've worked in web hosting for about 6 years and now I really understand DNS :)

[@castles990]

You're working for 6 in years in web hosting and you didnt know what dns was?

[@ilyadaemon]

@@castles990 for web hosting dns is a table like /etc/hosts :)

[@kaitlyn__L]

@@ilyadaemon plus you don’t need to know anything after the point where your web server network connects to your ISP!

[@sneedsfeed757]

Nice to see someone practically explaining dns lookup.Even I have never tried to do it

[@HopeRunsDeep22]

I have been studying for my Network+ Exam and am taking it on Monday. This video was so awesome to be able to relate some of what I am learning and actually see it in a real life situation to really get a better understanding of how it works rather than just reading about it. Awesome video! Thank you!

[@Kaepsele337]

I really like your style of explaining stuff completely. That really helps me with understanding, most other people would gloss over "details" like the root DNS servers.

[@BringbackTBC]

Sent this video to my networking professor to show to our class. This is the best explanation ive ever seen.

[@dantheman4700]

Can you please do a video incorporating NAT. You honestly have explained subnets, dns, and ip better than anyone Ive ever seen.

[@Hugh_I]

that would fit into the explanation of this video quite well. I'll try, probably not as clear as Ben could do it: His router has two addresses, one public IP address that is routable in the internet (i.e. his ISP advertises routes to that address via BGP) and one on his local network, that is not. Only his router knows what to do with addresses on that local network. Now when he sends a packet from a host on his local network to the internet, it gets send to the router (the default gateway). What a NAT router does is now in addition to just passing the packet on to his ISP's router, is to slightly modify it. It replaces the source address (i.e. the 'return' address that the host on the receiving end should send its answer back to) with his own public IP address. It also stores the info of where it actually came from in a table in memory. That way it looks to the host on the other end like the request was sent from the router's public IP address, not the local IP address of the host in Ben's network. Any reply will be sent back to that router's public address, that everyone knows how to route to. Once the router receives a reply, it'll look up in his table where the request actually came from and rewrites the destination address for that packet back to the original local IP. Now it can route that reply packet via the local network, and sends it to that local address. This allows one to use a big number of local addresses but still access the internet, even though the ISP gave you only one single publicly routable IP address. As Ben mentioned, IPv4 address space is limited (to about 4 Billion addresses), so NAT is an essential crutch to keep IPv4 working today, since that address space would otherwise be to small to address every device on the planet.

[@javabeanz8549]

@@Hugh_I not bad, a little bit simple, since the NAT typically ends up rewriting the port number as well as the IP address, but that's a minor point compared to the rest of your explanation.

[@kaitlyn__L]

@@Hugh_I my favourite is when there’s two or three layers of NAT, so port-forwarding breaks because it’s only aware of its own internal NAT table, and not the layers beyond itself.

[@waldolemmer]

I've finally watched all of your videos from beginning to end.

[@rymaples]

I feel more ignorant every time I watch one of his videos.

[@HandyFox333]

Same here.

[@rainbowbunchie8237]

Dig your feet in and don't give up.

[@opendstudio7141]

What is Facebook? The only one I remember is one from the movie ARMY of DARKNESS or Ash v.s. The Evil Dead. 👻🙃

[@ZeroUm_]

Haha, and this one is one of the most beginner-level videos he has around.

[@NoahK216]

@@ZeroUm_ I'm big into his breadboard PC series but not into networking. They're very different, this may be beginner but still wack for someone not familiar

[@yumnuska]

This is still one of my favourite videos, I think I’ve watched it three times now.

[@YandiBanyu]

My god, all this knowledge for free! Thank you Ben!

[@nathantron]

Great job explaining this by the way. It's extremely challenging to explain this in a simple way for everyone to grasp, but you manage to do it perfectly. Great work with the lab stuff too.

[@orchidejczyk]

plot twist: it was not simulated network, Ben just was recording while repairing facebook xddd he just needed few days to do montage : D also without recording he would repair it much faster! thanks for vid Ben!

[@anti7gn]

Ben took down Facebook just to create an explanation video.

[@jarred7721]

Damn this was really cool, I'm glad you took the time to explain this in such detail.

[@bassman87]

when you said AS number 7018 I immediately said AT&T. Ive peered with AT&T so many times now that their AS number is permanently engrained in my head. haha.

[@hariranormal5584]

I do know 174 is cogent and 6939 is HE, and 1299 is telia, too much interest on the hosting community so that's just stuck on my head xD

[@BobHolowenko]

​@@hariranormal5584 Years of working in carrier networks and looking at BGP tables has me reading these like maps...It's funny how numbers stick inside the human brain.

[@JonathanRockway]

I decided to look up who AS 1 is, and it's Level 3 these days. Ironically, they are not AS 3; that's MIT. They should broker a trade ;) (I used to work at 111 8th Ave, and Level 3 proudly had space on the 3rd floor. Always enjoyed that.)

[@minirop]

@@JonathanRockway then they move to England and everything breaks x)

[@meme-hj5rs]

@@minirop is it because of break-xit ?

[@alimibrahem8120]

Thank you so much Eng. Ben for this pro explination, I think you have to do another one for what happen yesterday.

[@mustafaark4762]

Again, a great video. Never disappoints man!

[@angelgutierrez8303]

Ben has a gift to make things simple! Should keep making videos..🎉

[@sophiemilton5939]

"Why was Facebook down for five hours?" A better question is why was ot ever up in the first place.

[@TheLamefeed]

Agreed, the time Facebook was down probably got families to actually talk with each other, and hopefully get a better relationship. I hope all these unsocial networks would cease to exist.

[@henrydorsett6076]

Technical question != interweb-social question But agreed in general :)

[@UncleKennysPlace]

@@TheLamefeed Actually, it made people crazy.

[@ELYESSS]

@@TheLamefeed back in my days we used sms instead of facetime

[@friendly0]

@@TheLamefeed pornhub usage increased massively 😂

[@doodmonkey]

I've been waiting for the technical analysis, ty

[@jaredteaches894]

For context, Ben Eater's expertise is actual computer networking. He speaks about his amazing journey on his podcast. He is absolutely brilliant and his story is inspiring.

[@mattym8]

Yeah, this is the subject he started out learning and his pro experience. The rest is just for fun.

[@williamd4883]

I know jack about IT and routing of traffic, but this video actually makes me think I have a glimpse of an idea of how some of this works. Thanks!

[@DaedalusYoung]

The fact that I missed the entire event says a lot about how much I value Facebook.

[@paulstelian97]

And WhatsApp and Instagram as well.

[@whynotanyting]

Only reason I knew about it is because my mom was flustered she couldn't reach me on Whatsapp... ugh

[@ornessarhithfaeron3576]

Only reason I know it is because I don't have the apps on my phone but I was checking if my professor answered my question in the private Facebook group we have for a lesson

[@MmeHyraelle]

My mom was crazy about accessing her cadle and resetted the internet modem because even if she understands its the first diagnostic step, she does not understand why and when to do it. You still had internet, mom. Just not those websites.

[@dieselgeezer18]

i didn't even notice it. My friends were talking about it and thats how i learnt about it

[@Fozzedout]

This was an excellent explaination of DNS and BGP and what occurred 👍👍👍

[@sim_only]

The level of intelligence, understanding and willingness to educate shown by Mr. Eater is genuinely off the scale. He should get into robotics (undoubtedly the next big thing) and change the world.

[@tgriff007]

This is how you do explainer videos. Terrific job.

[@deathsheir2035]

I learn a lot from your videos. May have to watch videos multiple times to get a firmer grasp on what you're saying, but my god have I learned a lot. Also, I don't think Facebook is telling the truth about why it was down for 5 hours. Then again, I never fully trusted facebook from the beginning.

[@WobblycogsUk]

I've been meaning to get around to trying to understand the basics of BGP for ages and now I do, thanks.

[@__8120]

"So I'm in the console on one of AT&T's routers that I just happen to have access to" I love that he just casually glosses over that and none of us even bat an eye. "Yeah ofc he has access to it, why wouldn't he, nothing to see here"

[@caleballen4721]

You know a person is a good teacher if they can communicate clearly about DNS

[@waldolemmer]

I like the nice illustrations And your way of teaching, I always learn so much from your videos :)

[@funkykong9001]

Thanks!

[@thuokagiri5550]

@beneater your grasp of content doesn't cease to amaze me

[@inspirated]

Thousands of outlets and channels covering the Facebook outage: zero knowledge and a clusterfuck of buzzwords. Ben Eater covers the outage and ends up creating a video that (informally) explains DNS & BGP better than almost everyone else out there. This is in fact a perfect video now to show to someone who asks "how does the internet work".

[@EdgarStgo]

Hats off @Ben Eater, you are a gifted man, the way you flow explaining all this is so smooth. I used to work as a Network Engineer but reached up to OSPF from the client-side, not ISP. I just learned a ton of useful aspects about BGP and DNS from the ISP Internet perspective. Cheers!

[@Levy1111]

Wonderful explanation, thank you. Could you in some future video show how did you prepare mock networks?

[@MrRahulKumarKandula]

Woww. Mind blown. Awesome explanation. Wish you could do these deep dives into current network related events. Thanks a billion for this video.

[@foxonrails4140]

This guy went from looking at a routing table in his LAN to a BGP table on Tier 1 ISP's network within 10 seconds. Damn.

[@ShainAndrews]

Nope...

[@b00lean]

Great job explaining! Thanks from Switzerland!

[@samiurkhan]

If I were to guess, they have something roughly inspired by OpenFlow SDN internally to propogate config changes. Configs store things like DOS protection thresholds, routes to key services (likely algorithmically generated and pushed based on an understanding of machines and distances between them). I suspect a config change they are talking about damaged the route generation.

[@myname377]

nohh. they just disconneced Mark Zuckerberg 10 years old laptop :)

[@samiurkhan]

@@myname377 fb hire this man right now

[@nickwallette6201]

LOL -- yup. The SDN controller was running on Phil's workstation, because that's the IP address it was configured to use when they were first configuring the dev POC. And unbeknownst to anyone, including Phil, it never got changed. So, when Phil shut down his workstation to do an OS update, just two minutes before a scheduled schema replication, the entire controller cluster went offline and started dropping routes. You know... or something like that. :-) If I've seen it once, I've seen it a thousand times.

[@kaitlyn__L]

@@nickwallette6201 damn you’ve genuinely seen that before? That’s depressing. I’ve had stuff like that happen with my NAS and various desktops, but to imagine professionals in a (public) company doing it… actually no, that makes perfect sense. The feet tend toward the well-trodden path and all that.

[@lightblue254]

Did not notice this was 30 minutes, wow, you really made it so easy to understand, thank you :)

[@danieladelodun9547]

14:20 but do they know you have access? 🤔 imagine getting rce on an at&t router just make a youtube explainer

[@rebmcr]

He could have a read-only account for auditing security, nothing he did on the live box needed rce.

[@BeepBeep2_]

You condensed the most important bits of my whole bachelor's degree into 30 minutes. Bravo.

[@codingstudent3163]

You are Inspiring, You are my Knowledge Angle, Bless me. Praneeth from India.

[@khealer]

Angle?

[@wol2231]

@@khealer I think they meant "angel".

[@jamesinonati7608]

@@khealer Don't be surprised if tomorrow there is a temple dedicated to Ben in India.

[@clonkex]

"knowledge angle" ahahaha

[@pallab-sharif-omar]

... You compiled this complex networking topic to an understandable pattern. You're magician indeed.

[@hansdietrich83]

So just like anything else in CS, the internet works on abstraction. It's a bunch of small networks, grouped into networks that are grouped into networks, ... Until you get to the ISP Level

[@Ceddi4199]

Thank you very much for your video. I learned a lot about the DNS process. Greetings from Germany!

[@ochanlee4414]

Bridging Gap Protocol :) Basically the reason for the outage was a human error that was propagated to >1000s of Facebook's backbone BGP routers through the massive use of network automation and management from a single pane of glass! It goes to show that automation doesn't get rid of human error but lets it propagate at a large scale...Don't forget the basics and think thrice before you hit that push button :) Excellent BGP and DNS overview Ben!

[@MrAntiKnowledge]

holy shit, that's much more in depth explaination that I got in Uni. The prof gave the impression, that it's just a single request and answer type deal. At no point did he discuss that the first server will only point you in the right direction, but not directy to your destination.

[@DanielCharry1025]

This is 1M subs bait, and I'm loving it!

[@bjrnsindrehydalsvik7750]

i ve seen alot of network courses and i ve not got so much clear cut information a actually understand so fast. Do more of this please!! :D

[@yokmp1]

I saw the thumbnail and thought 'yes, lets get that LAN built' Still was an informative Video even if like every major Tech-channel made one of these over the last days.

[@smitias_8474]

Came for speculation about Facebook outage but instead found outstanding explanation on topic I was always curious about yet was too afraid to touch at all. Much respect.

[@KarenTookTheKids364]

This was way different to Alex Jones explanation 🤣

[@KarenTookTheKids364]

@Toy Bingo

[@nickwallette6201]

Huh. You're right. I didn't hear Ben use the phrases "liberals" or "agenda" even once.

[@Mattboy300]

I have to say that the way you explain things in all of your videos by showing all the steps and distilling things into simple to understand chunks is really fantastic. If you were my college professor, you'd be my favorite.

[@SamBebbington]

I’m baffled, how have you managed to get such an incredible video up so quickly? Does this mean we’re going to start getting weekly videos?

[@darer13]

probably not. i think his discord/patreons help him with videos and it helps to have one video at the beginning of every month

[@grzegorzkowalski1507]

I love how you just casually answered my questions about the internet that have been absolutely boggling me my whole life and that I didn't know how to ask for

[@nicolascuervo9910]

Hey Ben, Here in Australia today there was a nation wide outage for a major ISP and the outage lasted 9 hours. It seems that bgp was the culprit. It would be great if you could explain to us how, because in this case no dns was involved