Building a Data Center Inside Your Laptop - Computerphile

Build a virtual switching network inside your own computer. Dr Richard G Clegg of Queen Mary University London demonstrates building a software network configured with Python.
Richard's Github link: bit.ly/C_RichardGC-OpenFlow
/ computerphile
/ computer_phile
This video was filmed and edited by Sean Riley.
Computer Science at the University of Nottingham: bit.ly/nottscomputer
Computerphile is a sister project to Brady Haran's Numberphile. More at www.bradyharan.com

[@xetera]

Parallel universe Alec from Technology Connections explained this one really well

[@detrage5483]

Not enough dad jokes.

[@fjh89]

Yeah, now I can't unsee it hahaha...

[@JohnMiller-mmuldoor]

Back to the future 2 Alec

[@Jake28]

Computer science Alec

[@telecorpse1957]

Parallel universe? Pssssh... That's just magic of buying too of them!

[@rachana481]

i love computerphile videos but one recommendation would be if the videos that get mentioned get linked in the description box it would make finding them a lot easier!

[@jamie_ar]

Added to Watch Later but just wanted to comment that Richard has a really, really excellent way of articulating his thoughts that is really engaging and lovely to watch; great stuff.

[@richardclegg8027]

Thank you. That is very kind of you to say.

[@jamie_ar]

@@richardclegg8027 not a problem, thank you for sharing your knowledge.

[@gasparem16]

Splendid Job guys.... I am still amazed by the quality of free videos. Particularly this channel. Keep it up, u're awesome.

[@MiSt3300]

Jesus, this is soo useful! Thanks a million! Love from Poland!

[@etziowingeler3173]

Amazing video. Very interesting and informatively explained. Thank you very much 💕

[@simonebrazioli2206]

Man, that's absolutely awesome! I'll play with that right now!

[@richardclegg8027]

Hope the github repos helps you.

[@Pickyricky69420]

I wish this guy was my professor.

[@LLoydsensei]

very nicely explained, thank you!

[@oasdfe1691]

thank you richard!

[@ananthakrishnankrishnan7287]

It would be really helpful if you can link the related/relevant videos in the description so that newcomers to the channel like me can easily get up to speed

[@saadhassan4031]

Thank you definitely going to try this as a pet project.

[@richardclegg8027]

I hope instructions on the GitHub repos are clear enough.

[@friend7120]

I wish I had this video before doing my Network Architecture exam

[@katarjin]

SDN makes a bit more sense now,thanks.

[@TheSam1902]

It's really weird, but Dr Richard G Clegg looks a lot like the Technology Connection channel's presenter (though a bit older). Brilliant videos :D !

[@TCraats]

I thought the same thing! Couldn't really put my finger on it till I saw your comment, cheers!

[@twoboxtoofurious]

Amazing! I want to try this

[@cazino4]

Really cool. What would be fantastic is if this could be used with a bunch of lightweight, local virtual compute hosts (i.e via process isolation). Then imagine having multiple physical boxes chained together that you could seamlessly deploy this compute and network topology to, a simplified OpenStack if you will...

[@richardclegg8027]

I really wish there were a multi computer version of mininet.

[@TheGreatAtario]

This seems like the kind of thing that must have existed for decades in proprietary form deep in the bowels of networking gear manufacturers

[@stefanhennig]

probably being invented again and again by every new senior engineer.

[@50PullUps]

David Bombal made a lot of excellent videos on this several years ago.

[@richardtwyning]

Really amazing video, thank you 👍🏻 The power of Linux 👍🏻

[@richardclegg8027]

Yes. I love the things you can get it to do.

[@ronjon7942]

This is great, nice work. This is something I could get lost in, but a couple questions - sorry if they showcase my ignorance: can you build sdn versions of actual switches like a Cisco 2960 and Dell 6248? And, aside from obvious design, testing and verifying upcoming bare-metal changes, where would this be used in a production environment....? Would it mash up in a sophisticated vm environment? Does anyone have a high-level example of their use in a prod environment?

[@jacobscrackers98]

Really nice and well-explained video. Are you going to do one on Huawei's "New IP" and it's implications for the internet? I would like to see that.

[@richardclegg8027]

There is a lot of buzz about it for sure

[@jacobscrackers98]

@@richardclegg8027 You can say that again!

[@jacobscrackers98]

Well. "Buzz" sort of sounds like the reaction has been positive. I wouldn't say that exactly.

[@MalongaModeste]

Don't you have a full couse on these skils Richard, great teacher

[@rajkishoreprasad7553]

I want an discussion on what is inside static and dynamic library and what they do

[@theNeuo13]

the door behind him reminded me with my room in Melton Hall at the University of Nottingham. By the way GNS3 can do this with GUI.

[@thomasdaem]

Love the faces that this guy sometimes makes

[@MrRobsn89]

What might be a specific use case for this ? I wonder how I could benefit from that, from a practical stance ...

[@klfjoat]

It's getting DNS packets because, by default, Linux ping does a reverse lookup for IP addresses. Using -n will disable that feature. Knowing this is the difference between an academic researcher and a sysadmin. :-)

[@richardclegg8027]

Doh. Obvious in retrospect.

[@CirkusBolgen]

Why a link shortener for the git-repo?

[@boenrobot]

MiniNet seems nice as a library, but if you want a graphical network simulator that you can set up actual VMs with, GNS3 is the best IMO.

[@THeMin1000]

mininet is not a simulator, at its low end it actually uses the openvswitchs, that actually routes packets. You can use wireshark to capture packets flowing through it as if it were real, helps for actual testing too. GNS3 ofc have its use but the scalability of mininet and ease of use due to the cli along with it being compatible with so many remote controllers(pox, ryu to name a few) make it stand out. Its especially useful for research (how I came to be familiar with it)

[@boenrobot]

@@THeMin1000 You can capture packets in wireshark from GNS3 too, so I'm not entirely sure I follow what's the difference, beyond the fact you can script out your topology, rather than draw it (which admitedly, is a cool feature in it of itself), and it uses less memory due to the lack of a GUI.

[@richardclegg8027]

GNS3 is really something different though. It is mainly a simulator though you can patch real devices into it.

[@saranobutt]

All I know is my dad builds secret paths to secret files deep in his computer thinking that I won't find his creepy pictures in there.

[@sp10sn]

No, pumpkin, he trusts that you will only invade his privacy to a reasonable degree. It's often hard for fathers to imagine that his children would be snoops. Disappoint... 😭

[@mintudas5724]

So very interesting!!@ 😌❤😌❤😌❤😌😌❤😌❤❤😌❤😌❤😌❤❤❤🖤❤❤❤😌😌😌😌😌❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤❤😌😌

[@TheJackiMonster]

This is using network namespaces in the backend, isn't it? I have written a relative short bash script the last weeks to setup network namespaces in a dynamically scalable topology for NAT traversal tests of a decentralized networking framework. The connections I used were virtual ethernet links but there are a lot of options, so I assume mininet is also made in a similar way. It's a really convenient tool for networking test cases to be honest.

[@THeMin1000]

yep mininet uses implementations of openvSwtichs which internally uses the os level net namespace isolation to model . But the thing about SDN is not being tied to the switch developer to build you specific features that you want. You can now get an off the shelf L2 switch if it supports openflow and implement all the higher layer logic in a controller. Allows the switch to pass packets a line speed too. SDN is a part of a bigger whole tho, with SDN+NFV you can have really interesting stuff. ;)

[@richardclegg8027]

Yes you are right. I did not want to get into the underlying tech. Network namespaces is part open vswitch is part. Lovely assembly of tech in mininet

[@OliverUnderTheMoon]

Curious about fault tolerance... if this is ultimately used in a production system how is stability of the Python process ensured and what happens if it goes down, the virtual switches just drop the packets they cannot query?

[@richardclegg8027]

What I demo is for sure not what you would put in production. If you want insight into that search for (say) Google SDN Faucet which is their enterprise version.

[@richardclegg8027]

A commercial OpenFlow router/switch will usually have a "traditional" system built in so it has failover.

[@OliverUnderTheMoon]

@@richardclegg8027 that explains it. Thanks for following-up with answers!

[@MonsieurBiga]

When I saw the thumbnail I thought it was Alec from Technology Connections with an aging filter

[@Jake28]

Exactly

[@sundhaug92]

This is using network namespaces, isn't it?

[@richardclegg8027]

Yup. Honestly such a cool concept.

[@jacobscrackers98]

I have a few questions: a) What's the point of the controller? Why does there need to be an external controller writing the rules for them based on code? Why can't the switches just be programmed to deal with unknown packets on their own? If it's possible to make a controller do it, it should be possible to make the switches do it. Right? b) Do real data centers have a controller or is that just a convenience to make programming a test network easier? c) The internet does not have any controller controlling the whole network. Does it?

[@richardclegg8027]

The controller has a view of the whole network and can make central decisions based on complete information. The idea is to centralise decision making for a section of network. There is no central controller for the whole internet. The idea of a central controller for sub networks is really a new thing.

[@richardclegg8027]

And yes many real data centres run SDN in some form. It is hard to be sure the exact mix of SDN and traditional because companies don't really release that information.

[@jacobscrackers98]

@@richardclegg8027That answers my questions. Thank you.

[@psycho717]

No disrespect intended, but when I saw the image for this video on Reddit as I scroll down, I thought this guy was a villain of some sort LOL

[@HMan2828]

You already have QoS you can configure in most routers... Granted, it's not as intelligent as what the doc is talking about here, but if all you want is to make sure P2P traffic doesn't slow down HTTP or HTTPS, or that your VOIP and game data always goes through low latency, just setup some QoS queues in your router.

[@richardclegg8027]

Very true. I was actually thinking of examples where they can take different routes.

[@Thermophobe]

looks like an alternative to ns3

[@richardclegg8027]

Yes -- different purposes. I also use ns3. This though lets you write a real OpenFlow controller you could put in a production network.

[@Thermophobe]

@@richardclegg8027 indeed! great video and thanks for the insight

[@Fanaro]

Awesome. It's kind of unbelievable that it took so long for something like that to pop up, though. Decades of painful learning experiences in the networking classrooms.

[@richardclegg8027]

Very glad it has helped you.

[@Stierguy1]

It didn't really. It's just catching on again in a new form.

[@swilwerth]

This could be useful for microservices oriented software designs. Can it do load balancing?

[@richardclegg8027]

Sure. An open flow controller like floodlight has loadbalancer modules built in.

[@lindhe]

In most circumstances, I'd say there's a lot better things available by default in orchestration softwares like Kubernetes or Docker Swarm. Open Flow is pretty much an academic software AFAICT.

[@richardclegg8027]

@@lindhe Kubernetes and Docker Swarm are really useful and very widely deployed but do different things. They are as you say, for orchestration so it's an apples to oranges comparison. You need to ask what's the network technology is underlying the orchestration. OpenFlow saw big scale real world deployment, most famously in Google's Wide Area Network so really it is unfair to say it is "pretty much academic". But really OpenFlow is the predecessor to other SDN technologies and if you're running Kubernetes on, say Azure, EC2 or Google Cloud then there's some form of SDN gluing it together -- just that's behind the scenes. My target here was to find things that people could play that demonstrate SDN technology rather than to show how to (say) deploy an orchestration system in a production network (that's not at all my specialty).

[@Null-si2fy]

Damn... he looks older

[@velho6298]

Best audio configuration in unfurnished room....

[@BenState]

So this is similar to Pfsense?

[@richardclegg8027]

I am afraid I have not used that myself but I am pretty sure pfsense is for a single box on a network rather than overall network control.

[@BenState]

@@richardclegg8027 Ok thanks, great

[@crazzzik]

This beats Packet Tracer any time of the day

[@richardclegg8027]

Packet tracer has its uses but I don't enjoy using it as much and I don't find it as useful for research.

[@crazzzik]

@@richardclegg8027 that's fair. Just having flashbacks of doing my college midterms in packet tracer and loosing marks because I renamed switches in the wrong order. By the looks of it, mininet should be enough to teach most of the core networking concepts, minus cisco cli (having CCNA in mind).

[@richardclegg8027]

@@crazzzik Ah... you should try debugging packet tracer for fifty different students in the classroom and half of them have renamed their switches in slightly different ways just like you did. But then, if you set up your real network in that way it would also fail so I guess it trains you for real life deployment. :-)

[@crazzzik]

@@richardclegg8027 I'd rather face issues from a real network and maybe remember a quirk than loose marks because checklist is bugged out. But I get what you are saying. Networking is complex and things rarely go according to plan. Comes with territory.

[@wuuht]

You need asics right???

[@nupstar87]

Gsn 3 ?

[@richardclegg8027]

Or ns3 -- a few things in that "simulator that can grab input from real sources" space but this isn't a simulator.

[@nupstar87]

@@richardclegg8027 sure willtry, i need something to show firewall works basics in technical class

[@AndreaFlorio]

This is great academically, but in real production network this stuff never took over :(

[@richardclegg8027]

I think it depends where you are. Big data centre players are using SDN. It is hidden if you are a user (just like you never need to know what algorithm the routers are using). So if you're using Azure, EC2 or Google cloud it is there in the background but you never need know. If you are in a small company then yes you are probably just using a traditional set up.

[@daledavis768]

Stuff like p4 is pretty common place though.

[@richardclegg8027]

@@daledavis768 exactly. P4 is open flow the next generation. Even more powerful but same original idea.

[@Stierguy1]

when inet/loopback has been around for a decade and just now catches attention

[@richardclegg8027]

Yes, none of this is new but I think a few people don't know about it. Mininet is network namespaces under the hood and honestly I *love* network namespaces. Genuinely a very cool hack.

[@antonygabriel5489]

I am Brazilian and i am learning english

[@misophoniq]

That's one damn narrow room! Where's he at? In the closet?

[@richardclegg8027]

Haha... It is just the slightly odd position I place my desk so I can look out of the window.

[@sundhaug92]

@@richardclegg8027 I think the white background is also making the distance to the wall is a bit unclear

[@misophoniq]

@@richardclegg8027 Thanks for the reply. ;-) It looked to me like there's walls immediately to the left and right of the door and these extend to the wall where the camera is. :-D

[@richardclegg8027]

@@misophoniq @sundhaug92 I swear they are not keeping me a prisoner in a corridor. :-)

[@MrHasie]

Frames.

[@FrankReif]

Not sure the labourer analogy is cool these days. But a brain in a jar is a pitiful existence too.

[@BunnyBunder]

Damn Richard, did they move your desk in the toilet again ?

[@vtbn53]

Why is the video title misspelling centre?

[@kytzu2608]

11th!

[@mattsadventureswithart5764]

Five thousand, six hundred and forty first... Sounds ridiculous?

[@garryiglesias4074]

Pour les français qui se sont perdus: non, ce n'est pas Didier Gustin...

[@dxutube]

Why have you spelt "centre" the American way?

[@jbcallv]

First!!

[@aurorapaisley7453]

Last

[@BrianMcGuirkBMG]

Second

[@aminejafur7225]

Third

[@sleepycatguy3059]

lol no views but 7 likes

[@ajbdkla3652]

First

[@LarryTheRoleplayerTM]

Why.... Just why

[@busterdafydd3096]

don't think much of Dr. Clegg