OpenStack Neutron Packet Walkthrough (DVR)

OpenStack Neutron Packet Walkthrough (DVR).
Twitter: @davidmahler
LinkedIn: / davidmahler
Links:
Me: / davidmahler
OpenStack Documentation: docs.openstack.org
RDO: rdoproject.org
Assaf Muller's Blog: assafmuller.com
Introduction to Neutron: • Introduction to OpenSt...
Linux Network Namespaces: • Introduction to Linux ...
Introduction to VXLAN: • Introduction to Cloud ...
Introduction to Open vSwitch: • Introduction to Open v...
Introduction to OpenFlow: • Introduction to OpenFlow
Some commands used:
ip info
ip a
see routing table
ip route
see policy routing rules
ip rule list
see a particular routing table
ip route show table (table id)
list linux bridges
brctl show
packet capture, disable resolution, show mac/L2 info, filter on icmp
tcpdump -i (interface) -n -e icmp
list iptables rules
iptables -S
iptables rules for the nat table only
iptables -S -t nat
see open flow rules for a particular ovs instance and flow table
ovs-ofctl dump-flows (ovs instance name) table=(table number)
watch open flow rules counters move
watch -n.5 "ovs-ofctl dump-flows (ovs instance name) table=(table number) | grep -v n_packets=0"
see fowarding db of ovs
ovs-appctl fdb/show (ovs instance name)
see ovs instances and their ports
ovs-vsctl show
see open flow port numbers
ovs-ofctl show (ovs instance name)
list network namespaces
ip netns
go to a bash shell for a particular network namespace
ip netns exec (namespace id) bash
list neutron routers
neutron router-list
see where a neutron router is hosted
neutron l3-agent-list-hosting-router (router name)
status of proxy arp
cat /proc/sys/net/ipv4/conf/(interface name)/proxy_arp

[@RedPoisonDartFrog]

This is a fantastic video, I have watched it numerous times. The first couple of times, I could not finish it because I kept stopping to do further reading on some aspect that I didn't understand. There is a LOT of information compressed into a relatively short amount of time here: OVS commands, reading flow entries, Linux network namespaces, IP Tables, tcpdump, etc. If you can complete this entire video and understand every CLI command, and especially if you can understand reading and traversing OVS flow tables, you will have gone a long way towards building a solid foundation in understanding Neutron.

[@DavidMahler]

Thanks for the great feedback/info about the video! This video took a crazy long time to make for the same reason. Def more advanced, compressed and faster than my earlier videos like intros to some of the topics you mention (network namespaces, OpenFlow, OVS, etc.)

[@mikescott3355]

I actually replicated your environment and followed along (although I used KVM running Ubuntu images as hosts, and used DevStack instead of PackStack/RDO. I found DevStack to be easier to install, but that's just me. Great learning experience!

[@abhilashmalik1030]

David Mahler, Without any exaggeration. This is really a very very helpful video in learning the n/w concepts of openstack. You explained every single detail meticulously. Thanks a ton for this !!

[@DavidMahler]

You are so welcome, thanks for taking the time out of your day to comment!!!

[@pstoianov]

I have been at official OpenStack Fondation summits which do not have such presentation (shame). It's crucial in such environment to understand each flow and your video shows exactly this. I'm sure many subscribed to your channel because of determination you showed. Keep doing such great work, I know it's not easy to do... Cheers,

[@DavidMahler]

Thanks so much! Yes - I'm not comfortable with any networking technology if I don't see how packets flow through them and what mechanisms are used to control/direct/manipulate flows. So I try to present that info. I owe a lot to Asaff Muller's blog in breaking a lot of it down for me. True words, not easy to do! ;-).

[@dinaahmed6014]

Totally agree with you

[@bmachkour]

Fantastic, very good job David !

[@DavidMahler]

Many thanks!

[@rmmhn]

This is gold, thanks for breaking down, again certainly cannot watch the entire video and understand in one shot !

[@DavidMahler]

Thanks!

[@comsgn]

Thank you for this video. It explains openstack networking much better than the official documentation.

[@DavidMahler]

Thanks for the comment!

[@theconfusedchannel6365]

Wow this is the best video I have seen on packet walk through. Very detailed and crisp explanation. Thanks a lot.

[@DavidMahler]

Thanks for watching!

[@anilmay18]

Very clear illustration of various packet flows. Good work David, I learnt a lot. Thanks.

[@DavidMahler]

Great! Thank You!

[@clikcspeed]

Thank you for the great videos - I have been hooked over the last 2 weeks. Your explanations are very detailed and clear. Thanks again.

[@DavidMahler]

You're welcome. That's great!

[@debaprasaddas610]

Excellent one on DVR, with very clear illustration.

[@DavidMahler]

Thanks!

[@oberix]

Great presentation. Looking forward for similar presentation with OVN :) You are talented to quickly explain networking stuff. I finally know how networking works in OpenStack :)

[@DavidMahler]

thanks!

[@akashpatwal8647]

This was one of the brilliant explanations I have come across.

[@DavidMahler]

Thanks Akash!

[@bandisrinivas1]

Amazing training video on DVR. The explanation is crystal clear with detailed demo. Thanks a lot.

[@DavidMahler]

You're quite welcome! Thanks so much!

[@vashuse]

Very detailed video and very great channel. These are outstanding videos that not everyone able to produce, I hope we can see more educational material like that. Ps: Please make a full video course!

[@DavidMahler]

Thank you very much!

[@thechahal]

Your videos are great so thank you very much for producing them. There is so much detail in this particular one that I will be re-visiting this video as I practice and get up to speed. I really like the diagrams and the detailed step my step walk through. Thanks!

[@DavidMahler]

You're welcome. That's great, I hope it helps!

[@SarfarazAhmad89]

amazing resources ! keep up the good work !

[@dinaahmed6014]

Thaks alot for your explanation which is simple and clear

[@DavidMahler]

You are welcome

[@chpvnkumar]

Thanks for making this video and for providing very detailed inside view.

[@DavidMahler]

You're welcome!

[@vipintripathi745]

You are great sir , Excellent video all of them

[@DavidMahler]

Thanks!

[@shashidhares5653]

Thanks a lot, David...Well explained..!! I can't get better than this...:) too good ..!!

[@DavidMahler]

Thanks!

[@MrFrankSpierings]

Thank you David for explaining this. It saves me a lot of reading time!

[@DavidMahler]

I don't like reading either :-). Thanks for commenting!

[@lscala2008]

You do good to the world. Congratulations for such a great explanation.

[@DavidMahler]

Thanks!!!!

[@jasontle]

Great video presentation. Thanks!!! My only comment is when you walk through the cli commands, it helpful that you show related high level diagram on a side so we know exactly which stage you are referring too

[@DavidMahler]

Thanks, that's helpful feedback. I thought of that and guess got worried it would distract, but I think you are right, it would be easier to follow with the side diagram. I'll think about doing that in my next video on git/version control.

[@avinashhegde5769]

very usefull for understanding packet flow in OPenstack env. across differnt VLAN/VM's.

[@DavidMahler]

Thanks!!

[@edvinhe2603]

Clear, Concrete & Cool. Best DVR video!

[@DavidMahler]

Awesome, thanks for commenting and watching!

[@AntoniosPikoulas]

Dear David, great tutorial, thanks so much.

[@DavidMahler]

Thanks, Anthony!!

[@ankurchamoli]

thanks again for another great video... to the point, quick and lots of information...

[@DavidMahler]

You're welcome, thanks for supporting by commenting!

[@miladjahandideh]

Great and full of detail video. Tnx

[@DavidMahler]

Glad you liked it

[@philiphills2811]

Very good, clear and concise.

[@mvpnitesh430]

Thanks a ton David. Awesome video, we have to appreciate for your patience and for your effort in making such a beautiful video.Thank you once again

[@DavidMahler]

Thanks, Parimala! This one did have quite a bit more work involved than my other videos .... and those already have an extensive amount of work behind them. You're very welcome!

[@mohamedhabas7391]

Thank you

[@DavidMahler]

yw!

[@olimpolimp2860]

Thank you a lot for such a great content. Hope to see more of them

[@DavidMahler]

Thanks!

[@bamburaja]

Great video. Thanks David.

[@DavidMahler]

Thank you!

[@selmison]

Excellent Video!!! Keep the good work!!! Thanks!

[@DavidMahler]

You're welcome!

[@chechobarbery]

Great explanation!!!!

[@DavidMahler]

Thanks!

[@dhanapala375]

This is a great video David and helps to have better understanding.... It would be great if you can add one more with basic's of how to create setup separate controller vm and compute vm's with virtual box with devstack.

[@DavidMahler]

Thanks a lot for the comment Dhana! To be honest I need a neutron time out ;-). Next one will probably be on version control.

[@omkartelee1]

A great video... cleared my concepts throughly... Thanks a lot..

[@DavidMahler]

You're welcome!! Thanks for the comment!

[@foodieustad]

David Mahler great explanation can you also make a video on mirroring traffic in openstack for same and diffrent subnet also multi region deployment. Thanks

[@Little18Boy]

Great great video. Thanks for sharing such a wonderful explanation on subject. I have been watching all the great posts of yours and follow that , this particular has lot of information in one , had to play several times to digest. Hope to see more and may be something on frame sizes and Ethernet header and overheads in ovs and kernel.

[@DavidMahler]

I need to watch it myself over as I forget some of the things I said :-). Yes, this one is way denser than any of my other videos. Thanks for commenting! That's a good point on frame size issues, thanks.

[@matiasmoglia]

Awesame videos David. Very very Thanks!!!

[@DavidMahler]

You're welcome!

[@lordparth]

very clear explanation ! well done

[@DavidMahler]

Thanks!

[@robertobelluco5413]

Excellent video ! Thanks!

[@DavidMahler]

You're welcome!

[@nagarjunam3584]

Awesome vedio. Need some time and have to watch for understand the concept.

[@DavidMahler]

Thanks!

[@balachandarraju]

Great Explanation

[@DavidMahler]

Thanks!

[@AlexeyStrelchuk]

Great video indeed! I also like picking commands that I wasn't aware of from it ) For OS+ODL integration you might want to ckeck for OPNFV

[@DavidMahler]

Thanks Alexey. I'll check it out, thanks for the info!

[@srinirengasamy5706]

Great video... thanks for sharing.

[@DavidMahler]

You're welcome!

[@casswang9402]

Thanks for your share! excellent video!

[@DavidMahler]

Thanks for commenting! You're welcome!!

[@abhikgupta7854]

Great work, David. I love your meticulous way of explaining. Would it be possible for you to detail your lab setup?

[@DavidMahler]

Thanks a lot! Ya I get the reason for that, I'll think about a separate lab set up video for something like this in the future.

[@yusranad]

i loved this presentation hopely next video you make new home lab tutorial for self network service openstack thank you very much

[@DavidMahler]

Thank You Usran!

[@davidgabriel1425]

Dear David, I appreciate very well your presentations and I learned so much from you. I have a question/problem related to creating two ubuntu instances and setting an ovs bridge in another VM. Here is the problem description: I have defined two instances called VM1 and VM2 and ovs bridge, each one of them is deployed in one Virtual Machine (VM) based on this simple topology: VM1 ---LAN1----OVS---LAN2--- VM2 I used the following commands, taken from some tutorial, for OVS: ovs-vsctl add-br mybridge1 ifconfig mybridge1 up ovs-vsctl add-port eth1 mybridge1 //eth0 is dedicated for Internet ifconfig eth1 0 ovs-vsctl add-port eth1 mybridge1 ovs-vsctl set-controller mybridge tcp:AddressOfController:6633 Then I tried to make the ping between the two VMs but it fails ! Could you please tell me how to fix this problem. Thanks in advance. Regards

[@MrTheAlexy]

why table 20 sets the tunnel to 0x49 but port 5 display 0a10C. Should not they match?

[@VinothKumar15]

Your videos very useful. Could you add openstack load balancer related video

[@DavidMahler]

Thanks!

[@karthikeyanmani4857]

Great tutorial, thank u so much :-)

[@DavidMahler]

You're very welcome!

[@rj511rocks3]

fantasticccc

[@DavidMahler]

Thanks!

[@---tr9qg]

🔥🔥🔥🔥🔥😳

[@jeffjia106]

you setup a three node openstack cluster by Virtualbox? VB doesn't support Nested Virtualization on my thinkpad x390 laptop, how you did it?

[@Neeseius]

In a test setup I did I completely removed neutron from the controller node, only the 2 compute nodes had networking. If I give the instances a fixed IP on the flat layer 2 external network, they can communicate north south. In my case I did not need a controller vswitch. Why is that? Not sure if it makes a difference but I did the native openvswitch firewall driver, completely eliminating the need for linux bridge. It's a tap on the VM and a tap on the openvswitch integration bridge. If you can' tell I am after simplicity. EDIT: I noticed one of my compute nodes contains the SNAT namespace, so perhaps one of the compute nodes assumed the role of the network node. But I still wonder if SNAT is needed if you give VMs fixed IPs on the external network. It would make sense they wouldn't need that anyway.

[@DavidMahler]

That makes sense to me - if you deploy directly into an external provider network you don't need SNAT anymore. Then you lose the advantages of virtual network segmentation like VXLAN and virtual routers - you are depending on external infrastructure like VLANs and are directly exposing VMs to the "outside world" (outside of OpenStack). Thanks for the comment on the firewall driver! Things change fast!

[@amaraojiji]

When looking for ipv4 addresses you can use 'ip -4 a'. It would skip all interfaces without ipv4 and skip all ipv6 garbage.

[@DavidMahler]

That's great, thanks!!

[@RenatNurgaliyev]

Great videos, thank you! Do you play with any OpenFlow controllers like ONOS or ODL, and integrate them in OpenStack?

[@DavidMahler]

Hi, thanks! No I haven't worked with ODL + OS integration before.

[@wongking5769]

Hi David，This is a excellent video! I learned a lot from it. And May you share the slide file the video showed?

[@DavidMahler]

Thank you!

[@suryanayani8319]

Hi David, Thanks for the detailed explanation. I got 2 doubts here. 1) When a packet is sent between 2 VM's in same network and residing in same compute node, how does the integration brigde(br-int) forward the packet. How br-int knows whether the VM is on the same compute host or on a different host 2) When a packet is forwarded from br-int to br-tun over patch ports what about the local vlan id's. I cannot see any vlan tags present on the patch ports in ovs-vsctl show output

[@DavidMahler]

Hey Surya - on same node it's just normal L2 forwarding like any switch works. For the patch port I believe it is just like a dumb wire, no vlan specification/pruning/etc.

[@doniparthiravikiran]

may i know where this VLAN numbers are defined(range)?

[@martin0google]

Hi David , Very Nice video for a fresher like to start with Open Stack Neutron Part , Everything is described precisely on this video . However you mentioned that you are running 3 separate CentOS to deploy open stack components on three diff VM's and provided the connectivity for the same . could you please help me by providing the OVF for those VM's ? Or is there a way where i can typically install the same as you use in the lab ?

[@DavidMahler]

Yes this an issue. I don't cover much deep dive of my setups in my videos, in the future I need to have a gist or something up that gives more details for those that want it on the setup.

[@feedbrain6257]

I am trying to follow at my speed:) Thx David. So Linux Bridge is for Security and OVS is for Tunneling? Why not implement Security to OVS?

[@DavidMahler]

Yes, I believe it was some incompatibility with IPTables and OVS, my knowledge has gotten outdated fast though - hooray for the pace of technology - I believe now with for example OVN this is not the case on the newest deploys. Requires more research on my part, I've been focused on network automation so slipped on Neutron a bit.

[@qwerty69600]

Another great video - thanks for posting. One question: the difference in readability between something like Cisco's IOS and Neutron is staggering. Neutron almost looks like the underlying mechanics are not meant to be read by humans at all, and instead parsed by some computer program. Is this the case? Secondly, if you were building an environment like this for real, is this how you would configure and verify everything is operating correctly?

[@DavidMahler]

Hey, thanks for the comment. Well this is open source vs. a proprietary/vendor system, both of which have pros and cons. Maybe an analogy is a playstation/xbox vs a self-built PC. The Playstation may be more plug-and-play while the self-built PC provides you a lot more control/options which gives you increased flexibility but also complexity. Just depends on what your goals are. Also I should note many folks (honestly myself included) spend much more time as operators on OpenStack - once the system is built you really just interact with Horizon API via CLI, GUI or software/scripting so you don't have to normally deal with all that complexity (until something goes wrong or needs to be changed in the underlying systems).

[@MukeshSingh-bo9bd]

Thanks David, just subscribed and sent you a request on LinkedIn. I will be deploying a muti-node Openstack cluster for VNF onbaording, a lot of network customization required in that.... Hope you will be there to help and answer few queries.

[@DavidMahler]

Thanks!

[@zufar_dhiyaulhaq]

Hi David, what is fixed IP mean? are fixed IP is instance associated with port created in the network?

[@mohamedshageaa]

Zufar Dhiyaulhaq fixed IP is when you create a port in a network and configure a specific IP to it using --fixed-ip option using command neutron port-create. Then while creating an instance, if the same port ID is mentioned, that accompanied IP to this port will be always attached to your instance as fixed IP.

[@zufar_dhiyaulhaq]

@@mohamedshageaa Hi are this is same like I create port in dashboard with spesific IP?

[@mohamedshageaa]

Zufar Dhiyaulhaq I am working on neuton release, where creating a network port via dashboard isn't an option. Maybe in later software releases you can have that option, idk.

[@zufar_dhiyaulhaq]

@@mohamedshageaa Hi thank you, because I am curious what the difference when creating port via dashboard (I assign static IP) and creating via CLI?

[@mohamedshageaa]

Zufar Dhiyaulhaq if it is an option in later openstack release, then both should be the same. Just please do not confuse it with floating IP assignment via dashboard. Best of luck.

[@enki5420]

r u mirantis employee ??

[@DavidMahler]

No

[@nursanto5758]

hello David. if using DVR, when all controller & neutron node goes down, is it distrub network in compoute node? and, is network in instances like east-west and north-south trafic still running normally? Thanks a lot.

[@DavidMahler]

I suspect east-west and north-south with floating IPs would still work (while anything new or removed wrt to virtual networking would fail), but you'd have to lab it up and double check.

[@biondigiorgiogmail]

Thanks for your time.. great video.

[@DavidMahler]

My pleasure!

[@viniciusrodrigues2180]

The vlan are locally significant, the use of these vlans doesn't impose a limit of 4096 tenants on each compute node? vxlan can have a very much larger number of tenants but what happens when the limit of local vlans on the compute are reached? Is there any way of using another technology locally?

[@FrankMartin111]

Hello dear David, amazing work ! I need your advice on what tools will I need as I am working on automatic custom network topology creation usind SDN and a RYU controller, the infrastructure I am working on uses DSL language for resource reservation and is mostly static, therefore I assume I have to work at a virtual level to do so, also is worth to mention that I am working with a real OFX environment with 4 virtual links and 4 virtual machine instances reserved, all the examples I can find so far are done in Mininet which is not helping. Thanks in advance !

[@yunakaof-usp]

Great Work !! Thanks for this video. :-)

[@DavidMahler]

yw!

[@varinderthalwal1826]

Thanks for sharing such an a amazing video. I tried to install a two node architecture where my controller node is a vm having single network interface with static ip (10.126.250.99) and the compute node is a physical server with single interface and static ip(10.126.246.141). The installation is complete and I can even launch the VM's but after launching the VM Dashboard is showing IP assigned to the VM where as the VM instance at its console is not showing any ip. After troubleshooting i found that the metadata server (169.254.169.254/2009-04-04/meta-data/instance-id) is not reachable. Question 1) I this all because my controller node and my compute node is in different domain i.e. (10.126.250.xxx and 10.126.246.xxx) or is it due to the single network interface? Also can you share videos regarding creating network, instance using dashboard. Question 2) Tried certain google attempts but could not find concrete answer for another question that " Can I have my controller node in one network domain and compute network node in another i.e. Controller ip 10.126.250.xxx and Compute ip. 10.126.246.141? The reason why I am asking this because I think if the node are having different IP domains then probably the openstack technique "Virtual Router" will not properly because the router can't be configured to route more the one external domains to the internal network domain of the Instance created and hence the vm instance will never get connected to the outside world i.e. available to host or other hosts. What is your comment ? Please help

[@varinderthalwal1826]

I am using self service networks with linux bridge

[@varunnair3174]

Hi David, thanks for the detailed video. I have just installed Openstack with the Linux Bridge mechanism driver instead of the OVS driver. Could you please try to make a similar video for the Linux bridge driver as well, or direct me to suitable links that discuss a similar topic. I do want to understand how neutron implements networking using Linux bridge, before moving on to OVS. Thanks!

[@DavidMahler]

Hi Varun, thanks for the message. I am probably staying away from OpenStack videos for a while.

[@TheShankarPrasad]

Awesome... Such a short video covering popular network scenarios. Your video is above and beyond the expectation. Thank you so much.

[@DavidMahler]

You're very welcome!

[@ranjeetbadhe446]

Excellent Video . Thanks to David Sir. I watched this video 14 times . There is enormous amount of technical information in this video. I get the topic here and go to Internet and dig more information. This became my ritual for nearly one month :) . Appreciate the efforts of David Sir.

[@DavidMahler]

Wow, that's great. Yes, this video is the densest one I have, probably by far. It certainly took a long time to create. I spent quite a bit of searching time myself to create it :-).

[@sahils5929]

As always a very nice video, thanks !! Just as a feedback, i felt you have rushed through this just a lil quicker than in your most videos :)

[@DavidMahler]

You're welcome Sahil! Also, thanks for the feedback, I'll definitely take that into consideration in future videos.

[@agelakas23]

excellent technical presentation/video! (...as always!) keep the good work!

[@DavidMahler]

Thanks!

[@gsvenkatesh81]

time for a new video david

[@DavidMahler]

Ha, true. I needed a long break after this one!