← Back to front page

Where does VXLAN belong?

A new year – new tricks. So all the major IT related news sites try to predict what’s upon us this year. I’ll will give my two cents on the topic. Based on what was shown to us in VMware World 2012 Barcelona, my guess would be on the next three areas making it big this year:

  • PODs, witch stands for pre-designed and pre-validated base data center configuration, such as Cisco’s flexpod with NetApp and Vblock with EMC , IBM BladeCenter Foundation for Cloud (smart cloud) etc
  • Flash & SSD based storage solutions (surprised to find out there are so many newcomers into the storage market addition to the big guns showing their newest options)
  • Software Defined Networks, Network Virtualization and VXLAN (Virtual eXtensible Local Area Network)

It would be fun to analyze all of these new fancy concepts & techniques but I decided to pic just one, so the rest of the blog I try to understand and explain my point of view about VXLAN and network virtualization by diving a little into the world of networking in virtualized environments. First of all, for most of us the concept is not clear what SDN and network virtualization is and is not, a good starting point to understanding these concepts is to read a nice short article by Bruce Davie here.

When we think about network virtualization in general, where does VXLAN belong in this picture? One of the design approaches building virtualized environments is that you would like to handle your physical network as a huge L2 resource. This is just fine when you are running a relatively small data centre, the bigger your environment gets the more headache you will have when extending your L2 domain. To overcome this issue new protocols has been implemented to generate an overlay on top of your current physical overlay. These protocols allows you to design your physical network the way you see best (read: as simple as possible) and the overlay will take care of distributing your L2 where ever you need it and more importantly only where it is really used. The competing (though not excluding each others) candidates to implement the overlay via tunneling are [STT](Stateless Transport Tunneling), [NVGRE|] (Generic Routing Encapsulation header for Network Virtualization) and VXLAN (Virtual eXtensible Local Area Network ). At the moment it seems that VXLAN has gained momentum thanks to Cisco and VMware’s implementation and inclusion to vSphere (vSphere 5.1).

Few words about a data centre network design, two basic design main concept has been competing agains each others for a little over decade (based on my recollection), a fat L2 design vs L3 to the edge design. Depending on vendor, you have been preached about the supremacy of one of the designs (or in some funny cases about both). So what VXLAN can provide you is a simpler, more manageable L2 physical network (ok, you better know Multicast really well or you’re entering into a world of pain… the kind of pain you get from debugging IGMP issues) because of the ability to separate of overlays (virtual vs. physical logical topology). How things have evolved lately It seems that for now a fat L2 design with L3 close to core is the way to go.

So now that I have a little understand of what you can accomplish with VXLAN, I was surprised to find out that I’m running out of VLANs and MAC addresses! Even worse was the situation with broadcast storms, those could bring down my whole network. At least this was the opinion of marketing people in VMworld. Skeptic in me started to think that can all this be true. First of all, I have not heard any of my colleagues to really run out of VLANs (I mean the 4k limit) nor MAC addresses. Any semi decent ToR switch can handle 32k MAC addresses, even excluding all the other devices requiring MACs, defining two interfaces per VM you would get around 15 000 VMs before you run out of MAC addresses per switch (and this is the case only if you have that flat L2 design extended throughout your whole virtual infrastructure). To me that is an insanely huge virtualization environment. The common sense in me would limit VMs & hosts per vCenter somewhere 2k VMs & 100 hosts (though I bet my estimates will grow semi annually) anyhow, I know silos are bad, but there are so many good things about keeping designs reasonable sized and in containers (what PODs are more than predefined resource containers anyway?). So obviously this is not the reason why would one implement VXLAN in their environments. So why then? Lets get back to the core features and definition of VXLAN, VXLAN is defined as a framework for overlaying virtualized Layer 2 networks over layer 3 networks. VXLAN thus enables you to move VMs all around your network without the limitation of the physical L2 design and if we forget the reality for a moment what this means is that in theory you could relocate VM even between data centers or different cloud provides! This is the feature that gets people excited (well, at least me). Of course the reality at the moment is far from this utopia of limitless mobility, but the future seems really promising (after you resolve few little issues like security and multicast traffic and VXLAN multicast groups synchronization between providers etc…).

When talking about software based networking features (tunneling, VPN etc) there are lots of discussions about overhead they introduce. This is my personal opinion, but I have not (yet) seen a virtualized environment which were completely CPU bound. So for me 5-10% (5% if you believe VMware sales persons) penalty for having the flexibility provided by VXLAN is practically nonexistent. Couldn’t find any 3rd party performance measurements on VXLAN with vSphere yet, but you can get an idea what kind of overhead you can expect from tunneling protocols from this article written by Jesse Gross, Ben Basler, Bruce Davie, and Andrew Lambeth. To generalize a little, I’m all for doing as much of network stuff on software (VPNs, Firewall, tunneling) and I’m happy to see that others are getting into this as well (VMware with their load balancer,FW and VPN combo, Cisco with their virtualized ASA for example).

In the end I would like to share few links about areas that I was not able to cover on this post… so if you who are really pumped about SDNs, not everybody are cheering for SDN at the moment, for example, check this article from Slashdot, a nice overview of virtualized networks can be found here and for a reminder that with current generation of virtualization environments, there are hardly any practical limits anymore between you and your dream data centre, check the latest configuration limits from VMware here – these are truly impressive!

Sincerely yours,


Matias Mäkinen works as a Data Centre Service Manager at Ambientia's Business Platform Services.

One Response to “Where does VXLAN belong?”

Please, leave us a message and we'll contact you.
You can also contact our Service Desk by phone +358 290 010 500 or email servicedesk@ambientia.fi.