I’ll be the first to admit that while the title is good for some sensationalist attention, it certainly is a tall order!  I’ve had a fairly long, and occasionally tumultuous, history with vCloud Director going back to Lab Manager and my time at VCE.   It was always a casual relationship though, and never a primary focus for me.  As a result, I never had to dive much deeper than theory craft and whiteboard architectures and certainly never got heavily involved in implementation.  With that little bit of context imagine my surprise returning not only to the VMware ecosystem, but right into the deep end with a specific focus on vCD!  With one month at the mothership officially under my belt, I decided it was high time to burn the midnight oil and get completely crisp on vCDs most complex topic, the network model, so I can do a good job advising customers (knowing what you’re talking about, while not mandatory, tends to help!)

A cursory examination of the VMware blogosphere, for example a quick search of “vCD network model”, will make it immediately apparent that this is a very challenging topic for most customers. There is a ton of power and flexibility here, but also lots of moving parts and complexity.  When I first learned to swim, nearly 1 million years ago at summer camp, I was hurled directly in to 15 feet of water.  The pain and humiliation of that lesson stuck and I like to impose it on others whenever possible.  To the deep end we go!

At first glance this looks pretty complicated but… Well… Honestly this is pretty complicated, but it actually does make good logical and architectural sense if it is approached with the correct perspective.  Once you survive the dive into the deep end, it has always been my opinion that it is time to dry off, and stick a tentative toe in the water allllll the way back on the shallow side.  For our purposes that means a quick and slightly unstructured review of virtualization in general (just a touch) and what vCD abstraction was designed to provide.

As we all know (I can’t imagine anyone who doesn’t know having made it this far anyway), virtualization is meant to allow lots of discrete instances of an operating system, known as “guests” run concurrently and share a single set of hardware, known as the “host”.  That’s great and from the point VMware first released GSX server back in 01 through today has pretty much revolutionized x86 computing.  In the early says, most virtualization effort was focused on moving from a physical model, to a virtual one, with little additional transformation.  So a bunch of physical Windows and Linux servers running on dedicated storage, compute, memory and networking became a bunch of virtual Windows and Linux servers sharing a pool of physical resources via the dispatching agent known as the “hypervisor”.

As the industry matured, it become increasingly apparent that a basic migration from physical to virtual, with no change in process and tooling, leaves lots of value on the table and lots of potential untapped.  If infrastructure is going to become code, which is really what virtualization provides at its core, then all infrastructure should become code and all infrastructure should be managed as code.  This concept is what sits right at the core of all of todays modern trends and their associated buzzwords: Software Defined Datacenter (SDDC), Software Defined Networking (SDN), advanced management and orchestration, and all the flavors of cloud rainbow (public, private, and hybrid).

At the same time we have seen technology rapidly evolving towards an increasingly software defined model, we have seen the market dynamics of IT transforming in a dramatic way.  Pressure from the business units on IT has increased both internally through the forces of consumerization and the demands of emerging millenial workers, and externally as the commoditization of IT by cloud providers has challenged traditional IT economics.  In an attempt to adapt and survive, IT is faced with a need to evolve quickly from being a builder of core infrastructure to a broker of technology services.  This is a dramatic shift from a real in the weeds bottom up focus to a true, business value focused, top down view.

There is a great image produced by Felix Bodmer that does a fantastic job of capturing the aspirational end-state for next generation IT service management in a single graphic:

OK I promised we were headed to the shallow end but we’re back in 15 feet of water right?  Well this doesn’t count because its a different topic!  Without going into a lot of detail, what we can see above is how IT will likely be operating over the next 2-5 years and what we see here is a catalogue driven self service model where much of the IT services are delivered via a utility model.  Consumers can request resources via a portal and have these resources provisioned and allocated automatically.  Configurations are tracked and updated, the lifecycle is managed, the platform is monetized and all of it occurs programmatically freeing IT resources up to focus on business challenges and strategic architecture and design while the plumbing operates “lights out”.  This is the nirvana model that everyone is aspiring to and under this model IT essentially becomes a service provider.

So what was the point of that tangent?  This bit of context on industry direction is super important because it provides a really good understanding of where VMware has invested and why.  vCloud Director was the formal productization, and extension, of the Lab Manager framework.  Why did VMware do this?  The answer is clear when we look at the above and understand where IT is trending.  Lab Manager provided a set of tooling that added multi-tenancy, of a sort, to vSphere.  Any service provider model requires multi-tenancy as a foundation.  As great as vSphere is at virtualizing hardware and brokering resources, it assumes a pretty flat earth model when it comes to the consumption of those resources by customers.  vCloud Director changes this.

With vCD,  resources pools within vCenter are allocated into vCD pools by way of a construct known as the Provider vDC.   These vCD pools can be separated into classes of service, and can be subscribed by any number of tenants who within vCD parlance are referred to as Organizations.  For compute, memory and storage, this process is fairly straightforward.  Really fast storage and bigger vCPUs can be classified as “gold tier” resources while slower storage and smaller vCPUs can be classified as “bronze tier” resources or what have you.  Organizations make requests allocating resources and are presented options based on the entitlements that have been configured for them.  Consumption of resources is monetized via chargeback functionality and isolation, and tracking, occurs at an Organizational level providing the foundation for multi-tenancy.  Where things get more complicated, however, is in the networking model, which brings us full circle to the point of this entry. After a nice long journey we’ve arrived at the beginning!

Referring back to the first image, we can see some interesting things.  Some are familiar, and some not so much.  At the base of the stack is one of the familiar bits – the old reliable virtual switch.  As anyone with server virtualization background is aware, the physical network attachment(s) on the host(s) are made available to the guests by way of an abstraction layer know as a virtual switch.   The virtual switch provides virtual ports, which are arranged into groupings known as port groups and this virtual network is bridged to the physical one with which it is associated.  Virtual switches come in different flavors, most familiar at this point, but worth reviewing:

• vSwitch – standard virtual switch.  Associated with a single host, containing one or more NICs.  This is the standard vSphere networking construct and is the most basic.
• Distributed vSwitch (DVS) – an extension of the vSwitch, leveraging vCenter, which extends a vSwitch across multiple hosts in a cluster of hosts in order to allow for a common management and port domain within the cluster.  The DVS is the building block for any of VMwares emerging advanced capabilities
• VXLAN – a further extension of the DVS, covered in these pages, as well as many others.  VXLAN allows the distributed virtual switch construct to span across multiple L2 domains by way of L3 encapsulation and save you some VLAN ids by being externally VLAN independent across the VXLAN group.
• Nexus 1000V (and others) – in a pretty neat nod towards openness (or pressure from Cisco), the VMware virtual switch layer is extensible by third parties via the vNetwork Distributed Switch API developed jointly by VMware and, big surprise, Cisco.  One implementation of a third party virtual switch is the Cisco Nexus 1000V.  As the name implies, the 1000V is a piece of virtual infrastructure designed to emulate how Cisco physical Nexus infrastructure would work potentially making life easier for shops which want to preserve existing operational roles and responsibilities and keep the networking folks busy even in a virtual world.  The 1000V is quite different than the vSwitch or DVS and is a multiple component model including Virtual Ethernet Modules (VEMs) installed within each host, which represent virtual equivalents of a line card in the physical world, and one or more Virtual Supervisor Modules (VSMs) which run independently as a virtual appliance, and integrates with vCenter, functioning as the equivalent of the switch backplane and management interface.  The VSM runs NX-OS and can be managed like a Nexus switch.  If it is offline, VEMs still operate, but cannot be reconfigured.  It can be made highly available by being deployed in pairs.  These days the Nexus 1000V can participate in switch extension via VXLAN. OK thats a lot (too much?) on the 1kV.  Can you tell I used to architect Vblock configs?

Any of the virtual switch configuration options allow for the creation of port groups which map roughly to classic VLANs in the physical world in that they provide a mechanism for layer 2 isolation.  In addition, VLAN id’s (802.1q) can be assigned to port groups.  Obviously for members of these port groups to be able to communicate with the outside world by way of the physical NIC the VLAN configuration outside of the VMware environment must match up.   Another thing to make note of here is that normal networking rules aren’t suspended just because we are in a virtual environment  This is a recurring theme and good to remember.  If we took two different port groups, created a virtual machine and gave it a virtual NIC in each port group, and turned on IP forwarding on that guest OS, we would now have routing via that VM between the two layer 2 domains.  Similarly, if we have a single NIC upon which multiple port groups, and their corresponding VLAN ids, have been deployed, the physical switch port to which this single NIC is connected will need to have all of those VLAN ids defined and trunked so that the virtual machines in those VLANs can communicate out to the outside world.

So now we have a configured vSphere environment with some flavor of networking (virtual switch) and we want to introduce vCloud Director.  We know that the idea of vCloud director is to build on vSphere abstraction and make those pools of resources available for subscription by multiple tenants (Organizations).  On the networking side, vCD introduces some new constructs by which the tenant consumes the network resources made available by the vSphere host(s):

• Organization vDC – the Organization vDC is a VMware virtual datacenter allocated to the logical Organization entity.  One of more “org vDCs” can be provisioned per Organization.  Resources are allocated from the Provider into the org vDCs.  These resources (compute, network, storage) are then utilized by the VMs which will be provisioned into the Organization.
• Organization Network – Within an Org VDC, one or more networks can be defined.  These networks can either be external, in which case their configuration allows virtual machines outside the organization to communicate in.  Essentially connecting the Organization to a kind of backbone.  These external organization networks can be connected either direct, in which case they are L2 bridged, or routed, in which case the network is NAT routed to the world outside of the organization using the flavor of vShield Edge provided as part of vCD and connected to both the external provider network and the org network.  NAT definitions at this point can be made for both source (org network egress) or destination (org network ingress).  Organization networks can also be internal.  These networks are isolated, do not connect to the outside world, and are useful only for intra organization communication.  This can still be useful, however, for replicating isolated networking topologies from a legacy environment (secure networks for compliance, isolated test/dev, etc)
  
• vApp – OK ok, technically a vApp isn’t exclusively a vCD thing obviously, but the vApp really spreads it’s wings within vCD, so it is worth reviewing.  Put simply, a vApp is a collection of virtual machines for which a management association, and corresponding set of metadata, has been created.  vApps can also be nested, which is neat.  Once defined, they can be managed as a holistic unit, templatized and exported.
• vApp Network – this is where things get interesting and also confusing.  If a vApp is a collection of virtual machines that have some management and configuration relation, then it stands to reason that a vApp Network is the network configuration shared by these virtual machines.  vApp network definitions live in the metadata associated with the vApp in vCD and are both provisioned and torn down alongside the vApp entity.  Similar to Organization Networks, vApp networks can either connect directly to an Org Network (forming a common L2 domain), NAT routed (via a vCD vShield Edge endpoint attached both to the vApp network and the Org Network) or isolated (allowing only intra-vApp communication between VMs within the vApp).  Reasons you might want to consider utilizing the vApp network construct would include any case where a legacy application is moving, as a unit, and needs to being its network configuration with it into the cloud.  This means its IP and MAC address info must persist as a result of either application design issues, or broader management and monitoring requirements (compliance, security, etc)

The relationship between these constructs is captured in the diagram below (worth at least 800 words in this case).  Everything is represented with the exception of an isolated vApp network, but you can use your imagination easily enough for that one:

In Conclusion

vCloud Director adds lots of power and flexibility on top of vSphere and vCenter by introducing true multi-tenancy.  It also brings with it a fair bit of complexity, particularly on the network side, but things aren’t so bad if you consume them in bite sized chunks.  In the end, the existence of these constructs, and their relationship to each other, has logical reasons that do make sense.  If you evaluate enough use cases, you will certainly find yourself using all of these pieces eventually, although that doesn’t mean that you will use them all in every case.

Hope this has been helpful in taking some of the mystery out of vCD networking.  As always, feedback is welcome!