Cloud Tip: CloudSwitch Networking and ESX

Posted by Damon Miller on Apr 02, 2010 at 10:05 AM

• Digg
• Reddit
• Delicious
• StumbleUpon
• Facebook
• Twitter
• LinkedIn
• Buzz

By Damon Miller

As Director of Technical Field Operations for CloudSwitch, I have the pleasure of interacting with most of our customers on a regular basis.  This is an integral part of my job since I’m ultimately responsible for making sure we provide a positive experience every time someone uses our product.  The past month has provided a fantastic opportunity to do just this, as we opened our beta program to the public on March 1^st.  We’ve learned a tremendous amount from our customers so I thought I’d take a moment to share some of that information to help everyone interested in using CloudSwitch.

First, there are some basic things to remember before using CloudSwitch to move servers to the cloud: 

1. Applications need to be virtualized before they can be moved into the cloud, and you need a virtual environment in your datacenter for deploying the CloudSwitch Appliance.
2. The virtual machines that you move to the cloud need to come from a supported version of VMware.  Today, this list includes ESX 3.5 & 4.0, VirtualCenter 2.5, and vSphere 4.0.  We can also access servers directly through a CIFS share.
3. When running the CloudSwitch Appliance in an ESX environment, specific networking options are required for connectivity to servers running in the cloud.

The last item—ESX network configuration—has caused confusion for a few of our customers, so I thought I would also take some time to explain what specific settings we require and why they are needed.  First, though, it makes sense to provide a very quick description of how the CloudSwitch solution works.

After deploying the CloudSwitch Appliance (CSA) into a VMware environment, users select internal VMs to be migrated to a target cloud.  I’ll leave the full architectural overview for a later article, but suffice it to say that the selected VMs are then migrated securely into the cloud—encrypted in transit and at rest.  Once the migration is completed, the CSA starts the selected servers within our Cloud Isolation Technology.  This technology allows unmodified VMs to run in the cloud provider of your choice and provides secure, layer 2 connectivity to the datacenter.  A layer-2 connection—or a network bridge—means that users don’t need to change any network configuration data such as IP addresses and netmasks, since this is transparent when connectivity is established at the data link layer.

After the migration, one of two things will happen:  network connectivity will work exactly as it always has or the migrated VM will appear to be down (from a network perspective).  As you might imagine, our goal is the former.  However, in some cases VMware’s network configuration can interrupt the flow of CloudSwitch’s network traffic.  Specifically, there are two Virtual Switch Policy settings which must be enabled in order for CloudSwitch to provide layer 2 connectivity.  These are:

• Promiscuous Mode
• Forged Transmits

So, what do these settings actually mean?

Let’s start with Promiscuous Mode.  The name Promiscuous Mode is actually a bit confusing, though if you’re familiar with the related network interface option this will kind of make sense.  When Promiscuous Mode is enabled, ESX will send all traffic to nodes whose interfaces are placed into promiscuous mode.  Ultimately, enabling Promiscuous Mode on a virtual switch makes that switch function kind of like a network hub.  VMware has essentially provided a way to see all traffic on a given Virtual Switch or Port Group.

Ok, but why does CloudSwitch need this?  The quick answer is because without this option enabled, the CSA won’t receive traffic intended for servers it has migrated into the cloud.  If ESX’s virtual switch functioned just like a physical switch this would not be a problem.  Physical switches build up a list of MAC addresses connected to them, effectively “learning” which network interfaces are where.  Without this intelligence, a node would never be able to communicate across more than one switch.  However, ESX assumes that the VMs attached to its virtual switches are always the final destination.  In CloudSwitch’s case, this is not true:  There are other servers “behind” the CSA—cloud-based servers launched by customers.

If enabling Promiscuous Mode was acceptable in all environments, we would be in great shape.  However, some of our customers have told us that they don’t feel comfortable with this configuration for their virtual switches.  We completely understand their position, because we don’t want that extra traffic either!  We’re only interested in traffic for the servers we’ve migrated into the cloud.  Thankfully there is a quick and simple solution.  The CloudSwitch Appliance can be configured with its own Port Group and Promiscuous Mode can be enabled only for this port group.  In this configuration, the CSA will never receive traffic from any other node on its switch since Promiscuous Mode applies only to itself.

There’s one more setting we need to discuss, and that’s Forged Transmits.  With this option enabled, VMware allows virtual machines to generate traffic using a source address that is different from the network interface connected to the virtual switch.  CloudSwitch isn’t actually “forging” anything; rather, we are delivering the traffic generated by a customer’s cloud-based virtual machines into the datacenter across the secure tunnel we establish using the CSA.  All physical switches support this behavior by default because, again, without it no network consisting of more than one switch would function.  To provide traffic from migrated virtual machines to customer datacenters, CloudSwitch needs the CSA’s virtual switch to function more like a physical switch and allow this traffic to flow correctly.  Enabling ESX’s Forged Transmits option on the CSA’s virtual switch or port group makes this happen.

Both of these settings can be updated from the Virtual Infrastructure client. The process for enabling Promiscuous Mode and Forged Transmits is described in a knowledge base article available from VMware.  Note that these settings can be applied either to an entire Virtual Switch or to a Port Group.  Port Group settings override Virtual Switch settings so if you do change them on your Virtual Switch and you still don’t see the expected traffic, be sure to check the Port Group settings:  They may be overriding your switch-level changes.

CloudSwitch’s goal is to provide a simple, secure way to use cloud-based resources without requiring any change to applications or operating systems.  A few basic requirements such as a virtual environment are required to make this possible.  Additionally, when deploying the CloudSwitch Appliance under VMware ESX, two specific network configuration options are required.  Hopefully this article will help users understand what’s needed to get started, and thereby ensure a smooth, successful product experience.

About CloudSwitch

CloudSwitch delivers the enterprise gateway to the cloud. CloudSwitch’s innovative software appliance enables enterprises to run their applications in the right cloud computing environment—securely, simply and without changes.

Network with CloudSwitch

• What We Do
• See a Demo