Moving to the Cloud: 5-Part Series
Moving to the Cloud: The Road Ahead
Digg
Reddit
Delicious
StumbleUpon
Facebook
Twitter
LinkedIn
BuzzOver the past few posts I covered a number of key points to consider as you plan to move to the cloud. These issues are based on our experiences with many public clouds, as well as what we have learned from working with enterprises adopting the cloud.
I hope it’s clear that today’s clouds are powerful resources that can be used to rapidly develop and deploy applications; they provide on-demand resources and true value. The challenges I outlined in configuration, storage, networking, and management really come into play when you try to integrate the power of the cloud with your existing infrastructure and processes. These challenges are centered on the fact that the cloud is separate from the data center – a problem that hits home when you want to utilize existing applications and rely on your existing services and infrastructure.
We believe that this hybrid model, where companies can use the cloud as a flexible extension of their data center, is central to the adoption of cloud computing, and efficiently addressing these problems is essential for cloud deployments to succeed. The technology we have been developing at CloudSwitch is designed to bring this vision to life. Software from CloudSwitch can now integrate your existing infrastructure with the power of the cloud while preserving your applications, tools and infrastructure investments.
As we look forward to the evolution of cloud computing, I expect the cloud will continue to play a larger, more significant role in enterprise IT. Cloud providers have shown they can rapidly iterate and improve their offerings in response to customer input and have been drawing from their experiences to develop new and powerful infrastructure and features. It has been exciting to be part of this evolution so far, and we’re looking forward to the continuing innovation and expansion of cloud computing.
To end this series, I’d like to leave you with the key principles that guide our technology and product development at CloudSwitch:
- Provide end-to-end security between data centers and clouds to protect all data and storage
- Enable existing multi-tier applications to move to the cloud without modification
- Integrate cloud deployments into the existing data center’s management tools and processes
- Eliminate cloud lock-in so you can move between clouds or back to the center as needed
With these principles in place, it becomes possible to resolve or eliminate most of the challenges I’ve outlined in this series, making cloud a much more secure and viable option for the enterprise.
Moving to the Cloud: Managing your Environment
This post is part of a series examining the issues involved when moving applications between internal data centers and public clouds.
One of the advantages of cloud computing is that someone else is managing the infrastructure – including the servers, network devices and storage systems, not to mention the data center power conditioning, cooling and fire suppression equipment. One of the costs of offloading this infrastructure is that the cloud becomes something different and separate from your data center. In most deployments today, the cloud is almost completely isolated from your data center, and this often requires changes in how you manage and interact with your applications.
So what does “management” mean in this context? I look at it in terms of provisioning resources and managing the infrastructure, operating systems and applications. Over the years a remarkable set of tools and processes has been developed to handle these tasks in the data center, and the challenge now is how you integrate all this investment with the new cloud deployments.
For provisioning, the cloud has a model similar to a data center virtualized environment, in that you can provision virtual resources from a pool of physical resources. However, the definition of these resources is dictated by the cloud provider, which means you have to adjust your processes to account for the capabilities and limitations imposed by the cloud, specifically CPU, memory and storage resources. To be successful in the cloud, you need to match the resources required for your application to the capabilities of the cloud (i.e., pick enough CPU/memory/etc. to meet your application’s requirements while balancing the costs of the cloud resources). If you already have a provisioning system, you need to expand and modify it to account for the cloud (e.g., add parameters to account for cloud capabilities, build connectors to the cloud API, tie into the cloud account mechanism, etc.). If you don’t have a system in place, you need to build a new process to access the cloud resources. The overriding issue for either approach is that there are no standards yet for cloud provisioning, so the work you do to tie into a cloud provider is not portable to another cloud. The promise of cloud products which offer multi-cloud capabilities is that they can connect you to different clouds using a common set of interfaces.
Managing your cloud infrastructure can be a lot of work. You need to integrate with an architecture defined by the cloud provider, using its specific primitives for working with cloud components. This requires tying into the cloud APIs for configuring IP addresses, subnets and firewalls, as well as data service functions for your storage. Because control of these functions is based on the cloud provider’s infrastructure and services, you also have to modify your internal processes and control systems to integrate with the cloud infrastructure management.
Even managing your operating systems as part of a cloud deployment presents challenges. Many cloud services provide “base servers” or templates that contain a simple distribution or OS, which are then used to build up your specific server/OS/application. This approach works well when the provider has the exact base server you want to start from, and you have a process in place to build from a running server. The challenge is that when you build up a server based on a gold image, it may: a) not match the base cloud OS version, b) be built from a non-running or base OS versus a fully-running OS (as required by most clouds), and c) use internal resources (boot servers, internal repositories, etc.) that are not available in the cloud. From a maintenance perspective, many organizations use central controls for updates (like WSUS for windows), and these services depend on access to data center networks and services. Since public clouds are running external to your data center, these services either won’t work, or need to be altered to run the hybrid environment.
Finally, the cloud creates additional complexity for managing applications. You almost always need to modify applications to accommodate cloud differences (virtual environment, networks and storage), which means that the applications in the cloud diverge from the “original” or base applications in your data center. You may also use third-party tools to help with integration into the cloud (such as VPN software, integration scripts, encryption software, etc.), which then need to be maintained. Each of these software elements has it its own lifecycle and update management, most of which apply to every image deployed into the cloud.
The management problems introduced by including the cloud in your infrastructure all have their source in the same issue – the cloud is something separate and different from your data center. This separation becomes clear when you consider the integration and management issues that span everything from provisioning to reengineering your applications to changes in lifecycle management. At CloudSwitch we’re streamlining and automating cloud management to eliminate most of these issues, and bridge the separation between the cloud and your data center.
Next: Moving to the Cloud: Series Wrap-Up
Moving to the Cloud: Key Considerations for Cloud Networking
This post is part of a series examining the issues involved when moving applications between internal data centers and public clouds.
Every enterprise has a unique network infrastructure for accessing servers and allowing applications to communicate between components. Various layers support the management of network addressing, deliver critical services, and ensure security. The infrastructure includes specific addressing (sub-nets), address services like DHCP/DNS, identity and directory services like LDAP, and firewalls and routing rules – all reflecting the specific requirements and evolution of the given enterprise.
Clouds are not different from the enterprise in this respect; they have unique networking infrastructures that support complex and flexible multi-tenant environments. Remember that the cloud providers have to control their networking so that they can route traffic within their infrastructure. More important, their design is completely different from your enterprise networking architecture, design, and addressing. This is not a problem if you’re doing something stand-alone in the cloud because you don’t care what the network structure is as long as you can access it over the internet. However, if you want to extend your existing networks and use your existing applications, there are serious discontinuities that have to be addressed.
First, you have to deal with addressing. The typical cloud provider will assign you a block of addresses as part of your cloud account. For example, Flexiscale and GoGrid essentially give you get a block of assigned addresses that you can attach to the servers you create. In some cases these are external addresses (meaning that they are public addresses that can be accessed from the internet), while in others, they are internal. In either case, they are not assigned as part of your addressing. This means that even if you manage to connect these resources to your data center, you need to build new routes and alter your services to allow these “foreign” addresses into your system. Amazon originally took a different approach by providing a dynamic system where an address is assigned every time a server is started. This made it hard to build multi-tier applications, requiring developers to design systems able to pass changing address information between application components. The new VPC offering partially solves this problem for connecting to the Amazon cloud, although some key challenges remain. Other cloud providers are investigating similar networking capabilities.
The next major issue with networking in the cloud is data protection. In your data center, there is a secure perimeter defined and developed by your IT organization that is comprised of firewalls, rules, and systems to create a protected environment for your internal applications. This is important because most applications need to communicate over ports and services that are not well protected and certainly not safe for general internet access. Since applications are developed for the protected environment of the data center, it can be dangerous to move them unmodified into the cloud. Under normal circumstances, the application owner or developer has to build protection on a per-server basis and enact corporate protection policies.
The loss of control of the infrastructure mentioned earlier has additional implications – in most clouds you can’t control the physical interface level. That is, in addition to assigned IP addresses, you get MAC addresses assigned to you as well. These addresses can change every time a server is started which means that the identity of the server (and associated IP addresses) cannot be based on this familiar attribute.
All of these networking issues are at play whenever enterprise applications require the support of your data center infrastructure – things like identity services, naming services, and access to internal databases and other resources. Because of this, your cloud resources need a way to connect to your data center, of which the easiest approach is a VPN. In building this solution, you need to design for routing to the cloud and provide a method for cloud applications to “reach back” to the applications and services running in your data center. Ideally, this connection would allow Layer-2 connectivity because a number of services require this level to function properly.
To wrap up this segment, networking, like storage, is a very important part of your IT infrastructure, and the cloud adds a number of interesting new variables to the design and operation of your data center environment. What’s needed is a well-constructed architecture and a good understanding of the limitations imposed by the cloud if you want to integrate successfully with the public clouds. Today, this can be a major barrier to cloud adoption since enterprises are understandably reluctant to re-architect their network environments or become knowledgeable about the complexities of each cloud provider’s underlying infrastructure. When designing your cloud strategy, make sure to select a migration path that addresses these issues and protects you from costly engineering projects and cloud risks.
Next: Key Considerations for Cloud Management
Moving to the Cloud: Key Considerations for Cloud Storage
This post is part of a series examining the issues involved when moving applications between internal data centers and public clouds.
The true challenges in storage and data management in the cloud result from the diverse and often unfamiliar processes and infrastructures offered by the cloud providers, including: new provisioning methods, storage properties, data population and transfer, and systems for data management (snapshots, clones, replication, backup). The cloud providers define the relationship between servers and storage and often impose constraints on everything from allocation size limits to the ways in which storage is managed. These are just some of the things you’ll want to consider as you start to think about integrating cloud computing into your existing IT environments.
I’d like to focus in detail on the complexity and variability of cloud provisioning and storage properties. There are different models for storage in existing compute clouds, with the most common being an “inclusive” storage model. In this model, each server comes with a certain amount of storage attached to it. The storage is a fixed capacity that is provisioned when you create the server from the pre-existing templates.
For example, Rackspace gives you disk space that is proportional to the memory (RAM) size you select. The smallest memory/disk combination is 256MB of memory with 10GB of disk. With each doubling of memory, the disk space is also doubled until you get to roughly 16GB of memory and 640GB of disk. With the new Terremark vCloud Express, you get a system disk that is predefined for each “template” server you select. For a standard Linux distribution, you get a 10GB system disk, for Windows 2K3 you get a 20GB disk and for W2K8, you get a 40GB disk. Terremark’s vCloud Express allows you to add additional storage as new disks, while others (like Rackspace) allow to “resize” your servers and storage to create a new server with a larger disk and copy your data into it.
Amazon offers several distinct types of storage within EC2. The default storage you get with each server you create in the cloud is called “ephemeral” storage. You then have the option of allocating and attaching Elastic Block Storage (EBS), and there is also an object store system called Simple Storage Service (S3). Ephemeral and EBS are standard “block storage” devices – meaning they are viewed and used as disks attached to your server (/dev/sdg in Linux, D: in Windows) while S3 requires an API or other tools to integrate with your systems. The good part about the EC2 storage offerings is that you have some powerful options as you build for the cloud; the hard part is mapping the proper resources to your applications and integrating this with your existing processes. Specifically, the base storage is ephemeral, which means that if you power-off the server, or it has a hard fault, all the data on that storage is lost. This means that everything on these drives (boot parameters, application updates, user data, logs, etc.) is subject to loss when you power off the machine. There are several methods of handling this situation: 1) Build your servers every time you start them from a formula or other sources such that you don’t depend on the base storage being persistent; 2) Use Amazon or third party tool sets to periodically “bundle” your servers into S3 (effectively taking a snapshot of the server); or 3) Attach EBS storage to your image and store your important data on persistent storage.
Turning to granularity, we find a wide range in the units or increments of available storage in the various cloud providers. There is the “included” storage mentioned above that is often based on the size of the server and the requested OS type. To add storage, we find cloud providers (such as Amazon) allowing 1GB increments up to 1TB, and others (like Flexiscale) allowing only fixed increments of 50GB/100GB/250GB. For Rackspace, you can resize both the server and storage according to the defined fixed ratios, but these are bound to memory and CPU so there is no independent scaling of storage. The bare-metal cloud provider NewServers allows iSCSI storage to be attached to your servers in 250GB increments. In the cloud these varied increments really matter, because you are paying by the GB/month and if you need just a little more storage, you could end up having to purchase 10x more storage than you need, or having to pay for more memory and compute than you need.
The conclusion we can draw is that there are numerous storage configuration options in the cloud, and these options become linked to the server “flavors” defined by individual cloud providers. Because you don’t have the same control or even mechanisms in the cloud as you do in the local data center, the manner in which you allocate, populate and manage data in the cloud will be different. The work you do to understand and map your applications’ requirements into cloud-based storage requires changing your processes to match those of the cloud, and often this work is cloud-specific.
Beyond configuration issues, of course, there are many other concerns. For example, with data management, you have to determine how you will get your data into the system, how to grow your systems and how to protect your data. Right now most clouds use template servers that you have to build up from a pre-installed base operating system using update mechanisms and then re-installing the application components. As for protecting your data, there are also many cloud-specific options available – from RAID-protected EBS in Amazon, to data snapshots and cloning, to backup services offered by companies like Rackspace.
The bottom line is that cloud storage can be simultaneously simple and complex – just like cloud computing in general. It’s simple to use if you just want to try something new; complex if you want to integrate cloud storage into your existing processes and infrastructures.
Next: Networking in the Cloud
Moving to the Cloud: What's Really Required
When we started talking with a wide range of IT managers and companies in early 2008, we quickly encountered a fascinating dichotomy – Cloud Computing is really easy / Cloud Computing is really hard. What made this so interesting is that the casual users were saying cloud computing was easy and the hard-core users were claiming that it was hard. Amazon and a number of other cloud providers have made major advancements since this time, but the “it’s easy / it’s hard” split still exists.
Today, if you want to use the cloud and deploy a server, it is really quite easy to “build” a server from the base templates offered by the cloud providers. There are consoles available to launch servers including providers' control panels (Amazon, RackSpace, Terramark), plug-ins for Firefox (ElasticFox), and third party products like RightScale. Start from a predefined image, add your edits, and poof – you have a server running in the cloud.
It becomes a lot more complicated when you try to integrate an application with multiple servers running in the cloud with your existing data center infrastructure. When I say infrastructure, I mean all of your existing networking, services (DNS, DHCP, LDAP, Identity), build processes, third party applications; basically, the whole of your IT environment that you depend on to make things work.
When you deploy applications in the cloud, they are running on an infrastructure built and maintained by the cloud provider. This means that there is a certain amount of control that is transferred to the provider –the underlying control and assignment of resources they require in order to manage their environment. You need to understand this new environment, select the appropriate resources, and adapt your application to it. But moving an application that’s been running in your enterprise infrastructure, with all its associated processes and relationships, to a cloud provider that has its own way of doing things is where using the cloud gets hard.
To highlight some of the difficult areas, we’ll examine a set of issues across a variety of cloud providers out there. Because there’s a lot of ground to cover, I’ll break up the posts into multiple parts dealing with storage, networking, management, performance, and security. We’ll start with storage since it represents the real identity of the server and all that is important to your application and business. Stay tuned.
Next: Key considerations for cloud storage
