data center
The Hidden Costs of Internal Clouds
Digg
Reddit
Delicious
StumbleUpon
Facebook
Twitter
LinkedInBy Ellen Rubin
A public cloud can provide access to computing resources that many companies would otherwise never be able to afford. The arguments for the cloud are well-known by now, but remain compelling—no up-front costs, virtually unlimited computing power on demand, and highly efficient pricing where customers pay only for resources used. There’s also less pressure on corporate IT departments that are charged with managing the infrastructure and budgeting for new equipment to keep up with demand.
But concerns about security and loss of control in public clouds have led to an alternative model—the internal cloud—that replicates the cloud environment inside the corporate firewall. Within these boundaries, enterprise users can provision computing resources as needed, using the cloud’s self-service capabilities while leveraging data center services. Internal clouds are often referred to as private clouds, but since private clouds can also be found in external environments, the “internal” designation is a more precise term for what we’re talking about here. (RightScale's blog post provides helpful definitions of the different cloud variants.)
With servers, applications and data within the enterprise walls, internal clouds can provide many of the benefits of cloud computing without the potential risks when the computing environment is provided by a third party. Unfortunately, the economics of internal clouds makes them inherently less efficient than the public cloud, especially as new technology makes the public cloud safer and more reliable. Here are some of the reasons why:
- Infrastructure costs: Deploying an internal cloud requires building out the infrastructure to support the needs of all enterprise users. In addition to acquiring the necessary hardware and software, this includes things like configuring the network, allocating storage, paying the electric bill, and providing square footage for the equipment. Plus, all of this infrastructure has to be managed and supported on an ongoing basis.
- Over-provisioning: Just as with traditional IT infrastructure, anticipating resource requirements for an internal cloud is difficult since applications run at varying usage levels. Some applications consume resources fairly steadily while others require occasional bursts of massive computing power. Companies may have no choice but to over-provision, where some equipment sits idle most of the time in order to have resources available for peak periods. (Meeting these short-term usage needs is one example where the elasticity of public clouds really pays off.)
- Building the management plane: As Jon Brodkin points out, building an internal cloud is about more than just virtualization. One of the key benefits of cloud computing is the self-service aspect, where users can access resources as needed via a self-service portal, which then execute in the cloud automatically without administrator intervention. Building this control plane is another substantial step when implementing an internal cloud.
For all of these reasons, the internal cloud carries a cost overhead compared to leveraging the resources and self-service framework already provided by public clouds. Ironically, much the same protection and control offered by internal clouds is becoming increasingly available in an external public environment. So enterprises may find that a public cloud may provide a more cost-effective home for many (or even most) of their applications.
Internal clouds can make sense for the most business-critical requirements or those that need specialized hardware or have regulatory compliance issues. Focusing internal resources on these types of applications would greatly reduce the investment required to build, sustain and grow enterprise infrastructures. While some applications may always need to run internally, many companies have dozens of others that could run more cost-effectively today in a public cloud, providing computing power on demand along with required security, agility and scalability.
At CloudSwitch, we provide technology that allows companies to take full advantage of the resources and savings that the public cloud has to offer, while integrating seamlessly with their internal environments. Applications deploy automatically, are free from lock-in, and stay tied into internal data center services. In short, the enterprise gets the protection and control formerly found only behind the firewall, and can combine the best of internal and external cloud offerings with maximum agility and value.
Why an 'Apps' Guy Moved to the Cloud
I have spent most of my career involved in the development of software applications. For the past seven years, I had the honor of leading SolidWorks, the # 1 mainstream 3D CAD company in the world. When I left in 2007 we had over 750,000 users and tens of thousands of companies using our software. They were designing products that you and I use in our everyday lives.
Through a lot of hard work of hundreds of employees and some luck we helped create a lot of value for these customers as well as our investors and business partners. In addition to good luck (which we admittedly had a lot of), we capitalized on a fundamental platform shift: Unix to Windows. This platform shift, combined with a business model shift (direct sales to indirect sales) created a significant amount of value. I sense the same opportunity with the cloud.
After taking some time off, I spent time with some local venture capital firms looking at a lot of new technologies, new products and business ideas. All of the start-ups (not just some, but all) were using the cloud for their infrastructure. Like canaries in a coal mine, this to me validated what I was observing: the cloud really is a new platform. Also, the cloud requires a business model shift in enterprise software sales since cloud services by their nature are low-cost and on-demand – unlike the traditional sales model of large, costly enterprise purchases.
I spent several months more looking at companies in and around the cloud space and became intrigued by the perspective shared by Ellen Rubin and John Considine, founders of CloudSwitch. They had a clear view of how they felt companies would want to take advantage of the cloud. This was evident in their approach: rather than try to take the clouds to the data center (which everyone appears to be doing), they wanted to start from the data center and help them extend out to the cloud.
Combine this simple, yet profound insight with a platform shift and a business model shift and you understand why an apps guy would be so excited to join the CloudSwitch team.
Why Cloud is at the Top of the CIO's Priorities
In the most difficult economic climate in decades, CIOs are reevaluating their strategies and looking for new ways to reduce data center costs and overhead while improving responsiveness to business requirements. Cloud computing has emerged as a much more agile and efficient approach than what companies have done in the past: adding more compute, storage and networking capacity or trying to get more out of what they already own.
Cloud computing did not emerge from a vacuum, but has its origins in three technology "megatrends" that most CIOs are already familiar with. These developments were all born out of the same need -- to drive down costs, simplify data center operations and allow IT to be as agile as possible. As these megatrends have become pervasive, they've helped put the cloud in the CIO's strike zone:
The drive to consolidate: Consolidating sprawling data centers has become a top IT priority as companies struggle with out-of-control costs for hardware, power, administration and service. Many companies have seen their data centers grow beyond anything they ever anticipated, with the result that in many cases they're not only running out of space, they're increasingly running out of power and cooling as well. In response, they look for innovative ways to reduce their data center footprints - to move out anything that adds cost and complexity, and takes up extra real estate.
The growth of virtualization: Many organizations now operate in virtualized environments, where applications can be quickly deployed to available resources, rather than assigning them to a specific physical machine. Not only does this optimize utilization of equipment, it allows IT to become much more responsive to the needs of the business.
Emergence of SaaS: The Software as a Service (SaaS) model has become widely accepted, in which applications are hosted by outside service providers that can apply specialized expertise, the right hardware and economies of scale. The idea of running certain apps outside the walls of the organization is recognized as not only acceptable but often preferable, where an external provider delivers the service just as well (if not better) than companies trying to do it themselves.
Cloud computing builds on these megatrends, and goes several steps further, providing new capabilities for enterprise computing:
- Not just consolidating the data center, but creating the optimum environment both within the DC and in the external cloud, to match changing demands for computing resources
- Not just virtualizing applications across internal systems, but across whatever environment is most appropriate and cost effective
- Not just software as a service, but enterprise applications running in the cloud on the cloud provider's infrastructure
The ability to run applications in the cloud promises to radically alter the balance sheet by which IT projects are judged, where initial capital expense and ongoing operating costs are factored against value delivered and how quickly resources become available. CIOs now have the opportunity to do something much more significant than make small incremental improvements -- particularly as new cloud deployment and management tools come to market. That's why more and more IT executives are making cloud computing a top priority as they plan their strategies for 2010 and beyond.
Has Virtualization Solved the Data Center Crisis?
Over the past several years, many IT departments have committed to virtualization as an antidote to the spiraling costs and inflexibility plaguing corporate data centers everywhere. By running applications on virtual servers and consolidating underutilized hardware, data centers can get maximum value from their equipment. Virtualization also makes IT more responsive to the needs of the business: rather than spending weeks or months to provision a physical server, a virtual server can be launched in minutes.
Virtualization was meant to be the solution to today's data center woes - but is it? While it brings much-needed flexibility and efficiency to an environment where these qualities were sorely lacking, virtualization alone doesn't cure the underlying problem and in some ways adds to it. Companies still have large data center infrastructure footprints to maintain, plus virtualization licenses, plus management issues introduced by virtualization - ironically adding cost as they try to reduce cost. Many IT managers report that the technical and management challenges associated with virtualization are hindering them from realizing its full cost benefits. They're still paying huge energy bills (those consolidated servers are working much harder than previously). They're still running out of capacity and need to keep buying more servers and storage. And over half of them are still building new data centers at enormous cost.
We're Not Done Yet
But virtualization is one step toward a larger goal, not the end of the journey. IT is in the middle of a fundamental transition from the rigid, siloed world of traditional data centers toward a more elastic, responsive model where needs are met far faster and more efficiently. And we're not done yet. While virtualization helps companies reduce cost and improve agility, the full promise of the new model plays out with the addition of cloud computing, delivering infrastructure on demand as an easily-accessible, cost-effective service.
Rather than perpetuating a bloated data center, the new model will allow companies to get out of the computing infrastructure business where appropriate, retaining only the portion that is essential to the enterprise. As the cloud environment becomes increasingly agile and secure, provisioning decisions will be framed by asking: Should we be really be doing this ourselves, or can someone else do it better and at lower cost? The majority of companies surveyed that are either using or actively planning to run at least some apps in the public cloud have started asking themselves the same question.
Some companies - particularly larger enterprises with the skills and scale to do it effectively -- are building on their virtualized environments to create private, or internal, clouds that deliver several of the benefits of cloud computing within the enterprise. Private clouds provide users with an elastic computing resource on demand and help make better, more efficient use of existing capacity. But IT departments still face many of the same fundamental challenges - they still need to buy, manage and grow the data center infrastructure on which the private cloud depends. As Gartner Group's Tom Bittman points out, for most enterprises, the private cloud is not the ultimate goal, it's another stepping stone to services available in the public cloud as they become available.
It's All About the Application
The real issue is determining where each application truly belongs. Some apps are simply not suitable for any cloud, while others, at least for the foreseeable future, belong in the private cloud. Some applications are candidates for the public cloud, but the appropriate services aren't ready yet. And some data center applications could be moved to a public cloud now or in the very near future.
While virtualization is a key step toward moving beyond the rigid data center, cloud computing takes you all the way there - which is why it's getting so much attention. With new technology from CloudSwitch under development, it may work for your enterprise faster than you think. Stay tuned.
Moving to the Cloud: How Hard is it Really?
Many IT managers would love to move some of their applications out of the enterprise data center and into the cloud. It's a chance to eliminate a whole litany of costs and headaches: in capital equipment, in power and cooling, in administration and maintenance. Instead, just pay as you go for the computing power you need, and let someone else worry about managing the underlying infrastructure.
But moving from theory into practice is where things get complicated. It's true that a new web application built from scratch for the cloud as a standalone environment can be rolled out quickly and relatively easily. But for existing applications running in a traditional data center and integrating with a set of other systems, tools and processes, it's not nearly so simple.
What's really involved when moving an application from your enterprise data center to the cloud? Let's say you've decided on a particular cloud, and you've identified the application you want to run there - now what? You need to consider a range of issues which can potentially turn the migration into a complex engineering project.
Migrating to the Cloud
Today's cloud providers impose architectures that are very different from those of standard enterprise applications. As Bernard Golden explains in his in-depth look at cloud computing, difficulty in migration is holding back uptake, and there aren't yet any automated tools to smooth the way. The result is lots of manual configuring, complex engineering, and trial and error before the enterprise application is able to run in the cloud. A whole landscape of specifications for OS versions, storage, networks, integration with other applications and databases - all those configuration steps that normally happen behind the scenes - have to be mapped to a cloud environment that is probably very different from what your IT staff is used to. It's the type of project that can tie up a development team for weeks or even months.
Keeping Your Data Safe
When data moves to the cloud, it moves beyond the reach of tools and mechanisms put in place over the years to preserve its integrity. In an environment characterized by multi-tenancy and decoupling between hardware and applications, cloud users need to be vigilant and understand the risks. (For a good introduction to cloud security issues, see David Binnings' article, Top Five Cloud Computing Security Challenges.) In brief, you'll need to make sure that the cloud provider has a level of physical security and regulatory compliance that meets the needs of your business and the specific application (for example, those with public information vs. confidential vs. compliance-regulated). You'll also need to consider what additional measures might be necessary to protect against potential threats, including protecting data in transit as well as at rest. It may also be appropriate in some cases to keep the database within your data center and put the rest of the application outside in the cloud.
Managing Dual Environments
After you finally get your application running in the cloud, you'll find another big hurdle: how are you going to manage it? The cloud and the data center are currently two completely separate environments, each with its own set of system management tools, and no meaningful way to integrate the two. Accordingly, your IT staff will need to learn and use each cloud provider's management tools and policies, in addition to the ones they already have. They'll also have to give up some of the control and visibility into an application and its supporting infrastructure that's available in the data center, at least in current cloud environments. (More details about the challenges of managing enterprise applications in the cloud can be found in Peter Loh's article in Cloud Computing Journal.) And as the cloud provider makes changes to their underlying infrastructure (for example, patching a version of their OS), the cloud version of the application needs to be maintained to meet this new environment, so it becomes even more different from the local versions over time.
What if You Want to Change Clouds or Move Back to the Data Center?
All that effort was just for one cloud! What if another cloud provider comes along with lower prices or better service? Since you've invested all that time to set up the application for one cloud, you're going to be very reluctant to repeat all the development and integration work to meet the new provider's requirements. Many companies also wish they had the flexibility to use the cloud to develop and test a new application (leveraging the cloud's benefits in agility and low cost for early research/prototyping/development), before bringing it back to the data center to take advantage of the production set of data and their corporate processes and infrastructure. Today, it's not possible to move an application between different clouds or back to the data center easily, with a few mouse clicks. For many companies, the goal is to create a federated environment of their data center with one or more clouds, and to move applications and workloads wherever is most appropriate.
The cloud offers a great opportunity for enterprise applications, but it's important to understand the work required before embarking on a migration, and how the cloud environment will integrate with the existing data center. CloudSwitch has been working hard to address these issues. Stay tuned for further developments.
RSS Feed