Cloud Predictions Beyond 2011 - 2: The need for a cloud abstraction model

If the cloud is to fulfill on its promise we need to start thinking of it as a cloud, not as an aggregation of its components (such as VMs etc.)

As mentioned in a previous post I‘ll use some of my upcoming posts to highlight some cloud computing "megatrends" that I believe are happening - or need to happen – beyond 2011. One of these would be the creation of an “abstraction model” that can be used to think about (and eventually manage) the cloud.  A nice setup to this was done by Jen-Pierre Garbani of Forrester, who in a recent post at Computerworld UK talks about the need to Consider the Cloud as a solution not a problem.
 

In this is he uses the example of the T-ford -which was originally designed to use the exact same axle with as roman horse carriages, until someone come up with the idea of paving the roads - to argue that customers should not “design cloud use around the current organization, but redesign the IT organization around the use of cloud  .. The fundamental question of the next five years is not the cloud per se but the proliferation of services made possible by the reduced cost of technologies”.

I could not agree more, it is about the goal not about the means. But people keep thinking in terms of what they already know. It was Henry Ford who ones said “If I had asked people what they wanted, they would have said faster horses." Likewise people think of clouds and especially of Infrastructure as a Service (IaaS) in terms of virtual machines.  It is time to move beyond that and think of what the machines are used for (applications/services) and start managing them at that level.

Just like we do not manage computers by focusing on the chips or the transistors inside, we should not manage clouds by focusing on the VM’s inside. We need a model that abstracts from this, just like Object Orientated models abstract programmers from having to know how underlying functions are implemented we need a cloud model that abstracts IT departments from having to know on which VM specific functions are running and from having to worry about moving them.

In that context Phil Wainwright also wrote an interesting post: This global super computer the cloud, a post that originated 10 years ago. First, it is amazing that the original article is still on-line after 10 years – imagine what it would take to do that in a pre-cloud era. Second, the idea of thinking of the cloud as a giant entity makes sense but I disagree with him when he quotes Paul Buchheit’s statement on the cloud OS: “One way of understanding this new architecture is to view the entire Internet as a single computer. This computer is a massively distributed system with billions of processors, billions of displays, exabytes of storage, and it’s spread across the entire planet”  That is the equivalent of thinking of your laptop as a massive collection of chips and transistors, or of a program you developed as a massive collection of assembler put, gets and goto statements.

To use a new platform we need to think of it as just that, as a platform, not what it is made off. If you try to explain how electrons flow through semiconductors to explain how computers work, nobody (well almost nobody) will understand. That is why we need abstractions.

Abstractions often come in the form of models, like the client/server model or (talking about abstraction) the object oriented model or even the SQL model (abstracts from what goes on inside the database).  Unfortunately the current cloud does not have such a  model yet – at least not one we all agree on. That is why everyone is trying so hard to slap old models onto it and see whether they stick. For example for IaaS (infrastructure as a Service) most are trying to use models of (virtual) machines that are somehow interconnected, which makes everything overly complex and cumbersome.

What we need is a model that describes the new platform, without falling into the trap of describing the underlying components (describing a laptop by listing transistors). The model most likely will be service oriented and should be implementation agnostic (REST or SOAP, Amazon or P2P, iOS or Android, Flash or HTML5). Let’s have a look what was written 10 years ago that we could use for this, my bet would be on some of the Object Oriented models out there.

PS Feel free to follow me on Twitter @GregorPetri and read this blog at blog.gregorpetri.com

Cloud Predictions Beyond 2011 - Part 1: Consumer Services Rule

In the past weeks we launched directly from the season of cloud events into what SysCon calls the Annual Predictions Bonanza. Gartner released its predictions on December 1 leading with "critical
infrastructure will be disrupted by online sabotage."  At CIO magazine Bernard Golden gave two
interesting points of view, one for vendors and one for users, and even CA Technologies offered insights into the changes we expect in 2011, including how "security will shift from being perceived
as a cloud inhibitor to becoming a cloud enabler."

So, what happens after 2011?  In a few upcoming blogs I will highlight some "megatrends" that I believe are happening - or need to happen - in the decade about to start. (Now, you may argue that the decade started a year ago, but starting to count at zero is very "old school IT" and "old school IT" is definitely not what we are going to see going forward.)
BIG IT becomes Consumer IT
Traditionally "BIG IT" represented the IT operations of large banks, governments and Fortune 1000 companies. These organizations were typically the first to implement new technologies, ranging from the first mainframes to powerful UNIX clusters and later rack-based systems. Many technology companies used the 80/20 rule -- that the top 20% of companies were responsible for 80% of
the overall Global IT spend -- to guide their strategy.  Today the total data processing at the
average stock exchange still dwarfs the number of transactions a phenomenon like Twitter handles, but online entertainment is rapidly catching up.


This really hit home while visiting a large hosted European data center a few weeks ago. There were some corners where you could still find enterprise servers zooming away, but the really big server farms and all the reserved open spots were dedicated to consumer-related services such as online gaming, mobile internet and messaging, and on-demand television. The rise of of these consumer services will cause unprecedented demands for cloud storage, cloud networking and cloud processing in 2011, but the average enterprise IT manager won't particularly notice. In fact, many traditional IT chiefs may still feel they are "BIG IT".  If you're interested in an analyst covering these new consumer areas then you may enjoy Om Malik's GigaOM site.


You could say that this trend of data centers becoming more and more consumer-centric is the top- down part of IT consumerization. The bottom-up part is employees bringing their consumer technology (iPhones, iPads, etc.) and expecting to use them while doing their job. The long term impact of this top-down trend will be that traditional BIG IT technology vendors will start to focus their R&D more on new, fast growing markets. Vendors with a running start in this new reality will be consumer electronics companies (like Apple) and technology vendors that grew up - or grew big - with the internet. As a result Enterprise IT will become a secondary market, a market where data center inventions and investments that were originally made for the consumer and entertainment market can be redeployed. Something to take into consideration when picking your strategic technologies and vendors for the next decade. 


Now consumer IT won't take over Enterprise IT completely during 2011, but the days that we made fun of hardware vendors that made more money on consumer printers and ink than on enterprise data centers are definitely behind us.

P.S. -- OK just one prediction for 2011.  In one of my earlier blogs I wrote about the four P's of Innovation - Problem, Ponder, Publish and Pilot. For Enterprise IT, 2010 was clearly the year of publications (just look at the number of blogs with cloud predictions). That would make 2011 the year of piloting. Check back for my next blog on what I expect the production period will look like.

Virtual Strategy - Virtually Right

With a private cloud strategy and dynamic data center you can quickly respond to rapid business fluctuations. But how do you get there?

This post was originaly published as thanksgiving weekend special at virtual-strategy.com.
In the article I discussed some approaches for building a dynamic data center that not only addresses complexity and reduces cost, but also accelerates business response time, to ensure that organization realizes the true promise of cloud computing, business agility and customer responsiveness.

Cloud computing presents an appealing model for offering and managing IT services through shared and often virtualized infrastructure. It’s great for new business start-ups who don’t want the risk of a large on-premise technology investment, or organizations who can’t easily predict what the future demand will be for their services. But for most of us with existing infrastructure and resources, the picture is very different. We want to capitalize on the benefits of the cloud ― on demand, low risk, affordable computing ― but we’ve spent years investing in rooms stacked high with hardware and software to run our daily mission critical jobs and services.

So how do organizations in this situation make the shift from straight-forward server consolidation to a dynamic, self-service virtualized data center? How do they reach the peak of standardized IT service delivery and agility that is in step with the needs of the business? Many virtualization deployments stall as organizations stop to deal with challenges like added complexity, staffing requirements, SLA management, or departmental politics. This “VM stall” tends to coincide with different stages in the virtualization maturity lifecycle, such as the transition from tier 2/3 server consolidation to mission-critical tier 1 applications, and from basic provisioning automation to a private/hybrid cloud approach.

The virtualization maturity lifecycle 
The simple answer is to take it step-by-step, learning as you go, building maturity at every step. This will earn you the skills, knowledge, and experience needed to progress from an entry-level virtualization project to a mature dynamic data center and private cloud strategy.

It’s called the virtualization maturity lifecycle, and it builds in four steps. Just like pilots start their training on small planes (going full cycle from take-off to landing) before they move onto large commercial jets, it is advisable for organizations to implement these virtualization maturity steps iteratively. For example, start a full maturity cycle on test and development servers before moving to mission critical servers and applications.
Start easy, by consolidating servers, to increase utilization and reduce your current carbon footprint. To ensure deep insight and continuity in support of the migration from physical to virtual, you might want to leverage image backup and physical-to-virtual restore tools that allow you to move your physical IBM, Dell and HP images directly to ready to run VM images for VMware, Sun, Citrix and Microsoft.

The next step involves optimizing the infrastructure. Apart from maintaining consistency, efficiency, and compliance across the virtual resources (which is proving fast to be even more complex in virtual than in physical environments), we analyze, monitor, (re-)distribute and tune our applications and services.
While optimizing, we also discover and document the rules we will automate in the next phase. Rules about which applications best fit together, what areas are suitable for self service and which type of services are most important. As you can imagine the answers to this last question will be very different for a nuclear plant (safety first) compared to an online video rental service (customers first), which it is why it is such an important step. If you skip this stage and go straight into automation, you’ll likely end up in the same situation that you’re in today, just automated.

A successful cloud strategy is all about agility and flexibility, and the next step in the virtualization maturity lifecycle helps take care of automation and the orchestration of your (now) virtual services. You can empower users to help themselves ― industrialize processes ― without calling IT for every service request. Automation has many advantages here. It is the catalyst to standardize your virtual infrastructure, integrate and orchestrate processes across IT silos, and accelerate the provisioning of virtual cloud services. Once the industrialized provisioning process is live, automation technologies can then also be used to monitor demand volumes, utilization levels and application response times and to assist root-cause analytics to help isolate and remediate virtual environment issues.

The final stage is the centerpiece of a cloud strategy, a position which allows you to manage the definition, demand, and deployment of IT services: the dynamic data center. Your now agile infrastructure, delivered from a secure, highly available data center, enables you to quickly respond to rapid business fluctuations. To reach a dynamic data center, you need to automate the entire process of service delivery from request to fulfilment. This includes centralized service requests, automating the approval process so that department heads can quickly approve or reject requests, a standard and repeatable provisioning process, and standard configurations.

This goes much further than the traditional dream of a “lights out” data center, which basically was a static conveyor belt-like factory where all labor was automated away. The dynamic data center is like a modern car factory, where robots perform almost all tasks, but in ever changing sequences and configurations, guided by supply-chain-lead orchestration.

The new normal  
As we all know, technology changes fast. This advancement in technology is creating a “new normal” where relationships with customers are increasingly in a digital form and technology is no longer an enabler or accelerator of the business― it has become the business.

This is a theme picked up by Peter Hinssen, one of Europe's thought leaders on the impact of technology on our society. He evangelizes this new normal, arguing that in a digital world there will be new rules that define what is acceptable for IT, including zero tolerance for digital failure, an era of “good enough” functionality (60% functionality in six weeks rather than 90% in six months), and the need to move your architectures―including your new cloud architecture―from “built to last” to “designed to change”.
The lifecycle approach described earlier may be just what you need to help align your IT organization to what Hinssen calls the new normal. First you determine where opportunities exist for consolidation and rationalization across your physical and virtual environments ― assessing what you have in your data center environment and establish a baseline for making decisions that take you to the next stage. Next, to achieve agility, you have to automate the provisioning and de-provisioning of virtualized resources, including essential elements, such as identities, and other management policies such as access rights.

The next step in delivering an on-time, risk-free (zero failure) cloud computing strategy is service assurance. You need to manage IT service quality and delivery based on business impact and priority — top-to-bottom and end-to-end. That includes, for example, delivering a superior online end-user experience with low-overhead application performance management, and end-to-end visibility into traffic flows and device performance. The new normal also needs to be secure. IT security management technologies must be applied against current regulations and end-user needs, which enable the virtual layer to be more secure.
All these factors combined ultimately lead to agile IT service delivery. With agility, you can build and optimize scalable, reliable resources and entire applications quickly. By embarking on the virtualization maturity roadmap, you can move closer to a dynamic data center and successful cloud strategy.

Any shortcuts?
This evolutionary approach may sound very procedural (and safe). You may also be thinking, is this the only way? What if I need it now?  Is there no revolutionary approach to help me get straight to a private cloud much more quickly? Just like developing countries, which have skipped the wired POTS phone system and moved directly to a 100% wireless infrastructure, a revolutionary approach does exist. The secret lies in the fact that – in addition to the application itself - the infrastructure required to deploy an application can be virtualized – load balancers, firewalls, NAS gateways, monitoring tools, etc.  This entire entity – the application and the required infrastructure it needs to be successfully deployed – can then be managed as a single object. Want to deploy a copy of the application? Simply load the object and all of the associated virtual appliances are automatically loaded, networked, secured and made ready.  This is called an application-centric cloud.

With traditional virtualization, the servers are the parts that are virtualized, but afterward, these virtual servers, networks, routers, load balancers and more, still need to be managed and configured to work with the other parts of the data center, a task as complex and daunting as it was before. This is infrastructure-centric cloud.  With full application-centric clouds, the whole business service (with all its involved components) is virtualized becoming a virtual service (instead of a bunch of virtual servers) which reduces the complexity of managing these services significantly.

As a result, application-centric clouds can now model, configure, deploy and manage complex, composite applications as if they were a single object. This enables operators to use a visual model of an application and the required infrastructure, and to store that model in the integrated repository.  Users or customers can then pull that model out of the repository, reuse it and deploy it to any data center around the world with the click of a button.  Interestingly, users deploy these services to a private cloud, or to an MSP, depending on who happens to offer the best conditions at that moment.  Sound too futuristic?  Far from it.  Several innovative service providers, like DNS Europe, Radix Technologies, and ScaleUp, are already doing exactly this on a daily basis.

For many enterprises, governments and service provider organizations, the mission for IT today is no longer just about keeping the infrastructure running. It’s about the critical need to quickly create new services and revenue streams and improve the competitive position of their organization.
Some parts of your organization may not have time to evolve into a private cloud. For them, taking the revolutionary (or green field) approach may be best, while for other existing revenue streams, an evolutionary approach, ensuring investment protection, may be best.  In the end, customers will be able to choose the approach that best fits the task at hand, finding the right mix of both evolutionary and revolutionary to meet their individual needs.

Reshaping IT Management – by cutting it into two halves?

The McKinsey quarterly just published an interesting and very readable piece on “reshaping IT for turbulent times”.  In the article they analyze what seems to be a dichotomy for today’s IT management: How to balance running an efficient IT factory with being a responsive customer focused provider.

In the article (which is freely accessibly after registering) Roberts, Sarrazin and Sikes describe two models, an efficient factory approach and a more enabling, innovation oriented approach.  However, their suggested approach of applying two models, splitting the organization effectively into two separate parts -- a mainstream factory and a boutique -- seems less than optimal. This split very much resembles the traditional split of IT into development and operations, something that is also turning out to be less than optimal and too slow for today’s markets. Hence the emerging of a new IT discipline called DevOps. 

It is understandable they use two models as traditionally efficiency and innovation require different approaches. There is an old analogy that makes this very clear. Think of the organization as a sponge. If you want more efficiency, you centralize (squeeze the sponge and any excess water pours out); however, if you want innovation and new ideas, you need to let go of the sponge, creating room to suck up water - new ideas. Squeezing and letting go at the same moment seems impossible.

Addressing both efficient production and customer responsiveness at the same time seemed an insolvable issue in traditional manufacturing, as well. Until management innovations - such as just-in-time (JIT) supply chain optimization - gave management the tools needed to address this. The main difference between the new supply chain and the traditional manufacturing-oriented approach was that the goal shifted from efficient production to effective end-customer delivery. This leads to vastly different decisions when put into an optimization model. The IT equivalent of this JIT innovation is cloud computing.

Splitting the IT organization into a back-office grinder shop and front-office boutique will turn out to be a temporary solution at best. Not just because a dual-model approach - almost by definition - prevents any optimization across the two, but also because experience shows that in cases like these, the low cost grinding part will soon move to a low-cost provider (for example, manufacturing moving to China), after which pretty soon the innovation part is likely to follow (again look at what is starting to happen in China). Traditionally the best innovation labs are near factories, except maybe for fundamental research (which most commercial have lost interest in or can no longer afford). 

It took the manufacturing industry several decades (and fierce competitive pressure from pioneers such as Japan) to make the transition to be both efficient and responsive at the same time. IT can learn from these experiences. The competitive pressure required to make such a transition has already arrived. Cloud computing enables users to bypass IT completely and source solutions directly from outside service providers, a practice sometimes referred to as “Rogue IT.” In my post "On empowered users, rogue and shadow IT, stealth clouds and the future corporate IT" I wrote on the valid need for IT to be closer to the business again, which in my view can be achieved without cutting it in two.

By taking an integrated approach - based on aforementioned IT supply chain thinking, with a large emphasis on sourcing – IT organizations will be able to both have their cake and eat it too.