The private cloud debate is building up steam, but is it worth having?

Slowly but steadily the debate in the blogosphere about private clouds is increasing. Now it is always good to see some debate, but is this a debate worth having? Will the cloud long term not be about other things than who owns a machine?

Under provocative titles like “Private cloud discredited, part 1”  and “Do We Really Need Private Clouds?”  the private cloud debate is building up steam. The first blog is actually called “part 1” because the author is sure there will be a part two, given the raging emotions and all the opinions being aired.

The second one is part of a very readable guest series by IT analyst avant la lettre Robin Bloor at Cloud Commons. Cloud Commons  is a cloud consumer rating service community site, like consumersearch.com and epinions.com but for cloud products and services, that CA Technologies helped initiate.

Now it is always good to see debate, and I vividly remember when we all got exited some years ago about Open Systems (with Open roughly being defined as anything running on Unix versus anything that was not running on Unix- including mainframes, AS/400's, HP3000's, etc. etc.). To be honest , that debate was maybe as productive as a debate about private versus public clouds may turn out to be. In my view the most important thing that the cloud can bring is namely that cloud (finally) decouples the application from the underlying infrastructure. As a result it matters a lot less where it runs (private or public). In a comment on Robin’s “Do We Really Need Private Clouds?” blog Jonathan Davis, CTO of DNS Europe, introduces a good example of that principle. Using a cloud platform (Applogic* in this case) his company makes it possible that applications can be deployed transparently and instantly over grids of compute capacity (re. discussion of grids, see Robin’s earlier post in the mentioned series) regardless of whether these clouds are private (hosted or internal), public or a combination (hybrid).

Where to start
The remaining Private versus Public question then is: where to start. Do you start with less sensitive applications on a public cloud and then expand what you learned to core apps onto maybe a private cloud, or v.v. do you start with a more sensitive app on a private cloud and expand to public when you feel that is proven and secure enough for that application.

Surprisingly (at least to me) there was some very clear guidance given in the cloud scenario session at last month’s annual Gartner Symposium/ITxpo**. Basically a kind of “comply or explain” approach was suggested there. It basically suggested: first explore whether a job can be done with a public cloud (“comply”), and only if there are valid and severe reasons to not go public (“explain”) than consider private. I’m paraphrasing so check with your analyst for exact wording and/or check out the free videos recordings of this year’s symposium at gartnersymposiumondemand.com  . Unfortunately the break-out sessions on cloud are not available for free there, but it does offer all plenary keynotes, all industry CEO interviews (Bennioff, Chambers, Ballmer, Dell) and … a recording of my session ;-).

During the symposium, Gartner also indicated that security concerns should be more seen as valid but temporary challenges to be addressed and overcome, rather than as a reason (or excuse) to discard public clouds. At last weeks Datacenter Summit one of Gartner’s lead analysts on cloud, Thomas Bittman, gave a slightly more nuanced view. Understandable as this wa sthe week of Thankgiving and the folks in the room were predominantly the guys running today’s private datacenters (Thanksgiving --> Turkeys :-) ). He highlighted some scenario’s where private clouds makes perfect sense (eg. stable, predictable loads). In his worthwhile and balanced session he also noted that the current emphasis on Infrastructure as a Service (where the Private versus Public debate mainly plays today) over Platform and Software as a Services comes from the fact that IaaS can run today’s existing applications and does not have to wait for a next generation of apps, as developing such new applications simply takes time.

A new generation of cloud applications
In my view this new generation applications will be very different from the applications we run today, which makes it even more important that these new generations of applications will no longer be tight to underlying infrastructures. New generations come with new applications, mainframes introduced OLTP, Mini’s or distributed systems (both open and proprietary) introduced departmental systems and later packaged applications like MRP and ERP and internet web systems introduced the age of e-commerce, where we started buying books and gadgets online and doing our banking on-line (remember when everything was called e-something). So “re-hosting" our existing apps to (either private or public) clouds is only a very small part of the long term cloud story. Not that next year, as I will address in my upcoming #IT2011 predictions, won’t be a very lucrative year for many vendors - including the one I work for - helping organizations move their existing applications to (hosted or internal) private clouds. Not to mention that 2011 is likely to be the year of the IaaS KillerApp TestDev-clouds, that make test/dev machines (a whopping 70% of the average IT organization’s machine park) available in a much more flexible, economical and ecological way. My college Marv Waschke wrote about this last week as a perfect way to gain experience with private and public cloud scenario’s 

But the big long term story in my view is that cloud will be ideal for a generation of new applications. Applications that allow organizations to collaborate with other organizations – so not the now much talked about in-company twitter and facebook clones that enable people to waste as much time at the office as they do at home. Through these new collaboration applications, organizations can take business processes that were traditionally done in house and source them as “as a service”. These processes can vary from bill collecting, invoicing, physical distribution, repair handling and HR to full manufacturing or product design. Having companies be able to specialize and offer these services to many organizations will enable them to achieve massive economies of scale. Note that many of these services which will be largely or completely information/software based. An example: Imagine the efficiencies of one company handling repairs for several large mobile phone manufacturers versus each company having to arrange their repairs themselves. Most phone manufacturers sell through the same resellers, use the same repair centers, source form the same Chinese factories, etc. etc. Hooking these up ones to a central platform used by multiple players can give an enormous platform effect. An early European player in the area is eBuilder.com (who BTW also run a cloud academy, but specific about these business aspects which they call “Cloud Processes for the Value Network”. At their site their head of product marketing also shares a handy list of 10 essential elements to create and run such a next generation cloud service). These types of new generation cloud applications will render efficiencies far beyond any pure IT savings or efficiencies imaginable (see also my earlier entry re. Gitex)

Now you may say that companies have started this move towards specialization and outsourcing of processes already some years ago. And you’re right, but so far they did so often despite of support in IT applications. Thanks to the cloud, IT can now become the big promoter, enabler and catalyst for this. In fact, I came across the scenario of "repair handling as a service" over a decade ago, while introducing XML for a previous employer. But back then the idea of bringing crucial functions outside the firewall and outside the realm of internal IT was just a bit too revolutionary. The fast growing acceptance of cloud computing as a model (often even more on the business side than on the IT side) is rapidly changing this. And to be honest, public cloud may clearly have an advantage over private cloud for such public services, as it is already located outside single companies proverbial firewalls.

PS many links in this one, if you read only one, make it Robin Bloor’s series at Cloud Commons, his havemacwillblog was one of the first IT blogs and this new series shows that experience does count.

*Disclosure: The cloud platform Applogic is a platform by 3Tera, now part of CA Technologies, my employer. DNS Europe spoke at the invitation of CA about their experiences with Applogic at VM World in Kopenhagen.
**CA Technologies is a premier sponsor of Gartner's Symposium/ITxpo and Datacenter Summit.

It is the season of cloud events

This week, Gartner is holding their Symposium/ITxpo to Europe in Cannes .  For some reason the industry seems to be cramming all their events in the two short months that fall between the summer vacation period and Xmas. To name just a few, I've presented at: 

360IT in London

VM World Europe

Gitex in Dubai

Cloud Expo Silicon Valley 

Next week the part virtual reality, part physical reality UP2010 conference will also take place. 

Gartner wisely choose to not use the cloud word in this year's symposium title "Transitions: New Realities, Rules and Opportunities", but it is clear to most attendees that the majority of these transitions will be cloud related. Some may even argue (or fear) that the cloud will replace a large chunk of traditional IT. Having done the opening session of the cloud conference at Gitex, together with Gartner's Jim Murphy, where he gave a short preview of the Gartner Cloud keynote, I can tell you that ignoring the cloud (because insecure, immature or risky in general) is not recommended. Enterprises will need to formulate a cloud strategy -with public cloud taking a prominent role - just as much as they needed to develop enterprise strategies for PC's and end-user computing in the past.

At the event I will be talking, in conjunction with guest speakers from CA partners such as Accenture, salesforce.com and SAP, about "the cloud is the answer, it is also the question".  The reason we invited these partners to present together with CA Technologies is that we feel that there are many areas where the answer will not come from a single company. Today's IT is complex and - at least for the foreseeable future - developments like cloud and virtualization will add to this complexity. That is why we asked Keith Grayson of SAP how to address continuous and automated  governance, risk and compliance (GRC)  in this increasingly complex world. Steve Greene, leader of salesforce.com's agile development, will share salesforce.com's experience in moving from waterfall to agile development projects and Dr. Hauke Heier of Accenture will discuss the Investment Portfolio Management (IPM) Framework, a diagnostic model to help CIOs and top business executives align IT investments (cloud or non-cloud) with business needs (this session was overbooked 200% last year, so we moved it to a bigger room, but please register early).

From the above, it will be clear that both the questions and the answers around the cloud will, to a large extent, concern people and processes and not just technology.  This aligns nicely with the little informal pole I held among the attendees of my presentation sessions at the earlier mentioned cloud events. This year most came to do some 'tire kicking': seeing who else was there (apart from the numerous vendors), learn which end users have actually done something in the cloud and to decide what pilots to include in next year's plan.  Most attendees were more interested in approaches and experiences than in the shiny, cool and technological advanced tools that many of the vendors were showing.  Maybe an idea for the organizers of these events to include specific tracks around these softer aspects in next year's editions (which no doubt will take place in the same month. Old habits are hard to loose). In my sessions I already started to focus on these softer aspects (as I will do in Cannes) and focused on people and management, rather than on technology. People, Process and Technology ... somehow it has a familiar ring to it.

Follow CA Technologies at ITXpo on Twitter at #caitexpo2010

On empowered users, rogue and shadow IT, stealth clouds and the future corporate IT

This blog wa soriginaly published at ITSMPortal.com
Cloud computing can be seen as an important enabler for more user and business empowerment. Traditionally we consideredany IT outside the IT department’s purview“Rogue IT” or “Shadow IT”.  In this blog we examine how further user empowerment may impact the future of corporate IT.

Rogue IT (sometimes called shadow IT or consumerism) is the phenomena in which employees outside the IT department deploy IT technology to achieve or automate certain tasks. Rogue IT is not a new phenomenon. But cloud computing is giving it a new runway and much better camouflage, since with cloud computing you do not need to secretly misuse a server that IT made available for another purposes or explain to your colleagueswhy you have all these computers under your desk.

The idea of IT outside the IT department is enjoying renewed interest. Ted Schadler of Forrester wrote an interesting article in the Harvard Business Review called “IT in the Age of the Empowered Employee”. A recent Forrester survey of 4000 US-based knowledge workers found that no less than37% are using do-it-yourself technologies . In his new book,Empowered, he calls these covert innovators HEROes — highly empowered and resourceful operatives. CBC news ran a feature and and CIO magazine picked up on the topic recently using the term Stealth Cloud.

Personal Experience
Personally I had my first encounter withrogue IT many years ago, during my first assignment in one of Holland’s largest multinationals. The company had a department called “Information Systems and Automation”, ISA for short, that used mainly mainframes to run corporate reporting and accounting.But there was also ISA-2, a divisional IT department, which ran operational and planning systems. Their platform of choice was PDPs and Digital VAX. At the production location where I worked (a massive manufacturing plant so far in the south of Holland that it was practically considered a foreign factory) we had ISA-3, a local IT team that supported office automation and printing using the then just emerging PC platform.  But all these were not considered rogue; this structure was a logical consequence of the fact that bandwidth at the time was so expensive that this was the best way to deliver IT services.

How easily we forget in the age of the cloud (and if your orgchart still looks like this it might be time to reconsider).  I met the rogue IT function called “ISA4” on the second day that I was working there. 
The plant at which I was stationed manufactured medical equipment for which it was necessary to calculate the exact trajectory of electrons. For this, the head engineer had obtained a portable (at that time state of the art) luggable, suitcase size UNIX system, that he took home at night so these calculations could be run while he enjoyed a good night’s rest. So far, so good. Were it not that on that system he also built a small inventory and quality assurance system that kept track of all the work-in-process in the factory. This data was manually typed into the corporate and divisional systems at the end of each month (these were “primary records” as my EDP auditing professor would point out). Now each of these pieces of medical equipment cost more than my car (my current car, not the piece of … I drove back then) and the final QA outcome (approved or scrap) was extremely important for the financial and commercial success of the manufacturing operation.  Yet, every evening this crucial data left the premises, only to arrive back, provided our lead engineer did not run into a streetcar on his way to work.

Moving on
Now this was many years ago.Since then, all systems have been replaced by bright and shiny ERP applications (first at the divisional level and later at the corporate level) and this factory has been consolidated, off-shored, outsourced and then insourced again and the type of equipment has long been replaced by a whole new generation of (you guessed it) digital technology. But I am sure that in that new factory,there are still users outside of the IT department building rogue solutions and applications, because that is what smart employees tend to do.

Not saying I am such a smart employee, but with most of the jobs I held since (in marketing, sales, business development, even some in IT) I managed to create my own rogue pet systems. It started by using a Windows help text compiler and the office CD writer to save time on faxing product information to 12 European offices each Friday night, then developing a Lotus Notes system for gathering enhancement requests, followed by a rogue intranet site (Hey, somebody gave me a fileserver password and IIS was already on there) and then a kind of precursor to Salesforce.com (a CRM intranet site for logging visit reports and forecast data). I somehow was lucky and my rogue systems never caught a virus, but a colleague of mine used an unpatched test version of SQL server for his pet project, that caught the famous SQL slammer bug and subsequently pulled down all SQL and network traffic in the complete enterprise (it was not a comfortable conversation with his boss and the head of IT the next morning).

Apart from these security horror stories, the problem with these industrious/innovative end-usersis that they (including myself) sooner or later get bogged down by the same thing that is slowing down IT departments: providing support and maintenance to what they created earlier. Thus, 70% of the overall IT budget is spent on “keeping the lights on”. As soon as a user has developed something really cool, for example, a way to use his iPhone to support his customers, five of his colleagues will want the same. Two of these colleagues may not have an iPhone but instead use a Blackberry (cutting our innovator’s development productivity in half, as he now has to build and support the functionality on 2 platforms). So he spends less time on his real job and basically becomes a type of IT person. This is why after a while –either when he gets fed up with the support job or when he moves to a new job (these industrious types tend to get either promoted or fired quickly, depending on the type of organization)-IT is called back in to … clean up the mess. Which results in the IT department having even less time or budget available to provide the type of innovations the business was looking for in the first place.

The cloud impact
The beauty (or danger) of cloud computing is that it allows business users to not only create such innovations by “hobbying around” on their phones or PCs. They now can go out and contract with outside vendors to create such solutions in a more professional - but still rogue-  way. Traditionally, business users had to go through the IT department for any such investment, as anything that had to run on the corporate network or servers had to be approved by corporate IT. With cloud computing the new applications however no longer run on the corporate network or servers, enabling these business departments to completely go outside.

Neither of the two described scenarios are desirable. We don’t want creative business people bogged down by maintenance tasks, nor do we want end user departments to contract with any IT vendor they like, bypassing any attempts we made at having any type of enterprise architecture in the process. But at the same time we do want this type of user led innovation to continue. Somebody (some department) however will need to guide and orchestrate these innovations. IT would be the logical candidate, but only if they can free up their time and resources away from “keeping the lights on” towards these more innovative tasks.  And again cloud computing, with its potential to deliver formerly complex IT tasks “as a service” to the IT department, may be just the recipe to free up IT’s time from the mundane, towards these  more innovative and differentiating endeavors.
End user computing

An interesting phenomenon in this context is end user computing and especially sharepoint. Here the IT department in many organizations seems to provide end users with a gun to shoot themselves with in the foot. Many users start enthusiastically, to find out after a year the maintenance is overwhelming, prompting them to abandon the project or start over. The fact that many of these sites are aimed at specific department makes this worse, as organizational structures tend to change yearly or even more frequently, rendering the sitesobjectives and design no longer valid. Now many IT folks will say: but then the design was simply wrong, if you mirror the orgchart, your system will always become obsolete, you should mirror the process or even better the data model as those tend to be much more stable. It is kind of a right brain - left brain thing. And that is exactly why it makes sense to involve the structured – systemic thinking of IT folks in helping users come up with solutions that match their needs. This however is only likely to happen if these IT folks are working inside these departments.

A well-documented case around this is the reshaping of IT done at Procter & Gamble by CIO Filippo Passerini. Under his leadership P&G outsourced a large part of their hard core infrastructure tasks several years ago. The majority of the retained IT people were reallocated(also physically) to work in business departments like marketing, product development, sales etc. Together with their business colleagues they started creating new solutions and approaches to both operational and strategic issues, like creating a closely monitored social media / advertising campaign as recently described in Fortune magazine (hardcopy only). In this way IT’score strengths like structured thinking and problem solving capabilities,start toplay a crucial role in the overall success of the enterprise again. This however only happens after a large part of the repetitive infrastructure related services that traditionally keeps IT busy,  are supplied as a utility, eitherby using an outsourcing construct or by leveraging the cloud.

Time for Hybrids
This approach of IT and business working closely together was one team instead of in silo’s seems only logical, but reality in a lot of industries is that IT actually became more siloed in the last decade. A case in point, personally I was one of the first graduates of a new curriculum called IT & Business, which consisted of – you guessed it - 50% IT and 50% business and economics subjects. The goal of the study was to develop hybrids, people able to straddle and connect business and IT (either working as IT person in a business department or as a business person in an IT department. After my study I started in IT in the pharmaceutical industry and quickly discovered that there were only two departments recognized by management as strategic to the successof that industry: sales and research (funnily enough in that order). IT came in a long range of supporting departments that included finance, manufacturing, logistics, HR, catering and yes, also IT. Being young and having unmatched faith in the power of IT technology I moved over to an industry where IT waspretty core (the IT industry itself) and rapidly found out that my study - that straddled IT and Business - also came in quit handy when trying to sell or market IT stuff to business people.

During those years on the vendor side I observed a distinct widening of the chasm between IT and business. While early in my career I found the IT manager often was the best person to talk to  get a fast understanding of what a company did (as he worked with many departments, often having developed the applications they used himself in some kind of 4GL language during prior years). With the proliferation of standard ERP packages, 3 tier client-server, java and service oriented architectures, IT became more complex. More about technology and less about what the company did.Some may even argue that the worst culprit in this scenario was Java. Traditional 4GL’s and even COBOL aimed to be like plain English, so from using those languages to speaking to users in plain English about their business was not a big step for most IT folks., something that cannot be said for JAVA and it’s typical user.

Objects may appear further away
Now some of my observations may have been distorted because during the time I observed this chasm growing larger, I moved on the vendor side from marketing 4GL based MRP to Object Oriented ERP, to enterpriseintegration using XML and SOA and more recently to IT management technology. However, when speaking to former colleagues in the application space,they agree the profession has become more complex and more about technology. Sure they work with users everyday, but mainly to make them stick to agreed project plans and to make sure they adhere to the predefined workings of the selected standard packages, often not to invent new creative ways to do something completely differently (although many of them would love to).

Another testament to this mind boggling increase in complexity is the fact that the IT management industry (the solutions needed to manage IT itself) has surpassed the application industry (the solutions used to manage business processes) in revenue about a decade ago. Companies are now spending more money on keeping It running then on doing business things with IT. A weird and worrying statistic. The cloud now however has the potential to change this. Both because of the abstraction from the technology that cloud, virtualization and it it’s sibling technologies can deliver, but also because  enterprises are starting to realize they need to take a distinctly different, more business outcome oriented approach to managing cloud IT than what became standard for managing traditional IT. As described in other articles I see a big role for a supply chain approach to managing IT as this will free up in-company IT talent to truly engage in business matters again.

Time to choose sides?
So, If you are currently on the business side you may - with the services companies deliver to their customers becoming more and more digital (in telecoms, media, finance, government, education, etc. etc.) consider letting more IT people into your ranks. Never mind we talk funny and have a bad hairday every day  - to name a few stereotypes. With your business becoming more about shipping bits instead of atoms to customers now is a good time to start adding more IT skills to your team (if only to keep your rogue innovators productive).

If your currently on the IT side you may want to read this thought provoking in-depth study on The Future of Corporate IT.  I came across in this blog, which is from a think tank consulting firmcalled the “corporate executive board”. In their five-year outlook for corporate IT they come to some astonishing conclusions: “The IT function of 2015 will bear little resemblance to its current state.  Many activities will devolve to business units, be consolidatedwith other central functions such as HR and Finance, or be externally sourced. Fewer than 25% of employees currently in IT will remain, while CIOs face the choice of expanding to lead a business shared service group, or seeing their position shrink to managing technology delivery”.Funny how that all started more then 25 years ago.