The Great Cloud Shakeout – A Look Back

Back in 2009 I posted about the “Great Cloud Shakeout” and the coming market consolidation into a few very large clouds. Nearly 5 1/2 years later and it’s about (long past?) time I took another look to see how I did. Back then I predicted that the market would be dominated by “mega CSPs” by the name of Amazon, Google and Microsoft. Note that this was during a period of Cambrian Explosion in the CSP market – it seems like everybody in the hosting business wanted to be a cloud provider.

Every telco, every hoster, every data center outsourcer, most systems providers and many, many startups are becoming CSPs.  After all, there have been thousands of hosting providers over the past several years competing for your business.  A few were huge, several were large, and most were small but often profitable.  I’m convinced this time it might be different — the cloud provider market will be increasingly consolidated with fewer opportunities for new entrants or profit from the tier 2 or 3 CSPs.  The APIs, data center economics and proprietary platforms will make cloud a much more consolidated market.

Further, I talked about a chilling scenario for all but the biggest players:

… where the number of new entrants and the hyper-efficiencies gained by the biggest [providers] will result in razor thin margins that can’t be met by most of the players going forward.  The pricing curve will drive adoption, solidifying the economies of scale by these mega CSPs.

Over the intervening years I can happily (for me, not for most CSPs) report that my predictions were spot on.

I predict that there will be many new CSPs over the next 18 months, but even before the new entrants stop coming many companies will exit the cloud business.  Some exits will be via consolidation/merger, but many will just pull out of unprofitable businesses in the face of blistering competition.  My take is that the great shakeout will be in full force in the 2012 time frame, with a bottom reached over the following 5-10 years.

Clouds came and went, but the mega clouds from the “Big 3 Cloud Providers” continue to grow and prosper. Other cloud providers may remain and have a chance to survive – but only if they develop strong niche positions (market, capability, etc.).

shakeout

 

 

p.s. note also the news recently about HP coming to terms with their inability to compete with AWS in the public cloud space.

Rethinking a stale IT governance model for the cloud era

The rise of cloud computing is a perfect opportunity to rethink your IT governance model in a fundamental way.

Read the first part of this column for a discussion of how to reshape IT governance to add value, rather than hold a business back.

Many enterprises place prohibitions on the use of Amazon Web Services (AWS), Google and other cloud services, despite the overwhelming evidence that these platforms enable innovation infinitely more than most of internal IT, and are every bit as secure as current systems — often, more secure.

Rather than the five guiding principles of the COBIT framework that encourage these prohibitions, IT governance should have only one: Exceed stakeholder expectations for agility, innovation, quality and efficiency to drive business value creation.

How is that done while ensuring the proper allocation of capital, the right level of risk management, and while sticking to the service-level agreements demanded by the business? By throwing away convention and being bold; by inspiring your team to let go of preconceptions and fears so they will follow you; by removing people in your team that aren’t capable of adapting to this new role of IT.

Read the rest of this article here…

 

Modernizing the IT governance framework for fun and profit

An overhaul of your IT governance framework can be fun and profitable. This may sound a little far-fetched, but bear with me.

IT leaders, including chief information officers (CIOs), must ensure proper governance over all aspects of the IT estate. IT governance focuses on five core principles, according to theISACA’s COBIT 5 framework:

  1. Meeting stakeholders’ needs
  2. Covering the enterprise end-to-end
  3. Applying a single integrated framework
  4. Enabling a holistic approach
  5. Separating IT governance from management

The problem with the IT governance model is that the first principle is often a victim of the requisite processes and policies for the other four. This focus on how to implement the governance framework to control and reduce risk has a problem — it makes IT unwieldy and unable to meet the needs of stakeholders.

The governance of IT investments is a mess, and IT governance is killing enterprise innovation, according to a Harvard Business Review report. That’s a bit heavy-handed, but the core premise of the article is correct: Far too many IT investments are tactical and not driven by value creation for the business.

Read the rest of this article here…

 

Reshaping IT organizations to fulfill a DevOps strategy

DevOps is an exciting and far-reaching shift in IT delivery. The promises are tempting: radically higher productivity, lower cost and more reliable systems.

So I guess it’s finally time for your IT organization to get on the development-operations (DevOps) bandwagon, right? Everybody’s doing it, so don’t delay. The big question isn’t whether, but how and where do you start?

First, go out and hire some DevOps people. Wait, wait — DevOps isn’t a job.

Okay, so you should create a DevOps group or department, right? A great team of people you can train up on DevOps led by a director of DevOps. Hold on! DevOps isn’t a department or function either.

Well, if it isn’t a role, a function or a department, what is DevOps?

Read the rest of this article here…

 

 

The {Private} Cloud of Despair

“Oh ye of little faith!”  That’s kind of the reaction I have been getting from some of you to my last missive on the End of Private Clouds.

Perhaps it’s the definition of “not too distant future” that has people confused. There will absolutely be a continuing investment by vendors and their enterprise customers in building and deploying private cloud solutions. Some value will be realized and we can help make sure that happens. However, at some point most customers will fall into a pit of despair and abandon their private clouds BECAUSE THEY CAN’T KEEP UP.

How many enterprises write their own DBMS? SFA/CRM? Operating Systems, Networking, HRMS, ERP, or other systems? How many enterprises custom design their own servers and storage devices?  You know at some point early in the market many of them did just that. Then a vendor solution came along and made continuing their investment a bad business decision.

The fact is that Amazon, Google, Microsoft and (perhaps) IBM are proving that they can run data centers and clouds far more effectively than most enterprises. And it’s only going to get worse for the in-house IT team. The sheer scale, cost model, and ability to invest in R&D for data center technology and operations solutions just favors the bigger players and the gaps are only going to grow.

There’s been some recent noise about the gap between OpenStack hype and actual enterprise adoption. Perhaps many IT organizations are struggling with the tension between their desire to build and operate something as cool as a cloud and their understanding of the bleak reality they are facing.  The ladder to cloud success is very high – and only the undaunted will attempt the climb. The rest will despair of the Quixotic nature of such a quest and will move on to greener pastures.

(c) 2013 John Treadway / CloudBzz / TechBzz Media, LLC. All rights reserved. This post originally appeared at http://www.cloudbzz.com/. You can follow me on Twitter @CloudBzz.

The End of Private Cloud – 5 Stages of Loss and Grief

It’s not today, or tomorrow, but sometime in the not too distant future the bulk of the on-premise private cloud market is going to shrivel into a little raisin and die. A very small number of very large companies will operate private clouds that will be, by an large, poor substitutes for the services available in public clouds. However, they will be good enough for these companies for some percentage of their workloads.

I have seen dozens of private cloud efforts by many large customers. Most are pretty weak shells of a cloud, not coming close to the economics or capabilities of even 2nd or 3rd tier public clouds. Comparing them to AWS, Azure or Google is like comparing my art work to a Picasso or Rembrandt. The only similarity is that I can still call mine art even if it’s atrocious. I can still call your cloud a cloud too – even if it’s expensive, inelastic, and lacking anything but the most basic of features. Some will be reasonable, but in the long run it’s a game you cannot win.

No matter how good you think you are, you’ll never have the resources, skills or need to be as good as Amazon. AWS deploys enough computing capacity every day to run Amazon.com when it was a $7B online retailer. How many servers will you rack and stack today? How many petabytes of storage will you deploy this weekend? How many features did you update this year (Hint – in just the first half of November, Amazon announced 27 enhancements, features or entirely new services!!!!).

In her seminal work, “On Death and Dying,” Elisabeth Kübler-Ross articulated the 5 Stages of Loss and Grief. I think it’s time to look at this for private clouds.

1. Denial and Isolation

Most large company IT organizations are in severe denial about that is going on in the public cloud market today. They think that if only they get their vCloud or OpenStack cloud up and running they can be just like Amazon. Or perhaps they still cling to the total fantasy that their internal data centers are somehow more secure than Amazon’s or Microsoft’s – companies that spend more money on InfoSec per day than most enterprises will invest over the next 5 years. The denial comes from a fear of change, fear of loss of position and career, or just ignorance. By the way – denial is a guaranty that the risk is real. Those who see the future have already shifted their careers to ride the wave instead of being destroyed by it.

2. Anger

One you start to understand what is happening, that your career plans and worldview are being overtaken by the cloud, it is natural to become angry and bitter. You’ll go out of your way to point out potential security or performance issues with public clouds, maybe blogging about the “what ifs” of outages and disruptions, attack vectors and dirty power grids. You can’t control this because your CEO is not going to give you $100M it will take to really build a private cloud. Or perhaps you’re a private cloud vendor looking for that exit that may never come. “Oh why did I waste my time on this market” you might cry when all of the exits have passed you by and you’re looking at an ever dwindling market with lots of dying startups trying to consume whatever oxygen is left. Perhaps you can jump on (off) the Cloud Liberation Freedom Front (aka “CLiFF”)…

3. Bargaining

The normal reaction to feelings of helplessness and vulnerability is often a need to regain control–

  • If only I had moved to public cloud sooner…
  • If only I had gotten better advice from IBM, HP, VMware, Oracle or Accenture…
  • If only we had tried to be more cloudy in our data center…

Secretly, we may make a deal with our higher power in an attempt to postpone the inevitable. This is a weaker line of defense to protect us from the painful reality. Do we pray for AWS to fail? Do we pray for a Google data center meltdown?

4. Depression

It’s over. Sadness and regret set in and we realize that there is nothing to be done. Our best laid plans are in ruins. The future looks bleak, where servers are getting older by the minute, turning off one by one in silent desolation. The staffing model for 2020 shows a drawdown to a skeleton crew just keeping alive the old legacy stuff that you can’t kill or migrate. It’s dull, sad, drudgery.

5. Acceptance

Not everyone will get here. Many have already, coming to the early conclusion that the future is and will be in the public clouds. Those that do get here before everybody else will have more opportunity, more reward, more fulfillment. The late arrivals may have to find other careers – like today’s laid-off mainframe programmer looking for a job at Facebook, it ain’t gonna happen dude. Many a former techie has found fulfillment and happiness in other fields – I even know one who went back to medical school and is a practicing oncologist. Pretty cool, eh? Even Julia Child didn’t start cooking until she was 50 – so your second career is nothing to fear!

In any event, once you understand that the public cloud is the future – and when you are over the denial, anger, bargaining and depression – you can start to make plans.

CIOs should start getting ahead of the curve, thinking very hard about whether or not that new data center plan is worth the investment. Why spend $30 million, or $300 million, on a fancy new data center that may lay empty in a decade?  Instead of investing $5m in a new private cloud, how about investing $5m in the InfoSec upgrades required to safely use a public cloud?

It’s only a matter of time. Resistance is indeed futile. The public cloud is the future.

(c) 2013 John Treadway / CloudBzz / TechBzz Media, LLC. All rights reserved. This post originally appeared at http://www.cloudbzz.com/. You can follow me on Twitter @CloudBzz.

It’s All About SDN

By Ben Grubin

HP’s announcement last week at Interop that they are shipping their SDN SDK and SDN App Store is merely one of the first salvos of a war that will likely heat up over the next 24 months. What was once the purview of marketing and start-ups, Software Defined Networking has now become the dominant strategy of HP, VMware and others to truly disrupt the current state of data center network architecture.

HP SDN Ecosystem

As they announced a few months ago, VMware is going further with their NSX-based SDDC (Software Designed Data Center) concept, which essentially treats the entire underlying network infrastructure as dumb pipes.

In this new world of SDN and SDDC, the never-ending list of features that Cisco, HP, Dell, Huawei, and others have used as the lynchpin of their competitive strategy in the Ethernet switching and routing markets is nearly irrelevant. Instead, what these new technologies demand is simplicity and speed, something incompatible with layering on hundreds of unnecessary features into the software that drives Ethernet switches.

In fact, layering is the underlying story here. While most network architects have tried to avoid networking overlays due to complexity and losing visibility into layer 2 and 3 architecture, SDN and SDDC are truly a network overlay that abstracts away the entirety of the underlying physical network.

Implementing SDDC means only two basic requirements for the underlying network: it should have as few hops as possible between any two points, and as much “symmetric” capacity as possible–meaning the capacity should be equally large between any two points on the network. Only with this design do you enable the broadest possible freedom at the overlay SDDC layer.

What don’t you need? VLANs, layer 3 routing protocols (OSPF, IGRP), and other such mainstays of the data center. All of this is handled inside the software layer, and with VMware NSX the virtual infrastructure.

All in all, this is an exciting movement towards simplifying the network layer and making it more agile and responsive to the needs of business. Having per-VM virtualized network components such as load balancers, firewalls, and switches means less specialized equipment and less capital outlay in the racks.

Is all of this going to be in production tomorrow? No way. There’s still some key hardware and software challenges that need to be solved to equalize the performance equation. However, if history is our guide, it won’t take long for those to be conquered.