Always Another Barrier

Cloud computing represents change which gets 20% of the world excited by the possibilities of the future while 80% dig their heels in and prepare for a fight.  As Fortune 500 CIO's have told me in priviate on multiple occaisions, cloud challenges their very value propositions which have largely been focused on building and managing complex IT operations.  One in particular told me under no circumstances was he going to commit career suicide by telling the CEO that the $10B he spent on building out modern data centers over the past ten years was folderol (I expect to see news of his termination in the WSJ in late 2012 or 2013).

The reality is that many IT organizations are still busy building barriers rather than figuring it out and trying to make progress.  Like daylight, wishing cloud away won't change the fact that it's here and making a huge impact.  Cloud is the engine behind incredible innovations such as mobile apps and the analysis of largescale data.  As the mainstream media reports on these successes the CIO's are being forced into the corner throwing any objection they can at cloud like an actor in a B rated movie tipping over boxes while trying to escape.

Following are a list of common barriers and strategies for success:

1. Security:  Security has never been a strong argument against cloud because the reality is any proprietary data should always be encrypted in transit and at rest.  Together that covers about 90% of the penetration space used by hackers to gain ilicit access.  New solutions are being developed to protect data during execution.  In reality no new security technologies have been created for cloud; all the existing models have simply been applied and so far the result has been enough to enable financial, health, and government secured data to be used in the cloud.  It's time to turn the tables on the CSO and ask them how rather than may I.
2. IT Cost Savings Focus: How can focusing on cost savings be a barrier?  Easy.  When it keeps people from considering that the biggest value cloud can provide is in revenue enhancement!  Cloud is a global scale collboration platform and despite all the smarts in IT, it takes the additional smarts of the Business to unlock that value.  Companies have leveraged cloud to create entire new markets let alone revenue streams.  Zynga's entire operational model does not exist without cloud, neither does Google's.  Increasingly it's not just the high technology companies that are reliant on cloud.  Kroger, a $4B grocer leverage private cloud to enhance its supply chain.  Charles Schwab leverages cloud to provide high wealth clients with portfolio management tools.  The success stories in the Fortune 500 are hard to come by because everyone is protecting their competitive advantage.  However all have one thing in common: they educated the business and enabling them to reimagine how they did business, sometimes down to their very core.  Cost savings is a good thing, but there are also examples like GE's entire procurement system, in the cloud for nearly five years, where business costs have been reduced, not just IT.  Additional thoughts are available in a previous post.
3. Enterprise Strategy: It's amazing to me how often this critical element is overlooked.  Most cloud efforts started as ground up movements led by developers just as virtualization efforts were led by system administrators.  However the value of cloud will always be trapped in silos unless an enterprise strategy is developed to spread the wealth.  Today there is no reason to consider any hardware dedicated even if it is, in fact, dedicated.  Moving to an abstracted resource model enables separation of operational and development activities from the hardware maintenance and execution.  Only through this model can we eek out every ounce of value from our existing investments, and do so with a single architecture which reduces complexity while maximizing flexibility.  I have yet to see or hear of a successful general purpose private cloud that did not have an enterprise cloud strategy behind it.  An enterprise strategy need not be all encompassing or even complex, but it should form a foundation through simple steps like adopting a definition of cloud, definining what the goals are of leveraging cloud technology, identifying the required skills and technologies, and identiyfing the low hanging fruit to maximize the opportunity for success out of the gate.  A little bit goes a long way.
4. Service Based IT: Cloud spells an end to silos and therefore an end to the political fiefdoms that ruled data centers for decades.  The business people need to be shown back to their seats in the audience and IT needs to put the right people in place to deliver IT services.  Having a service catalog is a great start, but the bigger change is to adopt a service mindset throughout IT.  Cloud, in the end, is a service model and therefore can only be governed as as service.  However nobody will allow this to happen unless they are getting the service they need.  An increasing number of IT leaders are taking traditional service courses from companies such as Ritz-Carlton and Disney.  In one large retailer IT leaders are required to work in a retail outlet to learn what it means to deliver world class customer service.  It's a real eye opener.  At a minimum focus groups with the business are necessary where IT observes from behind the mirrored glass a moderator led frank discussion of how well IT meets their needs.
5. Governance: Cloud is, in essence, a new model for technology which provides multiple abstraction layers within which the complexity of a given technology realm is entirely captures.  Developers need to know nothing about the hardware to deploy a service.  Data Modellers need to know nothing about the database engine to create the right schemas.  Most importantly the business needs to know nothing about how the applications work beyond the business processes they instantiate.  To this end the language of managing the business/technology interface needs to change.  Capabilities and challenges need to be expressed in business langauge, translated behind the scenes into the correct technical elements by IT.  The business needs to control what is done, why, and when.  IT needs to control how it's done and where.  It sounds simple on paper but we have executives with 30yrs experience with broken models where IT explains what can be done so business leaders water down what they want to its barest elements.  The result is something IT can accomplish in a timeframe that's too long for the business minimizing the value delivered.  Penetrating this barrier starts with the CIO sharing the new vision for a cloud enabled IT with their peers and including them in the journey.
6. Available Skills: Interestingly this barrier exists from the start but is stealthly enough to be ignored by most companies in their initial cloud endeavors.  Only during the debriefs when the intial enterprise cloud effort failed do people realize they were limited by their knowledge; they didn't know what the didn't know.  Like any other innovation, cloud requires a new way of thinking.  However unlike most, cloud covers just about every skillset that exists in IT making it very difficult for specialists to understand outside the limits of their own knowledge.  Administration, networking, security, data modelling, programming, architecture.  Like Prego, it's all in there.  To drastically reduce risk start with a strong, knowledgeable enterprise architect who is skilled in cloud computing.  Getting the architecture right, and having that knowledgeable person available to provide course corrections throughout implementation, is critical.  And it's those skills that are available from technology companies providing at least one way to get over the barrier. Additional thoughts are available in a previous post.
7. Data: It's true that data as delivered in current centralized architectures is a valid barrier to cloud adoption.  Applications require data and data is expensive to move.  Getting over this barrier requires a deeper understand of what cloud represents (decentralization) and the realization that a new data architecture is required (federated).  Accept that latency needs to be minimized, bigger pipes will be required, and data will be distributed and then figure out how to make things work through existing concepts like data staging, caching, deduplication, replication, and best of all delivering answers rather than raw data (i.e. push the computation out to the data).  More information is available in a previous post.  At a minimum make sure the data and compute power are co-located when maximum throughput is required.
8. Lack of Robust Development Tools: This is often a cover for enterprise applciation vendors to sling arrows against the open source programming foundation of cloud.  The reality with cloud is that simple scales and therefore many of our old assumptions about how to develop for the web need to be dropped forever.  AJAX, XML/JSON, OAuth, REST; these are the new tools of the web leveraged by companies from Facebook to Google to deliver their applications on a scale never before seen.  What needs to change is the application development model, and in the new model the tools are robust enough to meet the needs of the largest technology companies in the world.  Part of this revolution, against the wishes of Microsoft, IBM, Oracle, and SAP; is the migration away from .NET and Java/J2EE.  Languages such as Python, Ruby, PHP, and Scala are establishing a foothold in the development groups within financial services, healthcare, consumer packaged goods, and retail.
9. N+1: So the cloud is live and now it's time patch the operating system, or update the router tables, or deploy a software update.  Cloud requries a nearly maniacal approach to change management and automated testing.  Nothing should go into production without understanding its potential impacts.  Once something goes into production it needs to be scrutinized and reversed at the first sign of an issue.  New challenges will always arise but through diligence the size of the challenge can be limited.  A method growing in popularity is continuous environmental testing whereby tools test various conditions to create issues thus exposing weak points early enabling remediation efforts.  NetFlix has their Chaos Monkey, Amazon's efforts resulted in the startup OpsCode.  Clouds must be reliable and predictable and the best way to ensure they are is to introduce disruptions and figure out how to minimize their impact.

That's enough to chew on for now.  In grid / utility computing / cloud projects working with clients for the past 10yrs I can tell you first hand companies don't hit one of these barriers, they hit them all.  Cloud done properly is transformative.  Most companies today are focused on the celebrity of cloud "Hey, we're in the cloud" rather than driving value and as a result they're not seeing the returns of a Bechtel or NYSE who were early adopters.  However the limitation is in their vision and implementation, not the concept of cloud.

Cloud Drives New Thinking About Data Architecture

It doesn't take much experience in the cloud space to realize it is subject to the same limitations as Grid computing and for the same reasons. One of the biggest limitations is data; the size of data sets, the bandwidth and cost required to move it, and the introduced latency to process it. I have repeatedly heard Alistair Croll quote another cloud visionary that "next to the cost of moving data, everything else is free". It's simple, and it's true, yet when adopting cloud so few people seem to be aware of the importance to architect for this reality.

As Geva Perry points out the reality of cloud is that like most technologies it enters a company through the everyday knowledge worker and then wends its way upward as it delivers value until finally the CxO's become aware of its success; typically right before they engage in top down initiatives to bring the technology into the company. Most of these early adopters are software developers and systems administrators, both of whom are eternally on the lookout for better solutions that make their lives easier. Neither take a data centeric focus which results in sub-optimal solutions. And in cloud where the return can be so high, a sub-optimal solution still looks great masking it's inherent shortcomings.

As I've explained to many who confuse cloud with mainframes using the centralization argument it's important to realize there is a huge difference. Mainframes were about physical centralization. And if we proved nothing else, we proved physical centralization is bad thing from a disaster recovery point of view, a cost of operations point of view, a response time point of view, and several others that I won't detail. Cloud gives us the best of both worlds: logical centralization within a physically dispersed reality.

New models require new architectures. Since the fundamental value element of any computing system is the data being processed, it's natural to use optimize the architecture for data. In the cloud the optimal data architecture takes advantage of the geographic diversity leveraging virtualization concepts to logically manage the data in a traditional centralized model. The reality is no matter how much storage you can put in one location, you'll never have enough. And even if you did, you'd never have enough buildings to house it. And if you did, you'd run out of bandwidth to move it around and make use of it. We are in a data driven age where we're better at collecting than using data, and combined with mobile technologies in which every electronic device suddenly becomes a sensor of some type we're starting to sink under the weight. Telco's had it right with CLEC's, Carrier Local Exchanges, the distribution points that linked the home to the global network.

In the Smart Grid arena I helped one of the largest utilities to re-think their smart grid strategy. The original design called for bringing all the consumption data back from the smart meters to the data center. The primary challenge on the table was how to move the data - wireless technology? Broadband over the wire? I turned the tables by refocusing the discussion on the fundamental assumptions. Why did the data need to be transferred to the data center? The answer? The business requirement was to be able to tell every user the accumulated cost of their power consumed to date at any given point in time. Digging deeper we found the current mainframe could not meet this requirement being able to only process 200,000 bills each day; hardly the real time answer the business wanted. So I asked if we could flip the architecture taking a distributed control model from my powertrain engineering days with GM. I argued it would make more sense to accumulate the data at the smart meter or neighborhood, calculate the billing at that level, and then only access the detailed data from the data center when needed. Through research the "when needed" use cases were limited to billing disputes, service issues, and the occasional audit. Since only 1% of customers called each month, even if 100% required the customer rep to reach down into the smart meter to retrieve the detailed data it ws certainly eaiser and less of a load than bringing 100% of the data back to the data center. Frankly it was hard to argue against a distributed model which became the standard and has slowly replaced the original centralized model of smart grid touted for years as the answer.

I have advocated the same distributed architecture approach for use by mobile providers (accumulate usage on the mobile phone, execute the billing algorithm, and if the provider needs the detail download what you need when you need it). I have advocated a more generic version for healthcare payers, retailers, and within the supply chain touting the advantage of storing the data where it's collected.

The data management tools are in their infancy but there is significant work going on around the world on the subject. Consider that five years ago your options within the database world were limited to a cluster. Now we have sharding, high performance filesystems, highly scalable SQL databases plus a whole new class of data management from the map-reduce/hadoop world.

Embrace it or fight it the reality doesn't change. Data growth is exploding. Storage densities are plateauing. Is it better to learn how to hold your breath, tread water, or swim?

Yet Another Barrier to Public Clouds - Hacktivism

Public cloud providers like Amazon, Google, Rackspace and Microsoft are struggling to be relevant to the enterprise, and to the Fortune 500 in particular. At a recent conference when a keynote asked if people felt confident enough in public cloud storage to put their data in the public cloud, I was the only person who raised my hand (and that only because of bepublicloud). However sitting through the keynote by a founding member of the Cloud Security Alliance brought me to the realization that there is another side to security that will block the adoption of public cloud even once all the security issues are addresses and confidence in the secure public cloud storage surges.

One of the fundamentals of public cloud is that it uses the Internet for connectivity. Even the VPN solutions use the Internet. Connectivity is limited resource and with the thin margins in public cloud bandwidth is a heavily scrutinized, monitored, and protected resource. Similarly enterprises labor continuously to optimize network architecture and minimize the size of the pipes to the Internet. Enter hacktivism and its favorite tool of disruption, the distributed denial of service (DDOS) attack.

A DDOS attack is basically a flood of requests that hit a targeted range of internet addresses seeking to overwhelm the systems ability to respond. Small attacks take down a server, medium attacks take down a site, large attacks saturate the nework and take down an entire company. Essentially so much garbage is being thrown down the drain that eventually the system blocks up and nothing can get through. When this happens nothing goes in or out.

Imagine a bank, hospital, or any other company who begins to use public cloud for enterprise solutions. To the hacktivists it would be the same as inviting their disruptive methods into the data center. A DDOS attack could essentially take the company off-line unable to complete any transaction involving the public cloud. No more access to systems, data, records, images. I expect this is an issue already faced by salesforce.com and other SaaS providers who become the target because of who their customers are rather than as a result of their own actions. It would certainly make a prospect want to know who else uses the service in advance, but well beyond the concern of shared hardware and co-mingled databases.

I'm sure there are ways to architect around this, however it those will likely increase costs and complexity, the direction opposite the strategy of enterprises. Of course adding this issue to the litany of security concerns in the end only serves to decrease confidence in the public cloud.

Ouch!