Information, Technology and More

So Where Are the Mainframe Cloud Providers?

2012-01-22T17:30:00.000-08:00

The very first thing I'll admit is that I'm not a mainframe expert. I have steered clear of the behemoths my entire career. My primary motivation has been a desire to focus on distributed computing which is the antithesis of the mainframe. However I do have quite a bit of respect for what the mainframe offers in terms of throughput, virtualization, and parallel processing. Which is why I don't understand why there aren't any cloud providers offering mainframe capabilities. EDS was founded on the concept of sharing mainframe time between companies who did not need and/or could not afford a mainframe of their own. Outsourcing today at companies like Perot Systems, HP and IBM charge by the MIP; isn't that a pay-as-you-go model in the same way reservations based virtual private clouds are priced?

Here are my theories; perhaps someone smarter than me can tell me which one or ones are right, or what I'm missing.

The Need
Do all the companies that need mainframes already have them? Are the existing outsourcing models satisfying the needs of the market? Perhaps. However having done consulting work for companies who are mainframe based, I know there is a desire to have a variable allocation (and thus cost) of mainframe resources (most often measured in MIPS) rather than pay for a fixed allocation. IT needs prices to reflect workload and that cannot happen in a fixed price environment, especially one where, like cell phone plans, you pay a higher penalty price for exceeding your allotment.

The Architecture
Mainframes assume local access to data and processing power; a truly centralized model. Perhaps this architecture is the limiting factor. Companies would need to send their data to the cloud providers for processing. However this same problem exists in the virtual machine world of cloud so unless all mainframe apps are data hogs at the top 10% of the scale it has to be a wash.

The Cost

Owning and operating a mainframe is expensive. Specialists, air conditioning, power, and footprint are all on a larger scale with a mainframe than typical blade or "pizza box" servers. Perhaps there's not a viable economic model to deliver mainframe resources. However the costs of an internal or outsourced mainframe have to be comparable. At worst the cost of operating a mainframe at a cloud provider would be equivalent to an outsourced model. Although a premium would be charged to provide a resource on a "taxicab" model (or perhaps "rental car" model), it seems the cost avoidance opportunity to purchasing a new mainframe or supporting an existing mainframe would be of value.

The Benefit

Would the cost of elastic and efficient mainframe resources be high enough to justify the cost? Is there enough load in complimentary usage patterns to enable a mainframe to be fully utilized enabling a lower cost per unit resource? Traditionally mainframes operated in the 60 -70% utilization category implying tremendous potential to consolidate 1 in 3 netting a 33% reduction. It seems like the benefit would be significant.

In the end I believe the issue is leadership, or the lack thereof. IBM has never been a leader in cloud and has struggled to be relevant despite having created the concept of virtualization. I honestly believe the zSeries team at IBM has kept the mainframe out of the cloud discussion out of fear, uncertainty and doubt. Nobody has emerged on the z team to champion cloud with the mainframe. The arguments always seem to fall short, just after loading some version of Linux onto the z platform. If the home team can't support the concept and get excited, it's hard to expect anyone else to want to take the risk.

What's funny to me is how many older IT staffers equate cloud to the mainframe. To me it's a gross oversimplification that glosses over significant differences, such as the one between centralization and distribution of resources. However I can definitely see some similarities, such as not caring how the cpu, RAM, and storage resources are marshalled to satisfy a request. Some of that is through automation, some through the magic man behind the curtain (the support team). When a developer doesn't care, things are just available, it looks pretty cloud like.

An End of Year Prediction

2011-12-31T07:12:00.000-08:00

Before anyone gets too interested this is not a prediction for 2012, 2013 or even 2020. This is my prediction for global economic shift driven by some fundamentals of economics. First, information is the new currency. We have done a tremendous job over the past three decades of understanding trends and drivers of behavior by collecting data, creating forecasting models, and tweaking our models over time to account for the variety of unknowns. Some areas are easier to model than others with greater or lesser accuracy, but the reality is we now have a good idea what weather we'll be facing more than a week in advance, trillions of dollars have been made in the equity markets, and the progression of debilitating diseases can be stopped before it starts.

Second, money chases problems. If you're able to solve a problem it's likely you'll be able to find money from investors to grow and make your solution as available as possible. The fundamental problem here is that the problem has to be worth solving, and many are not such as purchasing animal food on-line (pets.com).

Three, as solutions are optimized they become commoditized lowering the opportunity threshold and thus profit margins, and at some point are subsumed by the "system" to which the solution was originally applied. Think of enterprise resource management, supply chain management, and customer relationship management as examples of technologies which are becoming commodities as their growth in the cloud SaaS market expands.

Four, computing power continues to grow at Moore's law. Even if it slows down it won't matter because everyone will agree what we can do today in computing will be dwarfed by what we can do in 10yrs, and that by what can be done 10yrs later, etc.

Five, gold is valuable because it is real. US dollars are less valuable because their value is based on the esoteric idea of the US economy. You cannot trade in a US dollar for its equivalent in economic activity, and every investment in US dollars rides the roller coaster of confidence. However gold has been and continues to be valued by everyone universally. If paper currency was truly valuable, it's value would fluctuate in synch with other valuables rather than being counter-cyclical.

My prediction is over the next 50yrs the US economy, followed by the global economy, will make a shift from providing services back to providing goods. I do not foresee a return to the 1950's manufacturing dominance of the United States. We are a global economy good, bad, or indifferent and other countries will participate. However as our focus on the value of information is optimized over the next few decades there will be a tremendous amount of money available, poured into the market to drive innovative solutions. Those solutions will move the technologies into the mainstream driving them toward commodities until, finally, the solutions will become indistinguishable parts of their parent "systems". All of this new knowledge will enable companies to design, build, and deliver better products. The computing power will be available to do things we have not even dreamed of today beyond real time monitoring of the social web to identify a purchasing opportunity and deliver a customized offer with details, pricing and near immediate availability all within a second or two. Information companies will have a difficult time competing because everyone will have access to the same information and analysis tools. Investment bankers, brokers, consultants will all become the professions of bygone eras as the world turns to differentiation through the real-time application of knowledge. The value of information will drop like US currency and the value of goods, things you can put your hands, will rise in value.

I believe the shift will be so seismic that companies will repackage services as products, services getting such a negative connotation. And how we consume those products will be such a bold new model we have a hard time wrapping our heads around the idea of nanobots and technologies augmenting our human nervous system.

What I don't predict is mass employment in this new model. Employment will continue to drop as productivity grows. Unfortunately that's also a reality of economics. However I believe children born after 2020 are going to have to be equally skilled in managing information and managing the production of goods.

Of course I could always be wrong....

Take an Inclusive Approach

2011-12-13T05:52:00.001-08:00

Most people have a difficult time wrapping their head around Cloud Computing because it's a multi-disciplinary disrupting technology. It crosses just about every technical and business barrier that exists from accounting to system administration, legal to storage, and HR to application development. Changes within each group are required to make cloud successful. Who should be responsible for the assets? How will systems be managed? What limitations need to be placed on data access? The questions go on and on and on...

Organizations adopting cloud technologies in the second wave, the fast followers, are learning from the early adopters that having a strategy from day one is critical to success. Everyone needs to be singing from the same sheet of music to ensure the investments make sense and will generate the expected returns to the business. However many organizations are taking a narrow approach in defining who is a member of the chorus. I have worked with many firms that from the start exclude critical organizations such as internal audit, legal, software development, and HR. The net result is often a private or public cloud with few or no applications, that violates policies, adds risk to the business, and cannot be supported because of a mismatch of skills.

Socializing cloud is not easy but it's required for success. Everyone in the organization, from the CEO down must understand how their life changes with the incorporation of cloud technologies. Most importantly the line of business leaders need to be educated in what is now possible via virtualization that was not possible before. It's time for the business executives to dream; to stop limiting their vision of the future to what they believe their IT department can deliver or technology can support. Thinking without limitation is a core of creating innovative solutions.

My minimum recommendation to prepare an organization for the transformation engendered by adopting cloud technologies is to execute the following:

Develop an enterprise strategy and roadmap
Educate everyone on what is happening and why
Buy 3rd party education (computer based training) on the basics of cloud and require everyone to take the classes, even the business people.
Create a cloud council with representatives from all the constituent groups to act as a clearinghouse for questions, collection point for input, and distribution point for sub-strategies.
Bring in skilled 3rd parties to augment your team because no matter who you are you don't have enough in-house cloud knowledge.

It sounds like simple logic, and it is, but it's overlooked more often than implemented.

SAS 70 Type II Is Not Enough

2011-10-28T11:46:00.000-07:00

It's interesting how so many cloud providers point to their SAS 70 Type II attestation. Having been through the process of a SAS 70 Type II audit for a SaaS finanical services firm I'll point out the major gaps that providers do not make clear about a SAS 70 attestation.

A SAS 70 Type II attestation is about financial controls, not operations, not security. Therefore if something does not impact the finances of a company it is considered out of bounds for the audit. Remember the AICPA (American Institute of Certified Public Accountants) governs the SAS70 auditing standard. I respect accountants and auditors but I have yet to find one who I can't make their head spin with technobabble.
There is no standard for the contents of the attestation. The SAS 70 contains self reported financial controls. Therefore each company, by shopping for an auditor, can intentionally omit germane information and still have an attested report.
The attestation report is not available to non-customers by rule. If it is shared with a customer it is a violation and that alone should make someone wonder. Even as a customer there is often a significant cost associated with getting a copy of a SAS 70 audit report (in the tens of thousands of dollars based on my experience).
The attested report simply validates that the organizations does what it says it does. During the audit it is possible to obfuscate issues because the audits do not contain a significant amount of process execution observation.

A more proper standard given the significant concern over cloud security is ISO 27001 focused on information security. And it is my understanding from friends in the audit industry better standards and guidance are coming.

Demand more; don't be satisfied with a SAS 70 Type II audit. And if you can't get your hands on it your missing nothing.

A Legacy Laced Cloud

2011-10-21T11:41:00.000-07:00

The times they are a changing! And with all of the external forces driving change into organizations, plenty of internal forces are ebbing away at the status quo. I wonder is cloud the impetus for change or the reaction to the need for change? I think the answer is both, but the former is a tactical answer while the latter is a strategic answer.

Cloud helps companies in dramatic ways as proven by early adopters. The primary use case for cloud technology deployment to date has been to reduce costs within IT. A second emerging use case is the time to market benefit as cloud resources are often easier to procure, configure, and/or build solutions atop. However in the adoption of cloud we need to have a pronounced, intentional plan, especially when it comes to legacy applications.

I have witnessed situations where dropping an application to a cloud environment increases risk more than value. Not all applications benefit from virtualization: stable applications with little fluctuation in usage of resources often look identical with the single benefit being the sharing of unused resources (which may be enough of a benefit to argue for cloud). However many of the open questions when moving legacy applications into the cloud (or any alternate platform) include:

software library support
level of coupling (operating system, libraries, other applications, utilities, etc.)
component versioning
development, test, qa, and staging environment setup and certification
test cases and automated testing
testing of the application and its associated components and certification
migration to production
training of developers and testers
update of disaster recovery plans
update of systems management services (monitoring, troubleshooting, backups, etc.)

This is the easy short list and assumes no change in functionality!

Finally lets not forget the law of unintended consequences. Something will go wrong as a result of the change that was not forecasted and is a direct result of the new architecture.

Minimizing these risks is possible through a methodical, structure migration program governed by the following concepts:

Establish a framework - define an application framework to align redevelopment and future development with the growing capabilities of a cloud, even if only a portion of the cloud functionality is available.
Challenge conventional thinking - there is no reason to assume clouds cannot benefit siloed applications, mainframes cannot participate in a cloud, CICS applications cannot be executed in a cloud, or that the end game is even cloud (it should be delivered value).
Crawl, walk, run - learn from the first efforts and cascade that learning into following efforts until an optimization loop becomes part of the DNA of migration projects.
Components or Containers - for each application determine whether its better to componentize, optimize the application for the new environment adopting a SOA model, or containeraize and keep the application isolated as a whole solution perhaps using web services to map inputs and outputs extending the reach of the application. In an earlier blog post I provide four options to handle legacy applications.
The learning curve - architects, designers, developers, and testers will all need to relearn significant parts of their job. If internal development is a core need make sure the necessary skills are identified, quantified and present within the teams.

Legacy and cloud can coexist in many forms. The reality is many applications do not deliver enough value to justify their migration to a cloud environment. The goal should be to maximize the value delivered which means applying cloud capabilities where there is a business case to do so.

The Dissolution of IT

2011-10-01T08:16:00.000-07:00

This post is based on one of the half dozen white papers I've written in my career which were too "out there" to be published. I wrote this one in 2007 about the time I left Diamond Management & Technology Consultants for what became a failed financial services SaaS startup. You have been forwarned!

In 2004 I went through the IBM certification process for Enterprise Architecture emerging as a certified Senior Enterprise Architect. Starting in 2001 I was pushing my architecture envelope to understand as much as I could about the business. I started questioning my mentor father about his experiences including his 30+ year history in Supply Chain Management. I started learning everything I could about cool sounding ERP processes: order to cash, quote to cash, and procure to pay. As I pushed my boundaries I found companies were doing the same; assigning senior business analysts as the liaisons to IT. I worked with many in partnership asking them to teach me about the business while I would deepen their understanding of IT. Each of these business experts I found had a common trait: all had an IT background.

As I applied my business knowledge I started getting accolades from clients about the robustness of my knowledge of their business. In project after project I became the lead business process architect as my approach and requirement to diagram everything helped senior leaders identify inefficiencies and opportunities. It was this experience combined with the revelation that my business partners all had IT backgrounds that I realized the end of IT was coming!

Not following? My hypothesis is that as we digitize businesses the people within IT, by necessity, become acutely aware of the ins and outs of a business process. As their job responsibilities grow they naturally become aware of the ins and outs of other business processes and the integration of how processes work together. They live the reality of the interconnectedness of business processes and their automation gaining an education on both sides simultaneously. By comparison their business colleagues gain additional depth on the business side but are largely shielded from the technology. As the pace of business continues to quicken, as it has throughout time, depth becomes less important giving way to breadth (i.e. a company needs fewer specialists but they are still needed, and more generalists). As if guided by Darwinian evolution, IT analysts are becoming the experts on the business as well as the technology used for its automation.

Companies are slowing realizing this largely untapped natural resource and have begun moving IT people out into the business vs the trend ten years ago to move business people into IT (which was necessary and a good thing). Therefore my prediction, if my hypothesis is correct, is that IT will slowly dissolve back into the businesses as the level of knowledge about technology takes a revolutionary step forward in a short period of time. The end result is an IT that is largely fungible between internal and external expertise. And cloud computing will help expedite this changeover through its concentration on automation.

IT will continue to exist, but its form will change dramatically! All the strategic planning and architecture will be done in what we now call "the business" and design, implementation, operations and program management services will be used (whether internal or outsourced) to execute on the plan. The CIO of today will become the COO of tomorrow and the CTO of today will be focused on one or more of the services and very likely will not be an employee. If I had to lay all of my cards on the table I'd argue the new role of consulting will be that CTO role and management of the services which will be executed by the traditional systems integrators.

If I'm correct, things to keep in mind include:

Vertical domain expertise will be required for those in IT who want to be on the forefront of the revolution
As a revolution I believe the critical mass will be reached rapidly and therefore believe the change will occur over the next 8-10yrs
Cloud computing will help drive this change
A natural synergy is the decoupling of internal technical expertise from its application which has hindered companies for decades from making great leaps
Legacy systems will not hinder the revolution because they're already in the mix due to the issues of cost, complexity and compliance
Demographic challenges in finding qualified people will drive the consolidation of positions that used to be split between the business and IT
Companies who get it right will benefit so tremendously they'll never turn back
The shift is already underway!

Rethinking IT for the Modern Era

2011-09-15T10:45:00.000-07:00

CIO's today need to ask themselves two questions. Would I have built my IT organization across the pillars of people, process, technology, structure and strategy to look like it does today? If not, what would I do differently? Why does a CIO need to ponder and pontificate on the state of IT? Because the pace of innovation continues to accelerate and we are rapidly approaching the end of steady state IT. It's time to Rethink IT and question all of our sacred rules.

For years IT has been focused on eliminating change in the name of stability, trying to reach the perfect steady state. However since the only constant is change IT has failed miserably in the mission, to the extent that many disagree it was a goal in the first place. I argue if change was the goal, then why is change within IT so difficult? Change management is not part of the core DNA of IT, but it should be. Instead IT does a bad job with change and a worse job with continual change. Yet change is the starkest reality of technology. I don't see anyone using Windows for Workgroups 3.11 or MS Word 5.0 anymore. Technology progresses and the pace is always quickening.

About 15 years ago the primary method for a customer to get support for a product was to call a 1-800 number. Quickly those requests starting coming in the form of emails in the mid 1990's followed by text chats and call-me-back requests in the early 2000's. Then in the mid 2000's we added social media such as MySpace, Twitter and LinkedIn along with discussion forums where customers could use a self-help model relying on other customers. In that same 15year time span many companies have, via bolt-ons and platform extensions, taken their original CRM solution and added functionality to support these new models. Still built on the original foundation of an inbound 1-800 call, we have added new staircases, wings, doors, chimneys, and floors to create the solution needed to serve the customer. What has resulted is something more analogous to the Winchester House than a family home. As a microcosm of IT, this approach of keeping legacy cores and extending solutions has created unwieldy, expensive, inefficient solutions. Rob Carter, CIO for FedEx, is on record stating that the greatest threat to FedEx is the complexity of IT. Rethinking IT, breaking down the old preconceived notions and forgone solutions, is the only way to address the critical mass of legacy IT.

Companies change their headquarter buildings more often than the applications which drive and manage their revenue stream. Architecture, for both buildings and software, is a managed planning step where decisions are made to support current needs and to the extent possible incorporate flexibility to meet undefined future needs. At some point in the lifetime of a building, or software, the limits of the architecture are reached becoming unsuitable for sustaining growth and must be overhauled or demolished to meet new demands.

Software, because it's viewed as inherently simple to change, is much more likely to be modified. Each modification makes the software a little bit slower, a bit harder to support, a bit more difficult to extend, and more expensive to operate. Add hundreds of modifications over decades and it's not surprising how many companies find themselves paying the legacy tax where changes, even minor changes, can cost millions of dollars. One aspect few consider is that little code is every eliminated. Multiple times in my career when asking why a process is executed we have found the answer in the code, the answer being because nobody told IT to eliminate that process or that algorithm. I contend there are millions of lines of superfluous code running every day which add to the ball of complexity and cost yet deliver NO VALUE! Worse, often those processes limit the business in what they can do which in turns limits the revenue generation capability of the company. There is a cost!

It's reasonable to expect that just as we moved through four technology iterations in 15 years (1-800 to email to text to social media), the next four iterations will take 10 years, and the next four after that only 5 years. IT can no longer afford the time and the business can no longer afford the cost of extending applications through bolt-ons. The only solution available today that provides the flexibility required is migrating to a services framework where new functionality is developed as a service available to other services as needed. Just as services replace monoliths (think ERP, CRM, SFA, MRP, etc.), application composition replaces application integration. The side benefit of moving to services is their natural alignment with cloud computing. In fact cloud computing as a holistic vision for IT is the execution of software services sitting on a virtualized hardware infrastructure. Amazing how things work together!

When the CIO resolves that no, the IT organization would look dramatically different if designed to meet today's needs, the first step on the Rethinking IT journey is under foot. Realizing the cost of not changing is greater than the cost of change means the second step is under foot. The remaining steps are to build the blueprint and roadmap and then effect the change with a particular emphasis on making change stick.

Of course the other option is to continue the status quo. Then it's time to ask question three: What happens to the CIO when IT is forced to forgo generations of innovation because of the time and cost required to update their ghastly complex legacy solutions?

Cloud in Healthcare

2011-09-05T08:38:00.000-07:00

A recent discussion on LinkedIn reminded me of my past exploits in deploying cloud solutions in Healthcare. I was involved in several projects with great organizations including Kaiser Permanente, The Diabetes Foundation, St Judes Children's Research Hospital, the LSU Eye Center and the Mayo Clinic. During these efforts we applied cloud in some really interesting ways including the following:

Diabetic Retinopathy: We presented on this at HIMSS in 2005. Our project addressed the rampant increase in diabetes among the poor of Louisiana. The two challenges were getting people to visit the doctor because the doctors were only available on certain days of the year, travelling to remote areas as part of their humanitarian efforts. The second challenge was finding qualified doctors who cloud leave their practices to travel. Our solution used low cost imaging devices to capture images of the back of a person's eye. It turns out the eye is one of the earliest ways to detect diabetes. The images were then uploaded to a cloud from which remote Doctors could identify early onset diabetes. Our goal was to improve service by using an automated filter to flag potential issues reducing the number of images reviewed by physicians. The approach resolved both challenges making the imaging locations available for a significantly longer time, some permanent, and didn't require the doctors to travel.

Long Term Digital Medical Image Archive: Digital medical images are captured in PACS systems which typically use expensive SAN technology for performance and redundancy. The challenge is that 98% of medical images are never used again after their first two weeks. As a result healthcare providers want to move the images to a low cost, long term archives as quickly as possible. HIPAA requires providers to keep the images for the life of the patient plus seven years, so retention is required. We developed a private cloud solution which provided an economical model for storing the images while also improving the ability to share the images between imaging centers, the hospital, ER/trauma centers, out-patient facilities and the primary physician's home office.

Electronic Medical Records: to be of use an electronic medical record (EMR), a folder containing a person's entire medical history from allergies and blood pressure readings to digital images and prescriptions, requires ubiquity in availability but strong security. Our solution used a cloud for the storage and management of EMR's. Each EMR was accessible through three methods. First, a person can grant others access to the account by creating logins. Second the person can grant temporary access by using their finger scan or RFID necklace. Third, a Doctor can request a limited version of the EMR in an emergency. Using WiFi enabled tablet devices the EMR's were visible via a browser from any enabled site (each site had it's own unique key which provided an additional layer of security so someone could not access the records from an unapproved location such as from home or the parking lot).

Disease Modeling: One area I found fascinating was the ability of medial researchers to create mathematical models to predict the course of a disease in a patient. They could identify with amazing results how weight, diet and even some genetic factors impacted the disease which could then be used to identify treatment regimens. Pharmaceutical companies could use the models to develop advanced chemical compounds. The compute power needed for even a simple analysis mandated the use of a cloud.

There is one other germane application I worked on however due to its proprietary nature and the potential value of the program I don't want to disclose it since I'm not the owner.

As I have repeatedly found, cloud really unlocks us from classical linear thinking in how we create solutions. We are in the early days of clouds and people way smarter than me will come up with solutions that will make us all stand in awe.

Time Value of Data

2011-08-31T11:47:00.001-07:00

Value is an inherent way humans quantify entities. Whether thoughts or products, ideas or tools, their value to us heavily influences how our brains classify the entity. Forks and spoons are of higher value than sporks, cars over horse and buggy, computers over typewriters, and so on. One early lesson we study in Economics is that money the value of money is not constant. A dollar today is, under normal economic condition, worth more than a dollar tomorrow. We call this the Time Value of Money to show that the timing of earnings and payments should be optimized to derive the greatest overall value.

To me it's a very gentle leap of faith to combine the Time value of Money with the idea that data is the new currency: I call it the Time Value of Data. Today we assume data has the same value on day one as it does on day infinity. For some data that's true. However I argue for the vast majority of data it's value starts at a peak and rapidly falls off nearing zero for the majority of its life until deletion (whether by intent or accident). For example how much I earn per hour today is something I safeguard however I much I earned per hour as a reservationist at Disney World in 1990 is pretty unimportant to me (the answer - $6.35/hr which was $1/hr better than the theme park jobs because of the typing skills).

When it comes to data we are pack rats loading the data onto our computerized pack mules. However one has to consider the security, efficiency, and monetary cost of this approach. From a security point of view should we protect data of low value with the same vigor as data of high value? Instinctively I say no but I can find no examples in the wild that show this rational thinking to be true. Valuing all data equally robs us of a valuable metric for finding pertinent data. We have to look through all the muck to find the golden data we need which impacts our ability to answer questions efficiently. And lastly there is a cost to storing data when it's value is at or near zero. Is my life going to be improved knowing the details of my mortgage from 2000 when I've refinanced twice since then? Not likely but keeping those papers has caused me to have to purchase yet another filing cabinet.

I don't have the answers; at this point I can only ask the question. I'm not sure how we define the value of data and how we differentiate the levels of value. I do, however, believe this is a larger issue of organizations not rationalizing the data they keep and use. It's easy to calculate the cost side, it's difficult to define the value equation. In money we use interest rates and years in a well defined mathematical equation to determine the current or future value of a dollar. A dollar is a single entity, data is all over the board, so they are not equivalent. However I do believe in one rule which I use at home. If I haven't touched something in 6mo it should go into the attic. If it's not used within a year then I have to question it's value. There are some pieces of data that we value forever such as baby photos. But like digital medical images, I bet 98% of data is never touched again after it's first few weeks.

But today we pay for it for it's entire lifetime.

The Role of Open Standards in Cloud Computing

2011-08-22T07:14:00.001-07:00

Over the past 12mo I have increasingly found what I feel is an obvious argument is mired in marketing hyperspeak. Cloud computing is predicated on open standards. There, it's been said as clearly as I can state it. Now to answer the question, why?

Probably the easiest way to answer this is by analogy. Remember back to the Betamax/VHS wars? How about CD/Digital Tape? Blu-Ray/HD DVD? The winner in each category had at least two of three benefits. First, lower cost. Second, media availability which is why anyone bought a device anyway. Third, ubiquity, availability in multiple formats. Believe it or not, technical superiority was not a driving factor. Beta was better than VHS, and HD-DVD better than Blu-ray. These three elements taken together drove market share which in turn determined who won the war and as such became the de facto standard.

So we proved that people like standards. Being the defacto standard meant being cheaper, having more media available, and being available in more form factors. Which in turn led to even cheaper devices... see the effect? What's amazing is that rather than fighting it out in the market to create a defacto standard, a standard can be set up front and thus drive the development of better solutions. (NOTE: open standards are simply standards for which there is no licensing fee. For example to use RAMBUS or JPG there is a licensing fee required and hence there's been a serious withdrawal from using either as a standard).

The goal of open standards is to drive interoperability, from day one, to maximize flexibility while keeping costs down. In turn the interoperability combines the efforts of all market entrants to achieve the three goals of driving down costs, increasing options, and enabling the development of new form factors. In fact open standards are everywhere. Your car has OBD-II, On Board Diagnostics 2nd Generation, which every automaker must comply with so a mechanic can connect a device and get a little help from your car in diagnosing a problem. HTML is another great example and there are plenty of others from email (SMTP, POP, IMAP) to managing IT (ITIL v3). In fact many of the technologies underlying cloud computing are themselves based on open standards. If we want cloud computing to be as ubiquitous and consumable as we all say, then we need open standards upon which to base interoperability. An isolated cloud is, in essence, nothing more than a large silo. Clouds need to have virtual boundaries which can be traversed easily and quickly as needs dictate.

So the real question is why aren't there open cloud interoperability standards?

The short answer is there are and more are coming. Standards evolve over time via learning. The original answer was the advanced advocates of cloud want to recoup their investment and make fat profits which is more easily done with proprietary technologies. However they almost all come around eventually. Several technology companies are very open to open standards including Rackspace, CA, AT&T, IBM, and HP. Others are coming late to the game like Amazon, VMWare, and Microsoft and still others may never join the team such as Google and Salesforce.com/Force.com.

Open cloud standards are here today in one form or another as evidenced by the Rackspace OpenStack and Eucalyptus. In fact OpenStack is quickly gaining momentum and warrants further investigation if you were previously unaware. In addition several governance groups have appeared beyond the original Open Cloud Manifesto, such as the OMG Cloud Standards Group, each working to establish open standards.

So when looking at cloud technologies keep in mind that proprietary solutions move you in the wrong direction. The days of proprietary vendor lock-in are not over. Open standards are out there so make sure they're part of your architecture and part of your technology selection criteria.

Learn From The Mistakes of Others

2011-08-21T07:57:00.000-07:00

In my career I've had some difficult projects but one in particular stands out as being my Bataan Death March. It was a project where you knew from the start that the journey would be painful and that everyone was kidding themselves in thinking the destination was anything like they envisioned. My responsibility was to help a marketing organization take the next step in rolling out their Enterprise Content Management solution. The list of project challenges is so long it's incredible to believe the project was even undertaken (which it was against my advice and the advice of several other experts). Here I will list the major pitfalls on the part of the customer, the consulting firm I worked for, and the technology to provide insight into what should have been done differently.

Day one I knew the project's scope did not reflect its challenges. The client was using the project as a way to effect an organization wide transformation to become a better customer focused company. First challenge: enterprise transformations do not happen through marketing. Transformations require the support of senior leadership, investment, and the acceptance of risk with defined risk mitigation plans and involve the entire company in one vision and one strategy. No foundation was in place beyond the feint, background support of the C-suite. I found repeatedly there was no structure, people or processes, in place to accept new responsibilities naturally resultant from the changes being made. This caused significant friction as work had to be done and there were no resources available. Second challenge: any transformation requires a well thought through and defined strategy. Other than having a vision, there was no strategy. Often during discussions when drilling down on a particular change the client was unable to articulate what changes should be made. Third challenge: project success requires competent people. Working long hours does not equate to success. The client project leader was ineffective at providing input to the timeline and managing client tasks. Due dates were meaningless and repeatedly missed without accepting the subsequent impacts to the timeline and cost of the project. The client subject matter expert was an off-site third party consultant spread extremely thin focused on maintaining their value add and bill rate rather than getting the project completed. The client technical leader was a junior resource with no architecture, application development or web experience who essentially played a very basic role as a go between with the hosting provider. All three client resources portrayed themselves as experts in the ECM space when, in reality, their knowledge was limited to content creation. In fact it was this arrogance that created the foundation for the technical challenges.

Based on the above items there was more than enough reason to not accept the risk of the project or run away when SOW negotiations took months while work was already underway. First challenge: senior consulting leadership was more focused on making a sale than doing the right thing for the consulting firm or the client. The engagement leader spent 1-2 days per week on the project and refused to listen to the highly experienced, dedicated directors on the ground every day. The engagement leader over rode concerns about the missing strategy and inability to articulate requirements; staffed the project as an off-shore development project with Java and .NET developers with no expertise on the ECM platform; and used the engagement to launch the new user experience team. One of the worst decisions was allowing the client to drag out statement of work negotiations for months while not being willing to secure the appropriate technical resources with the required ECM platform experience. The second worst decision was the continued willingness of the engagement leader to accept new responsibilities which could not be executed by the consulting team because they required client level decisions. Second challenge: misleading communication with the client. The client had no reason to believe the challenges were significant because, at the direction of the engagement leader, all communication with the client sponsor were closely held and minimal at best. Regardless of the issue escalated by the consulting team, the message to the client indicated everything was normal and manageable. The consulting team was pushed to be "creative" in finding ways to make progress but without making any decisions which the client properly reserved as their own responsibility. As a result the team created work, much of it not leading to a completed project. Third challenge: I tried to make it work when I knew the project would, and did, fail to meet its original scope, budget, and delivery date. At some point I should have engaged the other two directors and coordinated in intervention for the engagement leader escalating our concerns to the Quality Assurance partner.

And to make it a perfect storm, the underlying technology formed it's own issues. First challenge: the ECM platform used by the client had a unique architecture and implementation. Since many members of the consulting team had prior ECM experience they let their expectations drive their estimates rather than diving deep and realizing the uniqueness of the product. Second challenge: the ECM platform is relatively unknown and therefore expertise is rare and difficult to secure. It took several months to identify an expert and then several more weeks to get approval to get the person on-site. And then, because the project had been budgeted with off-shore resources, the cost of the expert pushed the project even further over budget. Since the client resources saw themselves as experts it was impossible without having the ECM platform expert engaged to change their minds about fundamental issues. Third challenge: the original ECM platform implementation was done incorrectly. The client had engaged a third party with no prior experience on the ECM platform to do the original implementation. In addition the third party did not push back on the client when requests were made for functionality that was dubious at best. As a result the solution had serious issues: difficulties in adding content, challenges in publishing pages, and areas of code which made no sense. To address this I architected a new implementation, having had prior platform experience, but required the added weight of the ECM expert to get the client to accept the architecture. The change essentially discarded the existing slow, unsupportable solution for a streamlined, very fast, very extensible solution which is currently in use addressing every issue the client had with the original implementation. An added benefit of the new architecture was it brought the off-shore resources back in to play as we were able to do development outside the platform in .NET rather than on the platform in its own proprietary structure.

Painful, eh? In the end the project cost two of the directors with the consulting firm their jobs, several people were given unflattering rating in the annual review process, and very little bonus money was paid out as the engagement leader looked for people on which to heap the blame. Conversely the engagement leader received a six figure bonus and is being considered for promotion to the Partnership. I guess it is good to be the king....

Unfortunately in follow-on projects little changed for the better as the engagement leader continues to be focused on revenue at the expense of people and the truth. In fact they have launched a new project with a new client using the same recipe for success with predictably the same results. However despite missing dates and going over budget the client is thrilled with the solution which, in the end, is the primary measure of success.

Yes, I understand you have legacy applications

2011-07-29T13:23:00.000-07:00

More than one CIO has explained to me that until someone comes to them with the solution on how to move their legacy apps into the cloud, they're not interested. Now that approach I find interesting. First, why isn't the CIO looking for the solution instead of waiting for someone to bring it to them? Second, does this mean they will continue to create new apps, which are the legacy apps of tomorrow, in a non-cloud manner perpetuating the problem indefinitely? And of course third, why would someone dig their heals in when it comes to changing the highest cost, least flexible elements of everything under their control?

The good news is that legacy apps are not precluded from cloud computing. The easy part is dealing with the legacy apps of tomorrow. It's becoming a defacto standard that all new development leverage a cloud architecture. Cloud architecture is a refinement of internet architecture which is a refinement of n-tier, client/server, and mainframe architectures. It's all about evolving as the technologies mature.

The hard part is what to do about the legacy apps of today. There are at least four options available: leave alone, enable with services, migrate key algorithms, and finally migrate. The leave alone option does not necessarily mean nothing can be done. For some mainframe apps there are other options. For example in my past we moved a large number of CICS applications from a mainframe to an HP9000 server with the benefit of higher throughput. In fact the Oracle Exalogic platform framework specifically identifies its ability to be used for CICS applications in the literature. And some applications can be moved with few changes such as a Smalltalk application ported from a mainframe to blades for a large brokerage to speed up execution. Never say never.

Option two, enable with services, is a pretty popular approach with those who will not wait. A large insurance company is working hard to enable its mainframe applications representing hundreds of millions of dollars in value to talk on the enterprise service bus (ESB) via services (I know because I designed the architecture). It is possible to keep that large investment largely intact and just change how it interacts with other applications. Through the ESB, a SOA concept, the mainframe apps have to respond in seconds in order to deliver web page results to prospective customers in a reasonable time, defined as no greater than eight seconds. Not only does this approach work, it can work well extending the life of applications to interact with the cloud at the software level while not participating at the hardware level (although I'll argue that with virtualization on the mainframe, they're just as much cloud as Zynga's games).

Migrating algorithms that are within applications rather than entire applications can deliver a huge benefit. Think of a pricing, scoring or inventory management application. The algorithm that helps management figure out what to do (the price, the score, or the shortages) can be move to, or replicated in, a cloud model enabling cloud applications to interact with the algorithm. To a cloud application that may be all that is needed, especially if that algorithm is exposed as a service. In good SOA design services should give answers, not expose raw data, and often all that's needed is the answer.

Finally migrating applications wholesale to cloud makes sense for some mainframe apps. However there is no requirement it happen today. The big issue has always been the business case; justifying the investment. CIO's have pushed application managers to put so much lipstick on these apps that they now resemble pigs. Bolt ons for integration, data management, identity management, internet access, storage tiering, systems management, and on and on combined with years of change requests have created systems so complex they carry a legacy tax. I have worked with several companies where just touching a mainframe application carries a minimum $1,000,000 fee. That's a pretty high buy-in, yet CIO's wonder why they're spending 70% of their budgets on maintenance. That first $1M is non-value added! CIO's need to expedite the sun-setting of these applications by building the case for change based on both the cost to change and the cost of not changing and accruing these legacy taxes. Sometimes you have to do what's right instead of what's easy and this is it.

The challenge to building the business case is the expectation that because computer code isn't physical it's easy to change. I balk at that argument with facts; if that were so why do software changes cost so much? I had a CIO challenge me to get the CFO to understand so I asked the CFO how much money had been spent to update/modify his highest producing factory while the CIO was to determine how much money has been spent to update/modify the most used application. The difference was staggering; the application costs were well over an order of magnitude higher although the plant was twice as old. So I asked the CFO if he were to invest that amount of money into operations, how much of the plant would have to change. He said he'd never do it, it wouldn't make sense. Instead he would build and outfit a new plant. Ah ha! So why do we use a different approach with software?

I'll admit that was a somewhat special case, but in the end it's all about foundation. Mainframe apps were built with a different architecture, intent, vision, everything. It's like investing in swords and armor and then being surprised when your enemy shows up in a kevlar body suit with his M4 and Seal Team Six tatoo. Ask the French how well their Maginot line worked.

Money Makes the World Go Round

2011-07-13T07:06:00.000-07:00

Like it or not money is a central part of all of our lives. As a consultant typically one of the first questions I'm asked is "How much?". I worked with a great client several years ago building their IT strategy to accommodate aggressive inorganic growth in the laundry room business. They operated laundry rooms at apartment and condo complexes and knew how to do it right. Their focus was on the management of the money and used it to diagnose any problems. First they could tell you how many loads would be washed and dried based on region and the number of units. They knew how much soap, water and electricity would be used. They could predict the required maintenance with amazing accuracy. Most importantly they knew how much money to expect from every washer, every drier, and every soap dispenser every day across their $1B organization. The EBITDA for each washer. The operating profit of each laundry room. Simply amazing!

Contrast this with the norm at Fortune 500 companies. It is a rare day when a client can tell me how much their technology costs per seat. I've been laughed at and asked why anyone would care. I've been told there are too many variables. Yet when I talk with CEO's about building a new business or offering a new capability the first thing they ask is "How much?". As a guide I typically ask "How much does technology cost per seat today and how many people do you estimate being involved?". Applying an average to generate a back-of-the-napkin estimate works pretty well. So the CEO asks the CFO or CIO who invariably don't know (and if not are not happy to be put on the spot) and invariably we end up taking their technology budget and dividing by the number of people. It's a start, but it's not transformative.

******************************** HINT ******************************

For anyone considering a technology rationalization effort, knowing the detailed costs is a fundamental requirement yet derails more projects than any other single cause based on my experience.

********************************************************************

Whereas the average number gives you a starting point it doesn't give you anything on which to provide an insightful analysis. If that number were good enough the next thing in line would be the contract, just sign on the bottom line. Yet what ends up happening is an exhaustive research project with consultants and IT finance gurus to get to the bottom of what everything really costs. Some times it becomes a project unto itself. Worst of all return in a couple years when the details have changed and you'll typically find you have to start all over again; nobody picked up the work. And I won't go into all the rocks you overturn exposing contracts being paid that are no longer required, missing maintenance on production servers, software licenses that are oversubscribed, and more that make you ask one question: "Who owns the money responsibility in IT?".

I cannot rationalize this lackadaisical approach by IT to activity based costing. On the one hand CIO's bemoan the increasing responsibility of IT with a shrinking or static budget. To me that means knowing the who, what, where, when, how, and why of every dime. Yet there's no transparency in the 2/3 of IT budgets which are operations focused. That tells me IT wants to deliver the message "we don't care what things cost" but somehow I've missed the legions of CIO's standing before their leadership to make that statement.

IT needs to know what things cost and track those costs for all the things it has. There are some tremendous tools out there to help with this but, and perhaps it's just me, I've seen an amazing lack of use. This underlies the larger issue of collaboration between the business and IT. The business talks in dollars, capital vs operating expense. IT has to use the same terminology to be understood, however that's very difficult when getting answers to simple questions like "What does that cost you per year?" or "What's the cost per seat?" take days, weeks, or even months to calculate.

I've been advocating this approach since I ran the IT department for a small call center outsourcer in the mid 1990's. We calculated the cost of everything at various levels (seat, program, location, company) which fed into our estimation process. As a result we could tell a client within a day or two of knowing their requirements the one-time startup fee and the ongoing fees within +/- 10% accuracy. IT was a big part of our business so we ran IT as a business.

This approach is now considered a best practice according to the MIT CISR researchers Peter Weill and Jeanne W. Ross as outlined in their book IT Savvy. So if you don't believe me, believe them, it will pay dividends and is a fundamental component of the success of IT.

Where's the Business?

2011-06-27T07:17:00.000-07:00

Despite all my attempts it appears cloud computing continues to be a technology topic, at least in the Fortune 1000. Over the past year secrets of cloud including Facebook's private cloud, Zynga's use of 12,500 Amazon EC2 instances, and NetFlix creation of their Chaos Monkey have made big company executives ask what are we doing? Hey Mr CIO are we doing these things? What are we doing that we can tout in Fortune magazine or at our next board meeting? Are we in the cloud? I wouldn't be surprise to find a few CIO's have broken down in heaving sobs or curled up on the floor with their thumb in their mouth.

The preconceived notion is that cloud is all about TECHNOLOGY. Guess what, it's not! In reality each of those examples, and the ones CEO's and CFO's trot out to "motivate" their CIO's have nothing to do with technology. Rather, each has to do with the business. It's the needs of the business which drove the application of the technology, not vice versa. Therefore the right question is not CEO to CIO but CIO to the CEO: "Here is what cloud enables businesses to do differently. How can we take advantage of it?"

As clouds are being built the enemy of efficiency, fiefdom building, is following right along. With the number of business savvy CIO's in the Fortune 1000 I'm surprised at how few have engaged the business in a discussion outside of cost savings. Great, run applications and all at a lower cost. But that wasn't the goal of Facebook valued at over $50B. Or of Zynga who is the fastest growing game company on the globe. Or NetFlix who singlehandedly put Blockbuster out of business and force the cable operators to take notice. Either the CIO's don't understand it which I doubt, nobody will listen, or too many people are busy building fiefdoms to protect themselves and therefore are unwilling to share their knowledge.

I met the CTO of Grooveshark, a great web based music delivery service, and we talked about their technology a bit. What interested me was his lack of interest. He had business problems to solve and that was his focus. He didn't want to talk technology.

I have said in front of clients, conferences, and on this blog that technology is the easy part of cloud. And I understand it's also the most visible thanks to the marketing of companies like Microsoft, Oracle and Google. However the value of cloud is in solving real business problems. The impact of cloud is in its transformative value; moving a company further into the digitization of its services.

Technology is interesting. Cost savings is great. But last time I checked companies are ranked by and investors take notice of revenue, profits, and growth.

So What Is the Public Cloud

2011-06-13T11:15:00.000-07:00

Before going to far past one of my recent posts on the evil of private clouds, I suppose I have to define what I mean by private and public. I take a very simple point of view on whether a cloud is public or private: who owns the assets. In private clouds the assets are owned by the consumer; in public clouds the assets are owned by an entity other than the consumer (called a provider). In my view of the cloud world, a public cloud can exist within the four walls of an organization in the same way a private cloud can be hosted externally. I see no other way to differentiate although many try:

Access: Many argue what makes a cloud public is who has access to the cloud. However the cloud doesn't really exist until something is provisioned. Once that thing (service, server, whatever) is provisioned you have a cloud but you also necessarily have restrictions on who can access the server. Amazon AWS, Microsoft Azure, all the vendors require logins and keys and such to protect their systems. And it's always possible to open up the gates and allow external entities to access a cloud within a company's data center. So really the differentiation cannot be access.

Security: Some argue its the security model and domain that make a public cloud. Public clouds are more accessible and therefore are more at risk. I point to all the penetrations at large companies by hackers to demonstrate at best there is no difference.

Multi-tenant: Many argue public cloud is multi-tenant. How is this different if its my competition or my sales department when the virtual machines and storage are all partitioned and sit in separate security domains? Again I see this as more similar between private and public clouds and less different.

The benefit of differentiating cloud models by who owns the assets is it plays into the focus of CFO's, covered in an earlier post, to drive IT to leverage other people's assets. IT needs move from a capital/asset intensive environment to one where costs are expensed and vary with consumption.

Therefore in my humblest of opinions using my point of differentiation, everything will move to public cloud. Why? The benefit of leveraging other people's assets!

Who's Holding Cloud Back? IT!

2011-06-05T06:32:00.001-07:00

Fear, uncertainty and doubt; also known as FUD. If it were up to me we'd change the name of IT to FUD because modern technology organizations are more about telling you what you can't do and why than figuring out how to do what's needed and doing it. FUD-IT. Hmmm...have to remember that one.

Today we face a talent crisis in Cloud Computing that it appears almost nobody has been preparing for over the past five years. Go try to hire a cloud experienced anything. Better yet, go hire a cloud experienced Enterprise Architect, Strategist, or Program Manager. You'll have an easier time replacing your C-suite. The talent is already employed and most companies are anchoring their talent to the ground with impressive golden hand-cuffs. Most popular right now are option grants which seem to run anywhere from $50k/yr to $100k/yr for top talent. And there are cash bonuses which are paid over time to entice people to stay, salary increases, the works. Hopefully my employer catches on...

For the past several months I've been blaming poor management by CIO's and their HR counterparts for not forecasting the need and establishing internal training and development programs to "grow your own". Although I'm not changing that position I realized after a few conversations just how formidable a problem such a plan would have to overcome. Developing cloud talent requires three key ingredients:

Willingness to learn
Desire to be broadly experienced rather than a specialist
Investment
Appetite for risk

Ooops, guess that's four ingredients. CIO's can be held accountable for the bottom two, but number 1 is squarely on the shoulders of the IT staff. Too many, I'll forecast the majority, of IT people today either don't have, have lost, or never had a willingness and desire to learn. They learn out of necessity, not desire, and as a result the majority have large blind spots. For some reason they resent the pace of technical innovation and the need to keep abreast. Perhaps some believe that you have to "relearn everything" every four years and figure what's the point (note: you don't have to relearn everything unless you stay a technician your whole life, for everyone else you realize the more things change the more they stay the same).

People are quick to give lip service when asked about being trained in cloud. We get several people who sign-up for lunch-and-learn sessions and attend 1hr presentations. However the number, more importantly the percentage who have stepped up and said "Hey, I want to learn about cloud" is so dismally small that when I ask HR they reply "Eh? No, nobody's asking about training". Talking with friends and colleagues we estimate about 2-3% of the IT staff have demonstrated, not talked about but executed on, a desire to learn about cloud computing. My favorite reason people give when we've asked why they aren't doing it on their own? Not enough time. Hmmm. Okay. Perhaps they don't realize that Detroit didn't have enough time to learn about economical cars, or Pittsburgh about producing low cost steel, or Las Vegas about the peril of building more homes than there are purchasers. I'll put a dallop of blame on IT and HR for not encouraging the training, in fact making it mandatory in some way, and financing part of it through paying for courses or giving people time or both.

When it comes to item two I see it as a 70/30 split with 70% on the shoulders of IT staff who want to specialize and 30% on IT/HR leadership for encouraging specialization for the past decade. I love when I talk to companies about their biggest problems in IT and they go on about changing technologies, business people who "don't get it", etc. yet when I ask them about their process for finding the root cause of an issue it invariably involves a team of 25 people. Why? Because everyone is a specialist so there's nobody who gets it from end to end. And of course it takes forever to find the actual problem because they all speak different languages. It's a problem the size of the Tower of Babel but one easily swept under the rug because it's only visible to IT.

So we have an under-educated group of technology consumers (can't call them practitioners until they've done it successfully several times) all scurrying around either telling everyone how lousy cloud is or trying to implement it without any help. What drives their thoughts? FUD. It's not just the naysayers; they're only the most obvious. Fear is the gensis of private clouds. Uncertainty keeps public companies from researching public cloud storage. And doubt keeps companies from applying cloud to production systems.

People filled with FUD need to remember this axoim: Lead, follow, or get out of the way!

And to those just tuning in - Private Cloud is a bad thing

2011-05-02T07:57:00.001-07:00

Private Cloud, in itself, is not bad. Rather what I find is that companies are willing to blindly follow the Private Cloud mantra while heaping their own expectations of an Amazon AWS or Microsoft Azure solution on top. As a result the deliveries fall well short of the perceived goals leaving companies in a no win situation. The cause of this problem is the vendor push behind Private Cloud solutions which naturally want to make it sound like a pancea for taming the infrasructure beast. In reality Private Cloud is in its infancy and is about 5-7 years behind Public Cloud in maturity.

Private Cloud implementations often mask or ignore the following:

Enterprise Cloud Strategy development
Governance
Migration costs
Application development costs
Learning curve
Disaster recovery effectiveness
Success metrics
Accounting, Tax, and Finance change
Legal impacts
Data integration

I can understand the lure of a "private" cloud: COST SAVINGS! Who can shake a fist at that? IT is expensive, not as expensive as the manual labor it replaces, but more expensive than free. Executives would prefer an IT that costs nothing to maintain where more can be done with less until an infinite workload is accomplished with nothing.

A cost savings focus is a myopic view of the future which necessarily puts more important agendas like growth on the back burner. My father, a retired Fortune 10 executive, always says "You can't cut your way to growth". Yet today the vast majority of Private Cloud implementations are driven by expected cost savings. So what do I believe is the right view? Cloud as a revenue generator, a driver of new business opportunities by engendering models not realizable with current IT structures. But of course this view threatens the very safe, reliable foundation of today's IT.

So let me point out how Private Cloud falls short on cost savings:

First, private cloud increases risk by moving companies to adopt a single vendor's proprietary technology. No vendor today has a complete private cloud in a box or offering. Rather vendors have gathered together technologies from across their portfolio to provide a glimpse of cloud but what is really just virtualization. Virtualization is good, but it's not cloud. I have yet to see a private cloud vendor offering that addresses the 70% of cloud that virtualization does not: software services, SLA based management of applications, automated scaling of resources based on application use and need, automated recovery of services, etc. Some of these things you can do with those vendor solutions, but it's an afterthought and what is offered is such a weak solution few if any have implemented it. Ask the vendor for a list of installations and it looks impressive. As a vendor for the list of clients who use 100 or more instances and the sheet staring back at you will be blank. Vendors are following the analysts too; if they could see they'd be leading rather than following.

Second, Private Cloud pretends to be a holistic solution but it's not. Again nothing is provided for in migrating applications into the cloud, and without applications all private cloud does is create larger silos. Most often there is no consideration for future integration with public cloud locking out the opportunity to access the lowest cost resources available. In addition Private Cloud requires a company to grow an expertise base that cloud as easily be outsourced where the implementations from an availability, security, and performance point of view are generally accepted as better. Once the cloud is up and running it should take very little effort to operate it; otherwise it's not a cloud. The money should be spent in building up application layer expertise in cloud which is much more expensive.

Third and most important, Private Cloud failures, and there are plenty, give cloud a bad name creating conflict for future adoptions potentially alienating one of the most valuable technological movements since the development of the relational database or the Internet. Companies are already having to rationalize cloud solutions that are only five and sometimes only three years old. Nobody wants to present to the CIO that a new solution is already in jeopardy. When I ask people why they think this happens almost 100% say its because of undetected security issues which is incorrect. The real reason is the unrecognized need, complexity, and cost of integration.

So to those who are learning I say welcome! To those are surprised to learn that private clouds could be a bad thing please do your own analysis. Cloud computing is one area where you need to do your research not only on the topic but also on those to whom you rely for knowledge.

One of the Hidden Challenges of Cloud - the Blank

2011-02-21T10:45:00.000-08:00

I spent many years as a developer in embedded systems, working on control systems to run everything from washing machines and ice makers to engines and transmissions. I learned in that time to make a product great required people who were able to forsee and avoid issues, and those who were able to debug and fix problems. As I transitioned into enterprise IT I found the same rules applied. That combination of talent is behind all great solutions from bridges and governments to iPod's and operating systems.

Considering the technological breadth and depth of cloud which crosses every technology boundary and impacts elements in every layer, no one person can be a one stop solution shop. It will take time and experience, successes and failures, before even the smartest gain the insight to know instinctively what works and what doesn't. And therefore like all new grand experiments we need to start off with the soundest foundation we can which gives us the greatest flexibility to adjust as we learn.

Inherent in this approach is having two talents at our disposal: the cloud architect who views cloud from the business point of view as a technology implementation of a business process; and the blank. It's not a sexy term, the blank, but it's very appropriate. Most organizations have architects, or at least an architect, who is responsible for ensuring technology is implemented logically according to a master design which aligns with the business strategy. These architects are always focusing on the future with limited feedback on how things are working today. However I have yet to find a company or person whose sole job it is to identify and resolve issues and inefficiencies; someone focused on today and how to make things better in the future.

Today when an issue arises it goes to a committee of people: application and data gurus, infrastructure engineers, architects, security shaman. There is no role responsible for capturing a problem, identifying the cost of the problem, then resolving the problem if its worth resolving. As a result the tools we employ to identify and debug problems is deplorable. Our infrastructure teams have gone the furthest down this route in both methodology with ITIL and tools including instrumentation and health monitoring. What about the other three fourths of the world: applications, data, and integration?

Cloud computing only works when the user uses the application. Whatever process it is the user wants to execute must come to a complete and acceptable resolution for the user to continue using the application. Cloud solutions are inherently complex leveraging a dynamic set of services across a dynamic infrastructure with a combination of static and potentially dynamic data sources. Yet for the most part the only tools available for tracing the cause of a problem end at the boundary. If the process doesn't end at the boundary neither can the debugging. If we can't resolve the problem quickly the application provides no value to the user, and tolerance continues to dwindle.

Today we rely on the hero model; that really smart person or group of people who combine intelligence and experience to sweep in and save the day. In the cloud we need to reset that experience to near zero while at least doubling the breadth of the issue. Our only option for survival is to start emphasizing the need for robust tools. However without a focal point, a role whose success depends on such tools, the tools will continue to stop at the domain boundaries: infrastructure, applications, data, and integration.

We need the blank, or perhaps a better name is Quality Architect, a role across domains responsible for ensuring our systems can be debugged, our knowledge is increased and captured, and for being the person in charge when a serious issue arises.

Today it's a blind spot we survive by luck and hard work. In the cloud with it's ability to rapidly scale and morph, that blind spot will become a killing field with the careers of well intended but unsupported IT experts littering the ground.

The CIO, the train, and the tracks

2011-02-09T07:03:00.000-08:00

In today's IT world the center of the universe is the CIO. They are tasked with ensuring IT is aligned with and delivers on the business mission. Today's CIO's are very business savvy with experience across multiple operations and support units. Often they know how the business operates, the underlying processes and compliance requirements, better than the line of business executives and sometimes better than the COO. However what has been lost over the past 15yrs is a fundamental understanding of technology. I recently had a CIO tell me at a conference "This job is great because I don't have to understand any of the technology stuff; I have a staff to do that. If I had to understand it all I'd be out of a job". What she doesn't realize is she has been laying the tracks to her own destruction for years by not learning about the technology and soon the Cloud Computing train is going to run her down, run her over, and keep on running.

A CIO needs to be a balance between business acumen and technology expertise. They don't need to be an expert in either, but they need to blend in on both sides somewhat like a chameleon. How many COO's know how to operate a stamping press, extruder or fork truck? Probably more than you realize but certainly not all. However they know what the equipment does, why it does it, it's value in the production chain, and often how it operates. I grew up the son of an operations executive in the food industry and I can tell you he knew how every part fit together. Why? Because when bad things happened he invariably got the phone call. Many of our new CIO's have never spent a day in IT prior to becoming an executive. Nobody appreciates the balance between the portal, application, enterprise service bus, and database and the underlying utilities, operating systems, processors, storage arrays, networks, and security appliances who hasn't served in an IT operations role. And think of everything I'm leaving out: architecture, quality assurance, requirements, SDLC, etc, etc, etc.

People today see technology as so complex and difficult to understand and yet they use it in their daily life without a moment of pause. The reality is technology is not complex in its implementation. The complexity is in developing new technologies, a business the vast number of CIO's are not and will never be involved in. So this false wall they hide behind is inexcusable. They should be a technology expert as much as a business expert. They should be held accountable for the architecture as much as the budget. They should be able to explain how a database operates as easily as how depreciation is applied to hardware.

We need to raise the bar!

The train is already on the tracks and it's accelerating. I predict cloud computing will end the careers of 50% or more of the current Fortune 1000 CIO's. Why? For several reasons:

Some will be held accountable for not researching and implementing these technologies when they started appearing in the 2001-2005 timeframe. Think of all the lost value that accumulated over the five year period from 2005 to 2010 as companies ignored cloud.
Many will simply be outpaced by their younger, nimbler competition who understand that CIO's need a balance of skills and can operate as their equal on the business side while still having deep technology knowledge and experience.
Most will be held accountable for a series of failed implementations because they waited to long or never did develop a cloud strategy. So many of the implementations today have been built to a vision with no supporting strategy. Already companies are scrambling to remedy siloed cloud implementations and rationalize multiple instances of the same cloud service.

Whether or not my prediction comes true is largely based on how the media covers cloud, how well CEO's hold their CIO's accountable, and vocal shareholders become. CIO's already have low tenure so my belief is it won't take much to tip the balance.

If I were a CIO I'd be sure to get educated, identify a successor, and get my finances in order.

Cloud is a Good Thing - Vendor Lock-in Is Not

2011-02-06T10:12:00.000-08:00

I equate vendor lock-in to anti-cloud. The entire purpose and value proposition of cloud rests on it having dynamically defined borders. Such an open architecture requires, by nature, strong standards to facilitate the interchange of data, processes, and policies. However underscoring the lack of available expertise in the market and the willingness of leaders to attack without understanding, companies are moving headstrong with short-sighted tactics (I dare not call them strategies for the amount of thinking support the tactic is barely enough to call it more than a whim!). Companies are simply mortgaging their future for benefits today.

The landscape today is Private Cloud. I have yet to see interoperability standards proposed by or supported by a Private Cloud vendor, whether a software provider such as VMWare or a service provider such as Amazon. It was only a few years ago where the cost of virtualization software was so high it was cheaper to add physical servers. Adopting a proprietary platform today is likely not only to lead to higher overall costs, but solution isolation without the adoption of interoperability standards.

As is most often the case I believe it will fall to the open source world to define, build and adopt open standards. Xen came in to existence to duplicate VMWare without the high cost, bloat, and closed architecture. Look at what Amazon AWS has done with Xen! A full slate of tools of in development in the open source community along with improvements in open source operating systems which will subsume many of the capabilities paid for in VMWare, Hyper-V and other solutions today. What today is a service or software will become a component and foundation.

I'm not ready to short the stocks of EMC (VWWare's parent), Microsoft or Citrix (owner of Xen), but I'm watching closely!

Private Cloud is Not Just Infrastructure

2011-01-31T06:58:00.000-08:00

I understand the push for Private Clouds. Major technologies incubate inside IT shops before they're moved to hosting providers and outsourcers. So it's no surprise to gain traction cloud needed it's internal equivalent and hence Private Cloud. The cynical side of me also recognizes that Private Cloud kowtows to the security wonks who see their jobs as preventing rather than enabling interactions. It also recognizes that no CIO wants to go to the CEO and explain the past several years investment in data centers now needs to be divested as everything moves to the cloud. As leaders continue to learn about the economics of public cloud and how they cannot, simply cannot be replicated in a Private Cloud, they've now started to reply "Hey, when I said Private Cloud I didn't mean it had to be in my data center. It could be outsourced." I understand it's hard to hit a moving target.

What everyone seems to be forgetting is that private cloud, like any cloud, is half infrastructure virtualization and process automation, and half application virtualization and process automation. A cloud does not exist with only one part. Yet reviewing the private cloud offerings of several software vendors such as VMWare and VCE show their definition is woefully short of a full cloud.

What about application service provisioning? What about the service discovery? What about service level management? What about session availability and continuity? None of these topics are covered by private cloud software vendors.

Where there is value, but it's more that "Private Cloud nesting underneath Public Cloud" flavor are the Private Cloud solution providers (CSC, Amazon ,etc.). They offer an entire stack, PaaS sitting on IaaS, on which one can deploy a private cloud.

A private cloud that only includes infrastructure is not private cloud; it's highly virtualized infrastructure. And sometimes it's not even highly virtualized...

Project Clouds

2010-10-18T09:29:00.000-07:00

As the debate over the value of Public Clouds rages on (not sure why, but it does), companies are just now starting to see the immediate value of cloud computing. I realize that people who run IT are risk averse (which is why there is so little innovation coming out of IT), resistant to change (which is why things rarely get better), and rarely comprehend the technologies they oversee (I won't expound on this one). However they have one tremendous trait their predecessors did not: they understand the business. This viewpoint is starting to leak into the thinking about cloud computing and how to take advantage of it. Rather than focus on fighting the governance, security, ownership, ego wars of change, cloud computing can be applied to projects!

The real driver behind project clouds is speed to market. Things have to happen quickly to seize opportunities. Sun Tzu teaches one to assume whatever one is doing, their enemy is also doing, and therefore the winner is often determined by the one who achieves the objective first. I have found over my twenty years in IT and consulting that the larger the company, the longer simple tasks take in IT. Nowhere is this more evident than in the time it takes to provision environments, applications, and other technology tools for projects. Today the business cannot wait for IT to go through all their risk avoidance, cost containment, change preventing processes.

Every project needs its own set of tools from document sharing to issue tracking and often more specialized, project specific tools. My most recent example. We had our cloud based project management, issue tracking, document sharing, and prototyping solutions up and running in one day. It took over two months for IT to provision the sandbox and development environments. Worse, the environments were over 30days late on a three month development timeline. Real world differences like these point to the need for using cloud solutions. Pay-as-you-go, instant availability and scalability, what's not to like? Adopting the cloud is a risk avoidance technique, it's the lowest cost possible, and it requires IT to make no changes.

And security is no longer an excuse for sitting on the sidelines. With billions of dollars in transactions traversing public networks and clouds already we've proven that existing security solutions are adequate to the challenges of the cloud.

IT can either enable this process, or fight against it. However nobody should ever choose to stand in the way of progress; history has not been kind to those who have.

Everything Should be Software

2010-08-31T10:12:00.000-07:00

If there's one lesson we've learned over the years it's that software is king. The tech titans of the world are Microsoft and Google, not IBM and HP. Hardware has its place but suffers from multiple obstacles including the laws of physics and the depth of education required to drive innovation. Innovation on the software side however can be as easy as doing something for the first time, and there are hardly any obstacles. As a result we have proven that software is significantly easier to build, manage, update and deploy than hardware.

The question I have is why does hardware exist in so many different forms today? Many have jumped on the bandwagon incorporating low-power processors with embedded Linux to create appliances from DVR's to EKG's and even engine control modules. What about inside the data center? We pretend today there is a big difference between the "pizza box" 2U rackmount servers with one or two multi-core CPU's and the big box 16-way multi-core machines which require a small power plant to operate. In the end the only difference between the two machines is which offers the lowest cost per performance unit.

Looking introspectively at what we today consider hardware or infrastructure applications many are nothing more than specialized servers. The true value is in the software, not the hardware. Examples include backup devices, switches, bridges, routers, DNS, directory services, and the list continues to grow.

Virtualization has taken us to the limit of the existing hardware architecture. The next step is the continued commoditization of the hardware where specialization is moved into software. The key enabler of this migration is the capability of the modern processor, both in speed and bandwidth. Many hardware devices have migrated to "appliances" and will soon be migrating to software packages. Why does this matter? Because we can leverage the existing virtualized environment to move, grow, and shrink these services just as we do application, web, and database servers.

I consider the FPGA (field programmable gate array) as the ultimate model for the data center. Such a data center would be composed of a generic set of servers configured on the fly to provide ALL of the services required from load balancing to routing, naming services to applications, data to system management. This move would improve resiliency, lower overall TCO, and provide the most efficient infrastructure possible at any given time. Considering the end of rotating storage is on the horizon, such a static hardware architecture would provide the ultimate in dynamic services!

The Secret Sauce

2010-07-19T11:07:00.000-07:00

Those of us who have been in IT over the past decade have seen a steady increase in the focus on business/IT alignment. Once the focus of Enterprise Architects, it's now become a part of everyone's job. But what exactly is the end goal? Ask the business leaders and the common answer is a more agile IT capable of making changes closer to the rapid pace of business and further from the lethargic, slothlike pace IT has followed for 30 years.

Offended? Well the reality is compared to the business IT is lethargic. The business has become nimble and I've finally discovered the secret to their sauce: assets. The business carries as few assets as possible. Everything today is an expense. Buildings, labor, equipment, everything. What was once capital is now leased resulting in fewer assets. Fewer assets to manage means greater agility. One specialty retailer I know of can put up a new store, from signing the lease to opening the door, in two days. They can shut one down in one day. Wow! Now that's agile.

How about IT? Most companies I know cannot setup a server, from request through completion, in two days. In fact I don' know of a company that can decommission a server in one day. See why the business thinks we lethargic?

So what we need on the IT side is more of that special sauce. We need to reduce our capital intensive asset driven footprint. Sure we already lease servers and data centers, but that's not enough. We need to adopt cloud computing and leverage other people's assets to deliver against our agenda. The best way to make money is to use other people's money. That's what cloud is all about. It's the opportunity IT has been longing for to gain the required agility to meet the business on their terms in their timeframes. From spinning up a new server or storage to rolling out a new application, timeframes need to be measure in hours and days, not months and years. SaaS solutions can put companies at the starting line on important enterprise solutions such as ERP, CRM, SFA, and MRP in days at a cost significantly lower than those opting for installed monliths like SAP and Oracle. And where do the savings go? Right to that bottom line!

Another secret? Innovation requires continuous tinkering and feedback loops. Product companies test, test, and retest. Focus groups, test markets, viral campaigns. It's all about iterations. Cloud can help IT in this approach as well, minimizing both the cost and time required for each iteration. More iterations means better results.

I always new the answer lay in the secret sauce!

Cloud Ain't Easy

2010-06-02T13:37:00.001-07:00

If there's one thing I've learned over the past decade working with grid computing and virtualization technologies is that innovation isn't easy. Cloud computing is an innovation initiative. It's not one thing but rather the implementation, configuration, integration, and automation of several things. It's not avaialble from one provider, off the shelf, in an easy to install and support package.

No. Cloud Computing is more like a jungle. Lots of custom scripting and some custom development. Lots of building interim and bridge applications. Lots of pushing vendors to deliver what you need rather than what they want to provide. Add to it the confusion over what cloud is (as recently as Vegas InterOp in April I heard someone state cloud was just another name for server virtualization) and it's no surprise it's a jungle.

Cloud only makes sense when it's done right. It needs a solid architecture at it's foundation. It needs a strategy and roadmap to ensure it's built from the start to deliver real business value. It needs to align with business and technology goals. And it needs to pay for itself where nobody wants to pay for infrastructure. It can be done, just ask Bechtel, but it's not easy.

First there's the skills gap. Many, if not most, organizations plan to go the cloud computing route on their own. The underlying strategy is vendor driven mistaking the vendor's best interests for their own. Whether it's Amazon, VMWare or Citrix I assure you tying to their strategy only ensures you'll be a profit generator for them. The most prominent skills gap is in enterprise architecture which is where cloud's rubber should be hitting the road. But we don't have enough EA's in the world and especially not enough with cloud skills. So instead companies forge ahead on their own with predictable results:
- lack of synchronization between technology teams
- lack of security standards and policies which address the technology footprint of cloud
- poor or missing governance structures
- growth of stealth IT in the form of expensed SaaS and even IaaS cloud services
- lack of understanding by the business how cloud can enhance what they do today

Second there's the focus on cost takeout. True, cloud optimizes the infrastructure stack which means more bang for the buck. But doesn't lower cost mean you can perform more iterations for the same budget? Learning is an iterative process so if we can learn more for the same dollar, don't we become smarter? Isn't this the foundation of innovation? Trying something new, tweaking it, and trying again until we have the better mouse trap?

Third there's the belief that IT controls technology. Technology goes wherever the money is and today with a credit card anyone can move to the cloud.

Fourth and most scary there's the "Cloud at any cost" crew who are innovating in the cloud without proper planning and strategy, exposing their companies to massive risk which cannot be quantified. Who owns the data? Is the company properly indemnified from patent infringement? Where does the data go? Who has the authority to ramp usage up or down? Is any regulated data involved? What if the cloud vendor disappears or goes bankrupt?

Cloud ain't easy. It requires planning and strategy. It requires both a business and technology point of view. Most of all it requires people to realize as a new, disruptive technology the old rules don't apply using the nice, easy to use cookie cutters we've created for the Internet.

Cloud simply ain't easy, but it sure is compelling!

But It's Not About the Technology

2010-03-31T07:23:00.000-07:00

Today the conversation on cloud computing continues having migrated slowly from definitions and scope to adoption and migration. The primary focus of IT professionals from the CIO and CTO through system administrators is cost reduction; maximizing the use of existing hardware. Secondary, in fact a very distant second whom only a few really seem to understand, is the application of cloud technologies to reduce complexity by drawing distinct lines of responsibility between the layers of the IT stack. Both goals are reasonable, even commendable, but they only breach the surface of the value of cloud computing. With Cloud, it isn't about the technology.

So what is it about? The business. As with so many other technologies that have come before we are again falling into that time honored trap of applying technology for it's own sake. Saving a few dollars here and there may sound like the interests of the business are first, but that's a very myopic point of view. Very few IT organizations have taken the time to explain cloud computing to their business leaders and engage them in the discussion. In fact I am actively looking for my first example so I can praise them, learn from them, and write about them. Stay tuned but don't expect any posts on the topic in the near future.

Cloud Computing provides a new model to loosely couple organizations providing them a dynamic, opportunity based model to share services and data. As the landscape in various industries shrinks through competition and acquisition pure head-to-head rivals fade replaced by the new model of "co-opetition". Companies today find themselves increasingly competing as well as cooperating with their partners. The concept isn't new but instead of being a rare exception it has emerged as a predominant model. How do you draw distinct lines in the sand to separate you from a competitor in one area when you want to share information and assets with the same company in another area. In the past we have created physical partitions which, by their static nature, often limited the growth of the partnership because the partitions either didn't or couldn't change easily. Workarounds become common which lead to wasted labor, inefficiencies, and increasing complexity. Cloud technologies with virutalization at the core provide the toolset required to dynamically map and remap resources. Adopt a cloud model whereby each partition becomes a cloud and the opportunities explode.

I am a firm believer the value of cloud computing lies in its ability to take collaboration further, extending our model of human-to-human cooperation between companies to our technology assets. Integrations between companies today are based on a steady state business process: accounts receivable, purchase order, shipment notification, etc. A new model will emerge, Cloud Keiretsus, whereby federations of companies will be loosely coupled with clouds providing the foundation. Data will move into, around, and out of the cloud as necessary to fulfill the objectives of the organizing principle whether it's to bring a product to market, manage a financial service offering, or develop a new drug. In this model when two or more organizations begin working together a collaboration cloud is provisioned as a foundation to facilitate the sharing of information; emails, prior work, document management, etc. As the relationship progresses and opportunities to share data emerge the cloud operates as both a repository and gateway managing the flow. New applications can be rapidly developed in the cloud to fulfill new needs and existing services can be exposed to automate processes as they cross over into the scope of the relationship.

An important advantage of this model is the ability to share and enforce standards, even when different, on both sides. Mapping the workflow of a process between organizations will enable compliance without requiring either company to abandon their own standards. Company A can map it's SDLC to that of Company B without explicit conversion. Exceptions where Company B has additional requirements can be mapped into the workflow of Company A. For example, such an approach would reduce the ramp-up time and cost of establishing the Program/Project Management Office for new projects and simplify the execution of the project.

As the world becomes smaller companies are pushed to cooperate combining resources to compete effectively. The next weakest link in the chain is the rapid integration of processes to ensure everyone involved is "singing from the same sheet of music." Cloud Computing provides a new architecture enabling collaboration between companies at the business process level. Those who see and take advantage of the opportunity first will be the best positioned to redefine their markets.

You need it real time? Really? Real Time???

2010-01-17T17:53:00.000-08:00

Business intelligence and executive dashboards are all the rage soon to be complimented, if not already, by customer dashboards. Although I'm no ERP specialist I get involved in all kinds of projects and one of the most glaring inconsistencies I face is the demand for real time data that changes daily or even weekly.

First we need to drop the term real time and focus on the term most recent. Real time to a guy like me from the plant floor automation/data acquisition/embedded systems/engine control module world means something TOTALLY different. Real time means just what it says, in the moment. No delays.

Second business people need to think about data with the fourth dimension of time. Why ask for a data update when the data hasn't changed? Whenever I hear the request for real time data I push back by asking "How often does the data change?" Sometimes it's hourly, sometimes periodic throughout the day. Most often its daily or weekly. Rarely does the data change minute to minute. More important to understand is how the data drive a decision. In one project we grabbed real time service data when all the executives needed was an hourly update. In return that hourly update was used to perform daily staffing analysis, and even then only when the service data exceeded set boundaries. Since the designers never asked, it was just assumed the data had to be real time. We changed from a real time posting to the executive information portal to an exception based alert and made everyone's life that much easier. This is one of hundreds of examples from healthcare to consumer packaged goods to retail that I have in my work history.

Why is this a big deal? Speed costs. It does in racing and it does in technology. The faster you want to go the more it's going to cost you. In IT we're not doing our jobs if we comply with the request just because the business "wants it". When we load up systems with unnecessary requests we slow everyone down; the opportunity cost of bad design. A great architectural option is the data intermediary which sits between the application and the data store to cache query and service results (if the data hasn't changed, the service response won't either)

Educate the business, add the time dimension to all data, and consider how the data drives decisions when building interfaces, services, reports, and dashboards. And the added side benefit? Less complexity!

We're Through the Looking Glass - Cloud Security

2009-12-29T11:54:00.000-08:00

I feel it's appropriate to consider an Alice in Wonderland world when thinking through the cloud computing landscape and it's security implications. Most security experts I know, including many CISO's at clients some of whom are quoted on the topic, appear to throw water on the burning desire of CTO's everywhere to go "cloud". It's understandable and, in my opinion, quite reasonable. Let's face it - the cloud isn't ready for prime time.

I have no argument against using the cloud for non-critical tasks but I tell clients day in and day out we are 2-3yrs from enterprise clouding (I love new domains where we can make up words!). Comments like that get me in the good graces of CISO's, at least until my next sentence, "You better get started now." What? Why do we have to get started now if the enterprise version won't be ready for 2-3yrs? Well because that's when it will be easy and everyone will have it - don't you want a competitive advantage? Well then put your nose to the grindstone and get that whole security thing figured out pronto so IT can move forward...

...or get run over, your choice!

With the cloud computing juggernaut gaining speed now is not the time for "No, but..." responses. What CIO's and CTO's need now are "Yes, if..." answers on how to pursue secure cloud services. We have lots of existing models, standards, and solutions so nobody can tell me the cloud is entirely unique. What it does present is a new architecture to which we need to plug in known solutions to known problems and some new solutions to cover feared gaps.

One of the biggest gaps clients identify today is data security. "How do I know my data is secure at a cloud provider?" Honestly I don't know in a holistic way but the old stand-by of encrypt data in transit and data at rest seems to pose the foundation of a solution. The immediate response, as the responder's face wrinkles so their eyes become nothing but slits in the creases of skin below their brow, "But that's too much overhead". Oh. So are we taking this security thing seriously or not? If we are then again, lets take our foundation and now get to work on the speed issue. Solving that problem involves the economics of speed where money is often the answer, governance so we don't speedily reach a cliff, and improving performance so the overhead of encryption becomes a round-off error.

Economically we don't have much of an issue. Cheap bandwidth. Cheap cloud storage. At $90k per 50TB of data storage at Amazon S3 we can afford encryption even if it increases our data sets by an order of magnitude in size. Governance is an issue but as we increase the use of automation in the cloud we should be automating governance as well. We need strong tools enabling us to enforce policies, especially on data which is hasn't been categorized.

How do we increase performance? Encryption takes time but if we can convince the cloud storage providers to provide hardware based encryption we can reduce the cost. Next we need a new way of thinking that takes advantage of cloud: lots of network bandwidth, storage services available on the fly, and ubiquitous availability.

How about applying a grid storage idea to the problem for data archival. Take a set of data and split it up into multiple chunks, each chunk with a sequence number, and encrypt it. Store a random set of chunks at three or more storage vendors and manage which data is stored where using a private index engine. Because each site contains a portion of the total data, the data is non-contiguous, and the data is random the value of the data at the site is dramatically reduced. A hacker would be required to hack all the sites, decrypt the data, and reassemble it to get the full picture. With the landscape being inherently more difficult to hack and the value of any independent data set being low a less onerous encryption method, such as one using 64bit keys, can be used.

Where could such a solution be used? How about monthly billing. Once the bill is generated and paid, the details are rarely if ever used again. Archive the data to the cloud. If it needs to be retrieved it can be, but its value is low to begin with for most hackers. Securing the data through obfuscation will make most hackers look for easier targets. Hacking is a numbers game.

So for all the CISO's out there consider that now is the time to identify the gaps and start looking at how to fill them in. One thing I can assure you of as we talk to CIO's and CTO's, the cloud computing train is coming and it's starting to build some serious momentum.

Be prepared to lead, follow, or get out of the way!

The Achilles Heel of Cloud Computing

2009-11-15T18:21:00.000-08:00

I think everyone understands the "Cloud" in cloud computing is an undefined network incredibly similar to, but not necessarily synonymous, with the Internet. What many apparently have not spent time thinking through is the impact building private and public clouds will have on the network architecture at most public companies. Three fundamentals which need to be revisited are: the Internet connectivity architecture, Internet bandwidth, and network security.

Many, if not most, large corporations consolidate their internet connectivity into a very few and sometimes a single point. I was working recently with a large non-governmental organization who has consolidated all of their internet access for primary and field offices into their Chicago data center. It's a great model for using the Internet, not so good for incorporating the Internet. In the world of Cloud Computing the Internet is less an end point and more one of several intermediate points during the execution of a function. Public clouds must be accessible at all times from any location to be of value. This mandate implies there is no single point of failure between the corporation and the public internet. A new architecture is required with many-to-one access to the internet instead of a one-to-one model. If New York cannot connect to the Internet it cannot jeopardize the entire corporation. And remember that Internet backbones do go down and more likely will due to ever increasing loads in the foreseeable future. If the public cloud function leverages data within the data center the reverse is true; multiple paths provide redundancy. I'm sure some will argue with me but I cannot make sense of data travelling from Denver to Chicago just to gain access to the Internet; it's an archaic model at best.

The second major issue is existing Internet bandwidth will have to grow. At the same time as traffic moves to the Internet it will move off of internal WAN's. We've grown accustomed to cheap bandwidth but with the explosion of WiFi, the coming of WiMax, and the growth of rich media on the Internet I expect those days are coming to a close. I expect it will be cheaper to run over the Internet than through private backbones which will help drive us to a more federated model for Internet connectivity.

Finally security as we see it today becomes problematic. How do we sniff packets between a user and a cloud provider when the company has nothing traversed in between. Surely we could route users through the corporate firewall but again, this defeats some of the economic model of cloud. We need better tools on the client side to help us manage the security aspects of this federated model. I'm not saying there aren't tools today, but those tools need to improve their automated detection, recording, and reporting capabilities to prevent attacks both inside and outside the company.

I've noticed over the past ten years a change in approach to networking. In the past bandwidth was managed loosely to ensure it was adequate. We've really tightened it down and now we need to be asking our bandwidth providers to provide more virtualized options enabling the rapid, automated scale up and down of circuits. We don't want to leave the network out of the push to move from a fixed to a variable cost model in IT.

Marketing Through the Cloud

2009-11-10T06:31:00.000-08:00

Cloud Computing promises to turn many conventional ideas on their head such as disaster recovery, data warehousing, and application development. However one area I see as having the opportunity to explode in value with Cloud is Marketing. I have always had an affinity for marketing from college where it was my minor (unofficially because as a Computer Engineering student we weren't allowed to have minors) through multiple interface roles from my internship at Eaton/Cutler-Hammer through my time in Sales & Distribution at IBM. The challenge of marketing is to know what the customer thinks and what the customer wants, often in advance of the customer. Much of marketing is driven by research; what works, what doesn't, who, what where, when, how, why. The only limitation is time because there is always one more question to ask. At some point that research has to be analyzed and made actionable through inputs to product development, sales, advertising, eCommerce and IT.

So where does cloud come in to play? First at the most base level with the enablement of rapid change represented by SOA. Once an SOA foundation is in place there are no limits to the reach of marketing. Social Networking. Semantic Web. Business Intelligence. Awareness. These are the future of marketing and all work better on an SOA foundation.

Social networking gives marketers the opportunity to find the influencers, tailor marketing messages, solicit feedback, observe from afar, and even seed new ideas. It's a human lab with no walls and no limitations. I have yet to see tools such as Facebook used for focus groups, or Twitter to measure interest, or harvesting forums for feedback. It may happen but as a user and an industry insider I've heard no discussion.

Semantic web technologies will be as important to Marketing as they will to supply chain management. The closer marketing gets to understanding the whole picture the better they can analyze data in the proper context and provide better input to downstream efforts. Why did someone purchase the product? What was the impetus for purchase? What made them think of the product? What was their first thought about the product? Good questions to ask, good information to know, but today all of this data is gathered post-sale through interviews. What if, via semantic technologies, customers were queried for this data and responded, all without realizing it? Capturing data real-time is always preferable to eliminate the issues of memory loss and filtering.

With the trove of new data available new business intelligence tools will emerge, and by having the data available in the cloud means the four walls of the data center will no longer limit how the data is analyzed. Specialty firms staffed with PhD's will offer services to slice and dice the data using their proprietary tools and provide an additional layer of context by bringing in additional 3rd party data sources. Business intelligence, real business intelligence and not the analytical reporting which passes for BI in many companies today, will emerge as a cloud service.

Awareness? What is that? Find me a better name and I'll use it but through cloud computing and semantic technologies we are a very large and important step closer to enabling awareness; the ability of a computer to understand. Although it will start with small steps, over the next 10 years computers will increasingly direct their own searches for data they feel is missing to assemble their own conclusions based on simple human queries. And when this happens we'll be ready to turn the corner in Marketing and become more proactive than reactive, able to predict events and prepare to seize opportunity. We'll be able to take our supply chain optimizations which can move the snow blowers to the states with the impending snow storms and extend that knowledge with who owns a snow blower ready for replacement, what size is needed based on the footprint of their property, who would benefit from snow removal services, who can be prompted to buy to use a remaining store credit. Targeted marketing will beging to take on the 1:1 reality we've been talking about for the past decade.

Cloud is a revolutionary technology which can be adopted in evolutionary steps making it unique and unavoidable. Cloud brings the world closer together eliminating some of our artificial barriers and bridging ones which are all too real. For Marketing this new capability will drive new thinking, new approaches, and new solutions bound only by their need to understand the customer better than the customer understands themself.

The Sun Sets on Disaster Recovery (Finally)

2009-11-07T18:18:00.001-08:00

Big changes have to come in small doses. For those of us fortunate to have several years experience with SOA and utility computing we see so many of the great things cloud can do and how it really addresses so many of the complexities within IT. I often explain part of the tremendous value of Cloud Computing is trapping complexity within layers of abstraction so we don't expose limitations. However I see one of the biggest killer apps for Cloud Computing, business continuity, as not only trapping disaster recovery within the infrastructure layer but doing away with disaster reactivity entirely!

First not every failure is a disaster. Failures can and do occur and we need to be smarter about how we engineer our solutions. Our focus should be on automated recovery; an option which becomes a real solution in a cloud world. If a service dies another service should be started. If hardware fails jobs should move to alternate hardware. Creating heat maps for failover can go a long way to identifying and targeting areas where failure recovery needs to be addressed and hopefully automated. But a disaster is a large scale failure for which we so often employ a different set of tools. Why? Primarily because of our legacy silo approach. If one silo dies we need to move data and jobs to an alternate silo. In a cloud architecture we don't have silos (even in a virtualized architecture the silo is logical rather than physical manifestation). So?

Once we architect our solutions to be service oriented and distributed from the start we lessen the impact of all failures from the simple to the theatrical. If we lose a data center that's bad. However if our solution is already load balancing across data centers, and we ensure by business rule we always have services available in each center, then our exposure is limited to in-flight transactions and sessions. If we have a true cloud infrastructure then we should already have the network bandwidth required to perform database mirroring in which case the disruption of the data center loss is as minimal as we have the ability to make it. Further advances will come to light in the next few years as databases tackle the federation issue and learn to manage data in logical instances rather than physical domains.

None of this happens, however, without intent.

We need a business case. What is the lost business opportunity per hour of downtime. For many business critical systems this value is calculable, and I argue if it’s not then there’s no reason for recovery. Our solution cost needs to be a small percentage of that potential loss. Today disaster recovery is EXPENSIVE: hot-sites and recovery contracts, tapes to retrieve from an off-site location and restore, staff to move around, periodic tests which always end with multiple failures. According to the Symantec 5th Annual IT Disaster Recovery Survey in June 2009, the average annual budget for disaster recovery is $50M. Consider that against the cost of 50TB of storage on Amazon E3: $90k. WOW! So in one fell swoop we can improve recovery speed and accuracy and reduce cost by elminating tapes, backups, tape recoveries, off-site storage, and the administration costs. And it only gets better!

Moving into the cloud we take advantage of all the tools and capabilities that already exist from service directories and virtual machines to provisioning and orchestration engines, schedulers and service level managers. We move out of Disaster Recovery and into Business Continuity. The focus shifts from recovering business systems based on a Recovery Time Objective and Recovery Point Objective to providing near seamless continuity via recovering services and virtual machines and cloudbursting to get needed resources. Is the cloud ready today? It's pretty darn close. Consider that Oracle's ERP solution will backup and recover from cloud storage. According to Symantec's survey three key hurdles in the virtualized world are storage management tools to protect data and applications, resource constraints which challenge the backing up of virtual environments, and that today 1/3 of organizations don't backup virtual environments.

Today or tomorrow business continuity brought about by cloud concepts is on the horizon and is a target every should be shooting for. It saves money and time and reduces risk. What's not to love?

What Should a Cloud Provide

2009-11-03T07:10:00.001-08:00

To me the single most important question I have never been asked is "What Should a Cloud Provide." I've been asked what do Cloud providers provide, and at what cost, and what solutions are available. But nobody has taken the question up a notch which to me means the focus is on application instead of understanding. We shouldn't limit our discussion to what's available. In such a nascent market we should define what we need, then tell the providers who can in turn react and build the services deemed most valuable to the market. So what should a cloud provide?

Nearly Unlimitted Bandwidth

It may sound impossible however cloud computing is predicated on having bandwidth available on demand. Workloads and data need to be shifted around the infrastructure at a moment's notice which means the network cannot be a constraint. This is a tremendous opportunity for the telco's to build out additional network bandwidth. More so this is a tremendous opportunity for telco's to deliver cloud services themselves out of their existing data centers.

Storage without Backup

Storage on a large scale is cheap. Amazon S3 provides 50TB of storage for $90k/yr. At those costs I can keep three copies of every file and eliminate ALL the costs of backup and recovery for less than the fully loaded cost of three administrators. And costs go down as volume increases.

Resources On Demand

This is the easy one but when additional processors are required they must be available, fully provisioned, within minutes if not seconds. From bare metal to fully loaded with application and all shouldn't take more than 15min. And although today we are bound by the limitations of processor type or operating system, now that technologists see them as barriers new solutions will evolve to minimize their impact.

On Demand Management Tools

Four important tools to learn and love are provisioning engines, orchestrators, schedulers, and service level managers. Provisioning engines which grab extra resources and put them into the correct pool when demand is high, and move resources out of the pool when demand subsides. Orchestrators observe the changes in traffic and determine when resources should be added or removed by the provisioning engine. Schedulers determine where jobs should run within the cloud and communicate with the orchestrators to ensure the required footprint is available. Finally the service level manager watches over all the applications and intercedes whenever an application threatens to miss its service level targets. At times this may require ramping down usage of non-critical applications in preference of critical applications.

Metering and Chargebacks/Billing

Cloud only makes sense with the economic model of paying by consumption when consumption and its associated cost are communicated. Usage must be metered and the costs must reflect the consumption. Although some costs will still make sense to allocate, the majority of costs must move to a chargeback model.

Interoperability

Clouds are clouds and as such the barriers between them are security based. Clouds must be able to share data, services, and infrastructure otherwise instead of cloud one ends up with a larger silo. The future value of cloud will be the new capabilities which are unlocked in such areas as collaboration, supply chain integration, business intelligence, and federated control. Security checkpoints will be important but should be the only barrier to integrating clouds.

Business Rules

Since a cloud is a consumption driven model there need to be business rules which govern consumption. Looking at healthcare today, what drove the high cost was the elimination of barriers to access. We made healthcare easy through co-pays and low deductibles. The easier something is, the more people will use it which results in a tilting of the supply and demand scale. In response either supply increase or prices will rise. So in the cloud we need business rules to govern consumption so it doesn't outstrip supply. What rights are required to execute an application? Who has the necessary rights? Who can grant rights? When will external resources be used? I believe that just as we did in the mobile telecommunications industry, at some point tiered pricing will enter cloud computing and rates will be higher during the day than overnight. When this change occurs it could have a dramatic impact on the costs of a cloud solution.

Three Cheers to Ubuntu

2009-11-02T13:47:00.000-08:00

"Sit Ubu. Sit." I can't remember what show it was but it always ended with a picture of a dog and that line. That's what I thought Ubuntu was when I heard about it the first time. Only later through continued pestering did I realize it was Debian fork in the mid 2000's. I'm a bit of a Linux bigot who preferred the Slackware release in the Volkerding days and moved to RedHat in the late 1990's. When RedHat went commercial I felt lost for a bit but picked up Fedora. Hearing how great Ubuntu was from friends I recently decided to give it a try on a Windows Vista machine with chronic problems and I was impressed.

Oh, don't misunderstand me, I'm not impressed with Ubuntu. I simply haven't used it enough but so far it seems remarkably like, oh, Fedora.

I was impressed with the stance Ubuntu has taken on cloud computing. Ubuntu brags on their site about the inclusion of Eucalyptus (Elastic Utility Computing Architecture Linking Your Programs To Useful Systems). Eucalyptus is an Amazon EC2 clone that works with EC2, S3 and EBS. Ubuntu takes an open source package that is relatively unknown, includes it in their server distributions, and makes the argument you should use Ubuntu as the foundation for your private cloud BECAUSE it will make you compatible with Amazon EC2 when you want to cloudburst. Now that's smart! People forget that Amazon is largely based on open technologies with Linux virtual machines running Xen hypervisor.

Pure GENIUS!

It makes me wonder where the relationship between Microsoft and Amazon sits. Microsoft and Google are competitors. In the growing cloud space Amazon and Google are competitors. But with Azure Microsoft is also an Amazon competitor, kind of, right? No in the sense that they use different platforms (Windows vs. Linux), but yes in that they compete for business. But Microsoft is so big perhaps Amazon needs to adopt "the enemy of my enemy is my friend" mentality and get Microsoft to build in cloudbursting capability to Amazon EC2. It would mean more licenses of Microsoft server products and address those Microsoft customers who don't want Azure, just more processor time. What happens when the concept of the operating system falls apart, as large tech heavyweights such as EMC and Cisco are starting to argue. Does that push Microsoft out of the picture?

Regardless of all the above, and perhaps because of it, choosing Linux is easy. Which distribution to choose? Really any because Eucalyptus can work with any distro, but Ubuntu has clearly seized the initiative.

My prediction is we'll hear RedHat announce they are working with OpenNebula...

Who Will Be The Expert?

2009-10-30T06:45:00.000-07:00

I've noticed a disturbing trend as we work to squeeze more and more computing technology graduates out of our universities. It appears we're dumbing the students down. Three quick examples, I have yet to meet a generation Y or Z who knows how:

1. to read and resolve a stack dump

2. how databases store data on the hard drive

3. how a network sends messages from one computer to another

Now I know there are some out there who do know, unfortunately I haven't met you. And I run in a circle of technology consultants whereas if I spent my time at Intel I'm sure more would know. Yet I work with graduates from our top technical schools: Michigan, Cornell, MIT, Virginia Tech, Stanford, Illinois, and Carnegie Mellon to name a few. Why are my expectations important? Here are the answers:

1. The development consultant could not resolve a stack dump and didn't because it was only happening on one developer's machine so they reformatted and reinstalled the software. However the cause of the problem was an automated update which changed the Java Virtual Machine. When the client moved to production the application wouldn't work because the operating system came with the later JVM and could not be downgraded. Downgrading the operating system meant losing crucial updates to improve database performance. Had they traced the stack dump they could have learned the cause months in advance and worked out an alternative instead of calling in all hands for several days and delaying their launch AFTER their public announcement.

2. The data architecture consultant did not understand the relationship of the files within a database so she incorrectly directed the support group to only backup some of the data files believing the rest were "configuration files and stuff". Turns out configuration files are pretty important for interpreting the data and as a result when the primary host went down hard, the data was unrecoverable because it was incomplete.

3. The security consultant didn't understand that Ethernet sends all packets to everyone so he believed the connection between two machines was point-to-point connection and therefore impenetrable. About 30sec of research would have taught him about promiscuous mode. It took the client many weeks and tens of thousands of dollars to define and implement a new architecture (nobody would take my advice to drop in SSL).

I have a cousin who is a developer for Microsoft and has had a lifelong passion for computing. During his degree program in Computer Science he NEVER learned Assembly Language, C or C++ instead being forced to focus on Java and its ilk. Java is ok, but it's got issues among them being it's heft. I spent several years in embedded systems and do not see Assembler being replaced anytime soon. At GM we used Modula, Assembler and C as the languages to program the Engine Control Modules. Lower level languages require you to understand how things like schedulers, pipelines, and memory work in order to maximize performance and sometimes just to make something happen (pointer arithmetic to manipulate memory for example). I encounter few Java developers who understand how memory is allocated and freed, how time-slicing is performed, how the cache operates and invalidates its contents.

I don't know everything. I know how to ask and how to learn. I'm never afraid to admit what I don't know. But I take great pride and work hard to know as much as I can. In each of these cases significant problems were stumbled into because the person in charge lacked breadth and depth. Why? Because none of them ever learned the core elements of computing; they were all one trick ponies.

So what is the solution? Luckily we'll have graduates in my program, Computer Engineering, and other engineering and science disciplines to act as the true experts. And those graduates will be swallowed up by the hardware, networking, telecomm, plant floor automation, aviation and embedded systems companies around the world. For the remainder of the world including consulting firms and corporations in non-R&D roles, I believe it's time to develop computing fundamentals courses to expand their view and understanding of computer technology. Companies today need people with multi-disciplined technology backgrounds to both lead large scale technology efforts and to provide guidance in troubleshooting. Find the good ones, expose them to a wider perspective, and most often they'll start to look at things with a different set of eyes which benefits everyone. Again, in my experience, I've had hundreds of conversations with people which end with "I never knew that. That's so interesting. Thank-you for explaining it to me." I guess that's why I've worked on projects including program/project management, IT strategy, Enterprise Architecture, application rationalization, software development, infrastructure architecture, business intelligence, data warehousing, systems integration, ERP selection, call centers, CRM, SFA, architecture modelling, requirements determination, and many more and in each one been considered the expert.

CRM, ERP, SFA, DW, and BI are not as challenging as operating system development, but they still needs experts!

The Value of Models

2009-10-25T06:26:00.000-07:00

I've had a theory for years which I always intended to research and proove during the course of post-graduate work. My belief is by understanding the core elements of computing (logic gates, transistors, magnetic storage, assembler, ethernet, etc.) makes all of their applications (databases, business intelligence, web architecture, cloud computing, etc.) easy to understand. Each computing technology core is composed of a set of models and I've found many models repeat themselves. We handle multiple simultaneous requests on processors, networks, and storage using the same time-slicing model which is the same way Client Service Representatives handle multiple chat requests. My theory is a person who learns and understands the models has the fastest route to gaining advanced knowledge in any one area and will have the broadest view.

Over the past 30yrs I've become a model driven person. I have taken the models I learned in college and continuously added new models or made existing models more robust to provide my core understanding. I've applied the same rules to business using my consumer side interactions (retail purchases, my bank account, etc.) as the foundation for models in each industry. What this model driven approach gives me is a head start whenever I encounter something new. I have found I can hit the ground in a new business vertical and be considered a technology and process expert within 90 days. My first goal is to understand, second to align one or more of my existing models, third to perform a gap analysis, and four to fill in the gaps. The result tends to be a very robust understanding.

I am often asked by my leadership where we can find more of me. It's not me, it's my way of learning and applying knowledge they want to replicate. But it all starts with an open mind. What I find in my competition, regardless of level or job type, is a very myopic view. I worked with a software engineer early in my career at Eaton/Cutler-Hammer who told me he didn't care about the hardware; he only wanted to know where the on/off switch was located. He felt understanding the hardware would take too much time and too much capacity. If only he realized the models between hardware and software are largely similar.

I work day-in, day-out with ERP and supply chain guru's, CRM experts, and people focused on Enterprise Transformation. What I find interesting is how many are one or two trick ponies. They are considered experts yet they cannot explain how things really work within their own domain, and certainly not to someone new to the domain. Perhaps they know the processes, which is paramount, but they don't understand the model. Every problem is different and I agree one can repeatedly apply the same approach to solving the problem, but too often consultants are trying to apply the same solution. When I dive in it becomes readily apparent the reason for pushing the same solution is that nobody really understands it but it's been proven to work. It's a best practice. The truth leaked out in mid-2002 while I was at PricewaterhouseCoopers Consulting when we were told to use the term "leading practice" in place of "best practice". Now that's some logic I can agree with because there is never one best practice.

As an Enterprise Architect I'm a modeller in a modeller's world. I find it interesting how businesses are now starting to unlock the power of modelling. In a recent internal discussion one of the 2010 technology trends discussed was the evolution of modelling in business to a primary focus. Perhaps not in 2010, but the fact that it was even a topic of discussion surprised me. I guess I'm lucky in that the way I naturally think is evolving as a better mousetrap. Hopefully it has long legs or my thinking continues to evolve.

Perhaps I should develop a course on modelling...

Google Makes an Important Step Forward in Cloud

2009-10-18T07:12:00.001-07:00

I'll leave my criticisms of the hypocrisy at Google between their "Do no evil" motto and their actions aside for now. I have to applaud Google for resolving one of three hurdles to the use of their Cloud products. Google is now releasing methods to transfer the data users have put into Google products such as Google Docs, Blogger, and Gmail out of Google's data center. Google calls it their Data Liberation Front and it has it's own webiste at dataliberation.org. Google will provide an easy to use method for exporting data from every product, in bulk, to the user's selected destination. Bravo Google!

Why is this important? The Cloud is predicated on a virtualized infrastructure (utility computing model) with a service based software layer (SOA). The combination of these two models creates a powerful foundation to provide the most flexibility in the most efficient manner. Creating arbitrary obstacles to moving data in and out of data stores, using application components instead of only full applications, and changing where the data and applications reside destroys some of the of Cloud. Cloud has to be bigger than Google, Amazon, Rackspace, IBM, or any other vendor. The emerging Cloud lacks a definition of data ownership. Ownership means the ability to add, change, delete and move at will and have it still be useful. Companies always seem to forget, whether is the old ASP model or SaaS, that they need to make sure their data can be exported AND imported into another tool, otherwise they don't own the data but rather have granted that ownership to the application/platform provider by proxy.

For the Fortune 1000 Cloud will start inside the data center, as it already is for large banks and a select few others with vision. To be relevant, public Cloud offerings need to enable, not disable, integration across the public/private cloud boundaries. Users need to own their data which is not the case when a cloud provider partitions it for them but ties it inextricably to their platform. Without ownership standards the Cloud represents nothing more than a new larger external silo, but still a silo.

What Google is doing will raise the bar for all providers which should go a long way to making Cloud more palatable from a risk point of view. The next steps should be enabling a federated data model so I can store highly sensitive data at home while using Google for the remainder of the data but all within a single data model. In addition Google can go further with enabling its applications as services for integration into other tools. Of course I expect Google will need remuneration and needs to think through how enterprise licensing will work because everyone knows you get what you pay for. As long as a service is free it also means at the mercy of the provider.

Oh, the other two obstacles Google needs to figure out? First, encryption of data at rest and in transit which I know is already on the drawing board and partly implemented in tools such as Google Docs. Second, interoperability standards to enabling the shifting of data and applications throughout the cloud. Where Google goes the public Clouds will follow.

A Road Made Longer by Doubt

2009-10-11T16:49:00.000-07:00

In 2003 I joined the IBM Grid & Virtualization team as the lead architect for Healthcare in the Americas. As a part of Systems Technology Group our job was to evangelize grid computing and virtualization technologies. We helped early adopters move further faster investing time and money to learn and have an impact. As the harbinger of new technologies we met LOTS of skepticism and dobut. Funny enough, just about everyone is moving that direction now, many having given up their chance at innovation by adopting the technologies first in their industries. Oh well. If I had a dime for every bad business decision about technology I've witnessed in 15+ years of consulting I'd be retired to my own island.

In 2003 grid computing was synonymous with high performance computing, a boundary we worked tirelessly to break down because it was arbritrary at best. In this endeavor I upgraded one of my computers to the latest Nvidia card, a company I have followed for years one because of its technology and two because two of its executives went to the same small engineering college I attended. When I researched the specs and looked at the calculations, and compared those to what a medical research institution was attepmting, I realized the video GPU had much more to offer than the general purpose CPU. I talked to a few fellow IBM'ers who agreed and had been looking at such uses for a few months. Getting my facts together I approached my client to propose we do some joint research into the use of the GPU for the calcuations.

Cost? Zero. I had funding in "blue money" ready to go. Delays? Zero. We had a functioning system but it was resource constrained. Receptiveness of my client. Also zero. No appetite.

The world has moved on and Nvidia has stayed its course now designing a new GPU architecture specifically to advance high throughput computing. It's a great idea, especially considering Nvidia's past architecture has enabled the sharing of resources across four video cards. Impressive to say the least!

I wonder who, beyond Oak Ridge, will bite. More so, I wonder how much faster we could have advanced important causes such as research into pharmacogenomics and protein folding if we had adopted this technology earlier. I have to believe it would have expedited the development, ultimately leading us to the same place but sooner. I don't know about you, but I'm interested in an AIDS vaccine and a cure for cancer BEFORE I die. I'm sure others are too. Isn't there a moral imperative that says research centers should be, perhaps, researching? I found they do as long as the topics aren't too politically sensitive. And for whatever reason the idea we presented was a political landmine. Too many people would have to agree. Too many people would have to approve. It would take too long. There was no guarantee.

Yes. But there is no guarantee in Research, or did I miss something.

Too bad for all the people for whom the vaccine arrives 1min later than needed; and to those whom it could have saved had it come to market on the earliest possible path instead of the one easiest to navigate. I guess that's why I left R&D after my internship in college and never wanted to go back (although I've been dragged, reluctantly, back through the halls of R&D a few times). R&D to me is all promise with very little delivery. Guess that's why we do so little in the United States these days. I didn't realize the reason for poor delivery was because researchers were afraid to do....uh....research.

Welcome to the Cloud Everyone!

2009-10-09T20:43:00.000-07:00

I find it interesting how some of us have always envisioned a computing world based on "cloud" technologies while others are just figuring out what cloud means. I started getting into virtualization technologies back in 1993 during an internship in the Artificial Intelligence Group at Eaton/Cutler-Hammer. A student in Computer Engineering at MSOE at the time, I used the FIDE package (Fuzzy Inference Development Environment) to do research on fuzzy logic. I could mimic the Motorola MC68HC11 micro-controller which opened my eyes to the realty of our compute stack. Each layer is an abstraction not for the benefit of the machine, but for the benefit of the human. A CPU cannot differentiate between code at the micro, operating system, or application levels. Packages, libraries, databases, user interfaces all look the same.

About this time I was asked by our neighbor back at home, President of Ameritech's business services division, what I felt the next big thing in computing would be in ten years. I was off on the timing but I replied "Hey, you guys in the phone company have lots of big computers. I think the future is running the applications I want on those machines and charging me only for what I use. You have way more power than I could ever afford because I only need it for a few nanoseconds at a time." My father, a first generation Computer Engineer, reminded me of the conversation earlier this year.

Post-graduation I worked for General Motors via EDS on the Powertrain Embedded Sytems and Controls team where I developed and managed the teams developing components of the Engine Control Module code as well as development utilities such as Cal-Tools and WinVLT. Again we used software to emulate hardware further cementing my belief that the hardware was, in some form, a commodity.

Fast forward to the Linux movement around 1996 when I saw the value of open source as a form of virtualization; consolidating the logic of various systems into a common, open application for everyone to learn and expand. Open source had few proponents in business but that didn't stop friends and I from trying to move an outdated call center outsourcer out of the mainframe age into the internet age. We didn't succeed, not for a lack of technical capability, but for a lack of salesmanship but that's another story for another post. We did succeed in demonstrating the value of open source, and once we did there was no going back (at least for us, the company, sadly regressed to a number of failed implementations including Lotus Notes - whose idea was that!?!?! - and eventually folded into the pages of history being acquired by an Indian firm).

So around 2002 I ran smack into an idea I called serverless computing. The idea is based upon that fundamental realization that most constructs of computing are for our benefit. So why, then, do we need servers? We invented the idea of a server to help us branch out from one system to two. Client. Server. Easy! But what if the client also acts as a server? Preposterous everyone told me. Crazy. Insane! Never! I submitted a paper to multiple outlets, including my employer IBM, but nobody would give it a second look (honestly I bet most didn't look at it a first time either).

Ok. I took a step back, did some research on grid computing and peer to peer networks and realized I wasn't wrong, simply nobody I talked to could see the light.

Well I welcome those who do. Important people like those at Cisco and EMC who have finally understood the operating system is a limitation. I expect now that they're thinking, that thinking will evolve and they'll realize it's not just the OS but how development environments, platforms, and the very concept of a centralized data center are significant limitations with inherent cost disadvantages. Let's move to a new model which frees us from the limitations of the physical which has always been the interface boundary for innovation!

Welcome to the Cloud! Welcome to the Party!

For more on Serverless Computing read the unpublished paper Serverless Computing.