There is a singularity in the future of cloud which I believe has guided its evolution since server virtualization became a topic in the early 2000's. However I only realized today that not everyone sees this inevitability. Cloud is about agility, efficiency, and elasticity where the low hanging fruit, cost savings, has brought a by-product of the transformation to the forefront as its value proposition. This in turn has focused our attention on the infrastructure side of cloud largely ignoring the original value proposition; the global collaborative applications composable from services and available anywhere, anytime, on any device. So today we see cloud as an enabler of new models for infrastructure consumption matching hardware needs to cost: on premise, off premise, private, public, hybrid, IaaS and even PaaS and SaaS. The product mix is typically viewed as mutually exclusive however in the optimal cloud it's not about one or the other, it's about having all of them. Only by maximizing our consumption option can we tailor available resources to our needs. I believe this road leads to one end state: an automated, autonomous cloud resource marketplace.
What on earth is that?
I admit I jumped to the punch line so let me make my case in a more linear fashion starting with an easier topic: the automobile. We use autos everyday to achieve many of our objectives: getting to work, picking up children, traveling to a client. At home we may own a car but when away from home we often have the same needs. And there are many who don't even own a car but need one from time to time. It's no surprise businesses have grown to meet these needs whether it's use a car for a few minutes (taxi), a few hours (asset shares like ZipCar), a few days (car rental), a few years (lease) or forever (purchase). The need for the resource drives the consumption model at any given time. That need is composed of a variety of factors that are contextually dependent such as where, when, for what purpose, for how long, with whom, etc. We balance these needs through prioritization and look to the market to provide the solution: a bullet-proof limo for the President, a rented mini-van for the family on vacation, a leased truck to deliver furniture, and a taxi cab to run downtown for a meeting.
We need the same market options in technology but so far there have been serious limitations. What companies need is the ability to benefit from technology without having to buy and become experts in it. However due to the limitations of the market, most companies have chosen to own and operate their technology. As a result the value of each dollar invested in IT is diluted by the cost of moving up the knowledge curve which does not lead directly to improved revenue or reduced costs. The first big step out of own and operate was mainframe time sharing which quickly led to outsourcing, a model of own and let someone else operate. It's such a small step there really isn't an equivalent in the more general market which probably explains all the pains behind outsourcing. Furthering the adoption of alternative models we've had some success with co-location (lease model), Application Service Providers and now Software as a Service (rental model), a we're nibbling at the shared asset model with public IaaS. However large scale adoption of a shared asset model is years away and there is no existing path to the taxi cab model due to issues of security, regulatory compliance, trust, and a healthy dose of fear, uncertainty and doubt (FUD).
Today we are at nearing the peak of the hill and it's time to start planning for what happens when we crest. Our journey into cloud isn't about adopting new models and leaving old ones behind. Cloud is about doing it all; the new and the old. Top to bottom. It's about the right resources at the right time in the right place. I believe a real cloud can deliver any kind of resource required from a mainframe processor to heat monitor's data. However getting to this ubiquitous cloud requires a fundamental shift.
As an infrastructure technology today, the vision for cloud is, at best, cloudy. The entire purpose of what we call Information Technology is to execute applications. To an application everything is a resource. Therefore it follows that from an application point of view, a cloud is all about the marshaling of resources (compute, memory, data, algorithm, bandwidth, security, etc.). There is no one size fits all answer, and as our applications become more user-centric by nature they will exhibit unique tendencies at the user level requiring micro-levels of optimization. Hence the need for all resource types and consumption models which enables cloud solutions to leverage the best available combination of resources at any given moment based on what's available. Continuous optimization becomes a scanning routine constantly looking for better resources or a change in the parameters. Inherent in this approach are all the benefits of cloud we know and love: scale-up/scale-down, pay as you go, no long term contract, etc. For lack of a better term I call this the Cloud Marketplace (NOTE: I am advocating the marketplace as an architectural pattern which would, by nature, enable the cascading of marketplaces to maximize the resource and consumption model pools.
Using the car analogy this approach would allow us to use the best vehicle possible for each task (go to work, visit a client, pick up the new widescreen TV, go out to dinner) rather than compromise by trying to fit all the uses to a single model. There is no convenient, economic model today that can get us a hybrid to get to work, a town car for the client visit, a delivery truck for picking up the TV, and a sports car for a night out. Luckily in the virtual world the only barriers are those we create.
A cloud marketplace would be governed by a set of business attributes (time to market, capex sensitivity, flexibility, self serve, service quality, criticality, compliance, IP protection, and continuity) and technical attributes (cost, location, availability, execution speed, latency & bandwidth, integration, reliability, security, scalability). Each application would include in its profile a predefined prioritized list of attributes to be used for by the marketplace. To a well designed cloud app there is no dependency on the infrastructure so the marketplace would use the attributes to marshal the necessary resources via the orchestration and provisioning systems to get the compute, memory, bandwidth, and other resources required. When the parameters change (time to market drops from high to medium) the system would look to modify the resource pool (move from $0.08/CPU/hr to $0.05/CPU/hr compute) to comply. The power of a marketplace model is in its inherent organization and optimization. Resources can be ranked by value based on real usage patterns providing clarity in capacity decisions.
No Fortune 500 company is going to rush out and close down data centers throwing away billions of dollars in investment to adopt this pattern. And they shouldn't. The concept naturally lends itself to a stepwise adoption. Once we have applications unbounded to resources (virtualized in some way) and can operate across more than one set of resources we have a need to manage and optimize. The marketplace can be built over time with self-identifying resources enabling plug-and-play consumption models. The more consumption models it supports the more resources made available for use. Once we have a marketplace we can expand our thinking on what constitutes a resource. Imagine a future where data is collected and stored at the endpoints, where compute power and connectivity are ubiquitous. Taking advantage of these resources in a marketplace is simple - just define their parameters and make them available. Without the marketplace concept the risk is likely too high to consider.
To get to this point we have several challenges. First we need to eliminate dependencies. Dependencies between resources. Dependencies between applications and resources. The fewer dependencies the better the marketplace operates. Second, we need better security tools able to define, execute, and audit policies throughout the execution cycle of an application. Third we need application management tools that are non-intrusive to the application which in turn requires a new standard for communicating application state. Fourth we need to address data management and security in a federated model.
To make such a model work requires a level of insight and automation well beyond where we are today. No such marketplace exists today and I'm sure it's not on any product roadmap. However I'm convinced it's the vision we need to work toward. Cloud is a global collaboration platform, nothing more but also nothing less. The power of cloud is delivered by the applications built to take advantage of the platform: Social Business, Real Time Analytics (aka Big Data), Mobility, and Collaborative Communications. The reason to adopt cloud is to bring our connected world to the next level: the collaborative world. Doing so requires a platform that delivers continuous availability, unlimited scalability, and optimal efficiency at all times. To do this cloud needs options and the ability to match need to resource. Anything less isn't a cloud; it's just another silo.
Friday, September 28, 2012
Tuesday, September 18, 2012
Where is Disney Going with Mobile Technology
In 2001 while working for a Big 5 consulting firm I was involved in a project to update Disney's content management. At the time my firm did tens of millions of dollars of consulting work each year for the Walt Disney Company. As one of the lead architects for our eBusiness team I was asked to sketch out a view of how the Disney Company could take advantage of mobile and web technologies to improve the guest experience at their theme parks worldwide. As a result I developed ten ideas which we presented to Disney leadership. Unfortunately SarBox came along and as the auditor for Disney we were suddenly no longer able to provide consulting services to avoid any perception of a conflict of interest. The good news is the ideas did not die and in fact have lived on in the form of Disney's NexGen project.
Although I have had no involvement in NexGen I have to believe either my ideas were shared with the team by Disney leadership as their view of the future, or it's another example of the convergence of thought such as when both Alexendar Graham Bell and Elisha Gray invented the telephone filing for patents just hours apart. However I am confident that my strategy document predates anything from NexGen.
Here are the ideas that as of now have been implemented through the web and mobile devices:
1. On-line reservations
2. On-line itinerary for trip planning
3. Robust electronic guide map with estimated queue times and parade times (now available as the My Disney Experience app)
4. Games to be played in line during longer waits
5. Location based themed scavenger hunt
The ideas not implemented
1. Disney.com email addresses for the public (who wouldn't want one?)
2. Electronic diary including drop-in professional photos and GPS location. Data would be uploaded to an online vacation portal (extension of on-line itinerary) with photo albums by visit including user uploaded photos, user diary, and complete GPS based walk-thru of the park for every visit (now this could be done via a Disney version of facebook). Each album would be available in hard copy (sponsored by Kodak) showing a map based on the actual GPS route followed mixing in user uploaded and professional images. There is a very simplified version of this available but only for the Disney PhotoPass pictures.
3. Location based dining reservations whereby table availability and reservations are managed by GPS location. If a person is too far to make reservation on time they can confirm or cancel and others nearby can be pinged with an open table offer. Intent was to decrease impact of voided reservations.
4. Bluetooth wallet for fastpasses. One person could contain the fastpasses for an entire group so there would be one check-in by one person. It could also represent room keys and park tickets as well. This idea could use RFID as a cheaper option however the interactivity would be reduced.
5. Reservation based attraction access (fastpass 2.0). The idea was to let people check into a ride via Bluetooth (now it could be done with a FourSquare app) at which point they would be given an estimated ride time. They would then be able to explore a given area while the system optimized the lines. Once a fixed place was defined the person would be notified of the time to enter the line. Similar to fast pass, this model would completely do away with traditional queues. When you entered the line your wait would be 10-15min every time. You could check in to multiple rides at a time and if conflicts occurred the system would figured out a coordinated plan of attack.
Realizing half of the ideas have been implemented, I'm starting to wonder if perhaps the second set will be. Perhaps not for idea #1. I doubt Disney will give away Disney.com email addresses at this point. I think it would have been a smash hit had they done it a decade ago but that ship has sailed. In fact I'm still amazed Disney stuck with go.com for all their addresses, as if go.com means anything to anyone anymore (in fact it never really did). I think idea #2 will happen whether Disney is involved or not; I've thought of developing the idea myself. I think Disney would benefit greatly from idea #3 which would enable them to get rid of their new solution, to charge a no-show fee on voided reservations. Idea #4 should happen in some form to make managing fast passes easier for people and groups in particular, and it sounds like at least some part of it is happening with FastPass+. My favorite idea, after #1, is #5. Nobody likes waiting in line. The guests don't like to stand around and move like cattle. Disney managment doesn't like having their guests waiting in line when they could be buying more trinkets or consuming more food and beverage.
It's good to see a great company like Disney moving in the right direction with technology. Properly applied mobility solutions can greatly enhance the guest experience. Disney theme parks have never been fast movers. Look at how long it takes between the emergence of a new character and any attraction based on that character. I hope they are thinking along the lines of my next generation ideas:
1. attraction enhancement - letting users select the back-story, storyline soundtrack, images, etc. of an attraction or interact with the attraction via a mobile device or RFID.
2. unencumbered views - stream parades, fireworks, concerts and special events so people with limited viewing access can enjoy them as well, even if they are on the other side of the park.
3. virtual wallet - no brainer! extends the bluetooth wallet to include credit cards too.
4. premium content channels - streaming of movies, games, etc. for a per day fee to keep children engaged.
5. virtual tours - providing content rich, location and context aware tours on a self-paced basis for a per day fee
The future will tell.
UPDATE 04-16-2017: I realized I need to give Disney credit for additional developments. The Magic Bands used RFID technology to act as a virtual wallet for park tickets, fastpasses, payment and room keys.
Although I have had no involvement in NexGen I have to believe either my ideas were shared with the team by Disney leadership as their view of the future, or it's another example of the convergence of thought such as when both Alexendar Graham Bell and Elisha Gray invented the telephone filing for patents just hours apart. However I am confident that my strategy document predates anything from NexGen.
Here are the ideas that as of now have been implemented through the web and mobile devices:
1. On-line reservations
2. On-line itinerary for trip planning
3. Robust electronic guide map with estimated queue times and parade times (now available as the My Disney Experience app)
4. Games to be played in line during longer waits
5. Location based themed scavenger hunt
The ideas not implemented
1. Disney.com email addresses for the public (who wouldn't want one?)
2. Electronic diary including drop-in professional photos and GPS location. Data would be uploaded to an online vacation portal (extension of on-line itinerary) with photo albums by visit including user uploaded photos, user diary, and complete GPS based walk-thru of the park for every visit (now this could be done via a Disney version of facebook). Each album would be available in hard copy (sponsored by Kodak) showing a map based on the actual GPS route followed mixing in user uploaded and professional images. There is a very simplified version of this available but only for the Disney PhotoPass pictures.
3. Location based dining reservations whereby table availability and reservations are managed by GPS location. If a person is too far to make reservation on time they can confirm or cancel and others nearby can be pinged with an open table offer. Intent was to decrease impact of voided reservations.
4. Bluetooth wallet for fastpasses. One person could contain the fastpasses for an entire group so there would be one check-in by one person. It could also represent room keys and park tickets as well. This idea could use RFID as a cheaper option however the interactivity would be reduced.
5. Reservation based attraction access (fastpass 2.0). The idea was to let people check into a ride via Bluetooth (now it could be done with a FourSquare app) at which point they would be given an estimated ride time. They would then be able to explore a given area while the system optimized the lines. Once a fixed place was defined the person would be notified of the time to enter the line. Similar to fast pass, this model would completely do away with traditional queues. When you entered the line your wait would be 10-15min every time. You could check in to multiple rides at a time and if conflicts occurred the system would figured out a coordinated plan of attack.
Realizing half of the ideas have been implemented, I'm starting to wonder if perhaps the second set will be. Perhaps not for idea #1. I doubt Disney will give away Disney.com email addresses at this point. I think it would have been a smash hit had they done it a decade ago but that ship has sailed. In fact I'm still amazed Disney stuck with go.com for all their addresses, as if go.com means anything to anyone anymore (in fact it never really did). I think idea #2 will happen whether Disney is involved or not; I've thought of developing the idea myself. I think Disney would benefit greatly from idea #3 which would enable them to get rid of their new solution, to charge a no-show fee on voided reservations. Idea #4 should happen in some form to make managing fast passes easier for people and groups in particular, and it sounds like at least some part of it is happening with FastPass+. My favorite idea, after #1, is #5. Nobody likes waiting in line. The guests don't like to stand around and move like cattle. Disney managment doesn't like having their guests waiting in line when they could be buying more trinkets or consuming more food and beverage.
It's good to see a great company like Disney moving in the right direction with technology. Properly applied mobility solutions can greatly enhance the guest experience. Disney theme parks have never been fast movers. Look at how long it takes between the emergence of a new character and any attraction based on that character. I hope they are thinking along the lines of my next generation ideas:
1. attraction enhancement - letting users select the back-story, storyline soundtrack, images, etc. of an attraction or interact with the attraction via a mobile device or RFID.
2. unencumbered views - stream parades, fireworks, concerts and special events so people with limited viewing access can enjoy them as well, even if they are on the other side of the park.
3. virtual wallet - no brainer! extends the bluetooth wallet to include credit cards too.
4. premium content channels - streaming of movies, games, etc. for a per day fee to keep children engaged.
5. virtual tours - providing content rich, location and context aware tours on a self-paced basis for a per day fee
The future will tell.
UPDATE 04-16-2017: I realized I need to give Disney credit for additional developments. The Magic Bands used RFID technology to act as a virtual wallet for park tickets, fastpasses, payment and room keys.
Tuesday, September 11, 2012
Ten Cloud Native App Rules and Lessons Learned
To my way of thinking cloud native applications are how we should have been building applications all along. My belief is rooted in how I believe apps should be architected which is not an opinion shared by all. I already know several people whom I respect who disagree with me. After several years of architecting clouds and enterprise applications and now having built bepublicloud.com, my own cloud native application, I'm convinced I'm right.
My rules for cloud native application development are:
1. DO NOT manage the cloud within the application. Application management is a separate domain and should be kept separate. If you cannot use one of the many existing tools to manage your application then build a management app, but do not consolidate it with the app.
2.Believe in the power of encapsulation; trapping the complexity of each layer within that layer and exposing only the simplest interface required to the layer above. Think low coupling and high cohesion. It's a great design principle for a reason! Plan for every service you use, because you will use some, to be replaced. Encapsulate them so they can be swapped out easily and painlessly.
3. Start with a multi-vendor deployment, don't just plan for it. There are lots of little things you need to learn about each cloud vendor. Some provide robust API's, some use third parties. Some have great support, some have next to none. Some have great management portals, tools, and/or API's while others are the bare minimum. And price is NOT an accurate indicator.
4. Test the speed and error conditions of cloud services before you use them, you might be surprised. I make calls to several third party cloud services via REST within bepublicloud and I was surprised at the speed of most, while one proved so unreliable due to the vendor's growth issues I couldn't risk putting it into production.
5. Expect the hardware and services to fail. The focus needs to be on continuous availability, albeit with degraded service when necessary. Netflix has the right model having multiple zones within each Amazon region, and multiple regions, so a user's access and experience will NEVER be offline.
6. Understand the long term ramifications if you decide to use vendor specific/proprietary services. As soon as a cloud application begins making calls to the hardware layer, whether to provision resources or provide a status, by definition you break cohesion and amplify coupling. In the real world the result is an application paired to a particular cloud. I believe the application should be portable across clouds. The upside is vendor independence. The downside is a whole lot more work building software services you could otherwise buy (email distribution, queue management, site search, payment processing, etc.).
7. Not all servers are equal in the cloud. Cloud server pricing is predicated on overprovisioning, selling more slices of a box than there really are. Most applications are input/output bound and waiting for instructions as users are figuring out what they want. Amazon differentiates with High-CPU and High-Memory instances, but not everyone does. Performance is the line of demarcation, so research by deploying apps on multiple instance types to get an idea of the ideal target environment. This is true of both public and private clouds.
8. Adopt a strong source code management solution that supports multiple users with remote capabilities. I have settled on Git having tried Mercurial and a few others. Git is by far the easiest to learn, most common used, and most robust. I can manage my codebase with ease, get friends to pitch in from time to time, and keep everything sane.
9. An agile approach works best leveraging working prototypes, continuous builds, and a focus on the user experience first.
10. Automated testing is paramount. Spend the time and money where it counts, on the user interface and feature/functionality. The heavy lifting (80-90%) of testing previous features with each new version (regression testing) can be done through automation.
In building bepublicloud.com the code base is PHP running in Ubuntu on Amazon EC2. The storage services come from Amazon S3, RackSpace CloudFiles, Microsoft Azure Blob, and Nirvanix (although is not priced competitively and therefore disabled as of now). In the few, isolated cases where for development speed I adopted Amazon services, such as their email service and queue service, the functions are encapsulated and optional to program execution. Sure the application would be degraded, however users would still have the access they need. As a result bepublicloud's code can run on any IaaS cloud, and I even have my choice of PaaS clouds which support PHP should I go that route. My design criteria was simple - no single point of failure in a world of failure. I trap every exception I can with recovery routines and robust log messages so I can manage the app without coupling to a provider's platform.
As a new service that is unmarketed, bepublicloud has very little traffic today, but I'm preparing for success just in case. My next step is to use memcache for session data so sessions are preserved as servers are added and removed. I am ready, when needed, to move to a cloud MySQL instance to give me rapid scalability. I won't use one service, but at least two.
I am now looking at virtualization on top of virtualization. To get the power benefits, is it possible to deploy linux containers on top of a Large Amazon instance and then divide up my resources as I need them. A large instance is four times the size of a small instance at four times the cost; however it also has more native storage, is ebs optimized and has high I/O.
There are plenty of use cases where an application is appropriately built on a Platform, especially applications with limited relevancy (specialized applications) or limited time (marketing). In those cases time to market trumps vendor independence.
Like all software development, the rules are really guidelines. There are always exceptions.
My rules for cloud native application development are:
1. DO NOT manage the cloud within the application. Application management is a separate domain and should be kept separate. If you cannot use one of the many existing tools to manage your application then build a management app, but do not consolidate it with the app.
2.Believe in the power of encapsulation; trapping the complexity of each layer within that layer and exposing only the simplest interface required to the layer above. Think low coupling and high cohesion. It's a great design principle for a reason! Plan for every service you use, because you will use some, to be replaced. Encapsulate them so they can be swapped out easily and painlessly.
3. Start with a multi-vendor deployment, don't just plan for it. There are lots of little things you need to learn about each cloud vendor. Some provide robust API's, some use third parties. Some have great support, some have next to none. Some have great management portals, tools, and/or API's while others are the bare minimum. And price is NOT an accurate indicator.
4. Test the speed and error conditions of cloud services before you use them, you might be surprised. I make calls to several third party cloud services via REST within bepublicloud and I was surprised at the speed of most, while one proved so unreliable due to the vendor's growth issues I couldn't risk putting it into production.
5. Expect the hardware and services to fail. The focus needs to be on continuous availability, albeit with degraded service when necessary. Netflix has the right model having multiple zones within each Amazon region, and multiple regions, so a user's access and experience will NEVER be offline.
6. Understand the long term ramifications if you decide to use vendor specific/proprietary services. As soon as a cloud application begins making calls to the hardware layer, whether to provision resources or provide a status, by definition you break cohesion and amplify coupling. In the real world the result is an application paired to a particular cloud. I believe the application should be portable across clouds. The upside is vendor independence. The downside is a whole lot more work building software services you could otherwise buy (email distribution, queue management, site search, payment processing, etc.).
7. Not all servers are equal in the cloud. Cloud server pricing is predicated on overprovisioning, selling more slices of a box than there really are. Most applications are input/output bound and waiting for instructions as users are figuring out what they want. Amazon differentiates with High-CPU and High-Memory instances, but not everyone does. Performance is the line of demarcation, so research by deploying apps on multiple instance types to get an idea of the ideal target environment. This is true of both public and private clouds.
8. Adopt a strong source code management solution that supports multiple users with remote capabilities. I have settled on Git having tried Mercurial and a few others. Git is by far the easiest to learn, most common used, and most robust. I can manage my codebase with ease, get friends to pitch in from time to time, and keep everything sane.
9. An agile approach works best leveraging working prototypes, continuous builds, and a focus on the user experience first.
10. Automated testing is paramount. Spend the time and money where it counts, on the user interface and feature/functionality. The heavy lifting (80-90%) of testing previous features with each new version (regression testing) can be done through automation.
In building bepublicloud.com the code base is PHP running in Ubuntu on Amazon EC2. The storage services come from Amazon S3, RackSpace CloudFiles, Microsoft Azure Blob, and Nirvanix (although is not priced competitively and therefore disabled as of now). In the few, isolated cases where for development speed I adopted Amazon services, such as their email service and queue service, the functions are encapsulated and optional to program execution. Sure the application would be degraded, however users would still have the access they need. As a result bepublicloud's code can run on any IaaS cloud, and I even have my choice of PaaS clouds which support PHP should I go that route. My design criteria was simple - no single point of failure in a world of failure. I trap every exception I can with recovery routines and robust log messages so I can manage the app without coupling to a provider's platform.
As a new service that is unmarketed, bepublicloud has very little traffic today, but I'm preparing for success just in case. My next step is to use memcache for session data so sessions are preserved as servers are added and removed. I am ready, when needed, to move to a cloud MySQL instance to give me rapid scalability. I won't use one service, but at least two.
I am now looking at virtualization on top of virtualization. To get the power benefits, is it possible to deploy linux containers on top of a Large Amazon instance and then divide up my resources as I need them. A large instance is four times the size of a small instance at four times the cost; however it also has more native storage, is ebs optimized and has high I/O.
There are plenty of use cases where an application is appropriately built on a Platform, especially applications with limited relevancy (specialized applications) or limited time (marketing). In those cases time to market trumps vendor independence.
Like all software development, the rules are really guidelines. There are always exceptions.
Sunday, September 2, 2012
The Cloud and the Pendulum
The single greatest challenge with cloud computing is the shear breadth of knowledge required to understand it. Servers, storage, networks, operating systems, application architecture, data architecture, integration, security, identity management, and on and on. And those are just the tip of the technical domains. Although today we see plenty of talk about cloud, the reality is almost all "private clouds" are really just mass virtualization. Very little has been done in automation and almost nothing with the applications, so calling these environments clouds is misleading at best. So why are we stuck at the crossroads of virtualization and cloud computing?
I attribute the stall out to the fact so few people are able get their arms around the concepts of cloud and get comfortable. We have seen the entrenchment of fear, uncertainty, and doubt across corporate America. I agree with new technologies a prudent approach is to take it slowly and deliberately making sure the risks are mitigated and lessons learned. What's confusing about cloud is the technology isn't new! Not a single piece of new technology has been invented for cloud. Cloud is the natural intersection of solving three pressing IT issues of the 1990's: a global corporate footprint, low server and storage utilization, and efficient code reuse. In fact the average IT department got pretty good at dealing with these challenges in the early 2000's, so why the cloud conundrum? Lack of understanding the forest vs the trees.
To me the focus on the trees is the predictable result of taking a specialist approach. Solving problems in engineering typically starts with functional decomposition; breaking the problem down into smaller problems which are easier to solve, and whose solutions can be combined to ultimately solve the original problem. What we forget is the solutions must be guided by a vision and strategy lest we choose a solution incompatible with our desired state. Developing this vision and strategy requires a complimentary set of skills, the breadth of expertise to see the forest.
Virtualization has exploded because it's narrowly focused. The only teams that are truly involved in virtualization, who need to understand it at the implementation level, are the data center systems engineers. They have added a new skill set in designing, implementing, and managing these virtualized environments composed of networks, servers, and storage. However very little imaginative thinking occurs within this domain because, to limit risk, most companies are aligned with proprietary vendor solutions. And even less imaginative thinking occurs outside the data center infrastructure side because people don't see the applicability. Cloud today is relegated in the Fortune 500 to data center consolidation, yet the value proposition of cloud is, and always has been, as a global collaboration platform.
What we lack in quantity today are the broad minded experts who can paint the picture of the future focusing on the art of the possible. As everyone has rushed to specialize a rare breed of professional has emerged in IT, often but not always coming from the ranks of the enterprise architects. This knowledge worker has been exposed to every element of the technology and business stacks. Jacks of all trades and masters of some, at some point in their career, by accident more often than intent, they had to learn the job next to them rather than the one above, then the next job over, then the next. What has emerged is a well rounded individual with the capacity to see the world from multiple points of view. Those whose areas of knowledge include hardware and software at their core; rounded by additional knowledge in security, networks, and data; tend to be the cloud experts. Tilt the balance toward software and data and you get Big Data experts, another broad technology. Tilt again toward software, mobile device and networks and it's Mobility experts. Rebalance on software and user experience and the expertise is Collaboration and Social Media.
Companies need to identify their experts and incentivize others to choose a path they might not otherwise. There is a dearth of talent in each of the most discussed technology domains because all are multi-disciplinary in nature. Specialists are still needed where the rubber hits the road, but as companies continue to outsource non-core capabilities, which increasingly includes data center operations, more of those specialists will work for solution providers.
Cloud impacts everything from finance, accounting and tax to HR, legal, and marketing. But nowhere is the impact greater than IT where the very composition of the talent pool has to change and change rapidly. Generalists will pave the preferred path to glory for the next generation of IT talent. And as everything is cyclical, certainly the pendulum will swing the other way just as we think we have it all figured out.
I attribute the stall out to the fact so few people are able get their arms around the concepts of cloud and get comfortable. We have seen the entrenchment of fear, uncertainty, and doubt across corporate America. I agree with new technologies a prudent approach is to take it slowly and deliberately making sure the risks are mitigated and lessons learned. What's confusing about cloud is the technology isn't new! Not a single piece of new technology has been invented for cloud. Cloud is the natural intersection of solving three pressing IT issues of the 1990's: a global corporate footprint, low server and storage utilization, and efficient code reuse. In fact the average IT department got pretty good at dealing with these challenges in the early 2000's, so why the cloud conundrum? Lack of understanding the forest vs the trees.
To me the focus on the trees is the predictable result of taking a specialist approach. Solving problems in engineering typically starts with functional decomposition; breaking the problem down into smaller problems which are easier to solve, and whose solutions can be combined to ultimately solve the original problem. What we forget is the solutions must be guided by a vision and strategy lest we choose a solution incompatible with our desired state. Developing this vision and strategy requires a complimentary set of skills, the breadth of expertise to see the forest.
Virtualization has exploded because it's narrowly focused. The only teams that are truly involved in virtualization, who need to understand it at the implementation level, are the data center systems engineers. They have added a new skill set in designing, implementing, and managing these virtualized environments composed of networks, servers, and storage. However very little imaginative thinking occurs within this domain because, to limit risk, most companies are aligned with proprietary vendor solutions. And even less imaginative thinking occurs outside the data center infrastructure side because people don't see the applicability. Cloud today is relegated in the Fortune 500 to data center consolidation, yet the value proposition of cloud is, and always has been, as a global collaboration platform.
What we lack in quantity today are the broad minded experts who can paint the picture of the future focusing on the art of the possible. As everyone has rushed to specialize a rare breed of professional has emerged in IT, often but not always coming from the ranks of the enterprise architects. This knowledge worker has been exposed to every element of the technology and business stacks. Jacks of all trades and masters of some, at some point in their career, by accident more often than intent, they had to learn the job next to them rather than the one above, then the next job over, then the next. What has emerged is a well rounded individual with the capacity to see the world from multiple points of view. Those whose areas of knowledge include hardware and software at their core; rounded by additional knowledge in security, networks, and data; tend to be the cloud experts. Tilt the balance toward software and data and you get Big Data experts, another broad technology. Tilt again toward software, mobile device and networks and it's Mobility experts. Rebalance on software and user experience and the expertise is Collaboration and Social Media.
Companies need to identify their experts and incentivize others to choose a path they might not otherwise. There is a dearth of talent in each of the most discussed technology domains because all are multi-disciplinary in nature. Specialists are still needed where the rubber hits the road, but as companies continue to outsource non-core capabilities, which increasingly includes data center operations, more of those specialists will work for solution providers.
Cloud impacts everything from finance, accounting and tax to HR, legal, and marketing. But nowhere is the impact greater than IT where the very composition of the talent pool has to change and change rapidly. Generalists will pave the preferred path to glory for the next generation of IT talent. And as everything is cyclical, certainly the pendulum will swing the other way just as we think we have it all figured out.
Subscribe to:
Posts (Atom)