Learn From The Mistakes of Others

In my career I've had some difficult projects but one in particular stands out as being my Bataan Death March. If successful the project would cement the fortunes of two rising executives; one client and one consultant. If failure prevailed, it would leave both executives careers significantly diminished. The project was to establish a beachhead for integrating the marketing efforts of several autonomous business units, cleverly disguised as an Enterprise Content Management implementation and web presence refresh. Sales were expected to grow as customers realized they could buy an end-to-end solution from the same company even though it might be composed of two or more brands.  With such a vast scope, and all the political challenges I cannot detail, it's incredible to believe the project was even undertaken (which it was against my advice and the advice of several other experts). Here I will list the major lessons I learned, and hopefully provide insight into what should have been done differently.

First Lesson: enterprise transformations are not a re-branding exercise
Day one I knew the project's scope did not reflect its challenges. The client was using the project as a way to effect an organization wide transformation to become a better customer focused company. Transformations require the support of senior leadership, investment, and the acceptance of risk.  Visions of a better future need to be seeded with the executives and given time to germinate, with the hope passion will build within each leader to make the future happen.  Instead we had the feint, background support of some in the C-suite, tepid support from IT at best, and a range from lackluster acceptance to abject opposition from the lines of business.

Second Lesson: any transformation requires a well thought through and defined strategy 
Without the support of the executives, the vision was empty.  As a result nobody wanted to participate in developing a strategy for the transformation.  And without buy-in and a strategy, there was nothing in place to guide the project as it hit critical junctures.  Often during discussions drilling down on a particular topic, nobody could articulate details on what needed to change, how, when and who would be the owner moving forward.  Without a target future state we found there was nowhere for new responsibilities emerging from the transformation to land.  This caused significant friction as we were on the clock, expected to make progress, but everything we did was dependent on client resources providing guidance and making decisions

Third Lesson: transformation success requires talent
All three client resources portrayed themselves as experts in the ECM space when, in reality, their knowledge was limited to content creation. In fact it was this arrogance that created the foundation for the technical challenges. And driving inexperienced, incapable people to work long hours cannot create success out of a vacuum.The client project leader was ineffective at providing input to the timeline and managing client tasks. Due dates were treated as meaningless and repeatedly missed without acknowledging the subsequent impacts to the timeline and cost of the project. The client subject matter expert was an off-site third party consultant spread extremely thin whose goals were not always aligned with our customer. The client technical leader was a junior resource with no architecture, web or application development experience who was quickly relegated to the role of a go-fer.

Fourth Lesson: success does not end with the sale, it requires client satisfaction throughout the project
Based on the above items there was more than enough reason to not accept the risk of the project, or run away when SOW negotiations continued for months after the project was already underway. But revenue generation was necessary as part of the business case for promotion, so the engagement lead ignored the concerns of his team.

Fifth Lesson: risk is inherent in every project, but it should be limited and managed
The engagement leader spent 1-2 days per week on the project but refused to listen to the highly experienced, dedicated directors on the ground every day. Blind to risks, within days he made four core decisions which pushed the project to the edge of the cliff for the remainder of its life:

• dismissed concerns about the missing customer strategy inability to articulate requirements
• used inexperienced, off-shore application developers for an on-site web overhaul and content management implementation
• delayed engaged the ECM platform expert until the statement of work was signed, which didn't occur until four months into the project (two months too late)
• accepted new responsibilities that required client level decisions

Sixth Lesson: communications must be transparent
The client had no reason to believe the decisions of the consultant threatened project success because they were never informed.  In fact they were insulated by the engagement lead who closely managed and controlled all information shared. Regardless issues raised by the consulting team, the message to the client was everything is normal and manageable. As the growing gap between expectations and delivery became increasingly difficult to mask, the engagement lead pushed the team to be "creative" and find ways to make progress.  Doing so quickly degenerated from difficult to impossible.  The team looked busy, but very little of the work actually progressed the project.

Seventh Lesson: take responsibility
Among the directors we knew we were out of options.  However because the engagement lead was also the best friend of his leader, and the engagement lead had driven a wedge between us and the client, we focused on trying to make it work.  We should have documented our concerns and requested a Quality Assurance review.  We had the right to, but doing so would also put our careers in jeopardy, especially considering the reputation of the engagement lead.  We shirked our responsibility and it came back to haunt all three of us repeatedly.

Eighth Lesson: know what you're getting into
To make it a perfect storm, the underlying technology had it's own issues.The ECM platform used by the client had a unique architecture and implementation. Since many members of the consulting team had prior ECM experience, we used that experience to drive estimates.  And because the platform was new and relatively unknown, it was difficult to find expertise.  It took several months to identify an expert and then several more for approval to get the person on-site.  And because the project had been budgeted to use off-shore resources, the cost of the expert pushed the project even further over budget. On-site, we worked with the ECM platform expert to address significant issues with the existing implementation.  He was able to validate my architecture as the correct approach, a contentious point with the client experts from the start which had stymied progress.  His documented expertise, supported by the platform vendor's own testimonial on his brilliance, wasn't enough.  We had to make the changes we knew were right and then show the client why it was better than their way.  Of course this didn't help our timeline, nor budget, but it was effective.  We essentially discarded the existing slow, unsupportable solution for a streamlined, very fast, very extensible implementation. Luckily our new direction changed some of the requirements enabling us to finally use those off-shore resources.

Unfortunately the client was in too deep to walk away or choose a different partner.  Follow-on projects made few changes for the better.  The engagement leader continued to focus on revenue at the expense of people and the truth.

From a deliverable point of view, the project is viewed as a huge success.  It delivered a better mouse trap and, for the first time, made the company appear to be integrated. However, as predicated on day one, it wasn't enough to appear integrated so the net impact on sales was negligible.  The project grossly overran it's budget.  Two of the consultant directors lost their jobs.  Several on the implementation team were given unflattering ratings as a way to spread the blame.  The executive on the client side was terminated and their team took a reputational hit.

And what happened to the consulting engagement lead?  Well of course he got that promotion, and a huge bonus.

Yes, I understand you have legacy applications

More than one CIO has explained to me that until someone comes to them with the solution on how to move their legacy apps into the cloud, they're not interested. Now that approach I find interesting. First, why isn't the CIO looking for the solution instead of waiting for someone to bring it to them? Second, does this mean they will continue to create new apps, which are the legacy apps of tomorrow, in a non-cloud manner perpetuating the problem indefinitely? And of course third, why would someone dig their heals in when it comes to changing the highest cost, least flexible elements of everything under their control?

The good news is that legacy apps are not precluded from cloud computing. The easy part is dealing with the legacy apps of tomorrow. It's becoming a defacto standard that all new development leverage a cloud architecture. Cloud architecture is a refinement of internet architecture which is a refinement of n-tier, client/server, and mainframe architectures. It's all about evolving as the technologies mature.

The hard part is what to do about the legacy apps of today. There are at least four options available: leave alone, enable with services, migrate key algorithms, and finally migrate. The leave alone option does not necessarily mean nothing can be done. For some mainframe apps there are other options. For example in my past we moved a large number of CICS applications from a mainframe to an HP9000 server with the benefit of higher throughput. In fact the Oracle Exalogic platform framework specifically identifies its ability to be used for CICS applications in the literature. And some applications can be moved with few changes such as a Smalltalk application ported from a mainframe to blades for a large brokerage to speed up execution. Never say never.

Option two, enable with services, is a pretty popular approach with those who will not wait. A large insurance company is working hard to enable its mainframe applications representing hundreds of millions of dollars in value to talk on the enterprise service bus (ESB) via services (I know because I designed the architecture). It is possible to keep that large investment largely intact and just change how it interacts with other applications. Through the ESB, a SOA concept, the mainframe apps have to respond in seconds in order to deliver web page results to prospective customers in a reasonable time, defined as no greater than eight seconds. Not only does this approach work, it can work well extending the life of applications to interact with the cloud at the software level while not participating at the hardware level (although I'll argue that with virtualization on the mainframe, they're just as much cloud as Zynga's games).

Migrating algorithms that are within applications rather than entire applications can deliver a huge benefit. Think of a pricing, scoring or inventory management application. The algorithm that helps management figure out what to do (the price, the score, or the shortages) can be move to, or replicated in, a cloud model enabling cloud applications to interact with the algorithm. To a cloud application that may be all that is needed, especially if that algorithm is exposed as a service. In good SOA design services should give answers, not expose raw data, and often all that's needed is the answer.

Finally migrating applications wholesale to cloud makes sense for some mainframe apps. However there is no requirement it happen today. The big issue has always been the business case; justifying the investment. CIO's have pushed application managers to put so much lipstick on these apps that they now resemble pigs. Bolt ons for integration, data management, identity management, internet access, storage tiering, systems management, and on and on combined with years of change requests have created systems so complex they carry a legacy tax. I have worked with several companies where just touching a mainframe application carries a minimum $1,000,000 fee. That's a pretty high buy-in, yet CIO's wonder why they're spending 70% of their budgets on maintenance. That first $1M is non-value added! CIO's need to expedite the sun-setting of these applications by building the case for change based on both the cost to change and the cost of not changing and accruing these legacy taxes. Sometimes you have to do what's right instead of what's easy and this is it.

The challenge to building the business case is the expectation that because computer code isn't physical it's easy to change. I balk at that argument with facts; if that were so why do software changes cost so much? I had a CIO challenge me to get the CFO to understand so I asked the CFO how much money had been spent to update/modify his highest producing factory while the CIO was to determine how much money has been spent to update/modify the most used application. The difference was staggering; the application costs were well over an order of magnitude higher although the plant was twice as old. So I asked the CFO if he were to invest that amount of money into operations, how much of the plant would have to change. He said he'd never do it, it wouldn't make sense. Instead he would build and outfit a new plant. Ah ha! So why do we use a different approach with software?

I'll admit that was a somewhat special case, but in the end it's all about foundation. Mainframe apps were built with a different architecture, intent, vision, everything. It's like investing in swords and armor and then being surprised when your enemy shows up in a kevlar body suit with his M4 and Seal Team Six tatoo. Ask the French how well their Maginot line worked.

Money Makes the World Go Round

Like it or not money is a central part of all of our lives. As a consultant typically one of the first questions I'm asked is "How much?". I worked with a great client several years ago building their IT strategy to accommodate aggressive inorganic growth in the laundry room business. They operated laundry rooms at apartment and condo complexes and knew how to do it right. Their focus was on the management of the money and used it to diagnose any problems. First they could tell you how many loads would be washed and dried based on region and the number of units. They knew how much soap, water and electricity would be used. They could predict the required maintenance with amazing accuracy. Most importantly they knew how much money to expect from every washer, every drier, and every soap dispenser every day across their $1B organization. The EBITDA for each washer. The operating profit of each laundry room. Simply amazing!

Contrast this with the norm at Fortune 500 companies. It is a rare day when a client can tell me how much their technology costs per seat. I've been laughed at and asked why anyone would care. I've been told there are too many variables. Yet when I talk with CEO's about building a new business or offering a new capability the first thing they ask is "How much?". As a guide I typically ask "How much does technology cost per seat today and how many people do you estimate being involved?". Applying an average to generate a back-of-the-napkin estimate works pretty well. So the CEO asks the CFO or CIO who invariably don't know (and if not are not happy to be put on the spot) and invariably we end up taking their technology budget and dividing by the number of people. It's a start, but it's not transformative.

******************************** HINT ******************************

For anyone considering a technology rationalization effort, knowing the detailed costs is a fundamental requirement yet derails more projects than any other single cause based on my experience.

********************************************************************

Whereas the average number gives you a starting point it doesn't give you anything on which to provide an insightful analysis. If that number were good enough the next thing in line would be the contract, just sign on the bottom line. Yet what ends up happening is an exhaustive research project with consultants and IT finance gurus to get to the bottom of what everything really costs. Some times it becomes a project unto itself. Worst of all return in a couple years when the details have changed and you'll typically find you have to start all over again; nobody picked up the work. And I won't go into all the rocks you overturn exposing contracts being paid that are no longer required, missing maintenance on production servers, software licenses that are oversubscribed, and more that make you ask one question: "Who owns the money responsibility in IT?".

I cannot rationalize this lackadaisical approach by IT to activity based costing. On the one hand CIO's bemoan the increasing responsibility of IT with a shrinking or static budget. To me that means knowing the who, what, where, when, how, and why of every dime. Yet there's no transparency in the 2/3 of IT budgets which are operations focused. That tells me IT wants to deliver the message "we don't care what things cost" but somehow I've missed the legions of CIO's standing before their leadership to make that statement.

IT needs to know what things cost and track those costs for all the things it has. There are some tremendous tools out there to help with this but, and perhaps it's just me, I've seen an amazing lack of use. This underlies the larger issue of collaboration between the business and IT. The business talks in dollars, capital vs operating expense. IT has to use the same terminology to be understood, however that's very difficult when getting answers to simple questions like "What does that cost you per year?" or "What's the cost per seat?" take days, weeks, or even months to calculate.

I've been advocating this approach since I ran the IT department for a small call center outsourcer in the mid 1990's. We calculated the cost of everything at various levels (seat, program, location, company) which fed into our estimation process. As a result we could tell a client within a day or two of knowing their requirements the one-time startup fee and the ongoing fees within +/- 10% accuracy. IT was a big part of our business so we ran IT as a business.

This approach is now considered a best practice according to the MIT CISR researchers Peter Weill and Jeanne W. Ross as outlined in their book IT Savvy. So if you don't believe me, believe them, it will pay dividends and is a fundamental component of the success of IT.