You need it real time? Really? Real Time???

Business intelligence and executive dashboards are all the rage soon to be complimented, if not already, by customer dashboards. Although I'm no ERP specialist I get involved in all kinds of projects and one of the most glaring inconsistencies I face is the demand for real time data that changes daily or even weekly.

First we need to drop the term real time and focus on the term most recent. Real time to a guy like me from the plant floor automation/data acquisition/embedded systems/engine control module world means something TOTALLY different. Real time means just what it says, in the moment. No delays.

Second business people need to think about data with the fourth dimension of time. Why ask for a data update when the data hasn't changed? Whenever I hear the request for real time data I push back by asking "How often does the data change?" Sometimes it's hourly, sometimes periodic throughout the day. Most often its daily or weekly. Rarely does the data change minute to minute. More important to understand is how the data drive a decision. In one project we grabbed real time service data when all the executives needed was an hourly update. In return that hourly update was used to perform daily staffing analysis, and even then only when the service data exceeded set boundaries. Since the designers never asked, it was just assumed the data had to be real time. We changed from a real time posting to the executive information portal to an exception based alert and made everyone's life that much easier. This is one of hundreds of examples from healthcare to consumer packaged goods to retail that I have in my work history.

Why is this a big deal? Speed costs. It does in racing and it does in technology. The faster you want to go the more it's going to cost you. In IT we're not doing our jobs if we comply with the request just because the business "wants it". When we load up systems with unnecessary requests we slow everyone down; the opportunity cost of bad design. A great architectural option is the data intermediary which sits between the application and the data store to cache query and service results (if the data hasn't changed, the service response won't either)

Educate the business, add the time dimension to all data, and consider how the data drives decisions when building interfaces, services, reports, and dashboards. And the added side benefit? Less complexity!

We're Through the Looking Glass - Cloud Security

I feel it's appropriate to consider an Alice in Wonderland world when thinking through the cloud computing landscape and it's security implications. Most security experts I know, including many CISO's at clients some of whom are quoted on the topic, appear to throw water on the burning desire of CTO's everywhere to go "cloud". It's understandable and, in my opinion, quite reasonable. Let's face it - the cloud isn't ready for prime time.

I have no argument against using the cloud for non-critical tasks but I tell clients day in and day out we are 2-3yrs from enterprise clouding (I love new domains where we can make up words!). Comments like that get me in the good graces of CISO's, at least until my next sentence, "You better get started now." What? Why do we have to get started now if the enterprise version won't be ready for 2-3yrs? Well because that's when it will be easy and everyone will have it - don't you want a competitive advantage? Well then put your nose to the grindstone and get that whole security thing figured out pronto so IT can move forward...

...or get run over, your choice!

With the cloud computing juggernaut gaining speed now is not the time for "No, but..." responses. What CIO's and CTO's need now are "Yes, if..." answers on how to pursue secure cloud services. We have lots of existing models, standards, and solutions so nobody can tell me the cloud is entirely unique. What it does present is a new architecture to which we need to plug in known solutions to known problems and some new solutions to cover feared gaps.

One of the biggest gaps clients identify today is data security. "How do I know my data is secure at a cloud provider?" Honestly I don't know in a holistic way but the old stand-by of encrypt data in transit and data at rest seems to pose the foundation of a solution. The immediate response, as the responder's face wrinkles so their eyes become nothing but slits in the creases of skin below their brow, "But that's too much overhead". Oh. So are we taking this security thing seriously or not? If we are then again, lets take our foundation and now get to work on the speed issue. Solving that problem involves the economics of speed where money is often the answer, governance so we don't speedily reach a cliff, and improving performance so the overhead of encryption becomes a round-off error.

Economically we don't have much of an issue. Cheap bandwidth. Cheap cloud storage. At $90k per 50TB of data storage at Amazon S3 we can afford encryption even if it increases our data sets by an order of magnitude in size. Governance is an issue but as we increase the use of automation in the cloud we should be automating governance as well. We need strong tools enabling us to enforce policies, especially on data which is hasn't been categorized.

How do we increase performance? Encryption takes time but if we can convince the cloud storage providers to provide hardware based encryption we can reduce the cost. Next we need a new way of thinking that takes advantage of cloud: lots of network bandwidth, storage services available on the fly, and ubiquitous availability.

How about applying a grid storage idea to the problem for data archival. Take a set of data and split it up into multiple chunks, each chunk with a sequence number, and encrypt it. Store a random set of chunks at three or more storage vendors and manage which data is stored where using a private index engine. Because each site contains a portion of the total data, the data is non-contiguous, and the data is random the value of the data at the site is dramatically reduced. A hacker would be required to hack all the sites, decrypt the data, and reassemble it to get the full picture. With the landscape being inherently more difficult to hack and the value of any independent data set being low a less onerous encryption method, such as one using 64bit keys, can be used.

Where could such a solution be used? How about monthly billing. Once the bill is generated and paid, the details are rarely if ever used again. Archive the data to the cloud. If it needs to be retrieved it can be, but its value is low to begin with for most hackers. Securing the data through obfuscation will make most hackers look for easier targets. Hacking is a numbers game.

So for all the CISO's out there consider that now is the time to identify the gaps and start looking at how to fill them in. One thing I can assure you of as we talk to CIO's and CTO's, the cloud computing train is coming and it's starting to build some serious momentum.

Be prepared to lead, follow, or get out of the way!

The Achilles Heel of Cloud Computing

I think everyone understands the "Cloud" in cloud computing is an undefined network incredibly similar to, but not necessarily synonymous, with the Internet. What many apparently have not spent time thinking through is the impact building private and public clouds will have on the network architecture at most public companies. Three fundamentals which need to be revisited are: the Internet connectivity architecture, Internet bandwidth, and network security.

Many, if not most, large corporations consolidate their internet connectivity into a very few and sometimes a single point. I was working recently with a large non-governmental organization who has consolidated all of their internet access for primary and field offices into their Chicago data center. It's a great model for using the Internet, not so good for incorporating the Internet. In the world of Cloud Computing the Internet is less an end point and more one of several intermediate points during the execution of a function. Public clouds must be accessible at all times from any location to be of value. This mandate implies there is no single point of failure between the corporation and the public internet. A new architecture is required with many-to-one access to the internet instead of a one-to-one model. If New York cannot connect to the Internet it cannot jeopardize the entire corporation. And remember that Internet backbones do go down and more likely will due to ever increasing loads in the foreseeable future. If the public cloud function leverages data within the data center the reverse is true; multiple paths provide redundancy. I'm sure some will argue with me but I cannot make sense of data travelling from Denver to Chicago just to gain access to the Internet; it's an archaic model at best.

The second major issue is existing Internet bandwidth will have to grow. At the same time as traffic moves to the Internet it will move off of internal WAN's. We've grown accustomed to cheap bandwidth but with the explosion of WiFi, the coming of WiMax, and the growth of rich media on the Internet I expect those days are coming to a close. I expect it will be cheaper to run over the Internet than through private backbones which will help drive us to a more federated model for Internet connectivity.

Finally security as we see it today becomes problematic. How do we sniff packets between a user and a cloud provider when the company has nothing traversed in between. Surely we could route users through the corporate firewall but again, this defeats some of the economic model of cloud. We need better tools on the client side to help us manage the security aspects of this federated model. I'm not saying there aren't tools today, but those tools need to improve their automated detection, recording, and reporting capabilities to prevent attacks both inside and outside the company.

I've noticed over the past ten years a change in approach to networking. In the past bandwidth was managed loosely to ensure it was adequate. We've really tightened it down and now we need to be asking our bandwidth providers to provide more virtualized options enabling the rapid, automated scale up and down of circuits. We don't want to leave the network out of the push to move from a fixed to a variable cost model in IT.