Storage Peer Incite: Notes from Wikibon’s May 8, 2007 Research Meeting
David Floyer presents At Your Storage Service! Storage services have been around for a long time, evolving from primarily media transport and secure archiving services to 'swipe the credit card' simplicity. This has significant implications for both user and vendor organizations.
The storage services market is poised for a rapid evolution as the regulatory environment places greater requirements on storage archiving, backup and related activities in businesses, as networking infrastructure technologies continue to evolve to support more complex large file structures and as users and suppliers become more familiar with the application-level realities of what's required from storage subsystems.
Storage services in many respects are not new. In the 1960's and 1970's, the first set of storage services were introduced namely as media services whereby vendors would agree to pick up, transport and store specific media (e.g. tapes, microfiche, etc.) for companies and store them in highly secure facilities. In the 1990's some suppliers attempted to evolve storage services to an online model and essentially provide 'data tone' for the enterprise. For the most part, however, these suppliers failed to secure a market presence as they did a poor job of reconciling the fact that the data being targeted for these services often had very high write to read ratios which made it difficult to move stateful data from one place to another in an online manner quickly and cheaply.
However in the last few years, we've seen new protocols for Web interactions, new standards for application development and new approaches to institutionalizing IT emerge that has made the idea of storage services much more attractive and economically viable. Big network pipes, new protocols for storage access and new conventions for metadata are furthering this market very aggressively and we will start to see new ecosystems for storage services emerge that play off the acceptance in the user community of storage services as a viable value proposition. As users consider what types of storage services to purchase and to depend on, they must focus on a few very simple issues. First is the number of users supported where the more users that require common services, the more attractive the storage service. Secondly, the read:write ratios of applications where the higher the read:write ratio, the more the application is a candidate for storage services. And finally, the degree of process integration associated with the application that ultimately leads to things like long lock times being placed on pieces of data even in a predominantly read environment.
As a consequence, as users evaluate their applications and suppliers in these terms, we'll start to see evolution in the storage services market, essentially revolve around three poles: 1) the availability of access method protocols and standards in those protocols as well as standards in metadata; 2) the organization of services that are increasingly just-in-time-like so that allocation can take place as near to a run time request as possible and 3) very importantly, the adoption of conventions for pricing models that will allow a user to truly compare and contrast the use of an external supplier with their own cost structure to make an informed decision about how to best organize storage capabilities within their organization.
Action Item: It's an exciting time in the storage services market precisely because the conventions associated with buying storage services are pretty well accepted. However to take full advantage of the rapid evolution in organization of storage service offerings, users must pay close attention to a few critical elements to ensure they get the right set of services now that are capable of adapting as their business needs evolve.
In many respects, IT storage organizations have been insulated from the market assault by high quality, cost-effective third party services. Networking, server hosting and application development organizations have all been impacted and forced to respond to market forces which necessitated dramatic changes in headcount, types of skills, and time spent around these increasingly service-oriented delivery systems. Attempts from so-called storage service providers (SSP's) in the late 1990's and early 2000's, that were trying to advance a vision of "Datatone," failed to make a dent on the status quo of storage services delivery within IT organizations.
All that is changing. Misguided attempts by SSP's to often target stateful block-oriented applications are being replaced by sets of pragmatic, low cost storage services from the likes of Amazon, Google, Yahoo and other consumer services firms going after the growing pool of unstructured data swamping organizations. In the immediate-term, storage organizations will be forced by end users to respond with much clearer positioning, packaging and pricing of both internal and third party storage services.
Action Item: IT storage organizations must assess cost structures, packaging and delivery and become more aligned with what's available on the open market. Storage organizations need to benchmark and model both business and consumer storage service suppliers and invest in capabilities that provide clear, flexible pricing options and usage metrics under one IT services umbrella. Only then will make versus buy decisions be clearly driven by business value as oppossed to inefficiencies in current storage delivery models.
Black box storage is going to disappear. The application owner will demand to make informed decisions about the storage services purchased based on the business value they will deliver. Three actions may avoid IT being part of the problem:
- IT should work with application owners to create on organization that provides an internal storage services market, and itemizes the costs for each service consumed. IT also needs to provide tools and expertise to show the value of each of the storage services to the business. For example, demonstrate that providing a snapshot every two hours will improve the time to recover from a failure by 10 hours an average of five times a year.
- IT should create the necessary structures to work with the business and agree criteria for qualifying storage service vendors. This should include auditing the delivered level of service, testing recovery and location of the data.
- IT should agree clear guidelines to application owners about the data most suitable to use external services, as explained the storage alert entitled "Storage services inside out".
Action: IT should anticipate storage services and organize to exploit both internal and external storage services. They should create the forums, tools and collective experience to ensure that application owners make informed decisions about storage services.
IT should avoid arguing why data should not be outsourced; the perception of protecting an empire will ensure that arguments, however valid, will be ignored. A better strategy is to suggest proactively where data should be held external to an organization, and to agree the business case, legal and qualification issues with the business. I suggest six main decision criteria:
- The larger the number of users and the greater the dispersion, the greater the potential of outside vendors to leverage capabilities to connect to data (e.g., Web marketing data).
- The higher the read to write ratio, the better suited the application to external storage services. High write rates and write ratios found in many transactional “business state” systems should usually be kept internal.
- Data not tightly integrated with many applications, and as a result having low write and read locking rates, will be more suitable for external services (e.g., email archiving)
- File or record orientated data will often be easier to externalize, because the data is easier to split from the application.
- Data that uses a storage service provider that can gain economies of scale by providing storage services for many clients, often in one industry. Cheaper costs just because the operators are based in India won’t cut it long term, and will result in higher switching costs down the road.
- Data that meets corporate and IT standards for the placement, protection and services access levels to information.
Action item: Technology considerations make applications that meet the above criteria like backup, “C” sites data services, personal files, email, email archiving, archiving, web sites, web advertising and marketing data and medical records likely starting candidates for storage services.
The rush of available storage services from consumer giants like Amazon, Google and Yahoo is in direct conflict with the tides of traditional enterprise storage offerings; and it is about to send the storage business reeling. Consider the evidence. Long sales cycles with highly-paid direct sales teams versus the swipe of a credit card. Swarms of systems engineers creating value by supporting the development of highly scalable, resilient infrastructures versus invisible, virtualized, always-on resources made available with the click of a mouse.
Indeed, while many enterprise storage companies rightly see this as an opportunity, the threat is just as large. The highly publicized use of Gmail at Arizona State University to replace existing email services and Network Appliance gear, redeploy storage admin headcount and delight users by increasing storage mailbox capacities from 50MB to 2000MB is just one example.
Storage services have evolved from primarily media services to failed attempts at supporting online B2B applications to today's consumer-driven online storage services. Recently we've seen companies like EMC, Symantec and others respond by announcing the intent to offer services in the backup and archive space, clearly ripe for the storage services picking along with other read-oriented enterprise storage applications. The old adage of "lead, follow or get out of the way" surely applies here as vendors scramble to position and/or understand what this all means to their businesses. Unlike many traditional storage ecosystem wars, however, the pending storage services battles will evolve less around technology and more around marketing.
Action Item: To get a fair slice of the storage services pie, suppliers must aggressively embrace the online delivery of storage services, either directly or via strategic partnerships. Transparent pricing and usage metrics are 'table stakes.' Vendors must make marketing a key priority and source of value creation to accelerate adoption with clear concepts, naming, segmentation, packaging, pricing and positioning.
As the range of storage service technologies and offerings expands, they will be positioned in a variety of ways by vendors, including as wholesale replacements for captive storage resources. However, while influential parties (e.g., finance) in some organizations may push for a broad substitution of storage subsystems by storage services, prudent strategies will recognize that storage services are better viewed as a complement to captive subsystems. Temporary capacity will be one important use of storage services, but other, specialized services will be adopted most quickly. In general, these other services will tend to be tied to the ability to offer highly differentiated cost and performance parameters, thereby becoming especially attractive to owners of specialized applications that are highly dependent on one, limited aspect of storage technology.
Action Item: IT organizations should promote the potential role of specialty storage services to their business as a means to foster both successful and appropriate adoption of these new capabilities.