Storage Peer Incite: Notes from Wikibon’s February 19, 2008 Research Meeting
This week's subject is creating an internal storage service, based on the outsourcing model. For shops with operational efficiencies and economies of scale equivalent to large outsourcers, this can create the best of both worlds. Creating a storage service does require the complete backing of senior management, which at times must be used as a club with parts of the organization used to buying from their own, non-standard vendors. However, it can allow the storage organization to establish and enforce corporate standards for vendors and equipment that will greatly simplify their operations and cut large amounts from the operating budget while maintaining complete control over corporate data, including security and access. This is particularly important for large organizations in highly regulated, carefully audited industries such as financial and medical. These organizations need to demonstrate compliance with very specific regulations concerning the handling of their data, which is often easier to do with internal services designed specifically to meet those regulations. Long term, it also allows the organization to ensure continuity of data storage and archiving, which can be difficult in an outsourced environment when the firm needs to change outsourcers. And it provides control over where data is stored and archived. Third world countries, which outsourcers use, may offer immediate cost advantages but often have their own long-term data security disadvantages. G. Berton Latamore
The basic premise behind a storage services architecture is thinking about infrastructure as a granular set of reusable services that can be invoked as needed by the appropriate business application. This approach allows organizations to optimize on cost, performance, recovery, and other metrics that are fundamental to business processes.
One way to conceptualize such an architecture is a three-dimensional cube with the following vectors, where infrastructure management includes a stack of resources from device to network to storage and server platforms all the way through applications:
- Infrastructure management (e.g. technology components),
- Information management – technology tiers (e.g. Performance),
- Data protection – (e.g. availability, classes of service).
Information management includes various performance levels built on tiered storage. – Data classification, global name space, retention/archive, and the data protection dimension accommodates a spectrum of protection methodologies from always on to generally reliable (backup, recovery, etc.)
Sets of various storage services are layered within each dimension – for example, provisioning services as part of infrastructure, copy services in data protection and archiving services in information management. The intersection of these dimensions and their corollary services will determine the technologies used, service levels, and costs. Importantly, not all intersections are viable. For example, the choice of iSCSI as a protocol today will not fit continuous availability tier 1 performance requirements and therefore would not be an option on the services menu.
What are the benefits of a storage services architecture? In addition to the lengthening of depreciation schedules, there are four primary benefits of this approach, including:
- It provides granular acquisition options for business lines with a high degree of cost transparency;
- It causes businesses and IT to make tradeoffs between cost and function and will naturally lead to more efficient use of storage as a resource;
- It minimizes the complexity of the technology portfolio and avoids “one off” solutions that can create migration problems down the road;
- It limits the number of suppliers further reducing complexity.
What drawbacks and organizational impediments will a storage services architecture bring? To be sure, there are some political and practical minefields users should consider with such an approach. First, a reduction in the number of technologies and vendor choices will limit technology options and create friction, as leading technologists will naturally want to integrate and exploit the latest innovations. As such, it is critical that organizations use their ‘sandbox’ lab specifically as a means of evolving the services model (i.e. their cube of offerings) so that the services offered remain competitive and current.
In addition, to make storage services effectively pay back, service level agreements and chargeback models must be in place and aligned, which will require deliberate effort, thought, plans, and actions.
Finally, the politics of consolidating storage services deployment options, while limiting the free-for-all mentality, will naturally disaffect certain factions within organizations that have become comfortable with high degrees of freedom.
What best practice advice should be considered? There are several actions organizations should pursue in regard to storage services implementation, including:
- "Construct service level agreements (SLAs)" and chargebacks consistent with services businesses with a one-time cost and an ongoing monthly fee that includes a technology refresh. Build switching costs (aka early withdrawal penalties) into the model and construct agreements over periods of time where lower pricing applies for longer contract periods;
- Limit the number of suppliers and negotiate on-demand contracts where vendors will install equipment but charge upon usage at the current price when that equipment is turned on, not when it was installed;
- Be aware that LAN-based backup will present the most difficult SLA challenge and consider disk-based backup for distributed networks; possibly even outsourcing to a remote service provider.
Most organizations will manage such initiatives as a cost center, or at the very least share some of the profits with business lines or invest back in the business.
Action Item: Storage in many ways has been insulated from the market forces of services-oriented architecture (SOA), software as a service (SaaS) and consumer-like business models. This is changing, and customers should evaluate the feasibility of constructing service-oriented models for storage which emphasize granularity, reusability, and cost transparency. This will allow organizations to make intelligent make vs. buy choices pertaining to an emerging set of managed storage services from a variety of service providers, including EMC, HP and IBM.
Users looking to provide Storage as a Service (STAAS) internally must understand that such a service must be productized, sold and marketed just like any product. STAAS must be run as a business replete with an organization consisting of teams dedicated to sales and marketing, product development, product management, product support, and a “finance organization.”
Of these, sales and marketing are, perhaps, the most foreign concepts to a traditional IT infrastructure organization. But they are also the most important. Their job is to sell STAAS based on the merits of reduced complexity, fewer vendors, better storage efficiency, faster provisioning, and lower TCO. Their customers are usually application managers or lines of business who are accustomed to owning their piece of the IT infrastructure – after all they paid for it. Or, they have just come back from an application software vendor meeting or conference and have been told that these applications run best with some specific hardware which is not part of the STAAS standardized offering.
We have seen that STAAS can be driven from the bottom up, top-down, or both. Bottom-up can be career threatening and can take years to get buy-in. Top-down often comes when there is a changing of the guard. Being driven from both levels often happens when service levels are consistently not being met and senior management gets frustrated enough to buy into the STAAS approach that has been championed by the STAAS advocates.
The second rather foreign concept is that of a STAAS “finance” organization. STAAS must, in essence, perform the magic trick of turning fixed-cost capital expenditures into variable operating expenses for its customers. As such, the STAAS organization must be able to generate a profit in order to have the working capital to purchase fixed resources (e.g., a storage array, a disk drive, or a network switch) and serve them up as a variable resources (e.g., allocated storage, bytes managed, bytes protected). Exactly how that profit is accounted for is problematic, but most STAAS organizations shoot for a zero-profit at the end of the year approach. The STAAS finance unit must also, of course, provide a billing and collection function that can hopefully leverage an existing chargeback mechanism. This only works if the business units pay real money for services rendered and not “funny money.”
One strategy that greatly simplifies this magic trick is to buy from vendors that offer “storage on demand.” Obviously, this just shifts much the capital expenditures to the vendor, but many are willing to do it, realizing they will become one of only two or three vendors supplying the standardized STAAS offering.
Action item: Organize and run storage-as-a-service like a business. Develop a top sales and marketing team backed by senior management. Work out the financial intricacies in advance including real chargeback. Plan on having many difficult meetings. Use vendors that offer “storage on demand.”
From the point of view of a business application owner or an IT application implementer, storage services interpose a layer of bureaucracy and take away a degree of freedom. Large organizations with a CIO for each division can represent the greatest challenge to continued employment for storage administrators.
Many managers find it fun to evaluate and decide every element of the technology stack, and fierce loyalties are built up to certain vendors. The benefits of storage services include reduced costs and complexity and are derived from economies of scale and a collective agreement to use the service. If senior management is perceived as allowing exceptions, they will be inundated with business cases and virulent lobbying. A process where storage administration has to seek renewed authority from senior management will result in the preparation of three envelopes and a resume. A storage services initiative cannot be optional or a recommendation; it has to be a mandate.
Action item: Storage services procedures must be free from bureaucracy and exceptions. Unless there is a clear unambiguous mandate from a senior manager with sufficient authority and control, human nature will sink a storage services initiative and the people who try to implement it.
One of the objectives of an IT organization is to streamline operations, thus reducing costs. Having well-defined services offered and communicated to an organization is essential to successfully achieving this goal. A Services-oriented Architecture (SOA) defines the services, components, vendors, and products and presents reusable services to the customer to consume.
By presenting only services and not technology, an IT shop can reduce the complexity in building a stable, reliable and robust infrastructure. We know that supporting a data center of multi-vendor, custom, one-off solutions for each application is costly, inefficient and resource consuming. Affecting a transformation to become a true service provider is non-trivial but significant as historically too wide a variety of constituents is involved in making technology choices.
What does it take to accomplish?
I believe there are a few key items that will make this transition successful, including:
- A limited portfolio of service offerings (i.e. reduced but capable). For storage, there are three top level services: data protection, information management, and infrastructure management (see Figure 1).
- Limiting the number of technology suppliers. For example, two array providers with a supporting suite of software based on SNIA standards, one tape supplier, one common resource management tool, and one or two fabric suppliers (yes they can interoperate). Provide direction via a roadmap and keys to success and allow them to deliver.
- Management support. This is critical, and I don’t mean signing the check, but buying into the concept, advocating it to customers and ensuring that you are successful in delivering on your service.
- A chargeback model that enables your group to acquire storage ahead of demand while providing a mechanism for refreshing technology and, if needed, acquiring human capital as workload increases.
I also recommend that you revisit the services portfolio and technology solutions every 12 – 18 months and your suppliers every 24 – 36 months. Assess each for success and what needs to be changed. Do they still align with your model? This way your management, suppliers, customers, and teammates all have a clear roadmap and know that their investment in your service architecture and has a positive return.
Action item: Architecting storage services is a way to transform from a cost necessity to a respected service group and value producer where infrastructure is an enabler, not an impediment to business progress. While the hurdles to this transformation are significant, once achieved they can make an IT organization much more cost efficient and productive. One tradeoff is less technology choice. As such, a self-funding model that charges consumers a modest fee to refresh the technology portfolio periodically will allow such initiatives to remain competitive and thrive.
With the advent of services-oriented architecture (SOA), Web 2.0, software (or storage) as a service (SaaS) and the general consumerization of IT, constructing a storage architecture comprised of clearly defined, re-usable services is starting to have some appeal. This approach seems to lend itself more to the big six (EMC, Hitachi, HP, IBM, NetApp, and Sun) than smaller firms because larger companies have a broad technology portfolio and the resources to slog out the contractual terms necessary to actually earn a profit on such an arrangement.
The premise of a services-oriented approach involves:
- Defining, communicating and offering sets of pre-defined services that can be purchased and consumed in a menu-like fashion;
- Charging (back) for services consumed with transparent pricing and usage metrics;
- Reducing complexity by winnowing the number of suppliers and "one-off" configurations that create migration headaches down the road;
- Evolving service offerings as business needs warrant and technology matures.
All this is designed to make IT more cost efficient and operate with a model much more like a managed service provider or Google (without the advertising). This type of messaging will have immediate appeal to line-of-business heads and senior management, but it's not likely to resonate well with many core technology constituents. This is because it sends a message to these individuals that the days of easily bypassing the corporate standard are gone. Many in the organization will have good relationships with a diverse set of suppliers and enjoy the flexibility of deploying "best-of-breed du jour."
So when should such an approach be proposed? There are at least three very related cases where this makes sense:
- An organization that is going through a cost-cutting transformation and rationalizing infrastructure is high on its priority list (think HP/Randy Mott);
- An organization with a new leader that has been empowered to break through inertia and implement change;
- An incumbent supplier has an exceedingly strong relationship with a customer and is in a strong position to deliver such services.
In all three cases, the sale will be made outside of the server room and much closer to the board room.
Action item: Selling services-oriented storage requires a broad technology portfolio and layers of services including virtualization, security, data migration, provisioning and dozens of other capabilities. These however are table stakes in marketing such a solution to an internal IT organization. Suppliers must target and secure senior management sponsors and champions from business lines, procurement, finance and the CIO to effectively persuade technology professionals to buy into a storage services architecture.
The benefits of storage services come from reducing complexity and increasing economies of scale. That means getting rid of arrays and migrating the data and applications. It means getting rid of software and changing procedures. It means implementing a sand-box to evaluate new technologies and refresh the storage services “cube.” It means a lot of work and push-back from users and IT staff resistant to change.
The business case should include an implementation plan. The plan must be detailed and include clear priorities and specific dates for removal of specific arrays and software. Delaying the detailed planning until after the agreement in principle to implement storage services is likely to result in little actual change and could actually increase complexity and costs.
One way of using the short window of leverage over winning vendors is to involve them in managing the process of cleaning house. A third party with a clear motivation of additional business as a result of success can reduce the friction of change.
Action item: The business case for storage services should include a detailed plan with priorities for reducing the number of array types and streamlining the storage procedures round a small set of storage management software. Consider using services from winning storage vendors to clean house and ensure the benefits are achieved by tying execution success to future business.