Recently I was introduced to an emerging storage company, Blackwave Inc. As I listened to a presentation by CTO Michael Kilian, what impressed me was his clear conviction and articulation of why the company came into in existence. It was not because it had cool technology (that was looking for a problem to solve) but because the company had identified a tangible need and applied technology to develop a meaningful solution. (The technology by the way is intriguing). Blackwave was founded in 2006, currently has about 40 people and has shipped product. It is nascent but highly energetic.
The company’s value proposition is based on the premise that conventional file systems and associated traditional hardware platforms are not sensitive to the requirements of unique data types such as Internet video. This application indifference leads to degraded performance such as longer latencies and stifled throughput. The team at Blackwave believe that this performance gap has created a substantial opportunity in the rapidly growing Internet video and content distribution market.
The real-world opportunity that the team identified provided them with the inspiration and guidance for the technology solution they have subsequently developed. The epiphany was the realization that a storage solution optimized for video and digital content delivery must have unique requirements that traditional storage was unable to deliver. Their claim is that they are able to deliver video from the disk to the Internet more efficiently, at a higher scale, and with a greater simplicity, which results in lower capital, operational, and management costs.
Similar flashes of understanding are commonly experienced by a number of innovative but nascent data storage companies but rarely by today’s storage establishment, or, if they are, they are often suppressed to protect the status quo.
Some of the Blackwave guiding principles are:
- Streaming video must be delivered to thousands of clients simultaneously.
- All content is not equal; different content requires different performance characteristics such as content-specific bit rates.
- Performance must be predictable to maintain video SLA’s and avoid periodic degradation.
- The system must have the intelligence to decide how to best deliver content based on its popularity.
- Systems must scale transparently to the application with minimum management impact.
- Eliminate the unnecessary sophistication and cost of traditional transactional storage and deliver the required performance at the right price.
- Coordinate distributed systems to act as a federated system.
With these principles as the guiding backdrop, the development team created a solution tailored to the Internet video market that is an integration of commodity hardware and unique software.
The Blackwave product suite consists of three models — the 8120, 8150, and 8170 — each with an increase in capacity and power positioned to meet local, regional distribution, or global distribution application needs.
Each platform contains the same basic modules:
- Storage Server: This is a 4U rack that support 24x1TB SATA drives. An embedded quad processor supports the 10GE connectivity used as the back-end system interconnect. Multiple storage servers can be stacked to a currently maximum supported capacity of 96TB.
- Access Server: A 1U server (minimum of 2 per system) with dual quad-core processors delivering 10Gb/s Ethernet connections, which enables connection to the Internet and to the storage servers. Each access server is capable of serving 5Gb/s of progressive HTTP downloads with additional access servers supporting Windows Media streaming or Flash Media streaming. (Note, storage and access servers can be added independently.)
- Interconnect: Two integrated switches. A 16 port, non-blocking switch for data and a 24 port switch for management and upload.
A fully configured 4x4 systems occupies 22U, has a capacity of 96TB, aggregated bandwidth of 20Gb/s and can support up to 30,000 users, depending on the bitrate requirements of the content.
Some of the interesting system innovations include:
- Inode and content data are not stored on the same physical disk. This technique eliminates or at least minimizes actuator thrash.
- System is sensitive to the video lifecycle and associated changing performance characteristics. What this means is that the system determines how to lay out the data and dynamically adjusts as use patterns are established to deliver the required performance. This performance optimization is achieved without administrator intervention.
- System-based decisions include the placement of data on the disk, with the content that requires the highest bandwidth performance located on the outer tracks of the platters. Throughput (content bitrate) requirements dictate the number of disks the data will be stripped across, and resilience requirements dictate the number of replications.
- Bitrate throttling is applied to match the content delivery to the actual bitrate required. This granular provisioning of transport resource prevents unnecessary and inefficient allocation of bandwidth and increases the systems ability to service a large number of concurrent users. Even the most modest of over provisioning of resources can cause a starvation of a request.
- While concurrency is a key attribute, predictability of service is equally important. When a request is received, the data management software determines if the request can be serviced by the available resource and without impacting any of the existing requests. If the answer is yes, the request is serviced; if no, the request is rejected. This is in contrast to other systems, where the willingness to respond to requests without understanding if sufficient resources are available can lead to a degradation in expected QoS. This is an example of the starvation issue referenced in point 4 above.
The Internet video storage solution from Blackwave is tailored for the growing video distribution market. It is a clear illustration of how user sensitivity and an early identification of unique market requirements can drive the rapid development of a dedicated solution with the attributes needed to solve specific customer issues. However as noted earlier, the company is in an early formative stage, and time will tell if their market execution matches their initial development success.
Action Item: Online data storage is increasingly moving away from the old paradigm that a single basic architecture will satisfy all data types and satisfactorily meet all use cases. Storage, management, and content delivery expectations and associated solutions are becoming more closely aligned with data types and usage patterns.
For the end user, if the cost of performance and an ability to predictably meet QoS expectations is a concern, then do not accept a solution that is a compromise. Look for a solution that is designed to meet your requirements; otherwise be prepared to accept a compromise in performance and a potential impact on the service you deliver.
A thought for my friends in the vendor community: If your solution does not address a real world issue that exhibits demonstrable customer pain, then rethink your development plans. Do not expect your product to resonate with your customers if its value is not obvious, and you have to convince them that there is a need.
Footnotes:
