Portal:Storage

Wikitip

Combining compression and data de-duplication

Wikibon estimates that the effects of applying compression to primary storage are additive when combined with traditional data de-duplication solutions (e.g. Data Domain, Falconstor, Diligent, etc). This is based on discussions with practitioners and an analysis of each technology.

As an example, let's assume:

• Primary storage compression ratio 2:1
• Data de-duplication ratio for backup stream 10:1

In a perfect scenario, the capacity reduction ratio for the technologies in combination would yield a final 20:1 data reduction ratio for the backup stream. In a worst case scenario, the data de-duplication ratio would yield 10:1, meaning compression has no additive effect.

Wikibon believes the 'typical' rule-of-thumb is that combined, these technologies will yield a roughly 15:1 data reduction ratio, assuming these base reduction ratios and appropriate data candidates. In practice, compression on primary storage is likely to yield 30-60% improvements in capacity and as such, in real world environments the combined or 'blended' ratio would be lower but still substantially higher than data de-duplication as a standalone solution backing up non-compressed data.

Wikibon believes that this rule of thumb should work for either in-line data de-duplication (e.g. from Data Domain, Diligent, FalconStor etc., and targeted at back-up and restore), or background de-duplication (e.g. the NetApp A-SIS feature which is suitable for finding duplicate 4K blocks in on-line storage).

View Another Wikitip

Featured Case Study

Virtualization Energizes Cal State University

John Charles is the CIO of California State University, East Bay (CSUEB) and Rich Avila is Director, Server & Network Operations. In late 2007 they were both looking down the barrel of a gun. The total amount of power being used in the data center was 67KVA. The maximum power from the current plant was 75kVA. PG&E had informed them that no more power could be delivered. They would be out of power in less than six months. A new data center was planned, but would not be available for two years.

read more...

Featured How-To Note

Storage Virtualization Design and Deployment

A main impediment to storage virtualization is the lack of multiple storage vendor (heterogeneous) support within available virtualization technologies. This inhibits deployment across a data center. The only practical approach is either to implement a single vendor solution across the whole of the data center (practical only for small and some medium size data centers) or to implement virtualization in one or more of the largest storage pools within a data center.

read more...