«
»

Dedupe Rates Matter…Just Not as Much as You Think

Posted by David Vellante in IBM, memeconnect, Wikibon on April 9, 2010


Not All Compression is Created Equally

In a recent blog post entitled: Ocarina Weighs in- Dedupe Ratios Do Matter, VP Carter George writes:  The dedupe ratio measures against what’s left after you’ve deduped. The percentage measures against the size of the data before you dedupe.  Both are valid measures.  It’s also true that some solutions do a better job shrinking your data than others.  Dedupe solutions that do a better job should be ranked higher when you are comparing solutions.  That said, comparing the claims made on vendor websites is not a very good way to find out who can actually shrink your data better.
Why?  Vendors lie.

I agree with Carter that Dedupe or compression or capacity optimization solutions that do a better job should be ranked higher. I also agree that vendors like to hype dedupe optimization ratios. The question is how to determine which solutions do a “better job.” I contend Dedupe optimization ratio is not the best measure.

For example, I’ll bet that on balance, despite the fact that Data Domain hypes dedupe ratios, its dedupe rates are higher than Ocarina’s. So should Data Domain be ranked higher? Not necessarily – because Data Domain is used in a different use case than Ocarina. Data Domain is all about backup where the opportunities to reduce capacity are greater than in primary storage or archiving use cases, generally.

When I applied this thinking to primary storage I realized capacity optimization technologies for primary storage are not created equally. Wikibon research suggests that inline compression is far more efficient and effective than alternatives for primary storage. What does that mean? It means that for the cost of the solution you get way more bang for the buck—if applied in the right use cases. By ‘way’ I don’t mean a little bit…I mean as much as two orders of magnitude better. Maybe this is completely intuitive to you but it blew me away when I realized what was happening in this technology space.

How did I come to this conclusion? The answer is CORE.

Update – 4-21-10:

I’ve received lots of feedback on CORE from a variety of sources that I respect. Here’s a summary:

*You need to include decompress rates as well.

*CORE too heavily weighs performance and as such is seriously flawed.

*The piece is seriously biased toward Storwize – its competitive advantages are not as high as CORE suggests.

*You need to do a full TCO and include operational costs.

My take – including decompress rates might change the conclusions; especially for ASIS so we should do that.

2 & 3 are closely related imo. Hard to believe any company has that much of a competitive advantage, isn’t it? But if you could embed something like Storwize into an array (a la ASIS), make it super fast and invisible…I think that would potentially change the game. I haven’t concluded (yet) that performance is less critical than the CORE calculation implies…but I’m very open to improving the calculations. I think that should be the area of discussion. To me…#3 above says that performance is not as important as CORE implies. My fundamental question is “why not?”  If time is money it would stand to reason that, in primary storage anyway, performance is absolutely critical. Where I admittedly struggle with this is a) ASIS is ‘free’ and b) what Ocarina does with different file types is really interesting and clearly adds value. The fact that you can do both during a batch window should somehow be factored.

On full TCO – I agree – that would be useful.

Background – CORE

Wikibon has developed a model to evaluate the effectiveness of data reduction technologies.

The model uses the concept of CORE, which stands for Capacity Optimization Ratio Effectiveness. It is a measure of the effectiveness of a storage optimization technology as a function of time and cost to achieve a desired capacity reduction. The bottom line goal was summarized by Wikibon member Mike Davis who reviewed the methodology and said:

…the goal of your proposal is to be able to rank ROI as defined by (marginal benefit)/(marginal cost).

Ask a CIO what’s more important – ROI or Dedupe ratios.

So we came up with some math to reflect that basic ROI concept and ended up with CORE. We vetted the idea to the Wikibon community and received some excellent feedback.You can read a full description of the math for CORE on Wikibon but basically it goes like this:

For a given capacity reduction technology, CORE is the capacity being reduced (S) times the percent reduction achieved (R) times the value of the capacity being saved (V) divided by the cost of the solution doing the reducing (C) times the elapsed time to compress the capacity (tc).

Here’s the formula:

CORE = (S X R X V) ÷ (C X tc)

The bottom line of CORE is the higher the number the more efficient and effective the technology…and the higher the ROI.

Applying CORE

Warning…these are rough figures and haven’t been fully tested. But they are incredible nonetheless. If you want to change one of the assumptions in the model make a comment in this blog or email me and I’ll happily re-run the figures. But I honestly don’t think it will change the conclusions dramatically.

We started with a primary storage use case which is a hot area of discussion these days and looked at eight different data reduction technologies that vendors claim to be appropriate for primary storage (see Table 1). We assumed a 100TB target capacity to compress. In addition, the table below shows our assumptions around % capacity reduction (Carter’s dedupe ratio), the time to compress (because time is money), the cost of the solution and, the resulting CORE (which is a measure of business value).

[UPDATE: 4/28/2010 - Primary storage is definined as on-line active data with a specified 'typical' read:write ratio; where a user writes data to a persistent medium and can read back that data tranparently. Transparency in this definition means when writing and reading data there is no disruption to a user's application experience at any time.]

Table 1: Effectiveness of Capacity Optimization Technologies for Primary Storage

*The higher the CORE the more effective the technology

*Time to compress values are rough estimates for the unit of capacity specified in the assumptions below—it has the single biggest impact on CORE

*NetApp doesn’t charge for ASIS – we took a percentage of the array’s cost

Here are the additional assumptions used in the model.

Table 2: Assumptions Behind the CORE Model

What Does this All Mean?

First it’s important to remember the use case here is narrowly defined as primary storage. I really hadn’t comprehended this whole space until we went through this analysis. Vendors brief you and they position their products as targeting primary storage and generally it makes sense. Given that caveat, here’s my take:

  • There is only one technology on this map (Storwize) that I would consider appropriate for true primary storage—i.e. active data.
  • The rest are really focused on stale data or secondary storage.
  • I believe the reason is that if you tried applying these technologies to active data you would run into some serious performance problems; which is why most of them operate post-process; and that is clearly demonstrated by this model.

There are several caveats to this analysis starting with time to compress. I used rough estimates in terms of time to compress a file. But even if I’m off base by a fairly wide percentage, it really doesn’t matter because in-line processing is much faster than alternatives. This is substantially what drives the CORE values along with the other factors cited.

Another caveat is NetApp’s ASIS and ZFS. Both are ‘free,’ meaning NetApp and Sun/Oracle don’t charge additional fees for the feature—making its ROI virtually infinite. But as we know nothing in technology is really free and so we had to make some assumptions about the actual cost of the feature when it’s being applied (in terms of resource consumption).

As well, most vendors on this list are careful about how they position their products. For example, Permabit is really focused on archiving use cases but the company’s technology can be applied to primary storage to increase the data reduction potential. Permabit has a “dedupe everywhere” mantra but its main focus is on archiving use cases; as is the case for most vendors on this list. The difference is Permabit is clear about that in its marketing.

It’s Not Only About the Reduction Rate

You can see this by looking at the following diagram. The chart shows capacity reduction rate on the X-axis and CORE on the vertical axis.  While the technologies vary quite dramatically in terms of dedupe or compression ratio, from a business value standpoint it really doesn’t matter. Why do I say that? Because when you factor in reduction ratio, cost, speed and efficiency – the real measures of ROI – wide swings in capacity optimization ratios have little impact on value (CORE). That’s bizarre but unless you don’t care about disruption to your IT shop, it’s true.

Notice that most technologies cluster around a relatively low CORE (i.e. a CORE value of less than 250)– although ZFS’s CORE is higher because compression is built directly into the I/O pipeline. But even ZFS is still too slow when considering the impact to the performance of applications. In the case of Storwize, the only real time technology we evaluated, the CORE is off the charts.

The bottom line is despite the way some vendors position their products, most should not be applied to what I think of as primary storage. This doesn’t mean there’s anything wrong with the technology—it just means it’s not appropriate for true primary storage applications.

Figure 1: In-line Compression is Orders of Magnitude “Better” For Primary Use Cases

I would say based on this analysis that any solution with a CORE of less than 1000 should not be considered for real time primary storage.   The only technologies I know of that are truly real time are Storwize and HIFN. There may be others and I need to do more research on this topic to develop a broader perspective. I’d be interested if readers of this post know any additional technologies that perform real time compression other than these two.

Additionally, as many in the Wikibon community have suggested, this analysis could go much deeper and include downstream effects on: a) other use cases like backup; b) operational costs (e.g. energy consumption) and c) performance and other capacity impacts. I agree. If you did that you would really get a better picture of the ROI, especially as you add in factors such as the ability to support heterogeneous storage.

The Winzip Factor

The performance issue is why I included Winzip. Everyone uses Winzip and while the inclusion of the technology in this analysis is hypothetical (i.e. you’d never use Winzip to compress 100TB) it is instructive to think about how Winzip works. You have a file to compress and you need space on the disk to compress and uncompress. If you don’t have the space you can’t use the tool. If you try to compress a file and you don’t have enough disk space you’ll get this error message:

In the case of Winzip the user doesn’t mind so much because you’re really compressing the files temporarily to store them and perhaps move them. Nonetheless, if the point is to save money by reducing capacity, one has to wonder: “If I need the disk space available to be able to decompress the files, how much money am I really saving on disk space?”

The obvious answer is I’ll never need to decompress all the compressed files that I have at the same time so as long as I have enough space available for the ones I want to bring back, I’m saving money. But users should be careful to understand the overheads associated with various capacity reduction techniques.

Conclusions

Capacity optimization for primary storage is becoming more mainstream; and there are many solutions to consider– including several we have not assessed here such as NTFS, HIFN and Oracle columnar compression. Don’t get sucked into debates about dedupe and compression ratios – as Carter points out in his blog – there’s lots of smoke and mirrors there. Each technology has its place however in-line compression is fundamentally more efficient and in theory can be applied to a wider variety of use cases. Technologies that operate post process do so because that’s the only way they can operate without disrupting IT operations.

Users should be mindful of this and be careful not to get confused by phrases like data reduction for primary storage. Primary in this case may really mean secondary.

This entry was posted on April 9, 2010, 12:58 am and is filed under IBM, memeconnect, Wikibon. You can follow any responses to this entry through RSS 2.0. You can leave a response, or trackback from your own site.

  • http://andirog.blogspot.com/ Anil Gupta

    I will suggest adding time to uncompress/reconstitute in your CORE too. Most people forget to consider dedupe impact on read/write speed. The main challenge deduping primary storage is not how fast you can dedupe or how much duplication you can squeeze out but how fast can you reconstitute. You will see an inverse relationship between dedupe/compression ratio and time to reconstitute.

    With primary storage, both speed of read and write important. With archive storage speed of read is relatively more important. With backup storage speed of write is relatively more important. Pick a dedupe/compression technique that fits your read/write profile not who can squeeze out the most and how fast.

  • dvellante

    That's an excellent point Anil and clearly affects ROI. Thanks for the input. I think it will further underscore the benefits of inline would you agree or am I off base on that?

  • Anonymous

    Interesting distinction and points made between deduplication for primary and backup. I’d have to agree that inline certainly is the most effective and beneficial a point also reiterated by Archie Hendryx’s “There’s No Duping the Reign of Data Domain” –
    http://opensource.sys-con.com/node/1346330

    The situation is that backup is one of the most redundant of workloads with up to 95% of a full weekly backup being data that’s been backed up before. I think that while the ratios for primary storage might be less, I agree that they are still significant, especially for unstructured data. We currently have databases which have up to eight copies and it’s here that reduction in primary storage can really reap some benefits.

  • http://andirog.blogspot.com/ Anil Gupta

    Not sure reconstitution rate has impact on inline argument. In the end, I would like to see an intelligent dedupe device that takes read/write expectations into consideration real-time.

    Majority of dedupe vendors are focusing on dedupe ratio/dedupe speed instead of intelligently managing dedupe and related functions based on incoming/outgoing data load while meeting read/write expectations of users/applications.

  • Pingback: Storage's 2010 Hottest Technology | The Storage Alchemist

  • emcguru

    So if I have a product that only gets 2% savings, costs 50K as long as it compresses in say 0.0001, then its far better than storewiz! (CORE=6000)

  • dvellante

    No emcguru…by my calculations the CORE in your example would be 600, not 6000– 1/3 the value of Storwize. However a 20% reduction at a speed of .0001ms would yield a CORE of 6000.

    So I would say yes…1/3 the reduction rate at an order of magnitude faster speed than Storwize would be worth more to customers…assuming you had a corpus of data large enough (in my example 100TB) and all the other assumptions I laid out.

  • wcurtispreston

    I applaud the effort to help quantify the effectiveness of the different capacity optimization approaches. I have several concerns about the model as published, and some of them suggest that calculating this is not as easy as you are trying to portray it.

    My first concern is that it doesn't take (IMHO) the most important performance factor into account: time to uncompress. As Anil Gupta pointed out, the impact on read performance is as paramount (if not more paramount) as write performance. In some models (e.g. NetApp's) data is deduped once (and at night when no one is looking) but is read over and over. Therefore, IMHO “time to uncompress” is more important than time to compress. If you factored in that number, my research shows that NetApp would have numbers equivalent to what you're showing for Storwize, as there is no significant change in performance when reading deduped data.

    And how do you figure into your calculation the fact that a solution like NetApp's only has to dedupe new blocks, where a solution like Storwize has to compress everything every time. Should time to compress be weighted so heavily if it's not happening during production hours? (Yes I know that not everyone has a Window in which to run post-process dedupe, but I do believe most do have one.) You also don't factor in that EMC's “0.5″ number only applies to older data — not the active data set, where ZFS, Storwize, NetApp (maybe some of the others).

    Your cost column is not fair at all. How can you possibly compare the net new cost of something like Storwize to something like NetApp's offering? You can't just buy Storwize; you have to buy storage with it. Storwize is made to put in front of a NetApp, Celerra, BlueArc, etc. So it'sthe cost of NetApp/Celerra/BlueArc PLUS a Storwize box OR it's a NetApp box. The way you're calculating cost severely slants this calculation to Storwize.

    BTW, the mere fact that you called it “time to compress” as opposed to “time to reduce” is one of the things that make this article read like a whitepaper paid for by Storwize. And the FUD section about Winzip seems a little slanted their way, too. Which of the solutions in the table behaves like Winzip (other than Winzip)?

    You mentioned that some stuff compresses better than it dedupes. Yes, and some stuff dedupes better than it compresses. (VM images, for example.) Compression isn't the perfect solution to every problem the way that this table makes it look.

    Finally, your numbers are so rough as to make the results of the table useless for comparison. (sorry for such a strong word, but I don't know what else to put there.) For example, does Ocarina really take 250 times longer to dedupe/compress something than Storwize? Really?

  • dvellante

    Thanks Curtis – I appreciate your input and perspectives. This type of interaction only improves the calculations.

    I am happy to factor in time to uncompress. It's clearly been a factor on peoples' minds and we could make some assumptions about read frequency – or to simplify; blend the two in some way. If it shows netapp has a higher CORE then that's fine with me– I don't care who wins.

    “And how do you figure into your calculation the fact that a solution like NetApp's only has to dedupe new blocks, where a solution like Storwize has to compress everything every time.” CORE doesn't account for that explicitly. We could make some assumptions about that as well and adjust the time to compress accordingly.

    “Should time to compress be weighted so heavily if it's not happening during production hours?” This is a fundamental question in my view. If performance is such a factor that you can only compress in off hours does that reduce roi? Maybe not if you have the batch window. I definitely struggled with this for ASIS and Ocarina. But it seems to me that if it can be done real time without the need to manage a window then it's an advantage.

    “You also don't factor in that EMC's “0.5″ number only applies to older data — not the active data set, where ZFS, Storwize, NetApp (maybe some of the others).” That's true. There is a mish mash of solutions here and that's one of the things CORE brought to my attention. I mean is it even valid to compare say a Storwize with an Ocarina solution?

    “Your cost column is not fair at all. How can you possibly compare the net new cost of something like Storwize to something like NetApp's offering? You can't just buy Storwize; you have to buy storage with it.” I disagree it is unfair. We attempted to take the cost of the compression component ONLY. With ASIS we took 20% of the cost of the array cost. Want to make it 1% – happy to make that change. 20% sounded reasonable but maybe it's high.

    “BTW, the mere fact that you called it “time to compress” as opposed to “time to reduce” is one of the things that make this article read like a whitepaper paid for by Storwize.” I chose compression b/c I've always used that term generically. Even Frank Slootman said Data Domain wanted to use the term. Time-to-reduce would be better I agree.

    “And the FUD section about Winzip seems a little slanted their way, too. Which of the solutions in the table behaves like Winzip (other than Winzip)?” I only included Winzip b/c everybody uses Winzip and can relate to it.

    “You mentioned that some stuff compresses better than it dedupes. Yes, and some stuff dedupes better than it compresses. (VM images, for example.) Compression isn't the perfect solution to every problem the way that this table makes it look.” The idea of CORE is to develop a single metric that can be applied to specific use cases that is more reflective than straight reduction ratios. Reduction ratios don't tell the full story. CORE is an attempt at developing a more accurate measurement.

    “…your numbers are so rough as to make the results of the table useless for comparison. For example, does Ocarina really take 250 times longer to dedupe/compress something than Storwize? Really?” Based on the research I did, yes – but I'm happy to be corrected.

  • dvellante

    Thanks Curtis – I appreciate your input and perspectives. This type of interaction only improves the calculations.

    I am happy to factor in time to uncompress. It's clearly been a factor on peoples' minds and we could make some assumptions about read frequency – or to simplify; blend the two in some way. If it shows netapp has a higher CORE then that's fine with me– I don't care who wins.

    “And how do you figure into your calculation the fact that a solution like NetApp's only has to dedupe new blocks, where a solution like Storwize has to compress everything every time.” CORE doesn't account for that explicitly. We could make some assumptions about that as well and adjust the time to compress accordingly.

    “Should time to compress be weighted so heavily if it's not happening during production hours?” This is a fundamental question in my view. If performance is such a factor that you can only compress in off hours does that reduce roi? Maybe not if you have the batch window. I definitely struggled with this for ASIS and Ocarina. But it seems to me that if it can be done real time without the need to manage a window then it's an advantage.

    “You also don't factor in that EMC's “0.5″ number only applies to older data — not the active data set, where ZFS, Storwize, NetApp (maybe some of the others).” That's true. There is a mish mash of solutions here and that's one of the things CORE brought to my attention. I mean is it even valid to compare say a Storwize with an Ocarina solution?

    “Your cost column is not fair at all. How can you possibly compare the net new cost of something like Storwize to something like NetApp's offering? You can't just buy Storwize; you have to buy storage with it.” I disagree it is unfair. We attempted to take the cost of the compression component ONLY. With ASIS we took 20% of the cost of the array cost. Want to make it 1% – happy to make that change. 20% sounded reasonable but maybe it's high.

    “BTW, the mere fact that you called it “time to compress” as opposed to “time to reduce” is one of the things that make this article read like a whitepaper paid for by Storwize.” I chose compression b/c I've always used that term generically. Even Frank Slootman said Data Domain wanted to use the term. Time-to-reduce would be better I agree.

    “And the FUD section about Winzip seems a little slanted their way, too. Which of the solutions in the table behaves like Winzip (other than Winzip)?” I only included Winzip b/c everybody uses Winzip and can relate to it.

    “You mentioned that some stuff compresses better than it dedupes. Yes, and some stuff dedupes better than it compresses. (VM images, for example.) Compression isn't the perfect solution to every problem the way that this table makes it look.” The idea of CORE is to develop a single metric that can be applied to specific use cases that is more reflective than straight reduction ratios. Reduction ratios don't tell the full story. CORE is an attempt at developing a more accurate measurement.

    “…your numbers are so rough as to make the results of the table useless for comparison. For example, does Ocarina really take 250 times longer to dedupe/compress something than Storwize? Really?” Based on the research I did, yes – but I'm happy to be corrected.

  • http://www.bostonboating.com skenniston

    Dave, good response, and Curtis – really? You comments go from thought leadership to slamming…

    First I will say I am a Storwize employee so take these comments here with that in mind.

    Dave ran his calculations by me as well, just like he did with the Ocarina guys. Storwize didn't “win” in every column. Optimization, people who do deduplication had higher overall optimization, (even though I did mention to Dave that we have seen that NTAP deduplication gets between 9% to 18% – remember, on primary storage there tends to be less deduplication but I digress). Here is the main point I think everyone is missing. CORE stands for Capacity Optimization Ratio EFFECTIVENESS! it is the “E” that is important and the one thing people seem to be missing.

    Let me ask, and PLEASE, put yourself in the customers shoes NOT the shoes of an analyst or thought leader for a moment.

    Company A comes in and says “I will sell you a solution that provides 50% to 90% compression (depending upon your data type) on your primary storage, that sits in front of your NAS appliance and has no impact on performance (reads or writes), and in some cases improves performance, will not require you to change ANYTHING on your application side, and 100% integrates with the Data Domain / Avamar solution you bought last year, and in fact make your dedupe faster.”

    Company B comes in and says, “I will seel you a solution that provides you 45% to 85% compression on your primary storage, and it too will not impact your operational performance as I will do compression once the data lives on the array, at night when your users have gone home and your backup jobs are done. Now this means that I will only be able to compress a little bit at a time, but over time it will all get compressed. Now, in order to read the compressed data, it will need to be re-hydrated before the application can read it, but once re-hydrated the application can access it. Oh, and by the way, when it comes time to use the Data Domain / Avamar solution, you will need to re-rehydrate the data in order for Data Domain / Avamar to have its affect.”

    Now I know the story told w/ Company B is a little over the top, but I am trying to disseminate enough to prove a point. Which technology is more 'EFFECTIVE'? Which would you choose to POC first for your data center? The answer is – IT DEPENDS. What is the cost, what is the environment, what is the use case?

    I too could argue that decompress is just as important. But if you assume that compress and decompress are the same or lets say optimize / unoptimize then you could assume similar numbers. Now, if they are dramatically different, then the vendor should be honest with their customers and tell them there is a difference. I can tell you, from a Strowize perspective, compress and decompress are the same. I don't know about the others. Curtis you say NTAP has 'minimal' performance impact. Again, put yourself in the customers shoes, what do you want, minimal impact or no impact?

    As far as cost of the technology – I think what Dave did was fair. He didn't have the cost for all of the NTAP storage, it was only a percentage. Also, with NTAP or EMC or other solutions that do post process, if you need 10TB then you need to buy 10TB and then the technology will compress after. with Storwize, if you need 10TB, because we sit in line – you only need to buy 5TB (at 50% compression) and you never need the disk space to decompress data on the disk. Again, what would you want for your shop? What solution has a higher “EFFECTIVENESS”?

    Also, integration with deduplicaiton. Over the past 4 years the industry has spent over $1B on deduplication for backup. If the primary storage guys, got to the backup guys and say, “Oh, all that money you spent last year is wasted because now we are using a different technology for deduplication on primary storage and there is no synergy with your stuff so you have to remove it and go back to disk based backup.” What do you think would happen?

    The point from my perspective – weather or not the spread from 1800 vs 250 is 100% accurate, is if Storwize was in a Gartner magic quadrant as the highest, up and to the right bullet, would we be having the same conversation? The issue is measure of effectiveness. In order to be effective in the enterprise you need to be:
    1) Fast
    2) Random access – to integration with deduplicaction
    3) No change to up front applications (remapping etc…)

    Curtis, I know we owe you a briefing so I will get that set up. The goal is not sway you but to make sure you understand that technically that what Storwize does is what is advertised.

    I would be happy to hear some comments on what you all believe the customer wants…

  • dvellante

    Curtis…I really respect your knowledge and appreciate you weighing in. Having gone back and re-read my post I think I addressed pretty transparently some of your concerns in the section on caveats and don't think some of your comments are warranted.

    I think the concept of CORE still applies. You can't take performance out of the equation – even if there are workarounds. All things being equal you'd always take a higher performance solution. What you don't want to do is compare use cases that require high performance with those that don't…which is clearly what I've done here by lumping all these solutions together.

    What I also learned from speaking with Carter George recently is that Ocarina is very hard to generalize. They have very fast solutions that solve certain problems and others that aren't as fast that solve different problems. Different compressors will give different compression ratios and operate at different speeds.

    But the bottom line remains the same to me. Value is a function of the amount of benefit divided by the cost of the benefit. To include time in the cost of benefit is not flawed, it's fundamental.

  • eemcguru

    at 20% savings running at 0.0001 ms, or 2% savings running at 0.00001 ms, I hardly find that to be a valuable proposition. The savings need to justify the overhead of another product. Just my 2 cents.

  • Anonymous

    Thanks again for weighing in and sharing your expertise Curtis. I hadn’t thought about the Storwize v ASIS angle but it’s pretty obvious isn’t it? ASIS is the gold standard from a business model standpoint.

    Your beef(s) are progress as far as I’m concerned. It’s exactly the type of feedback We’ve been looking for. CORE was published in Feb http://wikibon.org/wiki/v/Measuring_the_Effectiveness_of_Capacity_Optimization_Technologies

    But as is often the case with these things the hardcore feedback starts when you apply it.

    Before we toss the concept completely let’s keep an open mind for a minute. I’m happy to re-run this entire analysis. Floyer’s grounded due to volcanoes so I can use the help.

    Let’s look at your first point of optimize v un-optimize performance (I will change the wording btw). Let’s assume a read:write ratio of 80:20 and let’s assume ASIS reads at the same rate as Storwize. This reduces the ASIS time to dedupe by an order of magnitude but its CORE is still 78X (versus 720X) lower than Storwize. What does that mean? You said it best – if you can’t tolerate a significant slowdown in performance don’t buy ASIS.

    Ok…let’s look at your other beef– cost. BTW, the cost of 100TB of NetApp storage in the model is $150,000. I used $30K for the cost of ASIS. Too high? Let’s use the model to answer the following question:

    “At what cost for ASIS does its CORE surpass that of Storwize?” Answer: Assuming the 80:20 read:write ratio…$39.

    What does that mean? It means if you have a window at night during which the cost of ASIS optimization technology is less negligible because you don’t need the MIPS then ASIS is ‘better’ than Storwize.

    What is the cost of ASIS? I don’t know either but in mind mind it’s not free. Any more than Exchange 2010 DAGs are free. There’s always a cost to bundled function. HOWEVER if as a customer you believe it’s free then the ASIS core is literally infinitely better than any solution – including Storwize.

    Personally I think $39 is too low to assume but I can buy based on your input that 20% of $150,000 is too high.

    The great part (to me) about this discussion is we haven’t even talked about optimization rates. That was the original point of this post. They matter but not as much as vendor marketing suggests. Everyone is trying to run a play out of the Data Domain playbook “It’s the dedupe rate stupid” but CORE demonstrates the fallacy of that marketing.

    It’s what I like about models. Don’t like the assumptions…let’s change them.

    We could (and should) run similar examples with Ocarina in cases where optimization rates do matter more. I can imagine situations where an Ocarina compressor gets a decent reduction rate and Storwize won’t do anything. In such an example, despite the speed of Storwize Ocarina’s core would be higher.

    As I think you’ve underscored – it’s all about the application.

    More work to do for sure but I’m not convinced there’s a fundamental flaw in the calculation.

  • wcurtispreston

    @Dvellante

    I say again that I applaud what you are trying to do. I do think it may ultimately be too complex to quantify numerically, and I disagree strongly with some of the assumptions on which CORE is based.

    Let's face it. This comes down to Storwize vs NetApp. Let me state for the record that I'm a fan of both of them. I speak very highly of both approaches during the primary storage capacity optimization section of Dedupe School. They both get credit for being first movers and for having hundreds (thousands?) of customers actually using their products. I hear good things from both customers, too.

    So what's my beef? Your formula says Storwize is 720 times better than NetApp ASIS (1800 vs 2.5). And that is simply not true. Steve might argue (and does) that their approach is better, and we can discuss the merits of that argument. But any formula that results in one being 720 times better than the other has to be fundamentally flawed.

    You I and both agree that cost must be a part of the equation. What we don't agree on is what the cost of the ASIS solution is. Is it $1? I don't know. I am not aware of any increase in NetApp pricing when ASIS was added to the base OS. But what I know is with NetApp I have to buy 100 TB of storage and with Storwize I have to buy 100 TB of storage plus the cost of the Storwize box. To me, that means that one costs more than another and that is not reflected in your chart.

    Let me address some of your replies to my comments.
    “We could make some assumptions about that as well and adjust the time to compress accordingly. “
    The typical size of a NetApp snapshot (which would reflect the number of blocks that would need to be deduped that night) is about 1-5% of the size of the volume (1% for unstructured, 2-5% for structured).

    “If performance is such a factor that you can only compress in off hours does that reduce roi?”

    No it doesn't. It determines whether the solution is appropriate for you at all. If you have a 24×7 workload that cannot tolerate a significant slow down in performance some time during then night — DON'T BUY ASIS. Any good NetApp SE would say the same thing. If you have a workload that can tolerate some slowtime at night, then it doesn't affect your ROI at all.

    Storwize has different challenges that limit its use in certain situations. Put their box in front of datasets that don't compress very well (a lot of jpgs or tiffs). You don't get squat. Does that reduce your ROI? No, you just don't buy it for that application. It's ROI for the bulk of data is easy.

    Data Domain may have wanted to use the term compression but thankfully they didn't. There's a whole compression community that objects to using the term when applied to dedupe. (I found this out when I posted some blog posts on the topic.) Compression is compression and dedupe is dedupe. They are very different (although related) and you should not use one to refer to both. I choose to use a different term like “optimize” or “reduce” as they don't have existing meanings.

    I apologize if you think it was a low blow to say that it reads like a Storwize paper. I'm not accusing you of anything. I'm saying that the formula and the conclusions that you draw from it are so biased in their favor that it reads like it was written by Storwize. Any completely impartial 3rd party that read an article where one vendor is said to be 720 times better than its closest competitor, and more than 1000 times better than others would assume the article was written or sponsored by that vendor. Wouldn't you?

  • http://www.bostonboating.com skenniston

    Curtis,

    As I read it, I didn't come to the conclusion that Storwize was 720 times more effective than NetApp, rather it was, on a large scale, incrementally better. The values, at least when I evaluated CORE on the Wikibon site, lead me to believe this was more logarithmic.

    I am wondering if you were going to comment on my comments below. I am simply asking, if 'effectiveness' is a good measure of what customers would want to know about, and the functions that – real-time, that compresses before it hits the disk, and not require them to purchase the disk capacity for decompressed disk before you can compress the data – which is more effective? By how much?

    Again, and I agree with you, I applaud Dave for putting numbers to this versus Gartner that purposely avoids 'numbers' in their 'Magic Quadrant' so they can be surreptitious with vendors. When a vendor complains to their placement on the MQ, Gartner can say the placement is relative, I know, I was a vendor once that spoke to them. Assigning numbers is more a more concrete way, which provide real metrics on how they were achieved.

    I say to Dave, get this in the Wiki, lets debate what the real numbers for CORE should be both from a formula perspective and from a product perspective and revise. That is the great thing about the Wiki. I am not sure why this didn't happen with the content on CORE that is already there but it now looks like there are people who will add some more foundation to CORE – it can only make it better.

  • wcurtispreston

    @skenniston

    Where did I slam anything? I said nothing negative about your product and the strongest statement I made about the article and CORE itself was the values were so rough they were useless for comparison, and then I apologize for the use of that word, but I didn't know what else to use.

    You're telling ME to think of myself as a customer? Seriously, dude. You need to spend more time with me. That's all I do. I care more about the customer than anything else. And it was my concern for the potential customer/reader of this article that caused me to reply. I want ppl to compare your solutions, but I don't want them thinking that one is 720 times better than the other, because they're not.

    You also mention that CORE was run by Ocarina and Storwize. That may be the case, but it doesn't look like it was run by NetApp, and they are the vendor who I do not think is represented well here, which is why I'm coming to their defense.

    It wasn't 1800 vs 250. It as 1800 vs 2.5 or 1800 vs 1.5. As to your magic quadrant analogy, I'll run with it. When have you ever seen a Gartner Magic Quadrant where one vendor was in the far upper, right-hand corner and ALL other vendors were in the far, lower right-hand corner? NEVER. And that's what this chart essentially does to your product vs the others.

    It's been mentioned that CORE's been up for a while. I just saw it for the first time, so now I'm commenting. I'm not sure what else you would have me do.

    Now… On to your Vendor A and B comparison. You're right when you say “the story told w/ Company B is a little over the top.” It also contained at least one major inaccuracy about how ASIS works. There is no “rehydration” like typical dedupe solutions; it is an integrated feature of WAFL. I said it in my response to Dave, and I'll say it again. I've talked to dozens of very happy NetApp customers that are deduping very active data, like VMware images. (I talked to yet another one today.) They tell me there is no re-hydration penalty that they can see. Having said that, let's look at your comparison again.

    “Company A comes in and says “I will sell you a solution… that sits in front of your NAS appliance” (which means it is an EXTRA COST in addition to the NAS you already bought)

    “will not require you to change ANYTHING on your application side”

    Neither does Company B to my knowledge.

    “100% integrates with the Data Domain / Avamar solution you bought last year, and in fact make your dedupe faster.”

    This one I just don't get. I just don't see how the incremental benefit you would get by putting a Storwize box in front of a Data Domain box could ever justify it's cost. AND the Data Domain model only applies if you're not using OST, so your statement doesn't seem to apply. And I REALLY don't see understand it would work in conjunction with an Avamar box. They're not NAS.

    “Company B comes in and says, “I will seel [sic] you a solution that provides you 45% to 85% compression on your primary storage, and it too will not impact your operational performance as I will do the reduction once the data lives on the array, at night when your users have gone home and your backup jobs are done.”

    Of course their solution comes WITH the filer and your solution comes as an incremental cost to the filer, but sure. I'm with you so far.

    “Now this means that I will only be able to reduce a little bit at a time, but over time it will all get compressed. “

    This is EXACTLY the same as you. They will dedupe all new data (as you will compress all new data), but existing data takes some time to compress/dedupe. Just like your box will take some time to compress data that was written before you box was there.

    “Now, in order to read the reduced data, it will need to be re-hydrated before the application can read it, but once re-hydrated the application can access it.”

    No. It doesn't. See above.

    “Oh, and by the way, when it comes time to use the Data Domain / Avamar solution, you will need to re-rehydrate the data in order for Data Domain / Avamar to have its affect.”

    No, you don't. This is why I hate it when vendors tell me how they're better than the other guy. You obviously don't know their solution. You CAN, but do not have to rehydrate NetApp ASIS to back it up with Data Domain/Avamar, etc. You only need to use Snapmirror to Tape (which is free, BTW).

    –

    Look, my friend. I'm not trying to slam Storwize here. I'm saying that it's not easy as you and Dave are trying to make it to say that your solution is astronomically better than theirs. Their solution is good IF YOU HAVE A NETAPP or are willing to buy one. Your solution is good if you don't have a NetApp. But can you really tell me that your solution is so good that someone should buy a NetApp AND your box? I don't think so.

    BTW, I know your box quite well, I think. I've followed it since day one and I like it. I just don't think it's 720 times better than ASIS. Is that so unbelievable?

  • wcurtispreston

    It's a “score.” You got a score of 1800 and they got a score of 2.5. I mean, come on, dude! Anyone would look at that and believe that Storwize is 720 times more “effective” than NetApp. I don't see how you can say that something that got a score of 1800 is “incrementally better” than something that got 2.5.

    I did comment on your comments. Not sure why they didn't post. I reposted my comments.

    In a green-field situation you are right that NetApp (and other solutions) would require extra disk for that initial right — if migration time was paramount. I've never done this, mind you, but you could migrate 1 TB, dedupe it, migrate the next TB, dedupe it, etc. While I'm sure you look at that and say “you wouldn't need to do all that nonsense with our box,” and you'd be right. But remember that migration only happens once. It's the steady state to which I'm speaking. And it's the steady state where THEY have an advantage. You have to compress and uncompress every byte that is read/written every day. The only have to dedupe new blocks that are stored today, which is usually 1-2%, sometimes as much as 5% in application workloads. So while they have to dedupe at night and their process takes longer (not line speed like yours), they don't have to dedupe as much. Then when they're done there's (almost) no performance hit. People that have benchmarked it says that it's <5%, which most people are not going to notice.

    “Effectiveness” may be a valid metric. But I honestly don't know if you can ever get a formula that deals with enough of the situations to be of significant value. How do you account for different data types where each approach has limitations? How do you account for the fact that they have a major advantage if we're talking NetApp equipment and the reverse is true if we're talking non-NetApp equipment? How do you account for the fact that some variables are more important to some people than others? There are many little “knobs” like that I think are far too complex for a formula.

  • http://onlinestorageoptimization.com/ CARTER GEORGE

    Is the CORE Formula Flawed?

    I’m going to argue here that the CORE formula is flawed, but I want to say right off that the CORE formula has already added value to the community.

    It’s a great idea to try to measure the value and effectiveness of different data reduction solutions, and by putting a first shot at it out there, there’s already a vigorous discussion going on here that is going to increase awareness of the issues involved, and will probably lead to a new and improved version of the formula. I think we want to recognize the value that Wikibon has brought to both customers and vendors by starting this whole brouhaha.
    That said, yes, the CORE formula is deeply flawed.

    Read more here: http://tinyurl.com/34oy97g

  • Pingback: Online Storage Optimization » Blog Archive » Words of Wisdom: Let’s get Organized

  • Storagezilla

    >You also don't factor in that EMC's “0.5″ number only applies to older data — not the active data set

    Which shouldn't be the case anyway since the release of DART done in December supports compression of multi-TB sized active files and active VMDKs with integration into vCenter via a vCenter plugin.

  • wcurtispreston

    This is what happens when EMC doesn't brief me. I can only talk to what I know, and I can only learn so much by osmosis.

  • Pingback: Online Storage Optimization » Blog Archive » If You Do Dedupe on Primary Storage, Do You Have to Expand It to do Backups?

  • Pingback: Why does CORE fail? Part 1 | Security in an Unsecure World

  • Pingback: Gadget Newz

  • http://www.bostonboating.com skenniston

    Just one point of clarification – and I did speak w/ Curtis about this. Storwize would never sit in front of a Data Domain solution. The context is, and why I feel Storwize has a great deal of 'value' is because when you compression your primary storage (with Storwize in front of primary storage) then when you do perform your backups, you see the benefits in your deduplicated backups – and it is 100% transparent.

  • dvellante

    Thanks for commenting Carter. Let's keep the discussion going and see if we can't achieve some mutual objectives in the community.

  • http://onlinestorageoptimization.com/ CARTER GEORGE

    Dave, yes I agree, let's continue this conversation, its always healthy to hear new views and opinions.

  • Pingback: A Blog with no Comments? | The Storage Alchemist

  • Pingback: A Blueprint for Primary Storage Optimization | The Storage Alchemist

  • Pingback: From Tel-Aviv to Boston « Wikibon Blog

  • Pingback: IBM Squeezes Storwize into its Portfolio « Wikibon Blog