8 Reasons To Not Jump At XIV

Become a Member!

Why Register?

Login

Featured Research

Announcements

Technology Events

Home Profile Peers Wiki Activity Groups Feedback

8 Reasons to Not Jump at XIV

Currently 1.7/5 Stars.
1
2
3
4
5

rate this

Last Update: Aug 14, 2010 | 07:40

Viewed 44724 times | Community Rating: 1.7

Originating Author: RootWyrm

Please visit the Wikibon XIV Category to see other research on XIV.

Also...please see David Floyer's recent analysis of XIV which summarizes the Wikibon community's collective opinion.

This page has been flagged for Deletion

Community members have requested this page be deleted. Please review this article and its comments and provide your opinion in the comments section. You may also visit the 'discussion' page in the Vault (see tabs above).

This page has been flagged for discussion

Community members have requested a review of this article for accuracy and context. WARNING: Users should be mindful that the conclusions in this article have not been vetted by the Wikibon community. Please help us by reviewing this article and providing your suggested changes, if any, in the comment section or editing the article directly. You may also visit the 'discussion' page in the Vault (see tabs above).

To edit, please log in and hit the "edit" tab at the top of the page, make your changes in the dialog box, and click the "Submit" button when you are finished.

To see changes to this article click the Vault tab and view the history.

Note: The reliability and risk exposures expressed in this article are based on opinion and analysis-- They have not been validated by Wikibon members based on field results or experiences.

IBM has been announcing enhancements and updates to its XIV storage platform at a fairly steady rate, and the visibility of XIV has increased dramatically since it was quietly launched. IBM announced the latest update to this offering on November 10th, and David Vellante wrote an excellent blog post about why you should pay attention to XIV. But I’ve got eight very good reasons why you should think twice about buying and deploying XIV in your business.

XIV is immature. It’s a completely new design with a lot of new concepts. Think of it as the 1.0. While the ideas may be good, it is an extremely young product with very little real world testing compared to traditional arrays. IBM brought significant testing and development resources with the purchase, but XIV has continued to maintain its own sales and technical staffing separate from IBM’s Systems Storage group.
While XIV has achieved over 1,000 revenue ships, compared to products like the EMC CLARiiON, IBM DS4000 or the HDS USP families. this is barely even a drop in the bucket. That also means that it has seen far less testing in a much smaller variety of environments. IBM also hasn’t disclosed how or why customers are ordering multiple XIVs, or what exact tasks they are using XIV for. One of their example customers, Gerber Scientific, states that they use XIV for their entire production environment of six mid-range POWER5 systems for improved IOPS performance. But what they don’t say is more significant – what exactly XIV replaced, or how many XIVs they purchased. Other reference customers are new installs or expansion installs, leaving us without a way to compare to other solutions such as the HDS AMS2500 or USP-V, or EMC CLARiiON or DMX.
XIV is an extremely opaque storage platform. Administrators have no way to control or even view what disks and nodes are mapped to a LUN and must trust the XIV to do the right thing. While this method may work for some environments, and does work for other solutions like Compellent and 3par, this is a drastic change for existing IBM customers and storage admins who are familiar with IBM’s other offerings. It also eliminates the ability of storage administrators to manage workloads based on application or business priority. Competing solutions offer customers some form or means to prioritize based on LUN or host.
XIV still offers customers no scalability. 79TB is still the limit, period. You can now start lower, and build up to that ceiling, but not beyond. This makes XIV the lowest capacity controller in its market and price range. Expansion past 79TB means purchasing additional XIVs, which are separately managed, requiring administrators to manually balance loads across multiple independently managed and monitored XIVs.
XIV is extremely inefficient. 79TB of usable capacity requires 180 1TB 7200RPM SATA drives. Using traditional RAID6, the same number of disks offers about 115TB of usable capacity, and RAID5 can achieve 140TB, both with allowances for hot spare drives. The claims of 100% “effective” capacity are nice, except that they ignore other hardware and software methods to improve the effective utilization of traditional arrays, such as thin provisioning and similar differential snapshotting. XIV also lacks the capability to do soft or hard tiering, which means that an Oracle database runs on the same physical disks and has the same access priority as a Windows shared network drive. Compellent, 3par and Pillar all offer the capability to do soft or hard tiering and to configure the number of spindles involved in a LUN.
The XIV’s power and thermal efficiency is similarly deficient – 29K BTU/hr and 8.5kVA for 79TB make it the highest heat and power consumption of any SATA array of similar usable capacity. For comparison, 180 drives of any type in a DS5300 with EXP5000 enclosures is about 21K BTU/hr and 7.2kVA peak. That translates to peak power consumption of 40 watts per raw TB, besting the XIV’s claim of 43 watts per raw TB. When this is changed to usable TB, the XIV consumes 97 watts per usable TB, while the DS5300 using RAID5 with hot spares offers 140TB at 51W per usable TB.
RAS and data protection as compared to traditional arrays needs a great deal of improvement. It isn’t only about the “chance” of failures, especially as drives near their MTBF, and tend to fail in groups. Customers should not only be evaluating the chances of a failure, but also how any given solution handles failures when they occur. In this area, XIV fails every test put in front of it. Just how dangerous can a failure on XIV be? IBM XIV Could be Hazardous to your Career – Gestalt IT. Ultimately, the root of this problem is the inability of the user to add resiliency to the XIV by any way other than buying a second system and creating a mirror. XIV offers only one method of protecting data and offers users no way to control sparing other than to further under-utilize capacity.
The price to capacity and performance ratio of the XIV typically isn’t competitive with traditional arrays, especially when these are coupled with virtualization engines. Many customers will find that they can achieve equivalent performance and capacity with a combination of traditional arrays and products like the IBM SVC, or by expanding the Hitachi USP-V with AMS2000-series. Customers also frequently overlook competing flat-tier systems from Compellent, 3par and Xiotech. Most people have never even heard of these companies, yet they offer products which are substantially similar to XIV with equivalent or better features and performance.

It also stands out as the only storage system from IBM which has yet to be tested under any independently verifiable or auditable benchmarks; even the low-end DS3400 has been benchmarked under SPC, but XIV’s performance claims of “60,000+ IOPS” remain unproven by any independently verifiable means. Meanwhile the DS5300 has SPC-1 proven results of 58,159 IOPS from high performance FC disks in a mirror configuration, and 3PAR can boast SPC-1 proven results of 93,050 IOPS from the F400 Storage Server.

Product Maturity

To be fair, most of the points concerning data integrity and operational safety come from the first point I’ve raised – that XIV is an extremely immature product. Like any new technology, it will have teething problems. This is also why the relatively low revenue ships to date are a significant reason to think twice. While in one way these demonstrate customer confidence, they also expose of a lack of confidence. XIV has been available for more than two years, and total sales are only just over 1,000. That results in collateral damage to the cost-benefit potential of XIV. Your likelihood of finding a new problem is increased by the relatively low exposure, and a single failure can erase the cost benefits in an instant. While IBM’s service and support is generally well respected for a reason, any new product has a higher likelihood of needing engineering involvement, which may result in hours or even days of downtime for your business. I also recommend that users steer well clear of EMC’s Connectrix and the VCE VBlock system in their production environments as well. I have personally been “Customer Zero” on bugs that were missed in testing on other products three times, and it is not a place any business wants to be.

Criticisms of the Fundamental Architecture

However, XIV’s core failings existed before IBM purchased it. Its method for storing data and its fundamental design makes it incredibly inefficient and expensive to operate. Not only that, but it remains an unrealistic option for anything but limited application use because of its severely restricted capacity and performance. It remains technologically unfeasible to span XIV frames into single units; doubly so as the remote mirroring functionalities transport data across Fiber Channel or Gigabit Ethernet connections. Imagine attempting the same transport for all volumes across two frames – you would have no bandwidth for hosts. There are better options for interconnect such as dedicated Fiber Channel ports, 10 Gigabit Ethernet or InfiniBand, but they have yet to be implemented in XIV. Even the internal disk connectivity is Gigabit Ethernet.

This limitation extends into management; you cannot manage multiple XIV frames as though they were a group, because the XIVs do not form a logical group. Each XIV must be individually monitored and managed for both capacity and performance. For large capacity environments or anybody who has more than 79TB or needs more than one frame for redundancy or reliability reasons – cross-mirroring is a good example there – XIV is a management nightmare. Every ease-of-management feature is negated by requiring you to manually balance your disk loading between frames. This can be eliminated by putting an IBM SVC in front of the XIVs, but that adds substantial costs and eliminates many of the XIV’s software features including snapshots and time-delayed asynchronous mirroring, which are major selling points.

Sales Motion and Competitive Approaches

My introduction to XIV was in early 2009, where IBM presented it as a better technical solution than another IBM solution we were considering, and as extremely cost competitive. But there were many questions about XIV that IBM's technical representatives were unable to answer, were visibly uncomfortable trying to answer, or attempted to discount as invalid or irrelevant. This is not something anyone wants to see in a technical sales presentation. Performance was only ever given in vague terms, other than “bandwidth.” The presenters also avoided power, heat, and physical size concerns and questions – I had to go to IBM’s website for that information.

If I ask EMC what happens in a V-Max system when data blocks become corrupted, the answer should not begin with “that will never happen to you.” If I ask HDS what happens if the link between two USP-Vs fails completely, “we didn’t need to test that” is not what I want to hear. If I ask 3Par what happens in a multi-controller failure, the response should not be “that’s a silly question.” Those are unacceptable answers from any vendor for any enterprise storage. XIV has so far focused exclusively on marketing efficiency rather than functional efficiency in capacity and power and on marketing faults as “almost never” or “impossible” rather than ensuring XIV can better handle dangerous faults.

From a performance, power, and efficiency standpoint there is nowhere that the XIV does not fall short when put up against even other IBM offerings or competitors. In over two years of existence, no independent person or organization has published a single XIV benchmark. An IBM presentation (PDF; DEAD LINK-- SHOULD BE UPDATED OR REMOVED) from March of 2009 claims (on page 9) more than 160,000 IOPS from cache and approximately 30,000 IOPS from port, which is not the same as disk. IOPS from disk are conspicuously absent.

The next slide (page 10) shows the real story, random I/O workload. This is what typical Tier 1 is, and where IBM has decided to start positioning the XIV rather than the previously stated “Web2.0” workloads. Databases are highly random access, be it Oracle, DB2, SAP, Exchange mail stores, and so on. Long sequential throughput is a rarity. Random load in a fully configured XIV peaks at a bit over 16,000 IOPS. That would place XIV’s random workload performance closer to the IBM DS3400 than the IBM DS5300.

The pain continues from there on page 12 as response times for OLTP exceed 30ms before reaching even 15,000 IOPS with a half-configured XIV. Under SPC-1 testing, the IBM DS5300 never exceeded 12.5ms response times even at 100% controller load, making it’s worst case response more than 4 times better than XIV’s worst case of over 50ms. Even a fully configured XIV quickly exceeds the DS5300’s worst case response time at only about half the IOPS.

Conclusions and Opinions

Because of these points, my opinion is that it is likely that this generation of the XIV storage platform is an evolutionary dead end. It has reached a point where further scalability requires a new generation of hardware to realize greater capacity, ease of management, and performance. The current generation of hardware nodes does not have the physical space for a dedicated interconnect method between frames, except at the cost of host connectivity. Performance from large SATA drives is still painfully lacking in random workloads, and high utilization of the XIV has a dramatic impact on response times.

A new generation of hardware to resolve these issues will undoubtedly require a new generation of software. The current generation of software is deficient in resiliency and fault tolerance to an extent where a major rewrite is nearly a necessity. So while the XIV concept continues to have a great deal of potential, the current execution is still not ready for production Tier 1 use in most environments, and likely never will be.

This leaves the question of where XIV does fit. There is certainly potential for XIV as Tier 1 in environments that fit the capacity and performance profile, but to achieve true Tier 1 reliability requires purchasing a minimum of two XIVs and utilizing the mirroring functions, which only compounds the inefficiencies. Achieving Tier 1 performance for typical Tier 1 applications is simply impossible. While you may be able to get sufficient IOPS, you can only do so with massive response time penalties.

In my opinion, a single XIV only fits where you can afford downtime or data loss that can be restored from tape or are performing almost exclusively sequential reads and writes. Examples of this would be as a disk target for disk-based backups, Tier 3 and lower priority hierarchical storage as a last stage before tape, or situations where a storage outage does not affect operations. As a specific example, XIV fits many IBM Tivoli Storage Manager environments well with its mix of performance and capacity as a potential lower cost alternative to traditional arrays. Because of the way TSM operates, a disk failure doesn’t need to cause an actual outage, and the sequential nature of reads and writes are better suited to XIV. The caveat being that the typical 256KB block size for backup applications or VTL is a world apart from the tested 4KB blocks. Where a disk outage can or will result in an interruption, plan on buying two XIVs or stick to more proven technologies.

Action Item: XIV is completely different from conventional LUN-based arrays. Its technique of spreading data across drives in the array will provide good enough performance for a wide variety of workloads yet it may not be a fit for others. Users should take the time to understand the technology as as applied to their specific environments.

Footnotes: Power and thermal data for the IBM DS5300 taken from http://www-03.ibm.com/systems/storage/disk/ds5000/specifications.html

The comments elicited from a Wikibon blog entitled 8 reasons to pay attention to XIV and from this article have been included in a subsequent article The IBM XIV Storage Array Performance and Availability Envelope

Comments on '8 Reasons to Not Jump at XIV'

RootWyrm - thanks for the post. You make some good points for users. XIV has its place however and while it's new and immature it does have some happy users and will evolve.

Here's a case study David Floyer wrote last year on the topic fyi:
http://wikibon.org/wiki/v/Leumi_Bank_road_tests_XIV_storage_arrays

Posted By:David Vellante| Mon Nov 30, 2009 10:42
It's not clear to me the purpose of the article; if it's to be a balance to the "8 Reasons to Jump..." then perhaps the two together make for a balanced review?

There are a lot of assertions and assumptions about the architecture and the implementation and longevity and without knowing the context(s) and specific internals knowledge of the author(s) of the comments it's not clear whether or not they are valid. (Somewhat of the issue of Wikipedia vs. Citizendium.) It seems that the author(s) should be named to know.

Just my opinion....

Karl.

Posted By:Karl D. Schubert, Ph.D.| Mon Nov 30, 2009 07:51
Thanks for the comments Karl. I think we have to start having at the issues that require discussion. I will start. The note is long and maybe has to be broken into two notes.

The piece seems to be written by a practitioner that has good knowledge of storage generally...versus a competing vendor with an axe to grind. But I can't tell that will 100% certainty.

Within the main section... #3 "XIV is an extremely opaque storage platform." To me this is a) absolutely true and b) an advantage to many users. For example, every Compellent user I've spoken with says "I don't want to manage storage...the simpler the better." While it won't work for all apps, this approach has a place...and a growing place in storage.

Posted By:David Vellante| Mon Nov 30, 2009 08:01
The link to the performance paper no longer works; IBM appear to have removed it?

Posted By:Alex| Wed Dec 02, 2009 12:30
Hi all.

I'll see if I can find a cached copy of the IBM presentation. It was from a technical conference earlier this year, but I can't recall which conference. It was publicly presented though.

Also, I honestly wouldn't say I have an axe to grind with IBM as much as with the product and sales method. I'm actually very much an IBM guy (ask me about SVC, or DS family.) As I said, XIV has potential, but not as it is and not the way IBM has been trying to sell it. Being mindful of NDAs, what I can say is that they were trying to sell XIV over a DS8k for an environment that averaged over 60,000 random IOPS at around 40/60 read/write.

The point of the opacity that I probably should have clarified better is that really, XIV's opaqueness is unique in several ways. Obtaining performance data from it requires extensive scripting acrobatics - I was specifically warned TotalStorage Productivity Center could not collect performance data. A quick review of the documentation available on IBM's website indicates it's still limited to configuration data only. Statistics at the XIV are collected averages, and are only for cache. (XIV XCLI reference guide, ch.18 pp.309.)
Perhaps another point I should have been clearer about is what makes Compellent and other solutions different. Others offer more than one type of disk, and actively manage performance. XIV does neither of these things; one disk, and no active management of performance. If your Oracle LUNs are being hit harder, other solutions will let you or will automatically give it higher priority. On XIV, a long intensive read from a Windows share will disrupt the Oracle LUNs. Perhaps opaque is the wrong term for this, but I'm not sure of a better one personally.

I have no doubt had we taken IBM's word as gospel and selected XIV, we would have been very badly burned. Frankly, I would have given my right arm to have one for our TSM storage pools, because it does work for sequential loads. It works phenomenally. But as I said, IBM was trying to bill it as better than DS8k for random workloads - and I'm not the only one with that particular experience. And I have no desire to see other IBM customers burned, when XIV just isn't what they try to sell it as.

Karl, I honestly agonized over whether or not to publish under my name rather than my alias. The problem is that a lot of these points were born out of discussion with others at the organization at the time. While I am only speaking for myself, and as myself, it could damage business relationships even though I no longer work there. There was also some uncertainty regarding ongoing contractual obligations, so ultimately, I had to err on the side of caution.

Thanks for the valuable feedback, everyone.

Posted By:RootWyrm| Wed Dec 02, 2009 05:37
Most of the outstanding feedback is related to two factors: 1) Not every vendor submits SPC benchmarks...notably EMC and 2) XIV is positioned differently than other products in IBM's portfolio.

Both true.

Does anyone have anything related to the technical accuracy of this piece? If not we should remove the flag.

Posted By:David Vellante| Thu Dec 03, 2009 10:29
OK...well...I have a lot of concerns about this article.

This is going to come across as harsh, but if the article is intended to be factual and is aimed at people trying to make a decision -- rather than just being an opinion blog -- then these are important issues in my mind.

And, several of them *are* dependent on the author making themselves and their background known for us to put into context whether he/she has the qualifications to back up the statements and assertions they are making.

I'll start at the top.

1. …but XIV has continued to maintain its own sales and technical staffing separate from IBM’s Systems Storage group.

[kds] This is not uncommon in an acquired company with a high degree of complexity like storage systems. Consider all the acquisitions done by vendors such as EMC (CLARiiON, to name a well-known one). Whether or not they'll merge remains to be seen. BTW, IBM's Storage Systems Group is not the only group within IBM that sells or develops storage.

\\\\\\\\\\\\\\\\

3. …this is a drastic change for existing IBM customers and storage admins who are familiar with IBM’s other offerings.

[kds] Well, this does not a priori make this a bad thing. Many of the current storage systems mask to varying degrees where the data is actually placed on the disks. And, truth be known, if a disk assigns an alternate track a sector can be way far away from where you think it should be.

\\\\\\\\\\\\\\\\\\

4. XIV still offers customers no scalability. 79TB is still the limit, period. You can now start lower, and build up to that ceiling, but not beyond.

[kds] This is likely to be old shortly if it's not already; in non-NDA briefings they're saying that with the current hardware they're increasing capacity.

\\\\\\\\\\\\\\\\\\\\\\\\\\

4. …which are separately managed, requiring administrators to manually balance loads across multiple independently managed and monitored XIVs.

[kds] regrettably, this is generally the norm, anyways. Most people have not bought the argument that "the system" can manage where things go and move automatically. (See the author's earlier complaint about not knowing where everything is exactly. This is an inconsistency in the argument.)

\\\\\\\\\\\\\\\\\\\\\\\\\\\

5. …XIV is extremely inefficient. 79TB of usable capacity requires 180 1TB 7200RPM SATA drives.

[kds] This is more or less "mirroring" (loose replication mirroring) with "bay redundancy" that is about 2x actual space. So, same efficiency more or less as anyone running RAID 1 (or its variants).

\\\\\\\\\\\\\\\\\\\\

5. …Oracle database runs on the same physical disks and has the same access priority as a Windows shared network drive.

[kds] Actually, this can be managed I believe by how you set volumes up if you really insist on doing so. There is a big debate on many systems about whether fully random placement to start with and then adjustment as workloads change should be done. However, most applications are cyclic in nature so it's not clear how much work you'll have to do to keep balancing them if the storage admin insists on minute-by-minute watching the dials and turning the knobs.

\\\\\\\\\\\\\\\\\\\\\\\

7. …In this area, XIV fails every test put in front of it.

[kds] Argumentative without any real facts backing it up.

\\\\\\\\\\\\\\\\\\\

7. …Just how dangerous can a failure on XIV be? IBM XIV Could be Hazardous to your Career – Gestalt IT.

[kds] If you haven't gone and read this you should; it's a great one-liner with no facts backing it up in the blog. Period.

\\\\\\\\\\\\\\\\\\\\\\\\\\

8. …The price to capacity and performance ratio of the XIV typically isn’t competitive with traditional arrays, especially when these are coupled with virtualization engines.

[kds] OK…if this is so, there should be a table that shows the $/GB usable or some such to show that this is the case. Otherwise, this is assertion without fact to back it up.

\\\\\\\\\\\\\\\\\\\\\

It also stands out as the only storage system from IBM which has yet to be tested under any independently verifiable or auditable benchmarks…

[kds] There's a great difference between "tested" and "published." It's VERY unlikely that IBM hasn't run the tests. Many companies (including ones I've worked for) don't publish the data because they know how easy it is to create something that does well in the benchmarks but doesn't do well in real workloads. It's a twin-edged sword. And, they're only as good as the moment they're run.

\\\\\\\\\\\\\\\\\\\\\\\\\

XIV has been available for more than two years, and total sales are only just over 1,000.

[kds] For an enterprise-class storage subsystem, 1000 units in two years from GA is pretty darn good -- I know as I've worked for those companies!

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

…it remains an unrealistic option for anything but limited application use because of its severely restricted capacity and performance.

[kds] OK…verified background of the author is important to know whether or not this opinion has gravitas.

\\\\\\\\\\\\\\\\\\\\\\\\\\

…there were many questions about XIV that IBM's technical representatives were unable to answer, were visibly uncomfortable trying to answer, or attempted to discount as invalid or irrelevant.

[kds] This is bad sales representation and/or you got them presenting when it first came to IBM, perhaps? Either way, this is solvable by insisting that a knowledgeable person get on the phone with you to answer your questions to your satisfaction. Any potential customer deserves this and shame on any account team who does not make this happen.

\\\\\\\\\\\\\\\\\\\\\

From a performance, power, and efficiency standpoint there is nowhere that the XIV does not fall short when put up against even other IBM offerings or competitors. In over two years of existence, no independent person or organization has published a single XIV benchmark.

[kds] My comments about the benchmark, earlier, repeat.

\\\\\\\\\\\\\\\\\\\\\\\

The pain continues from there….

[kds] This does not add to the credibility of the article.

\\\\\\\\\\\\\\\\\\\\

Because of these points, my opinion is that it is likely that this generation of the XIV storage platform is an evolutionary dead end. It has reached a point where further scalability requires a new generation of hardware to realize greater capacity, ease of management, and performance. The current generation of hardware nodes does not have the physical space for a dedicated interconnect method between frames, except at the cost of host connectivity. Performance from large SATA drives is still painfully lacking in random workloads, and high utilization of the XIV has a dramatic impact on response times.

[kds] This is another place where the author's verified credentials to make such an assertion need to be made know. If they have not been involved in the architecture, design, and/or development of storage subsystems then this is an assertion that is not made from either a position of knowledge or experience re the "requires a new generation of hardware." Point in fact: IBM is saying in non-NDA briefings that the capacity is going up soon. (It's one of a number of examples.) Similarly for the response time comments.

\\\\\\\\\\\\\\\\\\\\

A new generation of hardware to resolve these issues will undoubtedly require a new generation of software. The current generation of software is deficient in resiliency and fault tolerance to an extent where a major rewrite is nearly a necessity.

[kds] Comments above about the hardware repeat about the software. If the author does not have the knowledge and/or experience directly on the development of these systems then these assertions must be questioned.

\\\\\\\\\\\\\\\\\\\\\\\\\\\\\

Achieving Tier 1 performance for typical Tier 1 applications is simply impossible.

[kds] Based on what fact(s)?

\\\\\\\\\\\\\\\\\\\\\\\\

And, I am signing my name to this review and feedback and you can easily find my background on the web.

Karl.

Posted By:Karl D. Schubert, Ph.D.| Fri Dec 04, 2009 08:34
Karl

I have a few concerns too; but those I would focus on are those where the identity of the author doesn’t change the facts. In the main, they’re in support of the piece. At which point, full disclosure is a good idea; I do competitive work for NetApp, and you can find me here; http:blogs.netapp.com/shadeofblue.

Your point that non-NDA briefings from IBM indicate that the 180TB raw/79TB usable will be lifted with the use of 2TB drives is fine; but that is the future, and the author reviewed what is available, not what might be available.

You also point to the XIV using RAID-X, and hence not being that inefficient in terms of capacity compared with RAID-1. Later, you dismiss concerns about failure of the XIV and the Gestalt IT article not supporting the assertion that the XIV has data loss risks.

That has been a proper concern for many, and the theory and practical research done on disk drive reliability tends to indicate that this method of doing RAID is decidedly risky.
Increasing disk drive sizes to 2TB will exacerbate that problem. Certainly, one undeniable feature of a second disk error during rebuild on the XIV is a complete loss of data, as there is only one RAID group in this system, not several.

I strongly believe that 1TB drives demand more than RAID-5, and I think you would be hard pressed to find anyone that would suggest that RAID-5 is OK for 2TB. RAID-6 or better is essential, especially in large aggregated RAID groups. RAID-X can be demonstrated to be no better (and probably worse than) RAID-1, which is a degenerate case of 2 disk RAID-5. This is demonstrably 100s of times less reliable at protecting against data loss than RAID-6.

The author was asking a pertinent question; what is the danger of a failure on an XIV? More to the point, what is the risk of failure? Beyond claiming that the speed of rebuild reduces the risk, it is not to the XIV team’s credit that there appears to be no satisfactory research or practice to support this claim. I would welcome their analysis.

Much has been made as to the performance of the XIV. Benchmarks are, at best, an indication of comparative performance in tightly controlled circumstances, and I understand that IBM may be reluctant to put the XIV through “benchmark Olympics”. But even when IBM’s internal testing was (accidentally?) made available on IBM’s public website, we find that it has now been removed. Now there’s nothing published, what is the XIV buyer to do when IBM make performance claims?

Some of your other points have substance, but outing the author is unlikely to change the questions he poses.

Posted By:Alex| Sat Dec 05, 2009 09:39
Alex,

I don't believe that they only way capacity increases are going to be realized are through larger drives; it's also the quantity of drives that are going to be increased and the cascading of racks.

All storage systems -- including NetApp's -- have data loss risks...starting with the disk drives and moving up from there. I agree that the more data stored and the more data on a disk the more catastrophic the failure. One of the reasons for using a "distributed mirroring" or "lazy mirroring" is that the exact sequence used to write the data on one copy is not the same on the other and therefore if there's a code sequence problem it's not likely to be experienced on both copies. That was the real point I was trying to make. I do believe that mirroring is VERY expensive and you certainly have to know what you're getting when you use it. The RAID type you use needs to be reflective of the data needs you have. On the other hand, I strongly believe that sooner or later it won't matter really what you choose -- you'll choose by the needs not by knowing the differences between the RAID types. People only depend on that today because the storage industry has taught them that (from earlier days).

I have not seen the research you mentioned about "...theory and practical research...." so if you have a pointer to it I'd appreciate getting it. I'll read through it.

While the author asked pertinent questions, I believe that he/she answered them with assertions and opinions phrased as facts. Outing the author lets the reader decide the basis for those assertions and opinions. Would you look differently on medical advice given from a doctor vs. a drug store (non-pharmacy) cashier who also takes that medicine?

Karl.

Posted By:Karl D. Schubert, Ph.D.| Sat Dec 05, 2009 10:10
....and, I should have added, that you don't know which is responding: the doctor or the cashier.

Karl.

Posted By:Karl D. Schubert, Ph.D.| Sat Dec 05, 2009 10:12
I've copied this discussion to the talk page of this article in the 'Vault.' We'll use that page for the purposes of discussing required changes to this article. Please paste the link below into your browser and hit the 'Discussion' tab to participate. Or click on the 'Vault' tab above and goto the Discussion page.

http://wikibon.org/vault/Talk:8_Reasons_to_Not_Jump_at_XIV

You are free to add comments here that don't pertain to requested changes for this piece.

Posted By:David Vellante| Sat Dec 05, 2009 10:34
As a matter of policy Wikibon does not require contributors to publicly disclose their identity. Many practitioners prefer to remain anonymous when contributing which allows them to speak freely and openly. Wikibon support that.

As such, it is up to the community to weigh the input of anonymous contributors based on the merit of the input.

Thanks for your help in doing that to improve this piece.

Posted By:David Vellante| Sat Dec 05, 2009 11:01
Okay, I work for a company who has actually tested XIV; the application was one of the most disk-unfriendly applications going and still IBM(XIV) were happy to put one of the floor. This is my opinion was a bizarre decision as the salesman involved was previously our EMC salesman and he should have known better. The application is pretty 100% read-miss, random I/O; XIV died on the ground.

Now this does not discount XIV as a valid disk array but it does highlight one of the weaknesses of the array. Where raw disk I/O is important with a cache-unfriendly workload, it will struggle.

The capacity discussions are interesting because as far as I understand it at the moment, the rumoured increases in capacity will come from increasing the size of spindles, not the number. So this potentially makes performance issues worse with I/O density decreasing.

Now I like IBM as a company but XIV was a company they did not need to buy. The DS8K and SVC are good solid in-house storage products, the rebadged DDN and LSI arrays are good arrays as well. XIV appears to have filled a gap which didn't need filling.

And yes we do need to look at RAID going forward, RAID-5 is not a choice I would make for mission critical apps running on 1 Tb drives but actually I wouldn't pick a protection scheme which meant a double drive failure could loose the entire array.

Posted By:Martin Glassborow| Sat Dec 05, 2009 12:58
Martin's comments are very informative...as are Karl's and Alex' above. I feel like we're getting somewhere and are forming a picture. Thank you and others for contributing; and thanks to RootWyrm (whoever you are) for hanging in with us on this.

I feel we have some raw material now to do a more complete assessment of where the strategic fit is for XIV.

Martin's points underscore the fact that XIV can't be tuned...period. And while any array will be challenged in a 100% random read-miss workload; XIV will croak.

Sales rep should get a "dope slap" for that one.

Having said that...XIV is cheap disk with simple recovery that spreads data everywhere to minimize i/o bottlenecks. LUN-based storage is not the answer for every workload and XIV has its place.

IBM has a complicated storage portfolio for sure. I think we have to huddle on the proper positioning of the product for users.

Posted By:David Vellante| Sat Dec 05, 2009 03:48
XIVs approach is nothing especially new, actually even the data protection approach which is often held up as being revolutionary feels very close the approach that Omneon take to the MediaGrid storage, with the difference that you can dial-up the number of copies on an Omneon.

Dividing an array up into chunks or even chunklets to use the 3Par terminology is getting close to the norm for most storage architectures. Then combining these chunks into something which we call a LUN is almost an irrelevance but it is a useful and familiar concept, so we persist with it.

There is probably no reason why in future that instead of allocating multiple LUNs which are simply a construct made of chunks; that a single data-storage-space defined on an application/service boundary should not actually become normal.

If we look at technologies which involve sub-LUN optimisation such as FAST v2, Compellant's Data Progression and others which I am aware of under NDA, these architectures do indeed become pretty much optimal.

What XIV have done and in my opinion, this it's biggest flaw is to design an array based around a single tier of disk which actually negates many of the advantages of such a chunk-based architecture. An XIV which could migrate it's chunks to faster disk would have value, an XIV which utilised a more sophisticated data protection method would have yet further value.

Unfortunately, this means that the XIV starts to look an awful lot like some it's competitors actual or proposed architectures.

And is XIV especially cheap...now that is an interesting question? I've seen prices which suggest that it is 30-50% more expensive that some of it's rivals from a pure Capex PoV. I've also seen the TCO model that XIV sales-guys pull out...my TCO would go up if I bought XIV.

IBM need to look at what's good about the XIV and integrate that into their existing architectures if they can.

Posted By:Martin Glassborow| Sat Dec 05, 2009 05:09
This article is clearly written by an EMC marketeer who is trying to block progress by twisting the facts and misleading customers so they would stick with their archaic complex and unmanageable storage systems.

Let’s view the 8 points with the real facts:

1. XIV is immature – their grid technology gave it a web resiliency so its immune from software and technician problems as represented with clean field reliability results and acceptance by major enterprises all around the world

2. XIV has achieved only over 1,000 revenue ships – they almost doubling their success each quarter and have now more then a quarter million drives in production at customer sites with the best performance and reliability results

3. XIV is an extremely opaque storage platform. – This is a big advantage of XIV since storage is becoming too complex to mange by humans.

4. XIV still offers customers no scalability. 79TB is still the limit – this is a limit of a rack but there is a common management that manage all of them from one screen which practically give it unlimited scalability

5. XIV is extremely inefficient. 79TB of usable capacity requires 180 1TB 7200RPM SATA drives – XIV compete successfully with traditional tier 1 systems which use 146GB and 300 GB FC drives so you get much better net capacity at XIV. On top of it its efficiency in Snaps, Thin Provisioning and no orphan space increase its practical usage a lot.

6. The XIV’s power and thermal efficiency is similarly deficient – the same as for point 5 which make it 4 times better in power consumption when comparing to similar systems which uses 146GB FC drive for getting the same performance and reliability

7. RAS and data protection as compared to traditional arrays needs a great deal of improvement – there is no other system that is imuned from software and technician mistakes as XIV. The fact of the matter is that it has the best quality results in the field with over quarter million drives in production.

8. The price to capacity and performance ratio of the XIV – XIV is a few times better in these aspects in relation to its most commonly usage as replacement for high end systems from EMC (VMAX) and Hitachi (Tagmastore)

Its pity that we let vendors using this forum for distributing their non based FUD’s

Posted By:Robert| Mon Dec 07, 2009 07:15
Hi Robert...I'm pretty sure the individual who originated this article is not an EMC marketing person. He (or she) goes by the screen name of rootwyrm, claims to have a first name of Phil and is an active participant in Twitter discussions.

Go to Twitter and see rootwyrm's Tweets and decide for yourself. Not likely an EMC marketer imo.

The read:write Web is a forum for anyone who wants to publish. Wikibon's protocol supports the concept of 'anyone can contribute' as long as people are respectful of one another.

We use a Wiki so we frequently edit/alter/improve as the community sees fit. To that end, it would be very helpful if you could provide some proof points of the claims you make; specifically on the following points above:

1 &7. Can you share field reliability data?
2. Again...can you point to performance and reliability results?
8. This point is interesting and one where there seems to be wide disagreement. We've requested that IBM provides some customers we can speak with to understand this better and confirm these claims.

Thanks for weighing in.

Posted By:David Vellante| Mon Dec 07, 2009 07:45
Robert:

As Carl Sagan once famously stated, “Extraordinary claims require extraordinary evidence.” Where is the evidence?

1. “immunity from software and technician problems as represented with clean field reliability results”; what results have IBM published?

2. “a quarter million drives in production at customer sites with the best performance and reliability results”; what results have IBM published?

3. “common management that manage all of them from one screen which practically give it unlimited scalability”; that is management scalability, certainly, but several distinct systems integrated at the glass does not represent storage scalability.

4. “tier 1 systems which use 146GB and 300 GB FC drives so you get much better net capacity at XIV”; the size of the drive does not affect net capacity efficiency.

5. “which make it 4 times better in power consumption when comparing to similar systems which uses 146GB FC drive for getting the same performance and reliability”; I would dispute your performance claim, given the IOPS characteristics of SATA, and would ask you to substantiate your reliability claim too. 146GB drives are unavailable from a number of manufactures, having been superseded by larger FC drives.

6. “there is no other system that is imuned [sic] from software and technician mistakes as XIV. The fact of the matter is that it has the best quality results in the field with over quarter million drives in production”; is nothing more than your assertion.

7. “XIV is a few times better in these aspects [price to capacity and performance ratio] in relation to its most commonly usage as replacement for high end systems from EMC (VMAX) and Hitachi (Tagmastore)”, but then so are racks of JBOD.

“Its pity that we let vendors using this forum for distributing their non [fact?] based FUD’s”; if you had answered any of the questions the article posed rather than restating the contents of the XIV brochure, that would have been helpful.

Karl:

http://www.cs.cmu.edu/~bianca/fast08.pdf
http://www.usenix.org/events/fast08/tech/full_papers/jiang/jiang.pdf
http://media.netapp.com/documents/tr-3574.pdf
http://www.cs.wisc.edu/adsl/Publications/latent-sigmetrics07.pdf

There are many more; note that much of this research is from thousands of systems and millions of disks drives over very long periods. Good stuff!

Posted By:Alex| Mon Dec 07, 2009 08:18
Alex,

Schopenhauer said: All great truth goes through three phases. First it is ridiculed, then violently attacked, and finally accepted as self evident.

For sure I took my responses from IBM brochures but we have 20 XIV boxes on our floor and I can testify that it’s “as advertised”!

Hi David,

I’m already violating our company policy by these comments so this would be my last posting. I suggest that you approach IBM marketing directly for the responses to your questions. As I wrote above, we already are having 20 of XIV systems with intention to consolidate all of our future storage needs on that technology. As I see it, it’s so disruptive in reducing cost and improving productivity that they are going to revolutionize the entire storage market and we shouldn’t be in ridicule position when people will quote us as an example for myopic view on technology.

Posted By:Robert| Mon Dec 07, 2009 06:40
Hello Robert...I'm fond of looking at new ways to solve problems and appreciate your input. Hopefully you don't see my comments as ridicule...I'm searching for the truth and I suspect this is a 'horses for courses' situation as others have suggested.

IBM is free and encouraged to participate in this effort and I'm sure will review whatever we formally develop as a Wikibon community opinion on XIV.

Thanks for your inputs and contributions.

Posted By:David Vellante| Mon Dec 07, 2009 06:44
Sorry for the long delay, things have been a little crazy and I wanted to ensure things were coherent.

Ask any EMC rep who's dealt with me; I'm definitely not in their corner. Frankly, they're overpriced and years behind the times. Before you get me started on PowerPath licensing. I'm sure they'd like me in their corner, but that's unlikely to happen any time soon. (Hint to EMC; stop making PowerPath a separately licensed requirement and I might look at your products more than once.)

Alex's first point is absolutely one of the most poignant; "what has IBM published?" The answer for XIV is - just about nothing. They published slides I cited as part of the March 19th System Storage User Group webinar. (I've been told that IBM reps should be able to provide the slides to customers on request.) Those slides comprised the entirety of data available publicly on XIV; everyone cites NDA or offers up only non-auditable claims.
If IBM were to perform SPC-1 or SPC-2 and publish the results, then we wouldn't have a hundred reasons to question every claim they make. Instead, they complain that benchmarks they helped develop are unfair or bogus and prohibit customers from publishing benchmarks. These are not actions anyone would want to see from any vendor making performance claims and also being a member of SPC.

Something else I keep hearing and keep seeing is people insisting on a very, very narrow comparison set for XIV. Saying it must be compared to TAGMAStore or USP-V and DS8000 with FC disk.
If we accept that XIV can do 60,000 IOPS at 80/20 read/write random, it's still not a competitor with those storage arrays. You're comparing 79TB to systems that scale to more than four times the capacity and double the IOPS (DS8300, 122K SPC-1 IOPS) or more than triple (HDS USP-V, 200K SPC-1 IOPS.)
To be honest, I don't think anyone can call it anything other than disingenuous. Yes, you're getting performance - claimed or real - at a fraction of the cost of those products. But the core measurements simply do not match up. If you were to compare XIV based on capacity and IOPS, you aren't putting it up against top end Tier 1 arrays. You're putting it up against things like the HDS AMS2500, which are an entirely different cost class.

Let's also not forget that XIV is not a new product. Or somewhat new product. It began in 2005 as Nextra XIV, so it has been on the market for 4 years.
I could tell you exactly how they tried to make it a deal we couldn't refuse, but I do know that would violate specific contracts I am still bound by, because it would require specific privileged information.

And let's go back to management and competing arrays. Everyone insists on comparing XIV to the wrong products. XIV doesn't compare to anything in IBM's portfolio. It compares to companies like Compellent, 3par, DDN, etcetera, as I said. These companies provide 'opaque' platforms, yet have still managed to consistently provide more information to administrators than XIV is willing to. Without making them too complicated to manage.

"Single pane of glass" - IBM Storage Manager provides me a single pane of glass for 128 DS4000 or DS5000 controllers in the exact same way. Show me how managing ten distinct systems from one point eliminates the problem I stated, that being that each XIV must be managed independently. It does not change that at all, it only makes it a single URL for those ten distinct systems. I have yet to see someone show me how I can manage 10 XIVs as a single unit, automatically balancing disk based on performance and capacity, short of putting them behind an SVC. Even within an XIV cluster, you're limited to slow mirroring with no cache coherency between XIV frames.

Cost savings; put it in numbers. I did, and certainly would be happy to do so with any other array I can get BTU/hr and power draw numbers for.
You can verify the watts per usable and total watts I posted yourself. It's simple math based on the numbers provided by IBM at their own website. And again, you need to compare it to products of similar performance and capacity - which may disagree with what the vendor wants to compare it to, and often does.

Certainly, I won't disagree that XIV is a good answer for some workloads. And your mileage may and likely will vary. However, I can only work with publicly available data - the same as anyone else not bound by NDA and in possession of one. And like anyone considering products for a storage refresh, claims without data are not facts. They're just claims. People don't buy storage sight unseen, and XIV asks you to do exactly that.

And certainly, Robert, I'm not saying XIV doesn't work for you. It works for some workloads. But it doesn't work for all workloads, which is a fact frequently deliberately left out of discussions involving it, along with any real performance data or relevant technical information.

I received the full technical presentation, and it contained exactly zero slides discussing IOPS or host bandwidth, only a lot of repetition about the backend bandwidth and how it magically made disks more reliable without explanation. By the end of the slides, the XIV reps were already writing up a configuration, and were genuinely shocked when we asked more questions. Only two non-technical questions ever received the promised answers.

IBM has yet to make public real information on XIV, or provide actual technical data rather than marketing part of their sales presentations. Because of this, there remains in my opinion a very firm need for potential customers to take a much more critical view, and insist on written guarantees should they choose to try XIV.

And again, I do feel it necessary to state, I am not anti-IBM System Storage, and I have no affiliation with any storage vendor.

Posted By:RootWyrm| Thu Dec 10, 2009 07:20
RootWym

We are using 2PT of XIV storage for our Oracle, SAP, Exchange and all of our other applications in production with an excellent reliability, performance, management and significant reduction in our costs while improving productivity. You should solicit some customer inputs for your report

Posted By:Robert| Thu Dec 10, 2009 08:39
Thank you Robert.

All...I have updated the Action Item and tried to convey the sentiment of the comment flow. I think we agree that XIV is not the best fit for all workloads but has its place.

We are in the process of organizing these comments to produce a piece that reflects the broad community opinion.

Would anyone be interested in participating in a Peer Incite Research Meeting on this topic? It's a 1 hour meetup with open conference call lines.

If you're interested in attending and/or participating actively, let me know in this comment stream or reach me on twitter @dvellante.

Posted By:David Vellante| Thu Dec 10, 2009 09:31
David,

Thanks for your invitation but our company policy wouldn’t allow me to participate in such call. BTW, we already got a copy of the article from EMC rep who is trying to get back into our account, so, combining it with the anonymity of the author, it raised my suspects of a foul play and they are trying to sneak out marketing FUDs under the community cover.

As for the summary of your article. “Users of this array should avoid high volume, random, small block workloads with poor locality of reference and rather target XIV toward applications where 'good enough' performance,” The reality is just absolutely the opposite and we are using it for EVERYTHING and we need excellent and consistent performance and utmost reliability which we can get ONLY from xiv. The article and other comments are based on false theoretical analysis where the reality is totally different.

I strongly recommend that you will check it out with some other XIV users, since, as any other disruptive technology, it’s sometime hard to grasp it theoretically so reality and real customer experience are the right course.

Posted By:Robert| Fri Dec 11, 2009 06:00
Thank you very much for the comments Robert. I've updated the Action Item with a fairly benign statement as a placeholder for now.

Just to clarify. Your point is you are seeing excellent performance in random read miss workloads...as well as other workloads...correct?

Can you help us with the following question so we can improve the Action Item? What's the one piece of advice would you give your peers about XIV?

Posted By:David Vellante| Fri Dec 11, 2009 07:31
We are a large shop and we are getting better performance in comparison to our DMX even out of random read misses since XIV always use 180 disks for any request while in our DMX with faster FC drives we were limited by the number of drives in a raid group. So the end result, for us, was no more "hot spots" and better performance even in such applications. For sure, theoretically, with different data layout in our DMX boxes we may be able to achieve better results but it's too cumbersome and almost impossible to do within our complex environment.

Thanks for updating the action at the end but the article itself is full with stuff such as "XIV only fits where you can afford downtime or data loss that can be restored from tape". As far as I know (from our own experience and from friends at other companies) this is the only technology that almost guaranties no downtime and no data loss! Per our experience. its reliability is much better then in our high end EMC and Hitachi machines.

We shouldn't allow such brute violation of our community (I suspect by EMC since they gave us a copy of it) without having the author reveal himself and without having any customers feedback as for any of the claims in the article.

Posted By:Robert| Fri Dec 11, 2009 10:17
Thanks Robert for the additional detail. Just so you know...the reason we have a discussion flag on this article is to clearly indicate to consumers this is a work in process. We are not done...not by a long shot.

We are compiling feedback and will publish a definitive community perspective. That's why I'm organizing a meetup (phone).

Unfortunately, for obvious reasons, several practitioners may have to stay anonymous on that call...but we welcome their input.

Posted By:David Vellante| Fri Dec 11, 2009 12:50
David,

It's true that "As a matter of policy Wikibon does not require contributors to publicly disclose their identity" but I can't find any other article in Wikibone without the identity and the background of the author. On top of it, the article is so cumbersome and complex that it got the lowest grade I ever saw in Wikibone. I don't see any way how we can fix it so the only option, in my mind, is the recycle bin and sooner is better since the fact that you flagged it doesn't change vendors' usage of it. We also should be very careful with anything we published since vendors are fast in the use of it to their advantage. I see it a lot as a customer.

Posted By:Robert| Fri Dec 11, 2009 07:49
Thanks Robert for the input. I agree it makes more sense for us to create a new note than it does to try and revise this one.

We did immediately flag the article because we knew it would be misused by zealous sales reps. I've also just updated the Flag Template to reflect a stronger 'Warning Label.'

I understand your concerns. Do you think immediately deleting the article is the right protocol? I feel as though RootWyrm's post has catalyzed a good discussion which is bringing to focus many issues around XIV.

The ideal solution would be to have a phone meetup on this topic. My thinking is we could invite a cross-section of informed individuals and Wikibon folks would make sure a definitive note gets created.

Would you be willing to participate as either an anonymous contributor or in a more private setting-- i.e. a closed forum?

Please email me at david DOT vellante AT wikibon DOT org or reach me on Twitter @dvellante.

Posted By:David Vellante| Sat Dec 12, 2009 09:41
David,

Thanks for your offer to call you or email you directly but, at this juncture (an article which most probably is a "foul play"), I believe that nothing should be done behind close doors and everything should be done in the open, so everyone can see and judged it to himself. I'll try to answer to questions from you as they related to our experience with XIV but without anything which can identify us.

I strongly recommend that you will solicit responses from customers beside me. There was one commenter, Martin Glassborow, who wasn't a user but tested the machine for random performance and rejected it. It's true that when he did his random i/o benchmark he got better results with FC Drives systems (as happened in our case) but he made a big mistake of rejecting it because, in the real world, our real problem is "hot spots" i.e. certain drives are getting more i/os then the rest of the system so everything is queued up to the user (It's out of the restriction of raid groups in standard storage systems). Martin may have a very small shop so he may be able to tune his system by different drives layout per every application but that is almost impossible to do at any medium to large enterprises

You asked me in the past the following 2 questions:

"1 &7. Can you share field reliability data?
2. Again...can you point to performance and reliability results?"

So, I don't know about the rest of the field but in our case (2PT of XIV almost 4000 disk drives) we have a clean reliability results (absolutely no downtime or data loss) and excellent and consistence performance. We run EVERYTHING on these systems.

Any more questions?

Posted By:Robert| Sat Dec 12, 2009 06:17
Umm, sorry Robert, you know nothing about our application at all. I do! There was no mistake in our rejection of the XIV platform, even IBM were happy with our decision. XIV was not appropriate for this solution. This was tested with a real world application with real world data; this was no synthetic application; during batch windows, it is 100% random, read-miss I/O; we are pretty much reliant on the speed of the spinning rust. SSDs would probably help.

And we are not a small enterprise.

And this constat suggestion of foul-play on Rootwyrm's behalf; he appears to be more than happy with SVC and his DS infrastructure; this does not sound like an EMC marketing shill. He has expressed an opinion which you do not like, that's fine but it does not make him an EMC lover or up to no good.

Personally, you need to get over yourself quickly because at the moment, you are coming across what I would call 'a true believer'; one who can only see what you know to be true. In the world of IT; it is very rarely this case. We buy lots of EMC, NetApp and IBM (but not XIV); we buy what is appropriate to our applications; some are extremely specialised, some are generalised. Some of the kit we buy, I'd never recommend to anyone without understanding their application.

Posted By:Martin Glassborow| Sun Dec 13, 2009 05:36
Martin brings up a point that is usually true in IT...that the workload determines the performance of an application, not the IT equipment. Certain solutions are a better strategic fit for certain workloads and what we're trying to identify here is the right strategic fit for XIV.

I was under the impression that 100% random read miss I/O was something to avoid with XIV, based on Robert's post and conversations with David Floyer. Intuitively it sounded correct-- no cache...SATA disk...slower performance.

Robert countered that his random read miss workloads performed better w/XIV...which did surprise me I admit. I find myself asking why that's the case.

What's the workload like? What's the scale? What's the previous array of comparison?

Robert's and Martin's/David Floyer's assertions are completely at odds-- which is why I'm proposing a discussion to better understand the issue.

Posted By:David Vellante| Sun Dec 13, 2009 07:35
Martin,

How do you overcome “hot spots” in your environment? How big is your environment and the application?

I agree that I’m “coming across as a true believer” but the same apply for all of XIV users and you choose the wrong path and by that deprive your business from the benefits of xiv.

As for the article, I think that there are a lot of “quite majority” out there who really see what it is as evidence from the article record low rating of 1.8

David,

You really need inputs from real users for real assessment of the system and, beside me, you got none so you can’t write “Robert's and Martin's/David Floyer's assertions are completely at odds” since they aren’t users so their assessment can’t be taken into account

Posted By:Robert| Sun Dec 13, 2009 11:08
David,

The article is full of sentences such as: "XIV only fits where you can afford downtime or data loss that can be restored from tape" and "situations where a storage outage does not affect operations" and "IBM XIV Could is Hazardous to your Career". There is absolutely no evidence for it from customers or the field and the only discussion was about performance (BTW, I saw only Martin's response and there is none from Dave Floyer).

If you don't want to remove this vendor' article then, Is it o.k. to edit out all of these sentences? I'll gladly do it. Also, it would be fair to add in the header a sentence such as: "there is no evidence from the field to the reliability and risk exposures expressed in the article".

Posted By:Robert| Fri Dec 18, 2009 06:54
Hi Robert. The nature of the Wiki is anyone can edit. Anyone that wants to see the changes can go to the vault and look at the history of the article.

Any change you make will be reviewed and so you are welcome to make any changes you see fit.

David Floyer will weigh in. We plan to do a piece on XIV. I've been recruiting folks to do a phone meetup which we'll probably do in early Jan to support the piece.

Would be great to have you on that call.

Posted By:David Vellante| Fri Dec 18, 2009 08:45
David,

How can I change the warning in the header to include: "there is no evidence from the field to the reliability and risk exposures expressed in the article".

Posted By:Robert| Fri Dec 18, 2009 09:20
Do we have a definition of "random workload"? The reason I ask is that some of the debate about XIV suitability or not for random workload may depend on how truly "random" the workload is. If the workload is random over the data within cache you will get stunning performance, if it is truly random over a 79TB set then probably not so good.

Posted By:Bazzer| Fri Dec 18, 2009 02:58
I think the best 8 reasons that the poster should have come up with are the wife and 6 kids he is trying to feed on an EMC salary, as IBM & XIV changes the way we all look at storage for ever. Much like VMware changed the game on virtualized servers several years ago..
Just because you have always done it your way doesn't make your legacy concepts the best direction for my future!
The points are "weak", in terms of what we REALLY care about when we buy storage for our business.

Posted By:LittleJoe| Fri Dec 18, 2009 03:15
Tony...and LittleJoe..you are free to hit edit and improve anything you see on the site. This is a community wiki. Please try to respect the dignity of the individuals who are contributing. Insults aren't very productive.

Just be aware that anything you post or edit-- just like this article-- will be scrutinized by the community. That's why there's a flag on this article because it's under review.

We welcome your contributions to make it better or help us develop a definitive Wikibon community opinion. We're organizing a meeting of interested people to help us do just that. You're both welcome to participate if you want. Please reach me on twitter @dvellante or just show up. As a registered member of Wikibon you'll get an invite.

For the record, the poster was responding to an article I wrote entitled "8 Reasons to Pay Attention to XIV."

http://wikibon.org/blog/8-reasons-to-pay-attention-to-xiv/

I don't think reasonable people would consider what I wrote EMC marketing.

Also...@rootwyrm-- the poster-- has shown to be someone that is willing to interact thoughtfully. I don't agree with much of what he wrote but reasonable people can disagree.

Posted By:David Vellante| Fri Dec 18, 2009 03:36
David,

The problem of editing it is that there is a need to delete almost everything since it's full of sentences such as: "XIV only fits where you can afford downtime or data loss that can be restored from tape" and "situations where a storage outage does not affect operations" and "IBM XIV is Hazardous to your Career". These sentences are in no respect to the dignity of the individuals who are using Wiki. The "smoking gun" for EMC hands here is that EMC is using this article everywhere.

As for your article http://wikibon.org/blog/8-reasons-to-pay-attention-to-xiv/ it was good to the community since you didn't made any biased assertions about the system so nobody suspect you as being a tool by EMC marketing or anyone else.

What about my question about how to add the warning sentence: ""there is no evidence from the field to the reliability and risk exposures expressed in the article".

Posted By:Robert| Fri Dec 18, 2009 04:33
Barry Mellish - great points.

Here's my understanding from talking to people like David Floyer and others.

The IOPs sustained on XIV disks is 60/2 times the # of drives. Simple. 180 drives in the chassis...every volume is spread across every drive and the # of IOs thru a SATA device is around 60 on average.

5400 IOPs going to disk. So the question is how cache friendly is the workload. If no cache hits...max IOPs are 5400. A 95% hit rate on the cache is 5400/(1-.95)-->108K.

At this rate the bottleneck will be the number of ports on the machine, which is not huge on XIV. Floyer-- pls check my math.

So in this context, the randomness of the workload is defined by the cache hit rate. The parameter we haven't considered here yet is block size. Larger block size means...more throughput.

So based on this notion...where XIV (and most arrays) will get crushed is lousy locality of reference. And XIV suffers more because it's 100% SATA.

This assertion has been supported by some comments here and questioned by others. Hard to believe XIV shines in 100% random miss workloads compared to some other arrays.

Posted By:David Vellante| Fri Dec 18, 2009 04:41
One other comment per Robert's point above is that in his case...it was EASIER to let XIV manage the spreading of data rather than reconfigure a DMX. This point should not be lost in this discussion.

Posted By:David Vellante| Fri Dec 18, 2009 04:51
Really, This is a Joke right…. Any person reading between the lines can see this blog is cooked and served on fine China. If indeed this is a place for free thinkers why don't you just let the nay sayers speak there mind. Instead of moderating them and discounting there comments. When someone doing research to make a decision for there company they are looking for truth not marketing. If you or anyone else that is wasting time on this site is confused buy my statement let me make it clear.

This is not a valid review Here are the reason.

1) You put up a weak review about how XIV is so great. The reason why its weak can be for a few reason either you don't fully understand what this technology bring to customers that need it.

2) You constantly validate comments about EMC fanboys and throw a few comment to try to keep your statement unbiased.

3) RootWym having your buddy throw up a blog so you add a bunch of remedial blather doesn't make me want to spend one more nickel on powerpath or continue to pay EMC for any of the other things that XIV has solved for me. If fact It make me rethink a lot of things that EMC reps have told me.

4) You comment are just comments biased ones. The main reason for this article is to strike doubt in potential costumers. (To those customers I would just stop reading because there is nothing of use here. There are some good points but this blogs clearly biased

So why am I even commenting on this wiki - Because I am waiting on a flight and have time to kill. A junior engineer sent this link to me. I don't blame him as he is learning he came from the NOC and is picking up some storage admin skills. Its a shame David that the very think that you say you are doing, spark intelligent conversation does not seem to ring well with people that have made an investment. I find it to be very unethical. EMC has send me many storage anarchist articles are well.

It seems that EMC is scared of technologies like 3PAR, XIV amount others. Why because they are more streamlined they don't spend money on a huge marketing department. They hire talent away from EMC that is a much better way to deal with you competitor.

I have a few fairly random application that now run on XIV and some on NAS and it works fine. The response time were very close when it was fist put on a CX after trying to scale the app out it created many hot areas and gave very unpredictable results. During the sales cycle EMC tried to discredit any new tech that I looked and tried to sell me more software to fix the inadequacies of the platform they sold me. Unbelievable not to mention that they tried to push Flash on a VMAX as a solution. If I am in the market for a replacement for a CX what made him think that I would spend that kind of money is beyond me.

I doubt you will understand my rant and this post will probably not make it. Because it is not "productive in you eyes"
Wiki's are a great way of sharing knowledge. It should not be a instrument for spreading inaccurate information. Ultimately the site will just loose credibility.

Or may be I will just put a wiki up on how to smell out someone that graduated from EMC marketing.

Posted By:Berk| Fri Dec 18, 2009 10:58
1 Really good reasons why this is not a useful review.

1) This wiki's is basically EMC marketing. Most of the facts are incorrect. I compete with IBM storage all the time and I believe my product is better than EMC but would rather not tarnish my company's name in this wiki. But with that said if you are reading this page for technical merit well.... Some of the statements made are very emotional many of the reviewers have posted their concerns.
To comply with David I edited my statement. But at least I am honest in stating by legion

David, Please don't take it as an insult its just the way I see it. Just my 2 cents.

--N

Posted By:Tony| Fri Dec 18, 2009 11:23
David,

You are responding only to "random performance" but most of the article is about reliability and risks. Why you let that garbage stay?

Posted By:Robert| Sat Dec 19, 2009 04:12
This discussion isn't generating any light on the subject of the XIV. Regardless of assertions that this is an EMC inspired piece, and the anecdotal assurances the XIV performs and is reliable, there is no verifiable evidence to support any of these statements.

1. IBM has made claims about performance. There are no benchmarks for the XIV, and the only performance paper from IBM on the subject has been withdrawn from public view.

2. IBM has made claims about reliability. Beyond IBM's assertion that rebuild times reduce risk, there is no evidence that this is true, and much research and experience that points to the opposite conclusion.

Perhaps someone from IBM needs to step up and fill the gaps.

Posted By:Alex| Sat Dec 19, 2009 08:21
I thought it would be useful to point out that Wikibon has published about 20 research notes related to XIV. I don't believe many of you have taken the time to review them and it's probably worth checking these out. We've followed the product closely since IBM's acquisition. We've profiled customers, done some cost comparisons and cited some best practices.

They can be found here--and are all editable (with the possible exception of the Lieumi Bank Case study which probably has been frozen).

http://wikibon.org/wiki/v/Category:XIV

I will post this link on the main article of this page as well.

It's important to remember the following:

1) This is an open forum...everyone can contribute, everyone can post and everyone can provide an opinion-- please be respectful.

2) We are a research Wiki. Our goal is to take the input from discussions such as this and develop useful information for practitioners. That takes time. It takes commitment and contribution from many people. We are not gurus...we don't have all the answers...we are facilitators trying to help users help each other to make better decisions.

3) As such we have policies to flag articles that could be misconstrued or used by vendors as blunt marketing instruments. We also flag research the community feels is work in process.

4) In our experience, practitioners are very savvy about vendor tactics and are quite smart about reading through the lines.

Please try to be thoughtful and help us advance the body of research on XIV. If you read the pieces we've posted you will see they do a decent job of explaining the architecture, its benefits, its drawbacks and strategic fit.

So here's where we are at. We are reviewing this piece and the information it contains...both the article itself and the comments. IBM has agreed to let us speak privately with some additional customers, which we plan to do in January assuming the customers can schedule the time.

We will then have a public meeting to discuss the community's current thinking on XIV and we'll publish research that reflects our views at that point.

You will all be invited as members of Wikibon.

Thanks for contributing and thanks for your help.

Posted By:David Vellante| Sat Dec 19, 2009 10:10
Something fascinating has been going on; I thought I would do so investigation into the IBM presentation that Rootwyrm linked to. That presentation certainly existed and I have a suspicion that it might well have been pretty much as Rootwyrm suggested; you see the beauty of Wiki sites is that you track changes and the beauty of Google is that it has a habit of caching things.

So if you look at

http://www.ibm.com/developerworks/wikis/display/usergroups/System+Storage+Technical+User+Group+Webinars

You will find an entry on IBM XIV Storage System Performance which is the only item without it's accompanying presentation. Now if you go through the edit history of the wiki, you will find that happened on November 30 2009, a few days after this was posted.

Unfortunately, there's not a cached copy on Google or Internet Archive as far as I can find. But it certainly looks like the presentation existed, existed on an IBM site and was deleted after this posted.

So before we go waving hands and accusing the original author of mis-doing etc; perhaps, we need to look at IBM as well.

Posted By:Martin Glassborow| Sat Dec 19, 2009 12:46
I believe I stated earlier this link was live at one point. It would be good to get a copy of the presentation if anyone has it.

Posted By:David Vellante| Sat Dec 19, 2009 08:52
Alex,

You wrote about XIV reliability “there is no evidence that this is true” but, is there any evidence from customers to the writing in the article such as "XIV only fits where you can afford downtime or data loss" and "IBM XIV is Hazardous to your Career”?

I read the opposite from user comments to this article.

Posted By:Robert| Mon Dec 21, 2009 05:35
Many thanks for all the contributions to XIV knowledge contained in the comments above. I have tried to summarize all the comments and my own research in a piece called "The IBM XIV Storage Array Performance and Availability Envelope" (http://wikibon.org/wiki/v/The_IBM_XIV_Storage_Array_Performance_and_Availability_Envelope) The executive summary is a follows:-

Wikibon classifies the XIV as a tier 1.5 array. The unique single-tier architecture of the XIV spreads the IO activity for all the volumes very evenly across all the drives. The result is very consistent performance for all volumes while using much lower cost SATA 1GB drives instead of FC drives.
The ideal application environments for the XIV array are:
*Well mannered applications with reasonable IO activity and cache hit rates
*A good number of applications using a significant number of volumes
*No single application requires extensive tuning

Contraindications for the use of XIV arrays include:
*The XIV array is dedicated to a few performance critical applications
*A large application with high IO rates and very low cache hit rates
*Where applications in general or specific applications have to have very low IO response times
*Where the very highest levels of availability and recoverability are required and can be justified

The benefits of the XIV architecture include lower costs, lower storage administrator costs and “good enough” consistent performance.

The drawbacks of the XIV array are that there is no ability to tune the array for a specific workload, and that close attention has to be paid to when the array cannot be filled with more volumes (adding additional volumes when the array is near its IO limit can have a big impact on the IO performance of all the arrays).

Traditional LUN-savy storage administrators will feel frustrated that they cannot use their skills. However, many CIOs will feel that the XIV offers an opportunity to avoid over-paying for traditional LUN skills.

There have been voluminous writings on the blogs and competitive presentations (as well as this piece) about the potential availability deficiencies of the XIV architecture. Particular reference has been made to the impact of dual drive failures. Wikibon considers some of the analysis above to be deficient, and not reflective of the positive experience of installations deploying the XIV. The availability design of the XIV is different to traditional arrays, but is appropriate for the specific architecture of the XIV, and results in similar availability to competing arrays. The detailed Wikibon analysis within the sections “XIV Availability Characteristics” concludes that the probability of having to reconstruct an array because of a dual drive failure is less that .05% over the five year life of an XIV, and is considerably lower than other risks that can results in array reconstruction on any array.

Overall we conclude that with its performance and availability envelope, the IBM XIV offers very good and sometimes outstanding price performance for most tier 1.5 and tier 2 workloads.

Wikibon would welcome further refinement and comments on the new piece.

Posted By:David Floyer| Mon Dec 21, 2009 07:41
I've posted a link to Floyer's research note at the beginning of this artciele as well:

http://wikibon.org/wiki/v/The_IBM_XIV_Storage_Array_Performance_and_Availability_Envelope

Posted By:David Vellante| Mon Dec 21, 2009 10:13
1) Immature? No, been in production since 2005. Yes IBM bought in 2008, but they bought an existing company. At least get that fact straight.

Dedicated sales force? Yes, and it's integrated with the thousands of other STG sellers. Is this different than say.... EMC? RSA Group, Avamar Group. Invista Group. Let's see. Nope, no difference. So what's the problem here? Don't create one where there isn't one, that's called FUD, and people are really starting to see through it.

2) I'm not even going to comment on #'s shipped as that's already been commented on by someone above and I agree with him that's it's a solid number.

IBM hasn't disclosed which customers are ordering multiples or what they are using XIV for.

Hmmmm really? Call Hitachi, EMC, and NetApp, and ask them for their customer list and what they sold them and what they replaced. This is nearly an idiotic statement you have made. I am however, not calling you an idiot. That's not my style.

But you're wrong there too. Multiple customers have said what XIV replaced in their own public statements. So... just because you didn't know that, doesn't mean it doesn't exist.

3) Admins have no way to control where their LUNs are. Really? Ask 100 admins if they care where their LUNs are if they get the performance they require. And don't ask the ones over 45 or 50 who have been doing it for 15 years and have their head in the sand. Ask the ones with 10 years or less solid experience with several virtualization storage models, and let's see the % of Admins who care where their LUNs are. When I was a storage admin years ago, I detested the manual balancing of 4000 LUNs in a certain very large system. We had to write a PERL program just to balance it over all the different buses, disks, back end. It was ridiculous then, and it's even more ridiculous now. CTO are cutting budgets for staff, and staff barely had enough time to do all that nonsense before, they now they have the choice of working 25% more hours a week, or trusting that virtualization and a well balanced algorithm is going to do a better and 10,000 times faster job of balancing. I know what I would choose. Simplicity over complexity.

4) Scalability. Sorry can't talk about this. But bide your time, you'll see your scalability. Rome wasn't built in a day.

"Do you want it right, or do you want it Monday?"

5) Tiering. Get off your dinosaur, and note that the wheel has been invented, and ILM was created to make money by software companies with no real purpose - and I think most people will agree that ILM has basically failed.

Are you saying XIV can't do thin provisioning and snapshotting that satisfies customer's needs? List a customer who says this. Good luck. You say it "ignores other hardware and software methods..." Sometimes you have to do things differently to possibly improve. Ever think of that? It is 2010, it's not 2005, it's not 2000, it's not 1995. Times were very, very different in each of those storage epochs - as I like to call them.

Or maybye you're the kind of person who doesn't like change, and would rather set it up, and forget about it. If that was the reality of a datacenter, that would be a potentially valid point. But I would say that 98% + just don't work that way.

6) I'm not going to get into a power discussion, especially since you're discussing IBM vs IBM products.

Plus - though power, space and cooling are recurring costs, they are not the only costs - whether you are right or wrong. You have administration time, maintenance time, migration time etc. And I think a thorough look at power, space and cooling would poke holes in your insinuation that XIV is very inefficient vs other products on the market.

7) Drives tend to fail in groups. Is that like H1N1 contagion locality? Are you thinking about RAID 5? Do some homework on XIV, I'm not going to do it for you here.

8) If price performance isn't competive, why are so many companies adopting it? Because Moche Yanai, the "father of storage" created it? Because IBM, a solid name for 100 years, with a HUGE sales, technical sales, and support force are behind it? Because XIV works?

There must be a logical reason or 2 behind why people are buying XIV.

Many people haven't heard of Compellent,, 3Par, or Xiotech? Are you kidding? Those companies haven't come knocking at their door? Someone in the storage/san/application team hasn't heard of them? Sure maybe if it's a TINY shop where they don't let anyone come in, that could be the case. I have yet to come across any storage admin who hasn't heard of everyone you listed plus 5 more that are even more niche. If they are in a storage purchasing position, they most certainly have heard of competitive products, otherwise, they shouldn't be in that position, and will make poor choices, and eventually be removed from the position.

----------------------------------------------
I'm wondering really if you're living in a vacuum. But as the saying goes, I may completely disagree with everything you say, but I will fight to the death for your freedom to say it. --- Not the exact quote, but I don't feel like looking it up.

Happy posting~

Posted By:theperfectsan| Fri Jan 08, 2010 02:09
Hi PerfectSAN,
I am actually an IBM Sales Rep and in an almost done Deal have been forced to spend ~3 hrs replying to this heap of not-understanding by rootwyrm. In my personal opinion, it is clearly a go at FUD, or it is a 3Par, Compellent or Xiotech effort to sail in XIVs wake and get attention themselves.

People need to understand that while XIV is based on Disks, too, this is almost the only thing making it look like a traditional, RAID-based, System.

This post is a disgrace to the complete forum which I now have spent some time browsing.
As customers tend to read the post but not the comments, the post should be deleted as uninformed.
It may have heavy impact on customer decisions and thus even legally attackable.
I am really pxxxxd as you see for having spent a lot of time dealing with this crap.

Any questions about XIV - ask.

But guys - do NOT take your 10% understanding of the box to market.

It is not a storage box as you know it, and some ways of thinking simply do not work here. Especially availability follows completely different paths, which we can and do explain to our customers.

Posted By:Yves Pelster| Wed Feb 24, 2010 11:00
I am a considering XIV. and I know that point 3 and 5 are a complete misuderstanding for XIV technology.

I think this article should be removed.

Posted By:Guy Smallwood| Sat Aug 14, 2010 07:19
Hello Guy...thanks for the comment. If you are considering XIV, please see this note (referenced in this article) which reflects the community's opinion as a whole:

http://wikibon.org/wiki/v/The_IBM_XIV_Storage_Array_Performance_and_Availability_Envelope

I'd be interested in thoughts on Guy's suggestion that this article should be deleted.
There are two schools of thought on this: 1) The article documents many of the criticisms associated with XIV and provides raw responses to these in the comments - which are further summarized by David Floyer's article or 2) The note does unnecessary damage to XIV and puts forth inaccurate information.

Please provide your thoughts on whether this article should be deleted.

Posted By:David Vellante| Sat Aug 14, 2010 07:30
HI David,

In the article point 3 concerns Where data resides.
While this is a consideration in other storage arrays the XIV where you are playing whack a mole with hot spots in the arrays. With the XIV; the XIV is the hot spot is one way of looking at it.

Point 5. This is comparing various RAID implementations against a reed-solomon implementation and then only using rules that apply for raid on both architectures.

So i would be lean more to your 2nd point. However I am new here and yeild to the senior members.

Posted By:Guy Smallwood| Sat Aug 14, 2010 07:42
David,
a lot of questions were asked, some were answered. I think that this page should stay; the comments allow people to draw their own conclusions. I don't believe that it does any real damage to XIV.

And do you really want to open the floodgates of vendors and their cronies asking for articles which offend them to be deleted?

Posted By:Martin Glassborow| Sat Aug 14, 2010 08:57

Revision ID	Author	Timestamp	Comment
30385	Dvellante	10 Aug 14 07:38:28
30384	Dvellante	10 Aug 14 07:37:59
30383	Dvellante	10 Aug 14 07:37:44
30381	Dvellante	10 Aug 14 07:23:23
26824	Dvellante	09 Dec 21 10:08:31
26817	David Floyer	09 Dec 21 08:03:03
26816	David Floyer	09 Dec 21 07:59:34
26815	David Floyer	09 Dec 21 07:55:41
26802	Dvellante	09 Dec 19 10:13:05
26801	Dvellante	09 Dec 19 10:12:36
26778	Dvellante	09 Dec 18 16:54:13
26640	Dvellante	09 Dec 11 07:26:19
26633	Dvellante	09 Dec 10 21:26:25
26632	Dvellante	09 Dec 10 21:12:41
26624	Meir	09 Dec 10 12:02:39
26391	Dvellante	09 Nov 30 19:08:27
26389	Bert Latamore	09 Nov 30 15:25:17
26373	Dvellante	09 Nov 28 16:08:45
26372	Dvellante	09 Nov 28 16:08:13
26371	Dvellante	09 Nov 28 16:07:23
26370	Rootwyrm	09 Nov 28 15:06:11
26369	Rootwyrm	09 Nov 28 15:04:29	Created page with 'IBM has been announcing enhancements and updates to its XIV storage platform at a fairly steady rate. The visibility of XIV has increased dramatically since it was q...'

Wikibon is a professional community solving technology and business problems through an open source sharing of free advisory knowledge.

Become a Member!

Login

Featured Research

Announcements

Technology Events

Product Maturity

Criticisms of the Fundamental Architecture

Sales Motion and Competitive Approaches

Conclusions and Opinions

Comments on '8 Reasons to Not Jump at XIV'

Post A Comment

most recent wikibon articles

latest wikibon blog posts

company profiles

wikibon community information