Contents |
Management Summary
Ryan Perkowski is UNIX infrastructure manager of a large financial institution running a mission-critical open-systems infrastructure environment. The backbone of his server installations are high performance pSeries servers from IBM running server virtualization with LPAR technology. The backbone of his storage are EMC DMX and CLARiiON systems, with a strong likelihood that he will install V-MAX and FAST 2. There are also significant numbers of Intel-based servers and other storage arrays from specific projects, mergers, and acquisitions.
Ryan and his team had four sets of storage management tools for the SAN, and Wikibon asked him about the tools, what they did for him, what were the best and worst features, and what was the overlap between the tools. He then gave us an insight into his future SAN management plans and how he has reduced the number of SAN management tools.
Ryan’s team uses SANscreen as the key storage management tool, replacing EMC’s Ionix Control Center (ECC) which they are discontinuing. They are using specific management tools from each vendor/array for control and for providing the raw data for SANscreen. NetWisdom is used for detailed performance analysis. They have found that that using the tools directly is more efficient and effective that using a front-end to try to give the illusion of a single screen. Figure one shows the relative roles of the different SAN management products at this financial services organization.
The financial analysis shows that the savings to the total storage budget for rationalizing the storage management software are about $0.5 million, 10% of the total storage budget. The net present value (NPV) for the four year savings is about $1.7 million.
SAN Management Tools
The tools that we discussed with Ryan were:
- ECC from EMC,
- SANscreen from NetApp,
- Storage Array & Switch specific management tools from each vendor,
- NetWisdom from Virtual Instruments.
ECC
ECC (renamed EMC Ionix Control Center) uses agents in the servers and can interface both with information extraction tools from the SAN components and interfaces with vendor array tools to initiate actions.
The advantages are that it is widely used, provides a good degree of customization, and is a good first stop for performance and management data. The ECC tool provides a common interface for management of all the arrays. It gives good array frame metrics (e.g., reads vs. writes).
The major disadvantages of ECC for Ryan’s team are that it attempts to cover a wide spectrum of functionality but does a poorer job in specific areas than more focused tools. Although ECC provided a common interface for all the different arrays installed, Ryan’s team found that on many occasions they have to use the native array tools and interfaces, and they soon found it easier to use the native tools all the time and stopped using ECC. Another practical disadvantage of ECC is that it requires agents, which often fail or go into a CPU runaway loop and require support (4 hours a day of a SAN management person was used to monitor failing agents in this 500-server environment). ECC performance data is limited; it does not help with data that reflects the user experience. Ryan says that all too often ECC gives answers that put the DMX array in the best light. Examples he gave were time to first bit instead of time to last bit (which is what matters to the application). Cache times and block size information are usually missing, and there is no audit capability or ability to manage change.
As a result, Ryan’s team is discontinuing ECC and using SANscreen, array vendor tools, and NetWisdom to provide a much higher level of SAN management that would be possible with just ECC.
SANscreen
SANscreen is a SAN management integrated suite provided by NetApp, and Ryan’s team has three components installed:
- Service Insight - provides near real-time visibility into the “storage services” delivered by the storage infrastructure. This is the base product.
- Service Assurance – assists audit and compliance with assuring path-based service policies, violations, and change-planning capabilities. This is especially useful to Ryan and his financial organization and saved significant time for his team whenever the auditors came.
- Application Insight – provides some sophisticated and unique methods of tying applications to the SAN resources and is especially useful to Ryan’s team as they try to link back to application user experience for SAL monitoring. Ryan’s team is also able to apply the detailed data from NetWisdom to the planning process for setting application-level SLAs.
SANScreen provides the overall SAN management function for Ryan’s team, as shown in Figure 1. It is used for monitoring SLAs, deciding where storage should be paced, analyzing historical trends, and deciding future array strategies and purchases. SANscreen provides the management glue between the individual array management tools and the very detailed analysis and history that NetWisdom provides.
Array & Switch Tools
All storage arrays and SAN switches have management tools that help to manage performance and configure the boxes correctly. Ryan’s group is well established and has good skills with these tools. Ryan believes that tools that sit on top of other tools can never have the depth of functionality of the component tools themselves. Given that they have to use the tools directly some of the time, Ryan’s group preferred to bypass a single central tool such as ControlCenter, and use the array tool itself directly.
NetWisdom
NetWisdom is a SAN I/O performance monitoring and troubleshooting product for complex, heterogeneous Fibre Channel (FC) Storage Area Networks. The FC network is tapped so that some of the light is sent to a hardware probe which analyses the data and creates a sophisticated performance database. In addition, software probes gather data from components (e.g., switch statistics) and the combination provides what Ryan refers to as a “scalpel” for zeroing into a problem area. It provides very detailed information of what exactly is going on in the storage subsystem from application to database to storage. It provides a very sophisticated level of troubleshooting and storage performance, and as a result of using it Ryan’s team has cultivated the trust of the server, application, and database groups concerning storage performance, as previously reviewed by Wikibon.
Financial Analysis
Table 1 shows the business case for rationalizing storage management software. The savings were the reduction in cost of software, the manpower cost of monitoring the agents for the software in 500 servers, and the manpower savings from eliminating software. The overall impact on the yearly budget is about $0.5 million. This represents 10% of the total storage budget. The net present value using 5% interest rate for the four year savings is about $1.7 million.
Conclusions
SANscreen has been very successful in the system management quadrant helping with audit management and tying SAN management back to application SLAs. In addition it provides the capacity management and overall storage management tools.
NetWisdom provides the performance data that helps system performance. For Ryan, the NetWisdom product has been successful, making him and his staff much smarter about the total system impact of storage, and gaining the respect of the server and database teams. Software probes are bought routinely as part of a major software development project to shorten development cycles by providing the detailed performance information. The cost of NetWisdom per terabyte monitored is higher, and Ryan’s team focuses it on key high performance databases and applications, with the ability to turn on a probe to investigate a particular troublespot.
Ryan’s team continues to use the array and switch based tools directly and sees no need for other tools to implement day-to-day storage management.
Overall, the most effective solution for this financial services company is a combination of SANscreen, NetWisdom, and standard array tools. As Figure 1 shows, there is minimal overlap between products. The cost impact of rationalizing the storage management software is a reduction in storage cost of 10%, or $.05 million impact on the yearly storage budget.The NPV for the four year savings is about $1.7 million.
Footnotes: Ryan Perkowski has since moved to become a senior consultant with Virtual Instruments.
Legal: © Wikibon 2010. This document is copyright protected by Wikibon and does not fall under the GNU general license terms for Wikibon.org. Links to this article from external sources are allowed, however any other re-distribution of this content for commercial purposes is strictly prohibited. Please contact Wikibon for more information.
Wikibon case studies are developed independently and their development is not initiated for or funded by any single company. Wikibon reports actual customer experiences and results with no attempt to emphasize any one vendor’s strengths or weaknesses.
Read the full disclaimer.