Contributing author: Bill Mottram

Introduction

Wikibon Energy Lab Validation Reports are designed to assist customers in understanding the degree to which a product contributes to energy efficiency. The four main goals of these studies are to:

1. Validate the hardware energy efficiency of a particular technology as compared to an established baseline.
2. Asses the potential contribution of software technologies to power savings, and validate the actual contribution in real world installations.
3. Quantify the contribution of the hardware and software technologies to a green data center.
4. Educate business, technology, and utility industry professionals on the impact of technologies on reducing energy consumption.

Our objective is to identify not only the hardware energy consumption but also the often overlooked and hard-to-quantify green software aspects of technologies. Wikibon Energy Lab Validation Reports are submitted to utilities such as Pacific Gas & Electric Company as part of an energy incentive qualification process.

Wikibon Energy Lab defines and validates the hardware testing procedures to determine the energy consumed by specific products in various configurations. As well, Wikibon reviews actual customer results achieved in the field to validate the effectiveness of these technologies based on real-world field-data analysis. These proof points are mandatory for the utility company to qualify a specific vendor's technology for energy incentives.

Wikibon Energy Lab Reports are not sponsored. Rather they are deliverables required by PG&E and other utilities as part of an incentive qualification process. As part of its Conserve IT Program, Wikibon is paid by the vendor to perform services associated with securing incentive rebates from utilities for end customers that acquire the vendor's technologies. To ensure this process is completely independent, Wikibon lab and field results are sometimes vetted by a third party engineering firm hired by PG&E or other utilities.

Wikibon only produces Lab Validation Reports for technologies that have been qualified for rebate incentives by PG&E or other utilities and have passed strict utility company guidelines. By adhering to this criterion, Wikibon assures its community of the independence of these results.

Executive Summary

A storage array consists of a large number of disk drives together with a storage controller. Many servers can be connected to a storage array, which allows any of the disk drives to be used by any server. Data is read from the disk and written to the disk with a read/write heads (Figure 1). The disk spins at between 7,200rpm and 15,000rpm. The heads are moved across the recording surface of the disk to different tracks where the data is stored.

Figure 1 - Components of Disk Drive

In traditional storage arrays all the disks are spinning all the time, even when they were not reading or writing data. About 80% of the energy consumed by a storage array goes to spinning the disks and moving the read/write heads.

Nexsan has developed software and hardware features called AutoMAID, a sophisticated technology to turn off individual drives in an array when they are not being used. AutoMAID has 3 levels, saving progressively more power. When a disk drive goes into idle state for 10 minutes, the drive goes into AutoMAID level 1. After 10 more minutes with no I/O activity, it goes into AutoMAID level 2, and after 10 more minutes into level 3. The delay is added to avoid “hunting” between AutoMAID levels, which would increase power momentarily when the disk became active.

Figure 2 below shows power required for idle disks and for each level of AutoMAID. These figures were determined at Nexsan’s laboratory on Nexsan equipment under the guidance and supervision of Wikibon Energy Labs. Wikibon certifies the accuracy of the data.

Figure 2 - Power Required for Hitachi 1 Terabyte Disk Drive with Different Levels of AutoMAID

The time that AutoMAID is invoked can be determined by running a standard report from each Nexsan array that itemizes the percentage of time each level of AutoMAID is used, as illustrated in Figure 3.

Figure 3 - Report on AutoMAID Performance from a SATABeast Array (Source: Nexsan, 2008)

Wikibon has developed a Wikibon Energy Lab Power Calculator for AutoMAID to determine the power savings and the rebates. If the data from Figure 3 is entered into the calculator, the savings are shown in Table 1. In this example, the savings would by 19,764kWh/year (39%).

Technical Overview

MAID technology was first introduced by Copan. In the Copan MAID storage technology a maximum of 25% of the total number of drives can be running (spinning) at any given time. This technology is suitable for workloads such as backup and archiving that use the drives in the same manner as tape drives. In these applications almost all the I/O activity is sequential; although this architecture imposes significant delays when accessing data on a randomly accessed idle disk because a drive has to be powered down and another powered up, the number of these data accesses are few and the delay can be tolerated by the application and users. However many if not most applications require the ability to make random access to all data on all disks, so the COPAN implementation of MAID cannot be applied generally.

Drive manufacturers such as Hitachi have recently introduced multiple powered down states. This was first introduced for laptops and has now been extended into enterprise SATA drives that are used in enterprise storage arrays. For example, Hitachi allows a drive to be in one of four power states:

• Level 0:
  • Normal operation at 7,200 rpm with heads loaded (un-parked)
• MAID level 1:
  • Heads Unloaded (parked, reduces wind resistance on heads)
  • 15% to 20% power savings
  • Sub-second recovery time
• MAID level 2:
  • Heads Unloaded,
  • Slows to 4000 rpm
  • 35% to 45% power savings
  • 15 second recovery time
• MAID level 3:
  • Stops spinning (sleep mode; powered on)
  • 60% to 70% savings
  • 30 to 45 second recovery time

Seagate has a SATA drive that allows the drive just to be powered off (MAID level 3), and Western Digital has a “Green Drive” that revolves slower (5,400rpm) and can also park the heads (MAID level 1). Nexsan currently deploys Hitachi and Seagate drives in its SATA storage arrays and may introduce other disk manufactures later.

Nexsan has developed software and hardware features called AutoMAID that allows control of when and how quickly each level is evoked and reports the amount of time that each of the drives is in each MAID state to the user.

Wikibon defined and reviewed the power measurements on the drives available on Nexsan’s SATABeast & SATABoy storage arrays, with and without the use of AutoMAID. Wikibon found that the key variables that determined the power usage were the type of controller (SATABeast or SATABoy), the number of controllers (dual or single) and the level of AutoMAID engaged. Table 2 gives the savings per drive from AutoMAID, which were derived from the detailed component measurements taken. The section below called “Power Measurement Methodology” gives additional detail.

Wikibon and Nexsan conducted a rigorous analysis of the benefits that Nexsan customers had achieved with the use of AutoMAID. The analysis showed that when AutoMAID is installed in a suitable environment, the disks will be stopped on average more than 50% of the time. Full details are given in the section “AutoMAID Effectiveness in the Real World” below. These savings can be verified with a report from each array that itemizes the savings, as illustrated in Figure 4.

Used in combination, these results represent a complete and accurate reflection of the actual power that would be consumed by Nexsan’s storage array products in a data center environment with and without AutoMAID, and the likely power efficiency savings that AutoMAID will achieve.

Figure 4 - Report on AutoMAID Power Efficiency from SATABeast/SATABoy Arrays (Source: Nexsan, 2008)

Wikibon Energy Labs developed the Wikibon Energy Lab Power Calculator for AutoMAID. The methodology used was based on PG&E methodology for calculating incentives. The data in this report is used in the calculator. The detailed inputs and outputs are shown in Appendix III.

Measurement Methodology for Power

The Nexsan SATABeast and SATABoy storage arrays are built from standard components and enable the power of different configurations to be calculated. The measurements were made at the component level; measurements were taken of the array controller (either single or dual) and the drives. Included in the measurements was the performance of the drives when idle and under a benchmark load and when different levels of AutoMAID were invoked. Table 2 above gives the impact of the measurements. Table 3 gives power consumption of the key components in the SATABeast and SATABoy configurations.

The measurements on the array controllers for AutoMAID level zero (i.e., normal operation) were made with a mixed workload of sequential reads, random reads, sequential writes, random writes, and idle state. All the measurements for AutoMAID levels one through three were made with the system in idle state.

The results of the three workloads were averaged to produce an overall figure of power consumption for each of the components of the Nexsan arrays. Wikibon believes that the benchmarks used and the power measurements made were done professionally and in good faith and that this measurement represents a good and reasonable estimate of the power consumed in the real world applications across a number of drives for “typical” combinations of applications found in a data center.

4.1 Equipment Measured

The Nexsan storage array and components measured were:

• SATABeast Dual Controller - GN60
• Controllers - 000402FC400D / 000402FC312B
• Chasis - 01754065
• 512MB cache per controller
• 42 x Drives
• 4 x 10 disk RAID 5 sets
• 1 volume per RAID set
• 2 x Hot spares
• DCNR - one Volume per port or Single controller 2 volumes per port

The load testing host equipment was:

• Dell 1850, Dual 3Ghz Xeon CPU
• 2 x Q-Logic 8GBp/s Dual port HBA's
• 4 or 2 ports Direct Attached in dual controller or single controller respectively

The drives measured were:

• Hitachi HUA721010KLA330 7200rpm 1TB, 750GB, 500GB SATA drives
• Seagate ST31000340NS 7200rpm 1TB, 750GB SATA drives
• Western Digital WD1000FYPS 5400rpm 1TB (not included in results as not yet announced by Nexsan)

4.2 Measuring Equipment Used

Amp measurements were made with a Fluke 600 A AC True-rms Clamp Meter 335, SN – 95980112. Volt measurements were made with a Fluke 189 True-rms Multimeter.

4.3 Location of Testing

The testing was done in Nexsan’s facility at Application Support Lab 1, Nexsan Technologies, 302 Enterprise Street, Escondido, CA 92029 USA (tel: 866.463.9726). The testing was overseen by Nitsan Tucker, at the above address and phone number.

Drive & Controller Measurement Results

The chart below summarizes the power consumption results from the drive and controller measurements taken by Nexsan. The detailed measurements are shown in Appendix II. The results of the analysis allow the power consumption of each of the drive components (drive chassis and each disk drive type) to be calculated. The final results are shown in Table 3. This table is used to power the incentive calculator for AutoMAID.

The characteristics of the benchmarks run to give the measurements under load are given in Appendix I below.

The controller and drive figures under load were used to determine the power consumption that would have been required without AutoMAID, the base case. The idle drive figures were used to calculate the reduction of power requirements as a result of AutoMAID and calculate the overall power consumption with AutoMAID. The difference between the two gives the direct drive power savings derived from AutoMAID.

AutoMAID Effectiveness in the Real World

Nexsan did a detailed study of eight customers who were analyzing the deployment of AutoMAID across multiple SATABeast arrays. The results of the study are shown in Table 4 below. What was interesting was the high variability of when AutoMAID could be deployed. The most interesting statistic was that AutoMAID could be deployed at some level 58% of the time. The other interesting statistic was that 80% of the time that AutoMAID was deployed, it was deployed at level 3, the highest level of power saving.

Table 4 - Study of Nexsan SATABeast Storage Users and deployment of AutoMAID

The following screen shots of the AutoMAID Utilization Tracking Log Screen were taken from eleven (11) production Nexsan SATABeasts and SATABoys with AutoMAID installed. They show the level of impact on energy efficiency that AutoMAID has had on these arrays. The results vary from very little impact to very high impact.

Figure 5 shows the log from a 42TB (42 disk) SATABeast system used for running various test in a LB. The 90 day log shows high energy saving, with the disks stopped 85% of the time.

Figure 5 -SATABeast System 42TB used for running various tests in a LAB – High Energy Saving – 90 day log

The array reflected in Table 5 was being used in a test environment. Over a 45 day period the disks in the system were stopped (AutoMAID level 3) 96% of the time. This is characteristic of many test environments, with long inactivity and occasional bursts of critical work.

Table 5- SATABeast 42TB System used for running customer application simulations in a LAB – High Energy Saving – 45 day log

The report from the array in Table 6 shows a smaller high-end production environment with critical databases where AutoMAID was not relevant. The disks were active 93% of the time, and the energy savings from AutoMAID were low to negligible.

Table 6 - 4TB System used in high end production environment with critical databases – Low Energy Saving – 30 day log

The array log in Table 7 shows the power saving results from a file and image content system for long-term archiving. AutoMAID is highly relevant. The disks were stopped 89% of the time, and the energy savings from AutoMAID were high.

Table 7 - SATABeast 42TB System used for holding file and image content for long term– High Energy Saving – 120 day log

The array log in Table 8 shows the power saving results from a small-company mixed-application environment with a mix of very active and non-active systems and a file and image content system for long-term archiving. AutoMAID is highly relevant. The disks were stopped 30% of the time, and the energy savings from AutoMAID were medium to low.

Table 8 - 14TB System used in daily production environment for a small company with a mixed application environment - Medium to Low Energy Saving – 60 day log

The array log in Table 9 shows the power saving results from an archiving system. Again, AutoMAID is highly relevant. The disks were stopped 85% of the time, and the energy savings from AutoMAID were very high.

Table 9 - SATABoy 14TB System used for holding archive data– High Energy Saving – 50 day log

The array log in Table 10 shows the power saving results from a backup and recovery system for a full production environment. This type of system is characterized by very significant activity for short periods of time, followed by inactivity for most of the time. This is an excellent environment for AutoMAID, and the disks were stopped 81% of the time. The energy savings from AutoMAID were very high.

Table 10 - SATABeast 42TB System used in backup environment- holding backups of production environments – High Energy Saving – 65 day log

The array from the log in Table 11 shows the power saving results from a system designed to test video games systems. This and other resting environments are excellent for AutoMAID, and the disks were stopped 90% of the time. The energy savings from AutoMAID were very high.

Table 11 - SATABoy 14TB System used for testing video systems– High Energy Saving – 75 day log

The array from the log in Table 12 shows the power saving results from a system newly installed and at rest. This happens frequently before data and applications are migrated over to the new system. This of course is a tailor-made environment for AutoMAID with the disks stopped 99% of the time. The energy savings from AutoMAID were very high.

Table 12 - SATABoy 14TB System at rest – powered but not being used – High Energy Saving – 2 week log

The array from the log in Table 13 shows the power saving results from a long-term content archiving system. As usual, this is a suitable environment for AutoMAID with the disks stopped 94% of the time. The energy savings from AutoMAID were very high.

Table 13 - SATABoy 14TB System used for holding content for long term– High Energy Saving – 50 day log

The array from the log in Table 14 below shows the power saving results from a testing and simulation environment. This again is a suitable environment for AutoMAID with the disks stopped 81% of the time. The energy savings from AutoMAID were high.

Table 14 - SASBoy 4TB System used for testing and simulations– High Energy Saving – 40 day log

Overall the percentage of time that AutoMAID was able to save power by stopping the disks varied from 3% to 99%. The majority of the environments were well over 50%.

Some environments impose software constraints that prevent deployment of AutoMAID levels 3 and sometimes 2. Almost all environments should be able to take advantage of AutoMAID level 1.

Appendix I – Benchmark I/O Definition

Table 15 provides definition of the I/O loads that were run against the components. The average I/O block size is 1 MB, which is a reasonable estimate of what is found in real-world usage of Nexsan SATABeast and SATABoy storage array deployments. This benchmark was run against all the array components and the power measurement (volts and amps) were taken. All the array components were also run in idle state, i.e., no I/O.

The overall mix for components load was defined as 2/3 load benchmark (as defined in Table 15), and 1/3 idle measurements. Wikibon confirms that this is an accurate estimate of the power consumption that would be found in real-world conditions when under load.

Appendix II – Detailed Measurements (VA)

Appendix III – Description of Wikibon Energy Lab Power Calculator for AutoMAID

The information for the Nexsan customer applying for the PG&E incentive is input in the blue boxes as illustrated in Figure 6.

Figure 6 - Wikibon Energy Lab Power Calculator Input - Customer Information

The equipment that will be installed is input in the blue boxes as illustrated in Figure 7.

Figure 7 - Wikibon Energy Lab Power Calculator Input - Storage System Information

The information about UPS in the data center where the storage equipment will be installed is input in the blue boxes as illustrated in Figure 8 below.

Figure 8 - Wikibon Energy Lab Power Calculator Input - UPS Information

The information about the cooling in the data center where the storage equipment will be installed is input in the blue boxes as illustrated in Figure 9.

Figure 9 - Wikibon Energy Lab Power Calculator Input - Cooling System Information

The information about the AutoMAID usage on the storage equipment will be installed is input in the blue boxes as illustrated in Figure 10. On initial submission, the usage is estimated using the pull-downs for daytime, evening and weekend AutoMAID usage. When the storage system is installed, the data from the AutoMAID Performance report (see Figure 3 for an example) can be input.

Figure 10 - Wikibon Energy Lab Power Calculator Input - AutoMAID Usage Information

The output from the Wikibon Energy Lab Power Calculator is illustrated in Figure 11. The Power Calculator provides a more complete readout when calculating specific incentive amounts.

[[Image:Nexsan Energy Efficient AutoMAID Technology image 30.gif|center|thumb|500px]|Figure 11 - Calculator Output]

Appendix IV – Description of Nexsan AutoMAID

In traditional storage arrays, disk drives continue to rotate all the time, which uses power and require cooling. High performance FC disks rotate at 10,000 or 15,000 rpm. Almost all high density disks (SATA disks) rotate at 7,200 rpm. Nexsan's AutoMAID (Automatic Massive Array of Idle Disks) energy saving technology transparently places SATA disk drives into different idle states to reduce power and cooling costs.

AutoMAID is granular to an individual drive or RAID set and offers multiple levels of energy savings. AutoMAID is user selectable, enabling users to determine the right trade-off between response time performance and energy savings.

Drive manufacturers have recently introduced multiple powered down states. This was first introduced for laptops, and has now been extended into enterprise SATA drives that are used in enterprise storage arrays. For example, Hitachi allows a drive to be in one of four power states:

• Level 0:
  • Normal operation at 7,200 rpm with heads loaded (un-parked)
• MAID level 1:
  • Heads Unloaded (parked, reduces wind resistance on heads)
  • 15% to 20% power savings
  • Sub-second recovery time
• MAID level 2:
  • Heads Unloaded,
  • Slows to 4000 rpm
  • 35% to 45% power savings
  • 15 second recovery time
• MAID level 3:
  • Stops spinning (sleep mode; powered on)
  • 60% to 70% savings
  • 30 to 45 second recovery time

Seagate has a SATA drive that allows the drive just to be powered off (MAID level 3), and Western Digital has a “Green Drive” that revolves slower (5,400rpm) and can also park the heads (MAID level 1). Nexsan currently deploys Hitachi 500GB, 750GB and 1TB drives, together with 750GB Seagate drives in their SATA storage arrays, and may introduce other disk manufacturers later.

We expect drive manufactures to introduce additional levels of power savings and much faster recovery times to access data. The faster recovery times will be particularly useful, as it will avoid potential impacts on the application.

Figure 14 illustrates the cycle of moving from a disk or disks in sleep mode, through a request for data and reactivation of the drives, through the cascade down through the MAID levels until the disk is in sleep mode again.

Figure 11 - AutoMAID in Action (Source: Nexsan, modified by Wikibon)

Nexsan Storage Array Specifications

SATABeast Technical Specifications

• Up to 42 SATA drives in a 4U rack configuration
• Dual 4Gb Fibre Channel SFP LC host ports per controller
• Dual iSCSI ports per controller
• Single or Dual active/active controllers
• Dual Power Supplies

Figure 12 - Nexsan SATABeast (Source:Nexsan, downloaded 9/28/2008)

SATABoy Technical Specifications

• Dual 2Gb Fibre Channel ports and dual iSCSI ports per controller or quad SCSI ports on a single controller
• Single or dual active/active controllers
• Up to fourteen SATA drives in a 3U rack configuration
• Dual Power Supplies

Figure 13 - Nexsan SATABoy (Source:Nexsan, downloaded 9/28/2008)

Action Item:

Footnotes: Disclaimer

This report was prepared by Wikibon. Reproduction or distribution of the whole, or any part, of the contents of this document without written permission of Wikibon is prohibited. Neither Wikibon nor any of its employees make any warranty or representations, expressed or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any data, information, method product or process disclosed in this document, or represents that its use will not infringe any privately-owned rights, including, but not limited to, patents, trademarks, or copyrights.

This report uses preliminary information from vendor data and technical references. The report, by itself, is not intended as a basis for the engineering required to adopt any of the recommendations. Its intent is to inform the customer of the potential cost savings. The purpose of the recommendations and calculations is to determine whether measures warrant further investment of time and/or resources.