Re-produced from Energy Star/EPA.
Contents |
ENERGY STAR® Enterprise Storage
Draft Specification Framework June 4, 2009 Please send comments to Storage@energystar.gov no later than Friday, July 3, 2009
Overview
This document describes the key building blocks that form the basis for every ENERGY STAR specification; these items are intended to provide the framework around which the EPA can develop an effective energy efficiency program for Enterprise Storage. The principal objectives for this ENERGY STAR specification are threefold:
(1) to encourage widespread adoption of appropriate hardware and software strategies to improve energy efficiency in enterprise storage systems,
(2) to provide purchasers with the means to identify the most energy efficient enterprise storage solutions for their specific end-use application, and
(3) to provide tools and information to designers and mangers looking to improve the efficiency of data center operations.
The purpose of each building block is provided under the subheadings below, along with EPA’s preliminary thoughts on how each may ultimately be incorporated into the Version 1.0 Enterprise Storage specification. At the end of each section are a series of questions aimed at generating discussion about the proposed approach. Please note that this document is not intended to be a comprehensive review of the ENERGY STAR perspective on enterprise storage, rather it serves as a starting point for EPA’s specification development efforts.
Stakeholders are encouraged to provide feedback on the specific concepts and definitions presented in this document, and are also welcome to submit comments of a more general nature. Communication between EPA and industry stakeholders is critical to the success of the ENERGY STAR program, especially in this early stage of the specification development process. Any and all creative suggestions for improvements to the basic ENERGY STAR approach outlined in this document will be considered for inclusion in subsequent draft and final specifications. ENERGY STAR representatives are available for additional technical discussions with interested parties at any time during the specification development process. Please contact Steve Pantano, ICF International, at spantano@icfi.com to arrange a meeting.
Building Block #1: Definitions
a. Purpose: Establish a set of definitions to explicitly describe which products are covered by the specification and which are not, and to clearly differentiate between Enterprise Storage and other ENERGY STAR product categories. Provide definitions for operational modes, key components and sub-classes of products, etc. Note that a product may not be qualified as ENERGY STAR under more than one specification – manufacturers must select the product category that best describes the product they wish to qualify.
b. Initial Approach: EPA prefers to make use of existing definitions that are generally accepted by industry. In cases where industry accepted definitions are not available or appropriate, EPA will work with stakeholders to develop acceptable definitions.
c. Preliminary List of Definitions1,2:
a. Storage Hardware:
1. Storage Media: The physical material that stores data. Storage Media used in Enterprise Storage applications may be electrical (e.g. solid state drive), magnetic (hard disk drive, tape drive), or optical (optical disc drive).
2. Storage Product: A system composed of integrated storage controllers, storage media and software that provides data storage services to one or more Computer Servers and/or other devices. While Storage Products may contain one or more embedded processors, these processors do not execute user-supplied software applications but may execute data-specific applications (e.g. data replication, backup utilities, data compression, install agents, etc.). This definition is specifically intended to exclude aggregating storage elements such as RAID array subsystems, robotic tape libraries, filers, and file servers.
3. Storage Controller: TBD
4. Storage Product Family: A group of Storage Product configurations where every configuration includes base components with the same or similar technical specifications and power specifications.
5. Blade System: A system composed of both a Blade Chassis and one or more removable Blade Servers or Blade Storage units. Blade Systems are designed as a scalable solution to efficiently package and operate multiple Computer Servers or Storage units in a single enclosure, and are designed for technicians to be able to easily add or replace hot-swappable boards in the field.
6. Blade Chassis: An enclosure containing shared resources for the operation of Blade Servers and Blade Storage units. These resources may include power supply(s) for power conversion, shared storage, and hardware for DC power distribution, thermal management, system management, and network services. A Blade
1 EPA is interested in information from stakeholders on industry standard definitions for those terms marked as TBD. Stakeholders are also encouraged to provide additional suggestions or clarifications regarding the proposed definitions.
2 The proposed definitions were compiled from a variety of reference sources, including the ENERGY STAR specification for Computer Servers and the Storage Networking Industry Association (SNIA) Web site. Chassis features multiple slots which can be populated with blades of different types.
7. Blade Storage: A storage-specific element that relies on shared resources (e.g., power supply, cooling, etc.) for operation. Blade Storage units are designed to be installed in a Blade Chassis, are hot-swappable and are incapable of operating independent of the chassis.
8. I/O Device: Provides data input and output capability to the Storage Product from other networked devices. Examples of I/O Devices include: Ethernet devices, InfiniBand devices, external RAID/SAS controllers and Fibre Channel devices. b. Storage Characteristics:
1. Capacity: The sum of the raw unformatted, uncompressed capacity of all Storage Media installed in a Storage Product.
2. Direct-connected: Storage designed to be under the control of a single host, or multiple hosts, in a non-shared environment.
3. Network-connected: Storage designed to be connected to a host via a network connection (e.g., Ethernet, InfiniBand, and Fibre Channel).
c. Other Data Center Hardware:
1. Computer Server: A computer that provides services and manages networked resources for client devices, e.g., desktop computers, notebook computers, thin clients, wireless devices, PDAs, IP telephones, other Computer Servers and other networked devices. Computer Servers are designed to respond to requests and are primarily accessed via network connections, and not through direct user input devices such as a keyboard, mouse, etc.
2. Blade Server: A Computer Server consisting of, at minimum, a processor and system memory that relies on shared resources (e.g., power supplies, cooling, etc.) for operation. Blade Servers are designed to be installed in a Blade Chassis, are hot-swappable, and are incapable of operating independent of the chassis.
3. Network Equipment: A product whose primary function is to provide data connectivity among devices connected to its several ports. Data connectivity is achieved via the routing of data packets encapsulated according to a standard protocol. Examples of network equipment commonly found in data centers are routers and switches.
d. Power Supplies:
1. Power Supply Unit (PSU): A self-contained component which converts an AC or DC voltage input to one or more DC voltage outputs for the purpose of powering the Storage Product. A Storage Product PSU must be separable from the main system and must connect to the system via a removable or hard-wired male/female electrical connection, cable, cord or other wiring.
2. AC-DC Power Supply: A power supply which converts line voltage AC input power into one or more different DC voltage outputs.
3. DC-DC Power Supply: A power supply which converts a DC voltage input to one or more DC voltage outputs. Any DC-to-DC converters (also known as voltage regulators) internal to the product and used to convert low voltage DC (e.g. 12 V DC) into other DC voltages are not considered DC-DC power supplies under this specification.
4. Single-Output Power Supply: A power supply with one primary DC output. Single-output power supplies may include one or more standby outputs which remain active whenever connected to an input power source. The combined power from all outputs other than the primary and standby outputs shall be no greater than 20 W in a Single-Output Power Supply. Note: Power supplies with multiple outputs at the primary voltage are considered a Single- Output Power Supply, unless these outputs are either, (1) generated from separate converters or have separate output rectification stages, and/or (2) have independent current limits.
5. Multi-Output Power Supply: A power supply with more than one primary DC output. Multi-output power supplies may include one or more standby outputs which remain active whenever connected to an input power source. The combined power from all outputs other than the primary and standby outputs is greater than 20 W in a Multi-Output Power Supply.
e. Operational States:
1. Idle: An operational state in which the operating system and other software have completed loading and the Storage Product is capable of completing workload transactions, but no active workload transactions are requested or pending by the system.
2. Active: TBD
3. Maximum: TBD
4. Full Load: TBD
d. Questions for Discussion:
1. How are Active, Idle, Maximum, and Full Load states defined in the industry?
2. What are the critical factors in determining if a Storage Product is idle? Are there other accepted terms for these states?
3. Are there other Operational States specific to Storage Products that will need to be defined in this specification? For example, how should EPA address data maintenance functions that may occur in the background while a product is in an idle state?
4. Are there other definitions required to identify certain types of Storage Products or components?
5. Are there any other sources that the EPA should review for variations of, or additions to, this list of definitions?
6. Are the power supply definitions for Computer Servers also appropriate for Storage Products? If not, how do these PSUs differ?
7. How prevalent are DC powered Storage Products? Do any Storage Products use power supplies directly integrated into the main system (i.e. not separable from the main system), or are PSUs always stand-alone (external) hardware?
Building Block #2: Eligible Product Categories
a. Purpose: Identify specific product categories to be covered by the specification based on the agreed upon definitions in Building Block #1. Clearly defined categories are particularly important where requirements may not be appropriate for products that perform distinctly different functions. It is also important to identify product types that are not eligible for ENERGY STAR qualification for reasons such as: use of proprietary technologies; limited availability of data; lack of differentiation with regards to product efficiency; or niche markets.
b. Initial Approach: EPA intends to develop or adopt a taxonomy to segment the enterprise storage market3. Once the market taxonomy is accepted, EPA will assess the enterprise storage market to identify opportunities to apply consistent ENERGY STAR criteria across multiple market segments. EPA’s intention in developing the Version 1.0 specification is to cover as much of the Enterprise Storage market as can be reasonably addressed in a timely manner, while maximizing the opportunities for energy savings. To this end, EPA may propose a tiered approach for the specification, concentrating on the greatest opportunities for energy savings in Version 1.0, and expanding the scope in later versions of the specification. Market segments with the greatest opportunity for energy savings will likely be targeted in the first specification release, while segments with less opportunity may be included in subsequent specification revisions.
EPA intends to explore the following types of Storage Products for inclusion in the Version 1.0 specification:
• Direct Attached Storage (DAS) and Network Attached Storage (NAS, SAN)
• Hard Disk, Tape, Optical, Solid State, and hybrid Storage Media
• Blade Storage
3 EPA is aware of several taxonomies that have been developed by storage industry stakeholders and intends to leverage this work to the extent possible. One robust example was developed by the Storage Networking Industry Association (SNIA) Green Storage Initiative, as part of the “Green Storage Power Measurement Specification.” This document can be downloaded free of charge from www.snia.org/forums/green/.
c. Questions for Discussion:
1. What are some additional means of segmenting the storage market?
2. Are there any upcoming technologies or product types in development which are not included on the list provided in section (b) and should be considered for inclusion in ENERGY STAR?
3. What portion of the enterprise storage market has the greatest need for an ENERGY STAR label to help customers identify the most efficient products? In what market segments can EPA expect to garner the most energy savings?
Building Block #3: Energy Efficiency Criteria and Test Procedures
a. Purpose: Once it is determined which products will be included in the ENERGY STAR specification, the next step is to identify metrics for energy efficiency performance. Metrics may address the efficiency of key components (e.g., power supplies), operational modes (e.g., Idle state), and/or whole system energy efficiency. Efficiency metrics must be supported by generally accepted test procedures. Further, while one efficiency metric across a broad range of products is preferable, EPA will evaluate the need to develop unique requirements for segments of the market if it can be shown that key product functions or purposes require additional energy. In other words, EPA does not intend to develop unique energy efficiency criteria for different storage products based only on differences in the underlying technology.
The efficiency metric(s) will ultimately be used to determine ENERGY STAR qualification. The ENERGY STAR program strives to set requirements such that 25% of the products available in the market at the time the specification becomes effective will be able to qualify.
Notes on Value Added Resellers (VAR):
In some cases, Storage Products may be shipped from the Original Equipment Manufacturer (OEM) to a VAR that then configures the device for sale to the end user. It is important to EPA that a product qualified for ENERGY STAR by an OEM, processed by a VAR, and ultimately installed and used at a customer site continues to meet ENERGY STAR requirements. The ENERGY STAR program for Computer Servers addressed this situation in the following manner: In order for the VAR to sell the Computer Server as ENERGY STAR qualified under the OEM brand name, one of two conditions must be met:
a. The end configuration sold by the VAR must have been originally qualified by the OEM; or
b. In the case that the end configuration has not been qualified by the OEM, the VAR must become an ENERGY STAR partner, and test and qualify the configuration. OEM partners selling Computer Servers to VARs must provide the VAR with a list of qualified configurations for that model, using approved components, which have been initially qualified and reported to EPA by the OEM Partner.
b. Initial Approach: EPA intends to adopt or develop a test procedure to measure the energy consumed by a Storage Product while performing a realistic and representative workload. This workload will likely include power measurements under both Idle and Active conditions. EPA would like to adopt an industry standard procedure, but if no suitable procedure exists EPA will work with stakeholders to develop one.
Once a test procedure has been identified, EPA will begin an effort to collect and analyze test data from product tests performed by manufacturers. This data collection is critical to the success of the program, since the test data will be used to inform the development of the final ENERGY STAR performance criteria. EPA understands that there are many software-based approaches to improving the energy efficiency of storage products. The benefits of virtualization, data deduplication, and other software-based data management techniques are well documented. These software solutions, perhaps even more so than the hardware itself, are heavily customized for specific customers and applications. Achieving maximum efficiency gains is highly dependent upon proper software architecture, implementation, operation, and maintenance by individual users. A key objective of the ENERGY STAR specification is to identify and reward storage solutions that seamlessly integrate software and hardware efficiency strategies that provide verifiable benefits without user intervention. EPA recognizes that electronic equipment in general, including that used in enterprise data centers, is typically used at utilization levels considerably less than the rated capacity of the equipment. For Storage Products this includes both the amount of data stored in a product as well as the data throughput into and out of the unit. Efficient operation at such part-load conditions is important to EPA.
In addition to system-level energy efficiency requirements, and based on lessons learned from the Computer Servers development process, it is EPA’s intention to develop energy efficiency criteria for storage power supplies. Power supplies have provided a significant savings opportunity in other ENERGY STAR specifications and are worthy of consideration for this product category. EPA intends to adopt the test procedure already used in the Computer and Computer Server specifications (i.e., the Generalized Internal Power Supply Efficiency Test Protocol maintained by the Electric Power Research Institute). EPA is interested in leveraging the work already performed on the ENERGY STAR Computer Server specification, as well as what has been done by industry groups and programs such as the Climate Savers Computer Initiative and 80plus. EPA will look to harmonize energy efficiency requirements, where appropriate, to minimize the number of different and competing standards in the marketplace. EPA is also considering the adoption of a “Net Power Loss” approach for power supply efficiency. This approached is also being considered for Tier 2 ENERGY STAR Computer Server requirements. Under this approach, EPA would either specify the allowable absolute power loss (in watts), or specify the minimum efficiency for a power supply under actual use conditions, rather than at benchtest load conditions (i.e. efficiency at idle and maximum power load of the storage system rather than at 10%, 20%, 50%, and 100% of the PSU rated output). Net Power Loss would address PSU sizing and redundancy considerations, whereas the current PSU test procedure for Computers and Computer Servers simply measure efficiency in a bench-test with no consideration of how a PSU is installed and operated in the field. Power supply efficiency, power factor criteria, and implementation notes from the Version 1.0 Computer Server specification, have been included below:
Table 1: Efficiency Requirements for Computer Server Power Supplies Power Supply Type Rated Output Power 10% Load 20% Load 50% Load 100% Load Multi-Output (AC-DC & DC-DC) All Output Levels N/A 82% 85% 82% ≤ 500 watts 70% 82% 89% 85% > 500 - 1,000 watts 75% 85% 89% 85% Single-Output (AC-DC & DC-DC) > 1,000 watts 80% 88% 92% 88% Table 2: Power Factor Requirements for Computer Server Power Supplies Power Supply Type Rated Output Power 10% Load 20% Load 50% Load 100% Load DC-DC (All) All Output Levels N/A N/A N/A N/A AC-DC Multi-Output All Output Levels N/A 0.80 0.90 0.95 ≤ 500 watts N/A 0.80 0.90 0.95 > 500 - 1,000 watts 0.65 AC-DC Single-Output 0.80 0.90 0.95 > 1,000 watts 0.80 0.90 0.90 0.95 Table 3: Input Conditions for Power Supply Efficiency Testing Power Supply Type Input Test Conditions AC-DC Single-Output 230 Volts, 50Hz or 60 Hz AC-DC Multi-Output 115 Volts, 60 Hz and/or 230 Volts, 50Hz or 60Hz DC-DC 53 Volts DC or -53 Volts DC
Additional Notes:
• Multi-Output Power Supplies: Note that AC-DC Multi-Output power supplies capable of operating at both 230V and 115V input shall be tested at both input voltages for purposes of ENERGY STAR qualification. ACENERGY DC Multi-Output power supplies capable of operating at only one of these indicated voltages must test only at the applicable voltage. Testing at 230V may be done at either 50Hz or 60Hz.
• 10% Loading Condition: All Single-Output power supplies must be tested at 10% loading in addition to the standard 20%, 50% and 100% loading conditions indicated in the test procedure.
• Fan Power: As indicated in the power supply test procedure referenced above, Multi-Output power supplies must be tested with internal fan power included in the measurement and efficiency calculation. Single-Output power supplies must exclude fan power from the measurement and the efficiency calculation.
c. Existing Test Procedures for Reference:
o EPRI: Generalized Internal Power Supply Efficiency Test Protocol
• http://efficientpowersupplies.epri.com/methods.asp.
o SNIA Green Storage Initiative: Idle Test Procedure
• http://www.snia.org/tech_activities/publicreview/GreenPower_v018. pdf
o SNIA Green Storage Initiative: Active Test Procedure
• Draft Pending
o SPEC: SPECsfs2008 Benchmark Suite
• http://www.spec.org/sfs2008/
d. Questions for Discussion:
1. Which operational modes (e.g., Idle, Active, Full Load, Maximum) should EPA address in the specification? In which mode(s) might the highest energy savings be achieved?
2. For various types or classes of Storage Products, what is the typical breakdown of energy consumption across operational states? E.g. an xx TB Storage Product typically consumes between xx and xx watts, and over the course of a year, xx % of the power is consumed in Idle state and xx% in Active state.
3. Are there any additional power consumption or efficiency test procedures that should be considered for reference in the ENERGY STAR specification?
4. Are there industry standard methods of measuring or specifying the useful work capacity or performance of a Storage Product? What other benchmarks might EPA consider for Storage Products?
5. It has been noted that the SPECsfs2008 benchmark might be an appropriate workload for NAS devices and is currently used by many NAS vendors. Is the benchmark suitable for more than NAS?
6. Are the power supply test procedures, test methods and levels from the Computer Server specification applicable to Storage? Is there data available that support these levels? What modifications might have to be made to address any of the unique characteristics of storage power supplies?
7. What are the typical loading ranges for Enterprise Storage power supplies? What are the typical redundancy configurations of these systems?
8. Would a specification for net power loss or efficiency over the load range be appropriate for Enterprise Storage?
9. Does the VAR sales channel play an important role in the Enterprise Storage market? Would the above requirements make clear how to qualify storage products through third-party sales channels?
Building Block #4: Information and Management Requirements
a. Purpose: EPA is interested in developing tools to facilitate the efficient design and operation of data centers. Improvements in the energy efficiency of Storage Products will lead to even greater efficiencies in data center design and operation, due to reductions in infrastructure requirements. To this end, EPA intends to take the Standard Information Reporting requirements and Data Measurement and Output requirements from the ENERGY STAR Computer Servers program and implement them in the Enterprise Storage specification. These requirements are intended to encourage proper capacity planning and the procurement of the most efficient equipment for a particular end-use scenario.
• The Standard Information Reporting requirements specify that manufacturers publish a Power and Performance Data Sheet for all ENERGY STAR qualified products. The data sheet provides information on the energy performance, advanced power saving features, and thermal characteristics of the product in a standard and accessible format. Templates for the Power and Performance Data Sheet for Computer Servers can be found on the ENERGY STAR website for Computer Servers at www.energystar.gov/NewSpecs.
• The Data Measurement and Output requirements specify that ENERGY STAR qualified Storage Products have the ability to measure and selfreport power consumption, utilization and input air temperature in an open, accessible format to interface with third-party management software. This information is intended to give data center operators more visibility into the real-time performance of their storage hardware to inform power management decisions.
b. Initial Approach: Following are excerpts from the Standard Information Reporting and Data Measurement and Output requirements from the Version 1.0 Computer Server specification, with minor modifications for Storage Products.
• Standard Information Reporting Requirements
o Partners must provide a standardized Power and Performance
Data Sheet with each ENERGY STAR qualified Storage Product. This information must be posted on the Partner’s Web site where information on the qualified model, or qualified configurations, is posted.
o Partners shall also provide links to a detailed power consumption calculator and to information on the power consumption of different system configurations.
o Partners are encouraged to use the standard data sheet template, but may also create their own template provided that it is identical in format and design as the referenced template, and has been approved by EPA. o Each Power and Performance Data Sheet must include the following information:
Model name/number, SKU and/or Configuration ID;
System characteristics (form factor, max storage capacity, power specifications, etc.);
System configuration;
Power data for Idle and full load, estimated kWh/year, link to power calculator (where available);
Additional power and performance data for at least one benchmark chosen by the Partner;
Available and enabled power saving features (e.g., power management);
Information on the power measurement and reporting capabilities of the device; and Select thermal information from the ASHRAE thermal report;
• Data Measurement and Output Requirements
o All Storage Products must have the ability to provide data on request over network connections on input power consumption in watts, inlet air temperature, and utilization.
o A service processor, embedded power or thermal meter (or other out-of-band technology shipped with the device), or preinstalled operating system may be used to collect and disseminate data.
o Data must be made available in a published or user accessible format so as to be readable by third-party, non-proprietary management systems.
o When an open and universally available standard protocol becomes available to report and collect data, manufacturers should incorporate the universal standard into their systems.
o Measurement Accuracy:
Input Power: ± 5% or ± 5 watts accuracy, whichever is greater.
Utilization measurements: TBD
Inlet air temperature measurements: ± 3° C.
o Sampling Requirements: Data must be provided as a rolling average with a period of less than 30 seconds.
c. Questions for Discussion:
1. What, if any, characteristics of the above requirements may not be appropriate for Storage Equipment? What data reporting capabilities do typical Storage Equipment currently posses? Are there industry trends towards the inclusion of these reporting capabilities?
2. What additional information specific to Enterprise Storage might be required for the Power and Performance Data Sheet?
3. How is utilization defined for Storage Equipment? What utilization information would be helpful to managers for procuring the proper equipment and integrating those systems into their data centers?
4. Are there any industry accepted communication protocols available for measuring utilization, temperature, etc?