Contributing author: Bill Mottram
Introduction
Wikibon Energy Lab Validation Reports are designed to assist customers in understanding the degree to which a product contributes to energy efficiency. The four main goals of these studies are to:
- Validate the hardware energy efficiency of a particular technology as compared to an established baseline.
- Asses the potential contribution of software technologies to power savings, and validate the actual contribution in real world installations.
- Quantify the contribution of the hardware and software technologies to a green data center.
- Educate business, technology, and utility industry professionals on the impact of technologies on reducing energy consumption.
Our objective is to identify not only the hardware energy consumption but also the often overlooked and hard-to-quantify green software aspects of technologies. Wikibon Energy Lab Validation Reports are submitted to utilities such as Pacific Gas & Electric Company as part of an energy incentive qualification process.
Wikibon Energy Lab defines and validates the hardware testing procedures to determine the energy consumed by specific products in various configurations. As well, Wikibon reviews actual customer results achieved in the field to validate the effectiveness of these technologies based on real-world field-data analysis. These proof points are mandatory for the utility company to qualify a specific vendor's technology for energy incentives.
Wikibon Energy Lab Reports are not sponsored. Rather they are deliverables required by PG&E and other utilities as part of an incentive qualification process. As part of its Conserve IT Program, Wikibon is paid by the vendor to perform services associated with securing incentive rebates from utilities for end customers that acquire the vendor's technologies. To ensure this process is completely independent, Wikibon lab and field results are sometimes vetted by a third party engineering firm hired by PG&E or other utilities.
Wikibon only produces Lab Validation Reports for technologies that have been qualified for rebate incentives by PG&E or other utilities and have passed strict utility company guidelines. By adhering to this criterion, Wikibon assures its community of the independence of these results.
Executive Summary
Virtualization and thin provisioning are two powerful storage optimization techniques that significantly improve effective storage utilization. This operational efficiency enables the storage administrator to store more data on less physical hardware while delivering substantial consumer operational and capital costs.
Xiotech has implemented virtualization and thin provisioning labeled Optimal Virtualization and Intelligent Provisioning, on the Xiotech Emprise 7000 series of storage arrays. Virtualization and thin provisioning technologies help solve the problem of the low utilization of storage by allowing capacity on different disk drives to be logically joined together into storage pools, and allocated when they are required rather than in advance. The storage capacity is shared by many servers and applications.
Xiotech’s implementations of thin provisioning is achieved by enabling the server owning the volume to automatically be allocated with additional storage in the storage controller when a utilization threshold is reached. The server always has knowledge of exactly how much storage is available. In contrast the competitive approach is to make it appear that the server has the real space, but only allocate space when it is actually written to disk. The same objective of reducing the number of physical disk drives.is reached by both methods.
Power savings from this technology are realized because the number of spinning hard disks required is significantly reduced. Example; See Figure 1, a 50% reduction is realized by increasing the data utilized from 35% to 70%. The assumed reduction ratio for incentive applications is a 35% reduction in spinning drives, discussed in detail later in section “Calculation of Energy Savings”.
The purpose of this report is to look at how Xiotech Optimal Virtualization and Intelligent (thin) Provisioning technology impacts the operation costs of power and cooling and to present a detailed analysis.
Measurement Methodology for Power
Baseline
The baseline for the power savings is the storage array configuration required if the virtualization and Thin/Intelligent provisioning software was not installed on the storage array. The power savings are calculated from the power requirements of the baseline array less the power requirements of the installed array.
Equipment Measured
The components of the Xiotech Emprise 7000 storage array were measured. The components were:
- Controller Node (2 required per storage array, running software release 8.01)
- Drive Bays (running ISE firmware version: 1.1)
- ISE with 2 x Performance Tier DataPac (20 Drives)
- ISE with 2 x Balanced Tier DataPac (20 Drives)
- ISE with 2 x Capacity Tier DataPac (20 Drives)
- FC Switch (Cisco 9124 running version 3.1.3a firmware)
- IP Switch (Dell 2724)
- ICON Management Appliance
The Emprise 7000 total storage array is composed of dual controllers, up to 64 drive bays, an FC switch, an IP switch and an ICON management appliance.
Location of Testing
The testing was done in Xiotech’s facility at 6455 Flying Cloud Drive, Eden Prairie, MN 55344 (telephone: 866-472-6764). The testing was overseen by Karl Morgan, Technical Marketing Engineer (tel: 952-983-2307).
Measurement Equipment
A BK Handheld Ammeter model 330B and a Chroma Programmable AC Source Model 61603 were used to measure the equipment.
Power Measurements
Wikibon reviewed the power measurements made on Xiotech’s Emprise 7000 storage arrays. These arrays are built from a series of common components, and enable the power of different configurations to be calculated. Wikibon has constructed a Wikibon Energy Lab Power Calculator (Xiotech V3) based on these measurements. The Power Calculator estimates the power used in the actual proposed storage system and the baseline system (the system required if no power saving software were available) and calculate the power savings. Wikibon is confident that this calculator will estimate the actual power used by the Xiotech storage arrays within 5% accuracy. The details are in the section “Measurement Results” below.
Methodology
The key component of a storage array that varies power consumption with different workloads is the drive bay, because changes in I/O rate changes the actuator movement on the drive. Most of the measurement effort was dedicated to the drive bay to measure ower consumption with different workloads.
The relevant properties of the disk drives are:
- Capacity - The amount of data held, measured in gigabytes (GB), where one GB is equivalent to 1,000 million bytes
- Spin Speed - The rotational speed of the disk, measured in rpm (e.g., 15K = 15,000 revolutions per minute)
- Transfer Rate - The maximum transfer rate from the disk, measure in gigabits (Gb)/sec, where one gigabit = 1,000 million bits
- Interface – The means by which the drive connect and exchanges data with the drive bay. The drive bays can have drives with either FC (Fibre Channel) interface or SATA (Serial Advanced Technology Attachment) interfaces.
The measurements were made with three different workloads:
- Sequential Read, with a 256K block size
- Random Write, with a 16K block size
- Idle (no data being transferred)
Read benchmarks for sequential were used for practical reasons, specifically it is much quicker to reset after a benchmark is run which accelerates the preparation for the next run. The main power consumed by disk devices is actuator movement and media rotation, which are similar for reads and writes.
The three benchmarks were averaged to produce an overall figure of power consumption for the Xiotech arrays. In Wikibon’s opinion, this measurement represents a good and reasonable estimate of the power consumed in the real world applications across a number of drives for “typical” combinations of applications found in a data center. The results of the comparisons are given in the Measurement Results section below.
Verification
Xiotech’s storage arrays have the ICON Manager software running in the array which tracks the storage reduction benefits that thin/Intelligent provisioning and virtualization achieve Details are given in the section below “Confirmation of Energy Savings with Virtualization and Thin/Intelligent Provisioning”.
Measurement Results
Measurement Summary
A summary of the results is shown in the Table 1 below. An average of three benchmarks with sequential, random and idle workloads was used to determine the power draw for each component. Where a measurement was not available, it was interpolated from the measurements on that component and/or other components.
Measurement Details
The detailed measurements were conducted by Xiotech with overall supervision by Wikibon. The detailed results are shown in Table 2 below.
Calculation of Energy Savings and Incentives
The Wikibon Power Calculator (Xiotech V3) is used to calculate the power savings, include the HVAC power saving.
In the example below, a Xiotech customer installs an Emprise 7000 with 30 ISE drive bays (10 of each type) containing 600 drives. Table 3 shows the output of Wikibon Energy Lab Power Calculator (Xiotech V3). The calculator estimates the power usage of the proposed system and the alternative baseline system without the Xiotech virtualization and thin/Intelligent provisioning software. The baseline system would require 35% additional drives to compensate. The output of the Wikibon Power Calculator shows energy savings of 112,000 kWh/year and budget savings of $87,102 over the life of the project. The recommended PG&E energy incentive would be $13,309 based on the NRNC program, using 2009 figures.
Verification of Energy Savings with Virtualization and Intelligent Provisioning Technology
Xiotech has a report as part of the ICON management appliance which shows the utilization of disk space. A sample output is shown in Figure 2 below.
The report shows the increase in utilization from an initial 40% (the utilization when the data was migrated to Xiotech storage array) to 95% over a period of one year. This level of utilization could never be achieved with a traditional storage array. The array would be unmanageable, servers would be crashing all the time, and the CIO would have been fired long ago. The reason for the achievement of this very high level of utilization is the virtualization of the resources, and the continued usage of Intelligent Provisioning.
The report can be run on request to confirm that the disk utilization that has been achieved from the installation of virtualization and thin/Intelligent provisioning. A useful report to verify the performance of early installations.
The screen shots below illustrate the process for creating a volume, and how the ICON Manager ensure that there is always space available. Figures 3 through 14 show how the whole process works. The first step is to create a new volume. Figure 3 below shows the initiation of that first step.
The next step is to define the volume size and other properties of the volume as shown in figure 4.
Figure 4 shows that a 50 GB volume was created as a D: drive on the server. (Note that this is all that is necessary to connect a 50GB disk to a Windows server. There was no work to do in windows itself.) Figure 5 and 6 show the completion of the process.
After “Create” is clicked, the volume is available.
Figures 7 shows the VDisk in the Xiotech Array Icon Manager software application. Select Properties to enable Intelligent Provisioning.
The Intelligent Provision tab is selected. In this example when the used space reaches 85% full, and volume is expanded by an additional 10GB. Every time the Vdisk gets 85% full then 10GB will be added to the disk. However once a 200GB size is reached the system will stop allocating storage and send an alert to the system. This is to control runaway expansion.
If a VDisk has Intelligent Provisioning enabled the Icon is changed to that circled in figure 9.
In figure 10 data is copied onto the D: drive with the XCOPY command.
From the server standpoint, all of a sudden the D: drive increased to “60 GB” size. This happened automatically. No storage administrator would be able to maintain an 85% level of utilization and ensure the systems continued to run normally.
Looking at the properties of the VDisk on the Icon Manager application you see this VDisk is now 60GB not 50. The Raid Count is now 2 (the auto expansion caused a Raid to be added to this VDisk).
Figure 13 shows the Icon Manager again, with the D: copying data, now only 14.21% free.
Figure 14 shows that lmost immediately, the VDisk size (Capacity column) has expanded to 100GB. The disk has copied data up to the point where there is 17.35% Free Space. If two more GB where added then 10GB will be added to the Vdisk again.
The key benefit of this is approach is that much higher utilizations can be achieved automatically, and this reduces the disk space wasted.
Energy Savings from Virtualization & Thin/ Intelligent Provisioning
Virtualization and Thin/Intelligent provisioning are relatively new technologies that significantly reduce the amount of storage that is required, and improve the utilization of that storage. A detailed description of the technology is given in Appendix I. The purpose of this section of the Energy Lab report is to estimate the level of saving that can be achieved with virtualization and thin/Intelligent provisioning.
Xiotech offer a “Call-Home” program that is used to help predict a component failure. It is subscribed to by a good percentage of its customer base. As part of the prevention analysis, the system captures metadata about disk usage and function usage from the service processor in each array connected to the system. There is metadata about each volume that has been created on the arrays. The database of this metadata allows this to be looked at on a customer basis, taking an average for all the volumes at each customer. This is a good way to look at the data, as the way a customer organizes volumes is usually consistent within a customer.
One key metric is the improvement in utilization that virtualization and thin provisioning achieve. For each volume there is a logical size (what the server sees) and a physical size (what has actually been written). In traditional storage arrays, the physical and logical sizes are very similar. With virtualization and thin/Intelligent provisioning, the physical size is significantly smaller than the logical size.
Wikibon performed a statistical analysis of data collected from 115 installations employing thin provisioning and found that the actual physical disk required to hold the data was only 33% of the logical data. From an energy point of view, 66% of the disks were not required. However, a closer analysis of the data showed that a few of the customers had very large gains. What is happening is that customers are finding new way of exploiting the virtualization and thin/Intelligent provisioning. For example, customers were making multiple thin copies of data many times each hour, so that they could recover back very quickly to an earlier copy of data if data was corrupted. This is very efficient with thin copying, because only changed data has to be written. The amount of data saved can be very high indeed.
However, from an energy rebate point of view, this is not a saving in energy, because the customer was not actually making physical copies before. To estimate the real power savings, Wikibon assumed that customers with saving in disk space of 50% or more had changed the procedures on how they were storing data. Wikibon took the subset of customers (n=42) that had less that 50% savings. The weighted average saving was 35%. Wikibon believes that this figure is a good predictor of the direct reduction in disk capacity that will be found in customers. This is the default figure that was used in the spreadsheets.
Appendix I –Virtualization and Thin/ Intelligent Provisioning Storage Technologies
The fundamental building block of storage systems is a storage volume. A storage volume can (and is usually) smaller that a disk drive, but can also be bigger and span multiple storage devices. The operating system “mounts” a storage volume to be able to read only or read/write to that volume. For example, a PC has one or more disk drives. A number of logical drives (equivalent to a storage volume) can be made from these disks drives (C: drive, D: drive, etc) and space allocated to them when they are created. The space on the disk drives is formatted and allocated to these logical drives when they are created. External volumes can be accessed from a PC over a LAN or WAN.
There are number of problems with this approach.
- The Space is allocated when the storage volume is created; a large amount of space is initially allocated but not used
- Over time, the data becomes fragmented as (for example) when data is deleted, but the space is not released
- It is difficult to move volumes around to optimize storage, as the application has to be stopped while the volume is moved. This limits the ability to reorganize the data and take advantage of spare capacity on the disk drives
- When a copy of a volume is required, the whole storage volume has to be copied
Virtualization breaks the connection between what is written, what is allocated, and what is unused. This concept was applied first by IBM in the 1960’s to the storage on the CP/67 server operating system, which later became VM with virtual disks.
Xiotech has integrated virtualization into the storage operating system. Virtualization allows a number of techniques to be used to reduce the amount of data that is actually stored, the most important of which is thin/Intelligent provisioning
Thin/Intelligent Provisioning & Intelligent Provisioning
Xiotech markets thin/Intelligent provisioning as “Intelligent Provisioning”. There are some differences in how it is implemented.
- With thin provisioning, a (say) 100 GB volume is created, but only (say) 10 GB of actual data is consumed and allocated, and the other 90 GB is available for all other applications to use. As the volume grows towards 100 GB, you need to add more disks into the storage pool. Thin provisioning best practice actually suggests that you create a much larger thin provisioned volume. Instead of a 100 GB volume, a 1 TB volume should be created so there is no worry about growth consuming capacity too quickly. Since thin provisioning doesn’t actually allocate the disk space until the data is written, you can have large volumes without any appreciable cost to your system.
- With Xiotech’s Intelligent Provisioning, best practice is to provision a smaller volume (say 20GB). When this 20 GB volume is created (like traditional provisioning) the entire 20 GB will be allocated and dedicated to that volume. However, as the volume grows the user defined threshold set at (say) 16 GB enables the volume to grow automatically. There is no need to reboot. Intelligent Provisioning best practices suggest that you create small initial volumes and higher thresholds based on the amount of actual capacity you have and the rate at which data is growing in your environment.
One of the advantages of Intelligent Provisioning is that all server file-system software will work with this approach. With the traditional thin provisioning approach there are some software environments that are either not supported, or would act in an non-optimal way (for example, Microsoft file systems do not use up deleted blocks if there is allocated space available, and does not have the ability to act on the fact that this is only virtual space).
Marketing will always make the most out of differences. However, Wikibon believes that thin provisioning and Intelligent Provisioning will give very similar savings in practice.
Disclaimer
This report was prepared by Wikibon. Reproduction or distribution of the whole, or any part, of the contents of this document without written permission of Wikibon is prohibited. Neither Wikibon nor any of its employees make any warranty or representations, expressed or implied, or assume any legal liability or responsibility for the accuracy, completeness, or usefulness of any data, information, method product or process disclosed in this document, or represents that its use will not infringe any privately-owned rights, including, but not limited to, patents, trademarks, or copyrights. This report uses preliminary information from vendor data and technical references. The report, by itself, is not intended as a basis for the engineering required to adopt any of the recommendations. Its intent is to inform the customer of the potential cost savings. The purpose of the recommendations and calculations is to determine whether measures warrant further investment of time and/or resources.
