Data Centers: Annotated Bibliography

ACEEE and CECS. 2001. Funding prospectus for "Analysis of Data Centers and their implications for energy demand." Washington, DC, American Council for an Energy Efficient Economy (ACEEE); Center for Energy and Climate Solutions (CECS). July 2001.

The paper includes an overview of data centers; discusses energy use, energy choices, and energy efficiency in data centers; potential impacts of data centers; Present and future regulatory issues; business opportunities in energy services.

Aebischer, B., R. Frischknecht, Ch. Genoud, and F. Varone. 2002. Energy Efficiency Indicator for High Electric-Load Buildings. The Case of Data Centres. Proceedings. "IEECB 2002. 2nd International Conference on Improving Electricity Efficiency in Commercial Buildings," Nice, May 27-29, 2002, Ademe (edt.), France.

Energy per unit of floor area is not an adequate indictor for energy efficiency in high electric-load buildings. For data centres authors propose to use a two-stage coefficient of energy efficiency CEE = C1 * c2, where C1 is a measure of the efficiency of the central infrastructure and c2 a measure of the energy efficiency of the equipment.

Aebischer, B., R. Frischknecht, C. Genoud, A. Huser, and F. Varone. 2002. Energy- and Eco-Efficiency of Data Centres. A study commissioned by Département de l'intérieur, de l'agriculture et de l'environnement (DIAE) and Service cantonal de l'Énergie (ScanE) of the Canton of Geneva. Nov 15, 2002.

The European study investigates strategies and technical approaches to fostering more energy-efficient and environmentally sound planning, building, and operating of data centres. It also formulates recommendations on how to integrate the findings in the legal and regulatory framework in order to handle construction permits for large energy consumers and promote energy efficiency in the economic sectors. Seventeen recommendations derived from the Transfer of the accord into an institutionalized legal and regulatory framework; Energy-efficiency policies for all large energy consumers; Preconditions, and prerequisites; Operational design of voluntary energy policies.

Anonymous. Automated Disaster Prevention in the Computer Room, www.intracomp.com.

Describe Consequences of an Environmental Catastrophe in the Computer Room, and Elements of an Effective Disaster Prevention System; Introduces the Justification for Installing an Automated Environmental Monitoring & Disaster Prevention System and case studies of An Interesting Application of a SAM System at NIH labs.

Anonymous. 2001a. Model Data Center Energy Design Meeting. Austin Energy, Austin, TX, Feb 12-13.

The purpose of the meeting was to gather information and brainstorm ideas for Austin Energy to use in marketing to data centers and in negotiations pertaining to the energy needs of data centers. Distributed generation manufacturers, data center development companies, engineering firms, consultants, research and development organizations, data center owners/operators, electric utilities, and Austin Energy technical staff attended the meeting. The Model Data Center Energy Design Meeting Report includes 1) Information about the Austin scenario; 2) Summaries of presentations and panels; 3) Recommendations and solutions; and 4) Listing of meeting participants.

Anonymous. 2001b. Symposium Digest. IEEE Emerging Technologies Symposium on Broadband Communications for the Internet Era: Symposium Digest. P. Winson. Radisson Hotel, Richardson, TX, Institute of Electrical and Electronics Engineers, New York.

Anonymous. 2002a. 7 x 24 Update: Design & Construction - Issues and trends in mission critical infrastructure design, planning and maintenance. July 23, 2002. http://www.7x24exchange.org/.

Security had traditionally focused on electronic intrusion from viruses and hackers. While electronic intrusion remains a hot priority, corporations also are anxious to protect the physical infrastructure of mission critical operations. Priorities of locations that must operate seven days a week, 24 hours daily include networking technologies, power, HVAC systems, security, and safety. Author pointed out that meeting these priorities requires the expertise found among various members of the facility planning team comprised of facility and risk management, information technology, architecture, engineering, construction and finance, among others.

Anonymous. 2002b. Continuous Availability Review (CAR), The Uptime Institute: Computersite Engineering, Inc. July 22, 2002.

An overview assessment of continuous environmental infrastructure uptime availability including a comprehensive site inspection plus a review of capacity constraints, potential downtime vulnerabilities, and concurrent maintenance capacity.

Anonymous. 2002c. End-to-End Reliability Begins with the User's Definition of Success, The Uptime Institute. July 22, 2002.

The gradual relocation of critical computing applications from centralized glasshouse mainframes to distributed desktop workstations is having a profound impact of many companies' availability and reliability requirements. Historically we have made tremendous infrastructure investments to provide continuous cooling and uninterrupted power for centralized direct access storage device (DASD) and mainframes. As critical applications have become distributed, the value of a PCs data stream can now exceed what is on the mainframe. Does this mean that the entire company should be put on an uninterruptible power system? End users are moving to PCs and workstations believing that they will have more control at less cost.

Anonymous. 2002d. Fuel Cell Installation Improves Reliability and Saves Energy at a Corporate Data Center. The California Energy Commission and the U.S. Department of Energy, Office of Industrial Technologies Best Practices, MAY 2002.

In 2002, Chevron Energy Solutions implemented a fuel cell installation project at the corporate data center of Chevron Texaco's corporate headquarters in San Ramon, California. The company undertook the project to evaluate and gain experience with fuel cell technology as a reliable, clean, quiet source of energy for critical power applications. The simple payback time would be less than 20 years. In addition, there is minimal environmental impact because the fuel cell has very low emissions.

Anonymous. 2002e. Industry Specifications: Cooling Compatibility Specification - Draft - Version 0.3, The Uptime Institute. http://www.upsite.com/TUIpages/editorials. July 22, 2002.

Computer and communications equipment that is "cooling compatible" has the capability of maintaining the consistent internal temperatures necessary for reliable operation while in a physical environment containing products from different manufacturers. The objective is to maintain the desired internal and external environmental conditions regardless of whose hardware is sitting beside, in front of, behind, or in the same rack as the subject equipment. Internal air circulation within the confines of the manufacturer's enclosure may be in any direction as long as the external and discharge conditions identified in this specification are satisfied. This specification assumes computer and communications equipment will be configured in "hot/cold" aisles on the user's raised floor. The "hot/cold" aisle concept is described in an Uptime Institute White Paper, which can be found at www.upsite.com/TUIpages/tuiwhite.html.

Anonymous. 2002f. Mechanical Systems Diagnostic Review (MSDR), The Uptime Institute: Computersite Engineering, Inc. July 22, 2002.

A comprehensive onsite assessment of mechanical system performance and the identification of design, operating, and maintenance issues affecting data center environmental reliability. Portable instrumentation is used to measure critical operating parameters including the actual cooling load and how existing capacity is being utilized.

Anonymous. 2002g. Site Infrastructure Operations Review (SIOR), The Uptime Institute: Computersite Engineering, Inc. July 22, 2002.

A Site Infrastructure Operations Review is a comprehensive onsite assessment of all site infrastructure operations including comparisons with industry best practices. The review looks at staff effectiveness and efficiencies, operating expenses, appropriate specialized training, overlying attitudes, and the critically important concepts of "saves" and "uptime effectiveness." In addition to an analysis of current conditions, the review presents detailed observations and recommendations on human-factor issues in support of uninterrupted site operation.

ASHRAE. 1999. ASHRAE Handbook - HVAC Applications: Chapter 16 Data Processing and Electronic Office Areas. Atlanta, GA.

Data processing and electronic office areas contain computers, electronic equipment, and peripheral equipment needed for data processing functions. Computer equipment has expanded beyond the traditional computer room to become an integral part of the entire office environment.

Baer, D. B. Emerging Cooling Requirements & Systems in Telecommunications Spaces. Liebert Corporation.

During the last several years, power density trends, and consequently thermal density trends in telecommunications spaces have become topics of increasing interest. This paper identifies several of the underlying drivers of these trends, project possible outcomes, and assess the impact on cooling system design for these spaces.

Baer, D. B. 2002. High density trends, Liebert Corporation. Liebert Corp., E-Mail: dan.baer@ liebert.com. 

Bailey, A. M. 2002. Accelerated Strategic Computing Initiative (ASCI) : Driving the Need for the Terascale Simulation Facility (TSF). Palm Springs, CA, June 4. http://www.energy2002.ee.doe.gov/Facilities.htm

Barista, D. 2002. Charging ahead - MasterCard's high-tech campus incorporates state-of-the-art data center designed to withstand any disaster, Associate Editor, Building Design and Construction.

Beck, F. 2001. Energy Smart Data Centers: Applying Energy Efficient Design And Technology To The Digital Information Sector, Renewable Energy Policy Project (REPP): Washington, DC., (November 2001 REPP).

Betz, K. 2001. The Dawning of the Age of the Internet, Energy User News.

The Internet may have been accused of escalating energy use, but it may reduce energy consumption, or at least reduce its costs. The author points out that the internet and deregulation were made for each other: 1) without the Internet to collect information and to make it easier to compare rates and procure energy, the idea of deregulation would be far less compelling and workable, and 2) without deregulation to open power markets, the Internet's ability to collect information and enable transactions would be of little use to the energy markets. Opportunities exist in energy efficiency in data center design.

Blount, H. E., H. Naah, and E. S. Johnson. 2001. Data Center and Carrier Hotel Real Estate: Refuting the Overcapacity Myth. Lehman Brothers: TELECOMMUNICATIONS, New York, June 7, 2001.

An exclusive study examining supply and demand trends for data center and carrier hotel real estate in North America. Lehman Brothers and Cushman & Wakefield have completed the first in a regular series of proprietary studies on telecommunications real estate (TRE), including carrier hotels and data centers.

Blumenstein, R., S. Thurm, and G. Ip. 2001. Downed Lines: Telecom Sector's Bust Reverberates Loudly Across the Economy

Impact on Jobs and Investors Is Proving Much Bigger than That of Dot-Coms - Dashed Dreams in Scranton. The Wall Street Journal.

Bors, D. 2000. Data centers pose serious threat to energy supply, Puget Sound Business Journal (Seattle) - October 9, 2000.

To cope with increasing energy demand from data centers, the author discussed feasibilities of two possible approaches: 1) energy industry approach by looking at alternative energy supply; 2) construction industry approach by looking at data center energy efficiency. To get there, it is worth investigating four distinct components. (I) Co-generation of power. Presently, standby diesel generators are required to maintain the desired level of reliability at most data center sites, but their exhaust makes most of these generators unacceptable for long-term power generation. (II) Fuel cells offer the promise of very clean emissions and the reasonable possibility for use as standby power. (III) Increased efficiency in data center power distribution systems. There are two separate items that are major contributors to data center power distribution system inefficiencies. The first, power distribution units (PDUs), are available with optional internal transformers that use less energy than the present cadre of K-rated transformers. The second, uninterruptible power systems (UPSs), come in a range of efficiency ratings. If the use of high-efficiency PDUs and UPSs are combined, they offer the potential of a 6 percent saving. (IV) Increased efficiency in mechanical cooling systems. In order to ensure data center reliability, mechanical equipment is often selected as a large number of small, self-contained units, which offers opportunities to improve efficiencies. (V) Reductions in energy use by computer, network and storage equipment. Computer manufacturers can do their part by creating computers with greater computational power per watt. They have been doing this for years as a side effect of hardware improvements, and they can do even better if they make it a goal.

Brill, K. G. 2002. We Have Met the Enemy of Site Uptime and He Is Us!, The Uptime Institute. July 22, 2002.

The author presented data showing that 54% of all reported site infrastructure failures were coincident with human activity. Suggested that education of management, their commitment, procedures, and better training will help to solve the problem and to protect the business.

Brill, K. G., E. Orchowski, and L. Strong. 2002. Product Certification for Fault-tolerance is Essential for Verification of High Availability. The Uptime Institute.

IT organizations report that 25% of all information downtime results from the interaction of computer hardware with its physical environment. These failures occur within the site infrastructure (uninterrupted power, stable cooling, fire protection, physical access, cabling and connectivity systems) or are caused by human error. Often, these failures are overlooked in a classical outage analysis because they deal with the physical environment and are infrequent. Although environmental faults total only 4% of all outages, when they do occur, the impact on information availability can be disastrous. Recovery from a facility environmental event can take several hours and many or all hardware platforms may be simultaneously affected.

Brousell, D. R. 1993. The Virtual Data Center. Datamation 39(8): 104-104.

Brown, E., R. N. Elliott, and A. Shipley. 2001. Overview of Data Centers and Their Implications for Energy Demand. Washington, DC, American Council for an Energy Efficient Economy, Center for Energy & climate Solutions (CECS). September 2001.

The white paper discusses data center industry boom and energy efficiency opportunities and incentives in internet data centers. Emerging in the late 1990's, data centers are locations of concentrated Internet traffic requiring a high-degree of power reliability and a large amount of power relative to their square footage. Typically, power needs range from 10-40MW per building, and buildings are typically built in clusters around nodes in the Internet fiber-optic backbone. During the development boom in 1999 and 2000, projects averaged 6-9 months from site acquisition to operation, and planned operational life was 36 months to refit. Even high energy-prices were dwarfed by net daily profits of 1-2 million dollars per day for these buildings during the boom, creating little incentive for efficient use of energy.

Callsen, T. P. 2000. The Art of Estimating Loads. Data Center (Issue 2000.04).

This article discusses the typical Data Center layout. It includes floor plan analysis, HVAC requirements, and the electrical characteristics of the computer hardware typically found in a Data Center.

Callsen, T. P. 2001. Internet Hotels - A Big Draw, Transmission & Distribution World, (Issue 2000.04).

Calwell, C., and T. Reeder. 2002. Power Supplies: A Hidden Opportunity for Energy Savings (An NRDC Report). Natural Resources Defense Council, San Francisco, CA, May 22, 2002.

The article discusses the efficiency of power supplies that perform current conversion and are located inside of the electronic product (internal) or outside of the product (external). The study finds that most external models, often referred to as "wall-packs" or "bricks," use a very energy inefficient design called the linear power supply, with measured energy efficiencies ranging from 20 to 75%. Most internal power supply models use somewhat more efficient designs called switching or switch-mode power supplies; and that internal power supplies have energy efficiencies ranging from 50 to 90%, with wide variations in power use among similar products. Most homes have 5 to 10 devices that use external power supplies, such as cordless phones and answering machines. Internal power supplies are more prevalent in devices that have greater power requirements, typically more than 15 watts. Such devices include computers, televisions, office copiers, and stereo components. The paper points out that power supply efficiency levels of 80 to 90% are readily achievable in most internal and external power supplies at modest incremental cost through improved integrated circuits and better designs.

Clark, M. J. 2001. Paralleling switchgear keeps California data center running smoothly during outages, Power Engineering, 105 (11): 146.

Cox, C., C. Fong, and J. Stein. 2001. Part II: Doing Business with HiDELs. Delivering Energy Services to Internet Hotels and Other High-Density Electronic Loads. E-Source: The E Source Technology Assessment Group, Platts Research & Consulting., Boulder, CO.

Craparo, J. S. 2000. Dell Time Equals Uptime!, Dell Magazines - Dell Power Solutions, (Issue 3 "Building Your Internet Data Center").

E-commerce has fueled today's economy with its continued expansion. Global Internet business is expected to grow at a compound annual rate of 100 percent to almost $3.2 trillion by 2003, and to $6 trillion by 2004, according to Forrester Research. Companies currently deploying an Internet business model are seeking answers to the challenges of managing successful, mission-critical data centers. Therein lies the challenge: to balance-and manage-the business growth and the demand for resources to create successful enterprises.

Cratty, W., and W. Allen. 2001. Very High Availability (99.9999%) Combined Heat and Power for Mission Critical Applications, Cinintel 2001: 12. http://www.surepowersystem.com

Elliot, N. 2001. Overview of Data Centers and their implications for energy demand. Washington, DC, American Council for an Energy Efficient Economy. Jan 2001, revised June 10, 2001.

EPRI. CEIDS Concept Paper: Consortium for Electric Infrastructure to Support a Digital Society.

EPRI. 2001. The Cost of Power Disturbances to Industrial and Digital Economy Companies. Electric Power Research Institute (EPRI), Palo Alto, CA, June.

E-Source. 2001. PROSPECTUS E-SOURCE Multi-Client Study: Delivering Energy Services o Internet Hotels and Other High-Density Electronic Loads. E-SOURCE, Boulder, Colorado.

The paper discusses types of data centers and their commonality: all are big boxes filled with computers, hard drives, switches, routers, and rectifiers, designed to store and channel digital information. The term "High-density electronic loads," or HiDELs for short, is created, with the following main characteristics: 1) Highly dense power requirements. HiDELs typically feature power loads ranging from 100 to 200 watts per square foot, but we are aware of several facilities under construction that will have a connected capacity of 300 watts per square foot. 2) Extremely high load factors. For these facilities, load factors are reportedly in the range of 80 to 90 percent. 3) High sensitivity to power supplies disturbances. These facilities face huge losses if they go down due to surges, sags, or outages. Nearly all HiDELs are equipped with extensive on-site electric storage and backup generation systems. 4) Rapid growth. The HiDEL developers we have spoken with are putting these facilities up as fast as they can. Qwest plans to triple the server farm real estate it currently has, jumping from 500,000 to 1.5 million square feet over the course of a single year. Inflow, one of the largest providers of collocation facilities, plans to more than double its capabilities in the coming year. In addition, these facilities seem to be widely dispersed, popping up near nodes in the fiber-optic network. The paper offer comments on whether or not HiDELs would become perfect candidates for a wide variety of services and products offered by energy service providers (ESPs), but points out that ESPs will likely spend billions of dollars over the next few decades installing generation, transmission, and distribution infrastructure; and that there are uncertainties about HiDELs future development in business.

Evans, R. 2001. META Report: Room at the Data Center?, Datamation: IT Management.

The paper offers guidelines on how to keep facilities needs manageable, in light of strong processor and storage growth that continues to challenge physical planners and facilities managers in the face of extensive infrastructure growth. Driven by centralization, ongoing growth, and sustained expansion of both storage and processor infrastructure, businesses are re-examining their short-term data center growth plans. Complicating assumptions of future space needs have been recent, dramatic reductions in space needs of storage and processing hardware (or "form factor technology") that have freed large tracts of data center real estate. The author offers the view that for the next five years planners should assume more "standard" technology form-factor rates of change for both storage and processors. More important, while base processor and storage form factor technology will see densities double every 18 months, adding complicated power supply and expanding interconnection requirements will throttle actual floor space improvements. On average, only about 14 percent to 17 percent of floor space will be freed up each year.

Felver, T., M. Scofield, and K. Dunnavant. 2001. Cooling California's Computer Centers, HPAC Engineering, (March 2001): 59-63.

An AHU design that conserve energy, improve IAQ, and increase reliability - indirect EC.

Feng, W., M. Warren, and E. Weigle. 2002. The Bladed Beowulf: A Cost-Effective Alternative to Traditional Beowulfs, Cluster2002 Program.

Authors present a novel twist to the Beowulf cluster - the Bladed Beowulf. In contrast to traditional Beowulfs, which typically use Intel or AMD processors, the Bladed Beowulf uses Transmeta processors in order to keep thermal power dissipation low and reliability and density high while still achieving comparable performance to Intel- and AMD-based clusters. Given the ever-increasing complexity of traditional super-computers and Beowulf clusters; the issues of size, reliability, power consumption, and ease of administration and use will be "the" issues of this decade for high-performance computing. Bigger and faster machines are simply not good enough anymore. To illustrate, authors present the results of performance benchmarks on the Bladed Beowulf and introduce two performance metrics that contribute to the total cost of ownership (TCO) of a computing system - performance/power and performance/space.

Frith, C. 2002. Internet Data Centers and the Infrastructure Require Environmental Design, Controls, and Monitoring, Journal of the IEST, 45 (2002 Annual Edition): 45-52.

Internet Data Centers and the Infrastructure Require Environmental Design, Controls, and Monitoring. The author points out that specifications and standards need to be developed to achieve high performance for mission-critical internet applications.

Gill, K. E. 2001. Telecom HVAC Cool Connections - HVAC in telecommunication facilities, HPAC Engineering, (Jan 2001): 29-38.

The "cold rush" of the infrastructure buildout in the telecommunications industry is in full swing. The author points out that the current trend (2001) toward larger facilities and higher cooling loads per square foot should continue into the foreseeable future.

Gilleskie, R. J. 2002. The Impact of Power Quality in the Telecommunications IndustryM. Palm Springs, CA, June 4.

The workshop addresses the unique issues and special considerations necessary for improving the energy efficiency and reliability of high-tech data centers. This presentation addresses impacts of power quality including voltage sags, harmonics, and high-frequency grounding in telecommunication industry.

Grahame, T., and D. Kathan. 2001. Internet Fuels Shocking Load Requests, Electrical World, Vol. 215 (3): 25-27.

This article discusses the implications of the increase for power demand by the Internet's traffic growth on utility planning, operation, and financing.

Grahame, T. J., and D. Kathan. 2001. Internet Data Centers & Electricity Growth - Will the Emerging Power Crisis Burn Your Bottom Line?, Broadband Wireless Business Magazine, Vol. 2 (No. 05).

Data center loads are rapidly becoming a political issue in electricity-short California, despite the importance of data centers to the information economy. The authors explored the context into which requests for large, dense electricity loads suddenly appear; how much electricity these telecom hotels are likely to draw. The authors pointed out that the electricity community could greatly benefit from estimates of future electricity needs, driven by technological change, which can be provided and aggregated best by the Broadband community of wireless and wire line carriers. The authors suggested there is a need for a technical committee that could map out internet growth and growth in storage needs; estimate how much electricity per some measure of internet use is needed today; and estimate how much electricity per measure of internet use would be required tomorrow. It would be a great service, because it would enable a better estimate of the internet's electricity use (and thus needs) going forward.

Greenberg, D. 2001. Addendum to ER-01-15: A Primer on Harmonics. E-SOURCE, Boulder, Colorado, September 2001.

The electrical distribution systems of most commercial and industrial facilities were not designed to operate with an abundance of harmonics-producing loads. In fact, it is only within recent years that such loads have become widespread enough for industry to take notice and to begin to develop strategies to address the problems that harmonics can create. By 1992, concern about the issue had grown sufficiently that the Institute for Electrical and Electronic Engineers (IEEE) developed and published its standard 519, "IEEE Recommended Practices and Requirements for Harmonic Control in Electrical Power Systems," which established an approach for setting limits on the harmonic voltage distortion on the utility power system and on the harmonic currents created individual power consumers. Since that time, the electronic loads that give rise to harmonic currents have grown dramatically and are projected to continue growing for the foreseeable future. This being the case, there is and will continue to be a market for technological solutions to the problems that harmonics can cause.

Gross, P. 2002. Needed: New Metrics, Energy User News.

Gruener, J. 2000. Building High-Performance Data Centers, Dell Magazines - Dell Power Solutions, (Issue 3 "Building Your Internet Data Center").

The introduction of Microsoft SQL Server 2000 is a milestone in the race to build the next generation of Internet data centers. These new data centers are made up of tiers of servers, now commonly referred to as server farms, which generally are divided into client services servers (Web servers), application/business logic servers, and data servers supporting multiple instances of databases such as SQL Server 2000.

Halper, R. F. 1985. Computer data center design : a guide to planning, designing, constructing, and operating computer data centers. New York, Wiley.

Hanley, J. K. 1990. Data Center Design Addendum, Datamation, 36 (3): 12-12.

Heller. Life-Cycle Infrastructure Risk Management : R&D Needs

Hellmann, M. 2002. Consultants Face Difficult New Questions in Evolving Data Center Design, Energy User News.

While few data center design projects are alike, there are always the twin challenges of "power and fiber." In addition, sometimes, even local politics and human factors. The paper suggested that the consultant should be brought in as soon as a business case is established so criteria can be established and a concept can be developed, priced, and compared to the business case. A planning is necessary before moving on to site selection and refine the concept and again test the business case.

Ho, E. 2000. Data Centers Meeting the e-Business Challenge, Dell Magazines - Dell Power Solutions, (Issue 3 "Building Your Internet Data Center").

The enormous impact of the Internet is creating a shift in business computing. Companies can no longer afford to focus primarily on their back-end systems and internal computing requirements. Instead, they must redirect significant resources to establish and enhance their entire Internet infrastructure, including the customer-facing applications. Already, these changes are having a major effect on business strategies, creating new criteria for success that virtually every IT organization will have to address. The focus of this paper is achieving 100 percent uptime with the integration of applications, tools, and infrastructure. Achieving 100 percent uptime requires effective tactics and strategies that address single-server bottlenecks, "scaling out" and "scaling up," single-site designs, and multi?site architectures. This issue provides insight and discusses best practices in uptime strategies for mission-critical e-business data centers.

Howe, B., A. Mansoor, and A. Maitra. 2001. Power Quality Guidelines for Energy Efficient Device Application - Guidebook for California Energy Commission (CEC). Final Report to B. Banerjee, California Energy Commission (CEC).

Energy efficiency and conservation are crucial for a balanced energy policy for the Nation in general and the State of California. Widespread adaptation of energy efficient technologies such as energy efficient motors, adjustable speed drives, improved lighting technologies will be the key in achieving self sufficiency and a balanced energy policy that takes into account both supply side and demand side measures. In order to achieve the full benefit of energy efficient technologies, these must be applied intelligently, and with clear recognition of the impacts some of these technologies may have on power quality and reliability. Any impediment to the application of these energy efficient technologies by the customers is not desirable for the overall benefit to energy users in California. With that in mind EPRI and CEC has worked to develop this guidebook to promote customer adaptation of energy efficient technologies by focusing on three distinct objectives. 1) Minimize any undesirable power quality impacts of energy-saving technologies; 2) Understand the energy savings potential of power quality-related technologies. These include: Surge Protective Devices (SPDs) or Transient Voltage Surge Suppressors (TVSS), Harmonic Filters, Power Factor Correction Capacitors, Electronic Soft Starters for Motors; and 3) How to evaluate "black box" technologies.

Intel. 2002a. Data center how-to: how do I build a successful e-business data center?, Intel Corp. © 2002 Intel Corporation. Aug 01, 2002.

The e-Business data center is a potential profit center for service providers. The ability to provide these robust services to customers raises significant new challenges for traditional service providers in the areas of performance, scalability, availability, and security. Proper planning is the key to success in any data center venture: 1) Facilities: including building, security, power, air-conditioning and room for growth, 2) Internet connectivity: performance, availability and scalability, and 3) Data center management: monitoring, support, and troubleshooting.

Intel. 2002b. Planning and Building a Data Center - Meeting the e-Business Challenge, Intel Corp. © 2002 Intel Corporation. Aug 01, 2002.

The paper discusses the keys to success of Internet Service Providers (ISPs) that include 1) Achieve the economies of scale necessary to support a low price business model; 2) Offer added value, typically in the form of specialized services such as applications hosting to justify a premium price. This document provides a high-level overview of the requirements for successfully establishing and operating an Internet data center in today's marketplace. It offers some of the key steps that need to be taken, including project definition, prerequisites and planning. In order to construct a data center that can meet the challenges of the new market, there are three basic areas of data center definition and development: 1) Facilities: including building, security, power, air-conditioning and room for growth; 2) Internet connectivity: performance, availability and scalability; 3) Value-added services and the resources to support their delivery: service levels, technical skills and business processes. The aim is to provide customers with the physical environment, server hardware, network connectivity and technical skills necessary to keep Internet business up and running 24 hours a day, seven days a week. The ability to scale is essential, allowing businesses to upgrade easily by adding bandwidth or server capacity on demand.

Isaac, R. 2001. Influence of Technology Directions on System Architecture. The 10th International Conference on Parallel Architectures and Compilation Techniques (PACT'01). P. G. C. Mateo Valero. Barcelona Hilton Hotel, Barcelona, Spain.

Kathan, D., and T. Grahame. 2001a. Internet Data Centers & Electricity Growth: Will the Emerging Power Crisis Burn Your Bottom Line, Part 1 of 2, Published in Broadband Wireless Business, Vol. 2 (5): 24-36.

This article explores the context in which Internet data centers' requests for dense electricity loads suddenly appear and how much electricity they are likely to draw.

Kathan, D., and T. Grahame. 2001b. Internet Data Centers: Demands for Electricity Proceed Unabated, Part 2 of 2, Published in Broadband Wireless Business, Vol. 2 ( 6): 44-47.

This article further discusses the impacts of increased power demand by Internet data centers on local and national electricity providers and regulators.

Kawamoto, K., J. G. Koomey, B. Nordman, R. E. Brown, M. A. Piette, M. Ting, and A. K. Meier. 2001. Electricity Used by Office Equipment and Network Equipment in the U.S., Energy: 50.

The authors examined energy use by office equipment and network equipment at the end of 1999. The study found that total direct power use by office and network equipment is about 74 TWh per year, which is about 2% of total electricity use in the U.S. When electricity used by telecommunications equipment and electronics manufacturing is included, that figure rises to 3% of all el electricity use (Koomey 2000). More than 70% of the 74 TWh/year is dedicated to office equipment for commercial use. Power management currently saves 23 TWh/year, and complete saturation and proper functioning of power management would achieve additional savings of 17 TWh/year. Furthermore, complete saturation of night shut down for equipment not required to operate at night would reduce power use by an additional 7 TWh/year.

King, R. 2002. Optimizing Data Center Energy Efficiency Results in Cost Savings, theWHIR.com.

While HVAC is a mandatory requirement in the Web hosting industry and serves a critical function, it is also noted as one of the most costly operational expenses in a data center. Indeed, HVAC has been estimated to account for between 40 to 60 per cent of power use in data centers, while servers usually account for 20 percent. Internet data center operators can optimize their cost savings by implementing energy-efficient technologies.

Koomey, J. 2000. EETD Study explores electricity use of office and network equipment, Environmental Energy Technologies Division Newsletter, V2(N2).

EETD researchers are studying the energy use and environmental impacts of PCs, networks, and other technologies of the new economy. They are doing this work with the primary tool of the new economy-the Internet-in collaboration with researchers at universities and private corporations. The Network for Energy, Environment, Efficiency, and the Information Economy (N4E) is the brainchild of Environmental Energy Technologies Division scientist Jonathan Koomey, head of a research group that studies the energy consumption of end-use technologies such as office equipment and household appliances.

Koomey, J. G. 2000. Rebuttal to Testimony on 'Kyoto and the Internet: The Energy Implications of the Digital Economy'. Ernest Orlando Lawrence Berkeley National Laboratory, LBNL- 46509. Berkeley, CA, August 2000.

Koplin, E. 2000. Finding Holes In The Data Center Envelope, Engineered Systems. Engineered Systems, September 2000.

The paper addresses importance of environmental control in data center facilities. Maintaining data center availability requires reliable infrastructure. A significant amount of this is devoted solely to maintaining stable environmental parameters. Only constant, thorough regulation and testing of these parameters ensures the integrity of the data center "envelope."

Laitner, J. A. S., J. G. Koomey, E. Worrell, and E. Gumerman. 2001. Re-estimating the Annual Energy Outlook 2000 Forecast Using Updated Assumptions about the Information Economy. Allied Social Science Association Meetings: Society of Government Economists Session - American Economics Association. New Orleans, LA. LBNL-46418.

This paper evaluates the potential influence of the emerging "Internet economy" on the nation's forecasted energy use and related carbon emissions. Our analysis shows small but significant differences in projections from just three modest changes in assumptions about the Internet and/or information-based economy. Other changes that could be important include reduced inventory and warehousing, reduced building construction, changes in other materials utilization, a greater trend toward outsourcing of energy services, and greater improvements in energy efficiency made possible by the new economy. Although the scale of impact for these initially analyzed effects is small, expansion of the scope of inquiry, as suggested by other recent analyses, may open up new and fruitful areas of research.

Ledford, J. 2001. Are California's Data Centers at Higher Risk? ASPNews.com Magazine, Issue 2000/04.

The news article argues that California might not be the ideal place for a data center, and that companies require an enormous amount of power, and data centers are not using the same kinds of back-up equipment that a utility company would use. Added to the power issues, California is well known for its natural disasters, such as wild fires and earthquakes. The biggest and most immediate data center issue - power resources - do not affect just California, however. Already, Manhattan has been threatened with a similar, though less-severe situation than California. The article addresses the reliability and cost issues of a fully redundant system for needs of disaster recovery.

Leventhal, L. A. 2001. Proceedings of Internet Appliance Workshop 2001. San Jose, California, Internet Appliance Workshop.

López de Vergara, J.E., J. Guijarro, and P. Goldsack. 2001. Modelling and developing the information to manage an Internet Data Centre.

New Internet Data Centres are emerging that contain thousands of servers. In the near future, they may contain tens of thousands. Such scale brings with it a fundamental discontinuity in the ways in which these Data Centres are managed. Administering this number of servers, all of them connected to the network, not only implies the control and monitoring of their performance, but also other aspects such as their fault-tolerance, security, accounting and configuration. This paper discusses the different possible approaches to define and prototype a data model to manage the configuration of such centres in an easy and flexible way so the management system can deal with the required scale.

Lyke, H. 1999. IT Automation: The Quest for Lights Out, 1/e, Prentice Hall Professional Technical Reference.

Mahalingam, R., and A. Glezer. 2002. Air cooled heat sinks integrated with synthetic jets. 2002 IEEE Inter Society Conference on thermal phenomena, Institute of Electrical and Electronics Engineers: 285-291.

Describe active air-cooled heat sinks using synthetic jet ejector for high power dissipation electronics.

Mahedy, S., D. Cummins, and D. Joe. 2000. Internet Data Centers: If Built Will They Come? Salomon Smith Barney, New York, NY, August 3, 2000.

Mandel, S. 2001. Rooms that consume - Internet hotels and other data centers inhale electricity, Electric Perspectives, Vol. 26 (No.3).

The article estimated that the amount of this data center space in the United States nearly doubled in 2000, totaling between 19 million and 25 million square feet by year-end, according to investment analysts. They say they expect another 10 million to 20 million square feet of new space to be added in 2001. Developers are asking electric utilities to supply the buildings with 100-200 watts of electricity per square foot. Since these data centers are new to the economy, there is little historical data on which to base estimates of electricity use for a facility. In addition, the dot.com world makes it difficult for the developer to say confidently how much electricity one of these Internet hotels will use. Source One estimates that tens of billions of dollars worth of electric infrastructure improvements will be needed for data centers over the next few years and that they will consume billions of dollars more worth of electricity. "The energy costs are as high or higher than the actual lease costs. Indeed, 50-60 percent of the cost of building a data center is for the power, including batteries, backup generators, and air-conditioning, as well as the cost for utility construction.

Mannion, A. 2001. Powering Mission Critical Enterprises. National Association of State Energy Officials (Energy Outlook Conference Presentations), Alexandria, Virginia 22314, February 25-28, 2001.

McCusker, T. 1991. Easing the Data Center Squeeze, Datamation, 37 (14): 53-54.

McGrath, J. 2000. Dell IT Data Center - A Standards and Best-Practices Perspective, Dell Magazines - Dell Power Solutions, (Issue 3 "Building Your Internet Data Center").

The challenges for information technology departments involve not only the effective and efficient deployment of the latest technologies, but also procedures and processes that ensure success. The standards and best practices used in the Dell data center reflect an on-going emphasis on providing robust, stable systems-efficiently and cost-effectively.

META. 2002. Dual Data Centers: A Practical Road to Recovery, IT World.com.

Mills, M. P. 1999. The Internet Begins with Coal: A Preliminary Exploration of the Impact of the Internet on Electricity Consumption. The Greening Earth Society, Arlington, VA, May 1999.

Mitchell-Jackson, J. 2001. Energy Needs in an Internet Economy: A Closer Look at Data Centers. July, 2001.

This study explains why most estimates of power used by data centers are significantly too high, and gives measured power use data for five such facilities. Total power use for the computer room area of these data centers is no more than 40 W/square foot, including all auxiliary power use and cooling energy. There are two draft journal articles from this work, one focusing on the detailed power use of the data center we've examined in most detail, and the other presenting the aggregate electricity use associated with hosting-type data centers in the U.S.

Mitchell-Jackson, J., J. G. Koomey, M. Blazek, and B. Nordman. 2001a. National and Regional Implications of Internet Data Center Growth. Resources, Conservation, and Recycling.

The electricity consumption of Internet data centers is a growing concern to utility demand forecasters, data center facility managers, energy analysts, and policy makers. Using estimates of U.S. computer room floor space and total computer room power density, we found that U.S. data centers in the aggregate require less than 500 MW of power, and use only a fraction of a percent (less than 0.12% in 2000) of the electricity consumed nationwide. In this paper, our order-of-magnitude estimate suggests that energy demands of Internet data centers do not represent an enormous new burden on the electricity industry as a whole. The fact that these facilities tend to be concentrated in certain areas, however, may mean that there will be significant regional electricity demands in some parts of the country.

Mitchell-Jackson, J., J. G. Koomey, B. Nordman, and M. Blazek. 2001b. Data Center Power Requirements: Measurements From Silicon Valley, Energy-the International Journal.

Current estimates of data center power requirements are greatly overstated because they are based on criteria that incorporate oversized, redundant systems, and several safety factors. Furthermore, most estimates assume that data centers are filled to capacity. For the most part, these numbers are unsubstantiated. Although there are many estimates of the amount of electricity consumed by data centers, until this study, there were no publicly available measurements of power use. This paper examines some of the reasons why power requirements at data centers are overstated and adds actual measurements and the analysis of real-world data to the debate over how much energy these facilities use.

Montgomery, D. B. 2002. Long-term Availability of the SurePower System Installation at the First National Technology Center. MTECHNOLOGY, Inc: 5.

The SurePower System as built for the First National Bank of Omaha meets and exceeds the terms of the contract, which requires an independent failure unavailability (IFU) of 10-5, with a target IFU of 10-6, measured at the distribution bus. A probabilistic risk assessment model of the as-built system, adjusted for field changes as described in the body of the report, yields an estimated IFU of 5x10-8. This corresponds to availability, excluding common-cause failures such as human error, of 99.999995%. The revised estimate of independent failure unavailability is slightly higher than the value calculated for the initial report. The initial value of 3x10-8, corresponds to an availability of 99.999997%. The small change is primarily due to the expected failures of belt-driven fans installed in the field. The calculated IFU is so small that a loss of power to the distribution bus due to random equipment failure is extremely unlikely over the 20-year anticipated lifetime of the system. Common-cause failures, including manufacturing defects, human errors, and catastrophic events not included in the mathematical model of the system, are much more likely to cause a loss of power than independent equipment failures. As outlined in the original report, a hypothetical common-cause failure with a frequency of once per century and a duration of one day will increase the IFU from 5x10-8 to 5x10-5, a 1000-fold increase in the likelihood of failure. The corresponding reduction in availability from 99.999995% to 99.9995% would result in failure of the electric power supply being about as likely as failure of the best available mainframe computers. Both this report and the original report discuss the mechanisms of common cause failures and provide suggestions for mitigating the risks posed by some of the identified common-cause failure mechanisms. SurePower continues to pursue a comprehensive course of training and reporting for all personnel who will operate and maintain the system. This will reduce the rate of human errors. The bank also has a responsibility and strong motivation to operate and maintain the equipment in a manner that will disclose and mitigate common-cause hazards to the system before they cause significant failures. This report suggests additional system-level testing and certain changes to the physical plant to eliminate some of the more frequent sources of failure.

Nordham, D., R. Reiss, and J. Stein. 2001. Delivering Energy Services to Internet Hotels and Other High-Density Electronic Loads: Part I: Structure of the HiDEL Industry, E-Source, a unit of Platts.

Pal, A., Y. Joshi, M. H. Beitelmal, C. D. Patel, and T. Wenger. Design and Performance Evaluation of a Compact Thermosyphon.

Thermo-siphons are a great prospect towards cooling of high heat dissipating electronics. The implementation of a compact thermosyphon for cooling of a Pentium 4 microprocessor in a Hewlett Packard Vectra PC is being presented in this paper. The thermo-siphon involves four components in a loop: an evaporator with a boiling enhancement structure, a rising tube, a condenser, and a falling tube. Experiments are done to assess the effects of working fluids and of system inclination on the performance of the thermo-syphon loop. Numerical simulations are performed to reflect on the efficiency of the condenser in natural convection condition and the effect of the system fan on the performance of the condenser.

Patel, C. D., C. E. Bash, C. Belady, L. Stahl, and D. Sullivan. 2001. Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Air Specifications. Reprinted from the proceedings of the Pacific Rim ASME International Electronic Packaging Technical Conference and Exhibition (IPACK 2001), © 2001, ASME.

Due to high heat loads, designing the air conditioning system in a data center using simple energy balance is no longer adequate. Data center design cannot rely on intuitive design of air distribution. It is necessary to model the airflow and temperature distribution in a data center. This paper presents a computational fluid dynamics model of a prototype data center to make the case for such modeling.

Patel, C. D., R. Sharma, C. E. Bash, and A. Beitelmal. 2002. Thermal Considerations in Cooling Large Scale High Compute Density Data Centers. 8th ITHERM Conference. San Diego CA.

A high compute density data center of today is characterized as one consisting of thousands of racks each with multiple computing units. The computing units include multiple microprocessors, each dissipating approximately 250 W of power. The heat dissipation from a rack containing such computing units exceeds 10 KW. Today's data center, with 1000 racks, over 30,000 square feet, requires 10 MW of power for the computing infrastructure. A 100,000 square foot data center of tomorrow will require 50 MW of power for the computing infrastructure. Energy required to dissipate this heat will be an additional 20 MW. A hundred thousand square foot planetary scale data center, with five thousand 10 KW racks, would cost ~$44 million per year (@ $100/MWh) just to power the servers & $18 million per year to power the cooling infrastructure for the data center. Cooling design considerations by virtue of proper layout of racks can yield substantial savings in energy. This paper shows an overview of a data center cooling design and presents the results of a case study where layout change was made by virtue of numerical modeling to avail efficient use of air conditioning resources.

PG&E. 2001. Data Center Energy Characterization Study. Pacific Gas and Electric Company (subcontractor: Rumsey Engineers), San Francisco, Feb. 2001.

Rumsey Engineers, Inc. and PG&E have teamed up to conduct an energy study as part of PG&E's Data Center Energy Characterization Study. This study will allow PG&E and designers to make better decisions about the design and construction of data centers in the near future. Three data centers in the PG&E service territory have been analyzed during December 2000 and January 2001, with the particular aim of determining the end-use of electricity. The electricity use at each facility was monitored for a week each. At the end of the report are a set of definitions, which explain the terms used and the components in making each calculation. The three data centers provide co-location service, which is an unmanaged service that provides rack space and network connectivity via a high capacity backbone. About half or more of the electricity goes to powering the data center floor, and 25 to 34 percent of the electricity goes to the heating, air conditioning, and ventilation equipment. The HVAC equipment uses a significant amount of power and is where energy efficiency improvements can be made. All three facilities use computer room air conditioning (CRAC) units, which are stand-alone units that create their own refrigeration and circulate air. A central, water-cooled chilled water system with air handlers and economizers can provide similar services with roughly a 50% reduction in cooling energy consumption. Energy density of the three buildings had an average of 35 W/sf. The cooling equipment energy density for the data center floor alone averaged at 17 W/sf for the three facilities. The average designed energy density of the three data centers' server loads was 63 W/sf, while the measured energy density was 34 W/sf. An extrapolated value was also calculated to determine what the server load energy density would be when fully occupied. The average extrapolated energy density was 45 W/sf. Air movement efficiency varies from 23 to 64 percent between the three facilities. Cooling load density varies from 9 to 70 percent between the three facilities.

Planet-TECH. 2002. Technical and Market Assessment for Premium Power in Haverhill. Planet-TECH Associates for The Massachusetts Technology Collaborative, Westborough, MA 01581-3340, Revision: February 20, 2002.

This study is pursued under contract to the Massachusetts Technology Collaborative, in response to a request for a "Technical and Market Assessment." It seeks to determine if the provisioning of "premium power" suitable for data-intensive industries will improve the marketability of a Historic District mill building in Haverhill. It is concluded that such provisioning does improve the marketability, however, not to a degree that is viable at this time. Other avenues for energy innovation are considered and recommendations for next steps are made.

• Many mill era buildings may not be suitable for data centers or other mission-critical facilities central to the needs of modern industries.
• Power reliability problems in historic mill districts may already be an impediment to economic growth. Both the buildings and the district level service may be in need of significant improvements.
• Premium power is an essential, but misunderstood element in modern business operations. Power outages and data center crashes caused by power fluctuations are major sources of business losses.
• Fuel cells and fuel cell-based data centers may be important solutions. Fuel cells sit at a nexus of three trends of vital import to cities seeking to improve the quality of business and resident life. These trends are the growing needs for reliable power, green (or renewable) power, and distributed generation.
• State and federal financial incentives are applicable to fuel cell project planning and installation.

Posey, M. A., and C. Munroe. 2000. U.S. Web Hosting Services: Market Forecast and Analysis, 1999-2004. Copyright 2000 IDC.

This report covers the market opportunity for U.S.-based Web hosting service providers that maintain Internet data centers in the United States and abroad. Web hosting services include rack space and Internet connectivity for servers that host the Web-based applications of business customers, including large enterprises, small and medium-sized businesses, and third parties that offer Web site and application services from their own servers and systems located in these data centers. In addition, hosting service providers offer management services covering the administration, maintenance, and monitoring of customer servers or sites and applications on shared-server platforms. Revenue includes recurring monthly fees for standard services and excludes professional services billed on a time-and-materials basis and the resale of hardware or software licenses. The report provides an overview and analysis of market structure, segmentation, and other characteristics, including typical service offers, pricing, and market strategies. This report also shows IDC's market sizing for 1999 and the first half of 2000 and forecasts market growth for the 1999-2004 time frame. Finally, the report presents service providers' 1999 and first-half 2000 market shares as well as detailed profiles of the top 20 industry players.

Reiss, R. 2001. Delivering Energy Services to Internet Hotels and Other High-Density Electronic Loads " (HiDELs) - A Summary of Presentations Summit of the E SOURCE Multi-Client Study. E-SOURCE, Broomfield,Colorado, May 1,2001.

RMI, and D&R International. 2002. Energy Efficient Data Centers - A Rocky Mountain Institute Design Charrette. Organized, Hosted and Facilitated by Rocky Mountain Institute, with D&R International, Ltd. and Friends. Hayes Mansion Conference Center, San Jose, California.

Rapid growth of "mission critical" server-farm and fiber-optic-node data centers has presented energy service providers with urgent issues. Resulting costs have broad financial and societal implications. While recent economic trends have severely curtailed projected growth, the underlying business remains vital. The current slowdown allows us all some breathing room-an excellent opportunity to step back and carefully evaluate designs in preparation for surviving the slowdown and for the resumption of explosive growth. Future data center development will not occur in the first-to-market, damn-the-cost environment of 1999-2000. Rather, the business will be more cost-competitive, and designs that can deliver major savings in both capital cost (correct sizing) and operating cost (high efficiency)-for both new build and retrofit-will provide their owners and operators with an essential competitive advantage.

RMI, and D&R International. 2001. Draft proposal for a data center energy design charrette - Efficiently powering the digital economy.

Robertson, C., and J. Romm. 2002. Data Centers, Power, and Pollution Prevention - Design for Business and Environmental Advantage. The Center for Energy and Climate Solutions; A Division of The Global Environment and Technology Foundation, June 2002.

Computers and other electronic equipment will crash at the slightest disruption or fluctuation in their supply of electricity. The power system was not designed for these sensitive electronic loads and is inherently unable to meet the technical requirements of the information economy. For data centers, which play a central role in the information economy, crashing computers cause potentially catastrophic financial losses. The same voltage sag that causes the lights to dim briefly can cause a data center to go off-line, losing large sums of money, for many hours. Data center owners and their power providers must therefore solve several related technical and economic electric power problems. These are: 1) How to assure high-availability (24x7) power supply with a very low probability of failure; 2) How to assure practically perfect power quality; and 3) How to manage risk while minimizing capital and operating expenses

Romm, J., A. Rosenfeld, and S. Herrmann. 1999. The Internet Economy and Global Warming - A Scenario of the Impact of E-commerce on Energy and the Environment. The Center for Energy and Climate Solutions - A Division of The Global Environment and Technology Foundation, Washington DC, DECEMBER 1999.

This paper explores the impact of the growing Internet economy on current and future trends in energy consumption. The paper addresses the need to understand the potential created for environmental gains and structural reductions in energy and resource use, as well as the need for certain industries to adapt to very large strategic challenges and opportunities. This paper reflects an analysis of currently available but incomplete data, and begins to construct some rough scenarios which hopefully begin the process of identifying opportunities and challenges for business leaders and policy makers and suggesting the directions of future research and initiatives. These dynamics will fundamentally shape the path to sustainability in the US and around the world.

Roth, K. W., Fred Goldstein, and J. Kleinman. 2002. Energy consumption by office and telecommunications equipment in commercial buildings, Volume I: Energy Consumption Baseline. Arthur D. Little (ADL), Inc., 72895-00. Cambridge, MA, January 2002.

ADL carried out a "bottom-up" study to quantify the annual electricity consumption (AEC) of more than thirty (30) types of non-residential office and telecommunications equipment. The study developed technically-detailed and carefully laid out AEC estimates for the major equipment types to assist in the planning of future research and development, deployment and standards programs. A preliminary AEC estimate for all equipment types identified eight key equipment categories that received significantly more detailed studied and accounted for almost 90% of the total preliminary AEC. The Key Equipment Categories include: Computer Monitors and Displays, Personal Computers, Server Computers, Copy Machines, Computer Network Equipment, Telephone Network Equipment, Printers, Uninterrupted Power Supplies (UPSs). The literature review did not uncover any prior comprehensive studies of telephone network electricity consumption or uninterrupted power supply (UPS) electricity consumption. Thus, this study is the first to address the energy consumption of both telephone networks and UPSs. The AEC analyses found that the office and telecommunications equipment consumed 97-TWh of electricity in 2000. The report concludes that commercial sector office equipment electricity use in the U.S. is about 3% of all electric power use. The ADL work also creates scenarios of future electricity use for office equipment, including the energy used by telecommunications equipment.

Schaeffer, H. 1981. Data center operations: a guide to effective planning, processing, and performance. Englewood Cliffs, N.J., Prentice-Hall.

Schubart, J. R., and J. S. Einbinder. 2000. Evaluation of a data warehouse in an academic health sciences center, International Journal of Medical Informatics, 60 (3): 319-333.

Snyder, R. G. 2000. Washington's data center growing pains, TechNews.com Magazine, (Issue 2000.04).

The Washington, D.C., Zoning Commission acted appropriately recently in placing an emergency moratorium on the development of telecommunications switching centers in the District. With requests for about 1.5 million square feet of such space coming in, the city had to confront the likely adverse consequences of having significant portions of prime real estate committed to uses that were not in keeping with its expectations for high-tech development. The use of its emergency powers is, of course, not the ideal way of handling land-use planning. The city was caught off-guard by the demand for data center space and did not have a planning structure in place to deal with these and related high-tech planning issues.

Spinazzola, R. S. 2002. The Next Evolution of Data Center Design: New Concept Is Energy Efficient, Cost Effective, Buildings - Free Newsletters from Buildings.com, Feature (June).

Today's approach to data center design has not changed for over 30 years, and therein lies the problem. The computer equipment has changed drastically, but the cooling approach has not. Current approach is to place all of the electronic equipment in either racks or cabinets sitting on a raised floor. In most facilities, the raised floor acts as a supply plenum to deliver conditioned air into the room to cool the equipment and can serve as wire management for power, data, or both. Air-conditioning units (ACUs) also sit on the raised floor and provide cool air into the raised floor plenum to cool the room and take their return air directly from the room. The existing method cannot effectively cool the heat dissipation generated by today's computers. The author points out that cooling the entire room to cool the equipment does not remove the heat from the computer cabinets fast enough. Today's computers dissipate heat at up to 10 times faster than what was produced just three years ago. The result is persistent hot spots and equipment failure.

Spinazzola, R. S., and Z. Menachery. 2001. High Delta T Cooling provides increased energy efficiency and reliability for data centers. RTKL Associates Inc., Baltimore, MD.

The HDTC server rack is engineered to reduce the energy required for cooling the server equipment by 16%. The patent pending design utilizes a specially designed airflow system that allows for a 40 degree F change in temperature within the cabinet and thereby cuts the airflow requirement in half. The HDTC rack is designed as an integral part of the cooling system. Cooling air enters directly through the bottom of the rack via integral supply fans to a specially designed vertical panel in the front door. Air is distributed across the electronic equipment, picking up heat, through a specially designed back panel. Return air discharges through integral exhaust fans directly to the computer room air-conditioning unit (CRACU). The cooling capacity of each CRACU is doubled when the air is delivered and returned directly to the cabinet. The doubling of capacity cuts the airflow requirements in half resulting in a 16% reduction in cooling plant energy. Fewer computer racks and CRACUs will be required. In addition to the lowering of energy costs, the use of HDTC racks has the potential to lower the cost of construction 7%. This additional savings comes from smaller building footprint, less raised floor area, smaller central plant equipment sizing and distribution systems including emergency and normal power and normal distribution systems.

Stein, J. 2002. More Efficient Technology Will Ease the Way for Future Data Centers. 2000 ACEEE Summer Study on Energy Efficiency in Buildings. Asilomar, CA., American Council for an Energy Efficient Economy.

Two years ago, numerous articles began to appear in the news media predicting that a surge of new data center development would overwhelm the U.S. electric power supply. That power crisis never materialized, probably for three reasons: many of the data centers that were planned were never built, many of the ones that were built ran at low occupancy rates, and those that filled up drew much less power than anticipated. The potential remains, however, that a future wave of data center development could bring on such a crisis. As the U.S. economy improves, consumers are likely to demand more Internet-based services. Furthermore, the amount of power drawn by the electronic equipment installed in data centers has been continually increasing. If these trends continue, existing data centers could fill up with equipment so power-dense that they stress on-site infrastructure, as well as local transmission and distribution systems. Although such stresses might lead to some local service disruptions, they will likely be short-lived, as new technology will improve the efficiency of virtually every energy-consuming system found in data centers, including servers, digital storage, and air conditioning. One early example of such a technology is a new Internet server that allows data centers to pack in eight times as many servers without pushing up their electric bills. Other more efficient technologies are sure to follow.

Stevens, L. 1989. The Open Data Center. Datamation 35(15): 55-56.

Sullivan, R. F. 2002. Alternating Cold and Hot Aisles Provides More Reliable Cooling for Server Farms. The Uptime Institute.

The creation of "server farms" comprising hundreds of individual file servers has become quite commonplace in the new e-commerce economy, while other businesses spawn farms by moving equipment previously in closets or under desktops into a centralized data center environment. However, many of these farms are hastily planned and implemented as the needed equipment must be quickly installed on a rush schedule. The typical result is a somewhat haphazard layout on the raised floor that can have disastrous consequences due to environmental temperature disparities. Unfortunately, this lack of floor-layout planning is not apparent until after serious reliability problems have already occurred.

SVMG. 2001. Silicon Valley Manufacturing Group Projections 2001. SVMG, San Jose State University.

This is the fourth Silicon Valley Projections report by the Silicon Valley Manufacturing Group. As in previous reports, Projections 2001 highlights the latest data in housing, transportation, education, environment, and energy, while providing perspective on future trends that will affect the economic health and quality of life in Silicon Valley. Each section is followed with a series of suggested resources and "opportunities for action" that are available for individuals and organizations interested in becoming more involved in issues affecting Silicon Valley. The objective of Silicon Valley Projections is to provide information and encourage dialogue between our communities and public and private sector leaders. Furthermore, SVMG encourages the development of regional partnerships that will facilitate a plan to prepare employers, employees, and their families for tomorrow, and ultimately improve the quality of life.

The Uptime Institute. 2000a. Heat-Density Trends in Data Processing, Computer Systems, and Telecommunications Equipment. The Uptime Institute, Version 1.0.

This white paper provides data and best available insights regarding historical and projected trends in power consumption and the resulting heat dissipation in computer and data processing systems (servers and workstations), storage systems (DASD and tape), and central office-type telecommunications equipment. The topics address the special needs of Information Technology professionals, technology space and data center owners, facilities planners, architects, and engineers.

The Uptime Institute. 2000b. Site Uptime¨ Procedures and Guidelines for Safely Performing Work in an Active Data Center. The Uptime Institute, Version 1.0.,

The procedures and guidelines offered in this White Paper are provided as a public service without charge by Computersite Engineering and The Uptime Institute in an ongoing effort to improve site infrastructure uptime. For many sites requiring 24 hour X "forever" uptime, the momentary failure of a single 20 Amp circuit breaker can result in hours of information disruption to end users and can easily cost millions of dollars in unrecoverable revenue. Extensive data collected by Computersite Engineering and The Uptime Institute consistently reveals that more than half of all site infrastructure failures are directly related to human activity. The data also indicates most failures would have been prevented if appropriate procedures had been followed. The majority of unplanned downtime events are coincident with technicians installing equipment, pulling cables, reconfiguring or troubleshooting site infrastructure equipment, or performing maintenance, as well as by accidents associated solely with the presence of "outsiders" in computer rooms and critical spaces. (Additional information about human interaction failures is presented in the White Paper "We Have Met the Downtime Enemy, and He Is Us!") However, the probability of human caused incidents is not justification to assume a "fix it after it breaks" mentality. Good maintenance programs pay for themselves on a ratio of four outages prevented for every outage caused. Preventing unplanned downtime is conceptually very simple yet difficult to achieve in practice, because personal accountability and responsibility is required of the people performing work.

Thompson, C. S. 2002. Integrated Data Center Design in the New MillenniumM, Energy User News.

Data center design requires planning and estimating future electrical needs. Designers must accurately predict space and energy requirements, plus cooling needs for new generations of equipment. Importance of data center reliability is discussed.

Turner, W. P. I., and K. Brill. 2000. Industrial Standard Tier classifications define site infrastructure performance. The Uptime Institute.

One of the most common sources of confusion in the field of uninterrupted uptime is what constitutes a reliable data center. As Internet website hosting continues to grow into a major industry, competing companies with data centers of radically different infrastructure capabilities are all claiming to deliver "high availability." With the explosive growth of the Internet comes an increased demand for computer hardware reliability. Information technology customers expect reliability of "Five Nines," or 99.999%. The authors address issues of investments a business makes to achieve "Five Nines" in its computer hardware to protect its mission-critical computing functions, and point out that investments must be accompanied by a solid understanding of how well their site infrastructure can support their availability goals.

Turner, W. P. I., and E. C. Koplin. 2000. Changing Cooling Requirements leave many Data Centers at risk. Computersite Engineering Inc.

As uptime expectations continue to grow and new computer technology replaces mainframe-chilled water with airside cooling, the performance demand on data center HVAC systems is rapidly increasing. Based on a number of recent mechanical system audits, we find many data centers are misusing the cooling equipment already installed, have more equipment than is necessary to provide redundancy, and waste significant amounts of energy. The most pressing mechanical issue we see facing data centers is a lack of integration between heating, ventilating, and air conditioning (HVAC) components and a related lack of understanding among HVAC professionals about the unique and changing thermal-dynamic requirements of computer hardware environments. Most data center cooling problems we observe, including outright failures would have been avoided if the HVAC hardware had been properly integrated into an effective system. You can expect this issue will have even greater impact as the new all air-cooled mainframe hardware incorporating CMOS technology arrives on the raised floor and requires substantially higher densities of fan-side cooling.

Wade. 2002. IT Sector Growth and Electricity Demand in Buildings. EIA

Waller, M. 1989. Planning Data Center Design. Datamation 35(21): 73-74.

Whit, A. 2002. Power Impacts on the Cost of Risk - Tools for Championing Power Quality and Reliability Initiatives from a Risk Management Perspective. 7X24 Exchange. Orlando, FL.

In these days of reduced IT spending, facility managers are finding it harder than ever to convince upper management of the need for power system improvements. However, the bottom line is that the availability of data storage and other high-tech equipment has surpassed that of the backup power systems engineered to support them, necessitating system upgrades. Moreover, the situation is exacerbated by the fact that the quality and reliability issues associated with the nation's electric grid are only expected to grow worse. So how can a facility manager's request for power availability improvements compete against marketing's "plan to increase sales 20 percent with just a small investment here," or operations' "plan to cut costs 20 percent with just a small investment there?" The answer may lie in a company's "risk management portfolio." Most large companies have a risk management organization or executive responsible for analyzing and approving risk reduction initiatives, as well as purchasing items like business interruption insurance. The facility manager who can speak in risk management term stands to receive better funding for power needs. While the CFO may not fully appreciate voltage regulation within the CBEMA curve, he or she will certainly respond to a numerical risk assessment showing that the corporate data center could be out of commission for at least one 16-hour window in the coming years.

Wood, L. 2002. Cutting Edge Server Farms - The blade server debate, new.architectmag July 23, 2002.

A blade is the industry term for a server that fits on a single circuit board, including CPU, memory, and perhaps a local hard disk. Multiple blades are plugged into a chassis, where each blade shares a common power supply, cooling system, and communications back plane. Multiple chassis can then be stacked into racks. By comparison, the conventional approach for rack-mounted servers involves only one server per chassis. A chassis cannot be smaller than one vertical rack unit (1U, or about 1.75 inches high). This limits you to 42 to 48 servers in a standard seven-foot rack. A typical blade chassis is much higher than 1U, but several can still be stacked in a rack, allowing upwards of 300 servers per rack, depending on the vendor and configuration. This compact design offers compelling advantages to anyone operating a high-density server farm where space is at a premium. Indeed, blades are the "next big thing" in servers, and it is probable that any given administrator will have to decide whether to adopt them in the near future.

Notice: This reference list is compiled by the Lawrence Berkeley National Laboratory, and is intended to foster information exchange concerning data centers. Comments or new additions should be addressed to Dr. Tengfang Xu.