Tier 3 Data Center Cooling System Design

Photo: T.Moumiadis, some rights reserved

Following the series of posts related to the Data Centers Cooling system design and based on the operation requirements developed and described in previous post, let's now dive into design details of a Tier III rated system according to Uptime Institute requirements.

The HVAC design of datacenters that follow the above requirements can be managed by assuring design, construction and operation compliance with one of the most industry recognized standards.

This is the Uptime Institute Tier classification.

The Tier classification of a data center can take four rates, namely:

Tier I, Tier II, Tier III or Tier IV.

The Tier class can be unique for a data center and describes criteria to differentiate four classifications of site infrastructure topology based on increasing levels (as numbers increase I to IV) of redundant capacity components and distribution paths.

Scope of this article is to give the HVAC design engineer initial guidelines on how the topology of the DC cooling system should be structured in order to comply with the UI Tier standards requirements. The standard of reference is Tier Standard: Topology [2] which can be requested for download from here.

Carry on with reading this post if interested on a Tier III data center requirements or check Tier I & II Cooling System design or Tier IV Cooling System design respectively.

Reference Standards and regulations

Guidelines, requirements and design criteria in use below are in accordance with the following references:

• ASHRAE – Design Considerations for Datacom Equipment centers / 2^nd ed. 2009,
• Uptime Institute – Data Center Site Infrastructure, Tier Standard: Topology / 2018,

Tier III – Design Criteria

According to Tier Standard: Topology [2] / clause 2.3.1, the fundamental requirements for a Tier III – Concurrently Maintainable Site Infrastructure are:

(a) A Concurrently Maintainable data center has redundant capacity components and multiple independent distribution paths serving the critical environment. For the electrical power backbone and mechanical distribution path, only one distribution path is required to serve the critical environment at any time.

The electrical power backbone is defined as the electrical power distribution path from the output of the on-site power production system (e.g., engine generator) to the input of the IT UPS and the power distribution path that serves the critical mechanical equipment.

The mechanical distribution path is the distribution path for moving heat from the critical space to the outdoor environment. For example:

• Chilled water piping,
• Condenser water piping,
• Refrigerant piping.

(b) All IT equipment is dual powered and installed properly to be compatible with the topology of the site’s architecture. Transfer devices, such as point-of-use switches, must be incorporated for critical environment that does not meet this requirement.

(c) Twelve hours of on-site fuel storage for ‘N’ capacity.

A few important points

Before dive into details of the cooling system design requirements in compliance to Tier criteria, keep first in mind a few more important points from the aforementioned standard [2].

Every data center subsystem and system must be consistently deployed with the same site uptime objective to satisfy the distinctive Tier requirements.

This standard requirement makes clear that certain Tier rating requirements shall be applicable to all mechanical, electrical and building systems that serve the IT space. So whatever we will discuss below about the cooling system for a Tier III data center design are equally applicable and essential for the on-site power production, UPS and storage equipment, fuel tanks and water storage (evaporative cooling) systems as well.

The Tier topology rating for an entire site is constrained by the rating of the weakest subsystem that will impact site operation. For example, a site with a robust Tier IV UPS configuration combined with a Tier II chilled water system yield a Tier II site rating.

So even if the team is quite careful and especially concerned about the cooling system design, but fail to implement the same criteria into the electrical or fuel supply systems, the overall rating of the data center will be lower than the expected.

Design of Tier III Cooling system

The cooling system design of a Tier III rated data center shall comply with the following requirements:

Redundant Capacity components – meaning that there is need for redundancy (backup) of any equipment of the cooling system including and not limited to components like:

• CRAC / CRAH units,
• Chillers, chilled water pumps,
• Cooling towers, condensing pumps,
• AHUs
• Split type DX cooling units,
• Makeup water storage tanks and pumps.

This is what we call “N+1” units configuration or in simple terms the total capacity of all equipment units to be used is equal to the datacenter cooling demand plus at least one more unit.

Multiple independent distribution paths – meaning that the following distribution networks:

• Chilled water piping,
• Condensing water piping,
• Refrigerant copper piping for DX cooling systems,
• Makeup water piping,

shall be designed in a way that the transferred medium (water, fuel, refrigerant) is able to reach any system unit (CRAC, AHU, Chiller, Cooling tower) from multiple paths. This should not be confused in any way to the well known ‘loop’ system design. Multiple independent distribution paths shall be deployed through two chilled water systems where all the terminal units are connected.

Refer to fig. 01 below where a ‘single distribution’ and a ‘redundant distribution’ chilled water piping networks serving the same computer installation are compared.

Fig.01 – the illustrations describe the case of a chilled water system supplying N+1 CRAC units through a (a) two pipe chilled water system constitute a single non-redundant distribution path and (b) four pipe chilled water system, which actually is a group of two independent distribution paths, always one active at a time. Case (b) is a piping system that can be part of a Tier III cooling installation.

Concurrently Maintainable – meaning Each and Every capacity component (CRAC, AHU, Chiller, Cooling Tower, pump) and distribution path element (valves, filters, check valves, regulating valves, meters, instruments, sensors), can be taken out of service for maintenance, repair or replacement without impacting the Critical Environment or IT processes.

A Tier III Concurrently Maintainable data center is an easily and always possible to expand, without disruption, data center.

A concurrently maintainable cooling system shall be designed so that:

All the capacity components are redundant through an N+1 configuration. So that if for example the cooling demand of an IT space is 300kW, the implementation of the system shall include 6 CRAC units of 60kW each in a 5+1 arrangement. There will always be one unit redundant. The same happens for AHUs, pumps, chillers cooling towers etc.

On the other hand, the piping network shall has all the required accessories so that each and every part or fitting will be possible to get isolated and removed or maintained without affecting the chilled water flow to the rest. The key point to design a concurrently maintainable piping system is the selection of valves positioning. Let’s explore together of how isolation valves shall be provided in a water piping system in order to achieve concurrent maintainability.

Consider the pumps system of Fig.02. These pumps serve a piping network where the flow demand is 200m³/h. The design provides a set of three pumps each one sized with a maximum flow capacity of 100m³/h. It comes out that we have a 2+1 configuration where two pumps continually run and the third one is backup. According to Tier 3 requirements, the pump as a capacity component is redundant. Also the concurrent maintainability of each and every component in the system is achieved by providing two valves in the piping section between each pump.

In that case for example assume that valve A is broken and has to be maintained or even removed. The team has only to turn off valves B, C and D and without any disruption to the data center cooling system, that element (valve A) can easily be maintained or replaced. The same happens with all the valves and elements of this arrangement. Make your checks and confirm the concept! Use this arrangement on any of your future pump sets design.

  Fig.02 – A set of three pumps in an N+1 configuration that serve a redundant distribution path (water flow and return is served from both sides). The provision of two valves between each section assures a concurrently maintainable system. A pump system arrangement which is fully compliable with Tier 3 requirements and is applicable to chilled, condense and make up water systems.

The same concept can also be extended to any piping system as shown in Fig.03 below. The arrangement of valves is in a way that any part of the system can be isolated and maintained, without any downtime of the cooling system. Again try to check each and every component and element and you will realize that there is always a set of valves that can achieve isolation of that component or element. Be careful with the example since it is not a Tier 3 system. The piping network cannot be considered as a multiple independent distribution path. It is just a concurrently maintainable system.

Fig.03 – Provision of isolation valves in a single distribution piping network (loop design). They are arranged in a way that allows concurrent maintainability of each and every capacity component or system element.

How to apply these requirements into the design

These requirements can be applied in several cooling type systems like shown in the schematic representations below. A Tier III cooling system can use split type air conditioning systems with refrigerant (Fig.04), or a water type system with CRAC units inside the IT space (Fig.05) or even more advanced solutions with chilled water and air handling units (AHU) that blow air within the IT spaces (Fig.06).

These systems configurations are quite common in the data center industry where redundancy of the 'capacity components' combined with the proper design of distribution paths achieve concurrent maintainability and system reliability that gives the system Tier III rate.

For your understanding explore the schematics below and try to identify all the principles discussed before related to capacity components, distribution paths and system maintainability.

Fig. 04 - A Tier III split system CRAC units developed with DX type systems make use of 'N+1' number of CRAC units in a configuration of N units running and at least 1 redundant. The respective external condensing units and refrigerant pipes all together combine a system.

Fig. 05 - A Tier III water system CRAC unit contain single or more running 'capacity components' all in a 'N+1' configuration. This includes a combination of water chiller, cooling tower, water pumps all together running at the same time and supplying chilled water to 'N' number of running CRAC units. Also the system has at least one more piece of redundant component for every type of capacity equipment. So there is a redundant water chiller, cooling tower, pumps and CRAC unit. Two 2-pipe chilled water systems connect all the chiller and CRAC units as 'redundant distribution paths'. There are provisions of valves in a way that two independent piping networks serve the system in a way that only one is active. Each and every component or element of the system can be maintained or replaced without system shutdown.

Fig. 06 - A Tier III water system Air Handling unit  contain single or more running 'capacity components' all in a 'N+1' configuration. This includes a combination of water chiller, cooling tower, water pumps all together running at the same time and supplying chilled water to 'N' number of running Air Handling units. Also the system has at least one more piece of redundant component for every type of capacity equipment. So there is a redundant water chiller, cooling tower, pumps and AHU. Two 2-pipe chilled water systems connect all the chiller and CRAC units as 'redundant distribution paths'. There are provisions of valves in a way that two independent piping networks serve the system in a way that only one is active. Each and every component or element of the system can be maintained or replaced without system shutdown.

Note that in case of water systems there are also requirements for the makeup water installation and the storage tanks. 

In general a Tier III data center allows all capacity components to be maintained or repaired with disrupting the operation of the critical environment. The same happens for the distribution paths (piping) and individual elements (fittings, valves and accessories). The operational risk that this configuration brings to the computer facility owner is limited but there still exist a risk of disruption during the scheduled maintenance on redundant components. 

I hope that you find this post interesting and educative.

• If you did like it please share it through the social so that more people can have access to it.
• If you have any questions or would like to discuss any special case, please leave your comments below. I will be happy to answer!