High performance computing (HPC) requires sophisticated cooling technology – but what type? Conventional air-cooling solutions don’t typically have the cooling capacity to manage the temperatures coming from high-density computing, but both rear-door cooling and immersion cooling are established as ‘HPC-ready’. So which is the best solution for you?
With great power comes great responsibility (for keeping temperatures down)
First of all, let’s quickly recap why HPC needs specialist treatment. A HPC setup includes clusters of CPU and GPU cores that give you much greater data processing capacity – the kind that makes mincemeat of endless data searches, scanning for patterns or insights, and even the modelling that goes into designing super-fast sports cars. HPC is growing in commercial use as more organisations make use of this incredible processing power to utilise and develop machine learning and AI applications.
The technical challenge of HPC lies in power and cooling. Rather than the typical medium-density power deployment of 3 – 7 kW per rack, HPC racks need more like 20 – 40 kW per rack. And where there is power, there is heat – which must be cooled effectively.
How can immersion cooling help?
Immersion cooling is more or less what it sounds like – those power-hungry CPUs and GPUs are immersed in a non-conductive liquid, which transfers the heat from the components into the cooling fluid.
There are two types of immersion cooling – single-phase and two-phase. In single-phase immersion cooling, the dielectric fluid is typically a mineral oil (akin to baby oil) that effectively draws off the heat from the machine. The coolant is then pumped to a heat exchanger where the heat is transferred to a separate cold-water circuit, so the coolant can return to the components to start over again.
In two-phase immersion cooling, a different type of cooling fluid is used that changes state from liquid to gas as it takes on the heat from the equipment. The gas rises, meets a condenser and ‘rains’ back into the tank, cooling the dielectric liquid again.
In both cases, cooling is very efficient in comparison to air cooling. Liquid transfers heat much more effectively than air and because it requires minimal energy input, it's green and cheap to run.
Pros of immersion cooling:
- Simple – very few moving parts, less to go wrong
- Effective – easily cools up to 100 kW per rack
- No fans – this reduces power consumption for both the data centre and the servers (server fans must be disabled prior to immersion)
- Efficient – a PUE of 1.03 is possible with immersion cooling
Cons of immersion cooling:
- Hardware modification is required to remove/disable fans and related systems
- Takes up more space in the data centre as racks are horizontal rather than vertical
- The coolant will need to be drained from the servers before you can carry out maintenance or upgrade work
- You may be paying for more cooling capacity than you need
What about rear-door cooling?
Rear-door cooling is another type of liquid cooling solution, but in this case the liquid is carefully enclosed in pipes and your hardware remains totally dry and accessible.
Cold water is sent to the back of the servers through a network of fully-sealed pipes. Again, water is a more effective conductor of heat than air, so when the hot air from the servers meets the pipes, the heat is transferred into the water, which is then carried away and cooled in a closed-loop system. Here at 4D we use adiabatic cooling, which makes the whole process very energy efficient – enabling us to achieve our industry-leading PUE of 1.12.
Rear-door cooling gives you that contact with the servers without having to modify the equipment or limit access in any way. It’s very efficient and can easily cope with the typical 20 – 40kW power density of HPC racks.
Pros of rear-door cooling:
- Effective – easily stabilises server temperatures
- Efficient – with adiabatic cooling, rear-door cooling is highly energy efficient
- No compressors – with adiabatic cooling there is no need for costly compressed air
- Accessible – rear-door cooling does not impact access to your hardware
Cons of rear-door cooling:
- More moving parts than an immersion cooling system, which could mean more maintenance
- Fans are still needed, which accounts for the increased energy consumption compared to immersion cooling
Which is right for you?
Both rear-door cooling and immersion cooling are very effective, efficient cooling methods. Your choice will likely come down to your budget, your power density requirement and the capability of your managed infrastructure provider.
- Your managed infrastructure provider - Any managed infrastructure provider looking to reduce operational costs and environmental impact at their data centre should be using rear-door cooling or an equivalent system because it is vastly more efficient than conventional compressed air cooling. Immersion cooling, however, is relatively new and not widely adopted among commercial data centres, which means you’re either narrowing the field of data centre options or you’re asking for new infrastructure to be established, which comes with a price tag.
- Budget - The price for either of these methods is only worth paying if you are looking at the upper end of the power density scale. But there is money to be saved by having one 30kW rack compared to three 10kW racks, so the value of these cooling methods might well have an impact on how you spend your budget.
- Power density requirements - For power densities up to 40 kW per rack, rear-door cooling is sufficient – and there is little point in paying for cooling capacity that you don’t need. If you see a future where your power density is likely to go beyond 40kW, immersion cooling will give you the safety net you need to expand processing capacity without having to add rack space.
Best of both
As a managed infrastructure provider, we are as keen as the rest of the ICT industry to find more efficient ways to enable our growing appetite for data without increasing our environmental impact. With that in mind, we have recently added hosting immersion cooling to our repertoire. An immersion cooling unit can be connected into out existed chill water system and run off of our highly efficient adiabatic cooling system, with the servers connected to our reliable power and network supplies.
This means we can now offer both rear-door cooling and immersion cooling for your HPC system – so you get to choose the solution that best suits you. If you’re not sure what that is, or you’d like to learn more about both, get in touch online or give us a call: 020 7183 0602.