Home Insight Is enterprise HPC moving to the cloud?

High-performance computing (HPC) has expanded beyond its original borders of laboratories and universities into the world of SMEs. Having traditionally been the sweetheart of data-hungry scientists, it is now proving its worth in the business world among companies looking for bigger and better ways to process their data.

Is enterprise HPC moving to the cloud - header

HPC is about much more than just crunching numbers. It can help you complete process-intensive tasks much more quickly than your typical desktop, such as:

  • Organising and analysing financial activity, medical records, and other important data
  • Processing 3D modelling programs for new product design
  • Running Machine Learning and Artificial Intelligence solutions

But bringing HPC into your business is a huge investment. This kind of hardware is very expensive and it needs careful treatment. And if you aren’t using it all the time, is it money well spent? These are the issues that have led some businesses to move their HPC operations to the cloud.

[Do you have an HPC hosting requirement? Get in touch]

HPC in the cloud

What does it mean to move HPC to the cloud? Basically, we’re talking about using HPC infrastructure that you don’t actually own – effectively renting HPC from the likes of Microsoft, Google and AWS, or from a data centre, who might also be able to integrate your cloud HPC with other services they are hosting for you.

1.  The benefits of HPC in the cloud

Using HPC in the cloud gives you a flexibility that on-premises HPC can’t offer. You use it when you need it, ignore it (and don’t pay for it) when you don’t. There’s no queue of people desperately trying to complete their workload by month’s end; you have instant access to however many cores you need and it’s entirely scalable to your operations.

2.  The negatives of HPC in the cloud

The negative aspect of HPC in the cloud has historically related to cost, resources and latency.

Cloud Latency – The delay between a request from the end-user, and their cloud service responding to that request. The complicated structure of cloud services can often lead to high latency, and when dealing with the increased data load of HPC, this issue is exasperated.

Even 12 months ago, users might have found the cloud latency unacceptable – but this is improving all the time. Likewise, resources are expanding quickly and so this is less of an obstacle to migration than it once was. However, cost is still a big factor. And while the cost of cloud HPC should be much less than investing in your own HPC infrastructure, it can still be prohibitive. Part of the problem is that the costs are an unknown variable and that makes it hard to budget for.

Perhaps due to the uncertainty of costs, wasted computing power in the cloud is thought to be in the region of 30 – 40% as users are careless with their cloud use.

Hybrid solutions:  Cloud & Onsite HPC

Of course, it doesn’t have to be an either/or choice. Some businesses will utilise HPC on privately owned infrastructure (either on-premise, or hosted in a data centre) and in the cloud, to deal with workload peaks and increased demand.

Other companies will maintain both cloud and private infrastructure and choose how to use them according to the size of the data set and where it is located. If the data is in the cloud, it often makes sense to move the workload to the cloud rather than take on the cost of moving data. Likewise if the data is stored in your private HPC it may be easier – and more cost-effective – to carry out the workload there. Running a hybrid cloud solution for HPC gives you the flexibility to do whatever is right for any given workload.

This hybrid cloud does require privately owned HPC infrastructure in the first place, which is unachievable for a lot of companies and their on-premise infrastructure. However, most will use a data centre that is able to keep up with the intense power and cooling requirements.

[Learn more about HPC’s infrastructure requirements]

An evolving relationship

If we’d written about this topic a year or two ago, the chances are we would have said something along the lines of ‘HPC users have been slow to move to the cloud’. It’s true that, for quite a while, HPC users were unconvinced that the cloud could offer them anything more than what they already had access to in their on-premises HPC.

Two things have changed – or are in the process of changing – since then. First, the big three cloud providers have upped their HPC game. Prices have come down. Resources have gone up. It’s easier to identify the benefits now than it was two years ago.

The second is that HPC users, as a group, have diversified and expanded. Their usage of HPC technology could as easily be ‘little and often’ as it could be ‘huge and seldom’. The ‘as-a-service’ model that has worked so well in other areas of IT is growing in popularity in this segment of the market as businesses come to realise the commercial benefits of HPC.

This being the case, a tailored approach to HPC is likely to be the trend going forwards. Enterprise users will gain a greater awareness of how, where and why they are using HPC and with that knowledge will come an understanding of the best way to manage their HPC use: what to do in the cloud and what they need to have more ownership of.

4D's HPC Capabilities

Across the UK, our data centres have been uniquely built for high density deployments, and our infrastructure reflects this.   All our racks are equipped with power supplies capable of hosting anything up to ultra-high density (around 40kW) without any additional power installation requirement.  The latest rear-door cooling technology comfortably copes with the cooling requirements of a HPC rack and can be installed upon request onto any of our existing racks.

4D’s high and ultra-high density capabilities provide a leading-edge HPC standard that can seamlessly integrate across other systems and services.

If you’re considering hosting HPC within a data centre, please get in touch with 4D or call us on 020 3962 0399 to explore our HPC solutions and how we can best help your company.

Advice on hosting high-performance computing