AI in Data Centers – Challenges and Solutions

HodlX Guest Post  Submit Your Post

The exponential surge in demand for AI-powered applications in recent years has necessitated a new approach to data center design, configuration and management.

Wall Street Journal estimates around 20% of the global data center capacity is currently used for AI purposes.

However, with over 77% of companies already using or exploring AI technology, traditional data centers may be going obsolete – fast.

The AI stand-off

Due to their complex algorithms and models, AI applications typically require more power and computing resources than others.

For example, a simple query on ChatGPT requires almost 10 times as much electricity needed to process a quick search on Google.

Traditional data centers are designed with an average density of five to 10 kilowatts per rack, but this increases to 60 or more kilowatts per rack to handle AI applications.

More workload and energy demands equals higher overhead costs.

In addition, data centers have to come up with alternative and advanced ways of dealing with cooling problems, vulnerabilities, security challenges and maintenance issues that can arise due to staffing shortages.

Then there is the question of environmental sustainability. Researchers estimate that GPT-3 generated over 552 tons of CO2 before it was even released for public use in 2020.

This figure is equivalent to the CO2 that would be produced by 123 gasoline vehicles over a full calendar year.

Unfortunately, unless these challenges are strategically and dynamically addressed, we may be looking at an infrastructural tight-rope similar to the GPU supply deficit.

The shortage of data centers fully equipped to handle the overwhelming demands of AI technology may ultimately slow down growth, promote monopolization of AI infrastructure and have serious implications for the environment.

Building for now and the future

To tackle these problems headlong, many companies are already implementing new measures.

These include using collocated data centers to reduce operational costs, promote scalability and ensure the availability of skilled on-site maintenance.

Data centers are also employing more advanced cooling techniques like liquid cooling, direct-to-chip cooling and immersive cooling, as opposed to conventional air cooling systems.

For new centers, design becomes paramount. For example, in 2022, Meta paused the construction of its $800 million data center in Texas to consider redesigning the 900,000-square-foot…