Fueling (literally) the AI Boom

Author(s): Aneesh Patil

Originally published on Towards AI.

Let’s take a moment to step back in time to our 5th-grade selves, a nostalgic #Throwback____ (insert today’s date) if you will. Picture ourselves in science class, perhaps doodling at our desks, daydreaming, or diligently listening to the teacher. Regardless of our activities, chances are we’ve encountered the term ‘greenhouse gas’ in one form or another. Just to jog our memories, greenhouse gases are those that trap heat within our atmosphere, leading to rising temperatures, global warming, and various environmental repercussions.

Among the array of greenhouse gases, carbon dioxidestands out as particularly impactful. Its prevalence in our atmosphere is largely attributed to its release during the combustion of fossil fuels for energy production. In fact, approximately 80% of the energy consumed in the United States is derived from petroleum, natural gas, and coal, all of which emit carbon dioxide when burned. This perpetuates a cycle where the combustion process not only generates energy but also contributes significantly to the accumulation of greenhouse gases in our atmosphere.

While the contribution of the information and communications technology (ICT) sector to global CO₂ emissions may seem like a drop in the ocean, it’s worth noting that it accounts for approximately 2%, as highlighted in a 2020 study by the International Telecommunication Union. However, this statistic may be just the tip of the iceberg. Since the study’s publication, significant shifts have occurred within the sector, particularly with the rise of artificial intelligence (AI), which has captured widespread attention and could potentially alter this emission estimate. Yet, despite AI’s burgeoning prominence, companies holding AI models have been notably reluctant to disclose data regarding their models’ energy consumption, often citing ‘competition’ as a reason for their silence. Nevertheless, numerous researchers have endeavored to provide rough approximations. For instance, a report from the New Yorker estimated that ChatGPT alone handles approximately 200 million queries daily, translating to an electricity consumption of around 500,000 kilowatt-hours.

On doing some math, we get

=> 500,000 kWh / 200,000,000 requests

=> 0.0025 kWh/request = 2.5W/request

To provide some context, let’s consider a standard household lightbulb, typically rated at 60W. Leaving this lightbulb on for an hour consumes approximately 0.06 kWh of electricity. Now, when we compare this to ChatGPT, each query made to the model is akin to leaving a lightbulb on for roughly 3 minutes (0.05 hours). It’s essential to note that this calculation is an estimate specific to OpenAI’s ChatGPT.

Among the major Large Language Models (LLMs) in the market are Google’s Gemini, Meta’s LLaMA 3, and Anthropic’s Claude 3. When combined, their daily query volume could reach approximately 600 million, translating to 1,500 megawatt-hours (MWh) of electricity consumption. To put this in perspective, according to the U.S. Energy Information Administration, the average household in the U.S. consumes 11 MWh of electricity annually. Therefore, the energy used by these LLMs in just one day would be comparable to the electricity consumed by approximately 140 households over the course of a year. If this does not seem baffling enough already, Dutch researcher Alex de Vries performed an in-depth study in 2023 to explore trends in energy appetite for AI models and the results are shocking. With the current trajectory, by 2027 NVIDIA will have an estimated 1.5 million server units running AI workloads which will observe 85.4–134 TWh of electricity consumption annually. This is just NVIDIA’s servers! With the competition in the AI hardware space proliferating, NVIDIA’s market share is anticipated to be lower in 2027 than what it is today so we can expect more than 2 million server units running AI workloads at the minimum based on de Vries’ study. On a yearly comparison, 134.0 TWh is equivalent to the amount of electricity consumed by Argentina, a country with over 45 million people.

So, AI is only scraping the surface when it comes to tech’s growing appetite for energy. Think about it — cryptocurrency, edge computing, virtual/augmented reality, robotics, and the Internet of Things are just a handful of technologies poised to demand vast amounts of computational power once they hit the commercial stage at the scale AI enjoys today. And with ongoing research in each of these areas, it’s not far-fetched to imagine them revolutionizing our daily lives in ways we haven’t even dreamed of yet.

But here’s the kicker: all this innovation comes with a hefty energy bill, and considering the limitations of resources and the environmental impact of electricity generation, it seems like a daunting challenge to keep up. Yet, amidst these concerns, there’s reason for optimism. There are promising and sustainable alternatives emerging to challenge our current energy consumption practices.

What could serve as a solid starting point? Well, if you’re wondering if companies can work towards optimizing for a more power-efficient hardware architecture, you might be onto something. A study published by Harvard University states that “a large portion of their energy use is spent simply passing data back-and-forth between chips”. In other words, a computer chip might need to fetch data from an external memory bank and it does so via signals; the larger the distance between the chip and the memory bank, the larger the amount of energy consumed. Gage Hills, an Electrical Engineering professor at Harvard, is researching ways to prevent a chip from sending signals to larger distances to save up on energy. One way is by stacking multiple layers of an integrated circuit in three dimensions and cutting out the need for passing any data outside the chip. The drawback to this approach is that when building these layers, there is a possibility of melting wires in one of the lower layers due to high temperatures. Engineers at the University of Virginia are exploring a way to cut out the intermediary by establishing direct contact between the chip and the memory bank using a material called hafnium oxide. However, in a natural state, this material is not ferroelectric (stores and releases binary information), which is an important characteristic of a chip. Nonetheless, the pursuit of a more power-efficient computer chip is ongoing, with promising advancements like Groq’s Language Processing Unit (LPU) chip. Groq has achieved remarkable energy savings (10x!) by storing all the required data for computation on the chip’s static RAM (on-chip memory) and minimizing external memory access. It’s worth noting that while Groq’s chip boasts low latency and power efficiency, its 230 MB SRAM capacity may seem insufficient for running a large language model (LLM). Moreover, SRAMs can be more expensive than having an external memory bank (e.g. high bandwidth memory) so scaling it could also prove to be cost-ineffective. To take an example, we would require well over 200 such chips to be able to run a 140GB LLaMA 70B model. While Groq’s chip presents groundbreaking advancements, it may not fully address long-term energy consumption concerns, especially as AI workloads grow in complexity and non-deterministic behavior. Therefore, instead of solely focusing on internal measures to mitigate carbon footprints, exploring opportunities to transition energy production toward safer, alternative sources could offer a more sustainable solution.

Renewable energy stands as a pivotal contender in the transition away from reliance on natural gas and fossil fuels for energy production. As indicated by the aforementioned chart, 13% of U.S. energy consumption is attributed to renewable sources, with wind, hydroelectric, and solar power leading the charge. Given that renewable energy sources emit no direct greenhouse gases, they represent a compelling option for long-term investment by tech companies. Notably, industry giants like Amazon, Microsoft, and Google demonstrated their commitment by investing in solar and wind energy plants for their data centers as early as 2021. Amazon’s recent announcement in early 2024 further underscores this momentum, revealing the completion of 500 solar and wind projects capable of generating up to 77 terawatt-hours (TWh) of energy. Strategically locating renewable energy plants alongside data centers worldwide capitalizes on geographical strengths, optimizing energy production. Yet, despite significant strides, the question remains: why hasn’t the transition to renewable energy occurred more rapidly? In my view, two major obstacles hinder this shift: inconvenience and costs. Unlike fossil fuels, solar and wind energy are contingent on ideal weather conditions, limiting their availability on demand. While energy storage solutions like batteries offer potential, their current capacity and cost present challenges. Despite these hurdles, the promise of renewable energy remains strong. However, widespread recognition of its value necessitates a profound paradigm shift. As per inspirecleanenergy.com, one other reason why we are not making the switch yet is due to the “lack of education surrounding sustainable energy”. While we work towards new technological advancements in this space, let’s look into the last (definitely not the least) alternative energy source.

Although nuclear energy has faced contentions and controversies over several decades, it’s challenging to deny its potential to profoundly shape the world. The process of generating nuclear energy typically involves firing uranium pellets, primarily consisting of the U-235 isotope, with neutrons. These neutrons are absorbed by uranium atoms, triggering the creation of U-236, an unstable isotope that releases heat and radiation energy. This phenomenon, known as ‘fission’, initiates a chain reaction as additional neutrons are released, subsequently striking other U-235 atoms. The resulting heat from this chain reaction is immense and is utilized to produce steam, which, in turn, drives turbines to generate electricity. Notably, this process produces none of the harmful byproducts associated with burning fossil fuels and ranks among the top two cleanest sources of energy in the United States.

This chart illustrates that nuclear energy has played a significant role in mitigating 476 million metric tons of CO₂ emissions. This number is comparable to getting rid of one-third of all the cars on the road in the U.S. Nuclear energy boasts several key advantages, including high energy density, reliability, and land efficiency. For comparison, consider the energy output per kilogram of each source:

+-----------+--------------------+---------------+
| Coal | Mineral Oil | Uranium |
+-----------+--------------------+---------------+
| 8 kWh | 12 kWh | 24,000,000 kWh |
+-----------+--------------------+---------------+

Referring to a previous example, if one query takes 0.0025 kWh, then one kilogram of each fuel has the capacity to service these many queries:

+-----------+--------------------+------------------------+
| 8 kWh | 12 kWh | 24,000,000 kWh |
+-----------+--------------------+------------------------+
|3200 queries| 4800 queries | 9 ,600,000,000 queries |
+-----------+--------------------+------------------------+

2 million–3 million times more with nuclear energy! This shows that nuclear power overpowers fossil fuels in terms of better energy density per unit amount and lower carbon emissions. But how does it stack up against renewable sources like wind, solar, and hydropower? Renewable energy’s reliability hinges on factors like sunlight, wind, and water flow, which can vary, unlike nuclear energy’s ability to consistently produce energy around the clock on demand. This reliability offers a stable price for end-users such as households and factories, a significant advantage. Furthermore, in terms of land footprint, wind farms, and solar plants require 360x and 75x acreage to output the same amount of electricity as a 1000 MW nuclear plant. This could lead to potential damage to natural habitats, loss of biodiversity, and environmental disruption. Nuclear energy is also starting to get a lot of traction from industry stalwarts. In March 2024 Amazon Web Services (AWS) bought a 960 MW nuclear power plant from Talen Energy to strengthen their position in alternative energy and power their data centers. Moreover, the CEO of OpenAI, Sam Altman, has substantial investments in nuclear startups — Oklo and Helion Energy. While Oklo is building power plants for nuclear fission (process described above), Helion Energy is taking the nuclear fusion route. Nuclear fusion is the process of generating energy by ‘fusing’ two lightweight nuclei (primarily hydrogen isotopes) to generate enormous amounts of energy. Fusion could generate 4x the energy per kilogram of fission, however, containing this amount of energy is a technological challenge that is being researched. With the upsides out of the way, there are also potential downsides that can come with the nuclear energy approach. From a capital perspective, nuclear power plants require large amounts of upfront costs to build, develop, and manage the advanced infrastructure. From an environmental perspective, nuclear energy releases some waste materials as a byproduct that could remain radioactive for thousands of years. The presence of radioactive waste in the environment can contaminate the air, soil, and water while also posing severe threats to human health as well. Radioactive waste releases radiation as they decay which can cause genetic mutations and damage to DNA cells. Moreover, there have been unfortunate accidents at nuclear power plants such as Chernobyl and Fukushima that have impacted several people and the environment around them. With so many compelling arguments to be made for and against each plausible solution, how do we pick one?

Well, I don’t think we need to narrow it down to a single solution. I believe there is more power in leveraging the strengths of each solution to transition into cleaner, safer energy solutions. For instance, nuclear power could work great as a base load power due to its ability to be available 24/7. Renewable energy sources could be integrated by optimizing for the geographical area. Whenever the conditions for wind, solar, and hydropower are favorable they can be made use of while toning down nuclear parallelly. Moreover, within renewable sources, hydropower is more commonly used for energy storage therefore it could be an alternative when wind and solar energy are not an ideal option. It would also be beneficial if electricity grids were built to be more interconnected to ensure flexibility in transmission. Another option would be to implement ‘load-balancing’ near data centers that run large-scale AI applications. This would mean when the traffic (demand) at a data center is lower than usual, then energy demands would shift to a non-critical workload or would cater to the requests when renewable energy conditions are more ideal.

In conclusion, the potential for transformative breakthroughs in the energy sector is evident through ongoing research endeavors focused on enhancing various aspects of technology and infrastructure. From optimizing computer chips to advancing AI algorithms, and from refining renewable energy storage to addressing challenges in nuclear fusion containment and nuclear waste disposal, each avenue of exploration holds promise for reshaping the energy landscape. Moreover, improvements in the interconnectivity of power grids stand to enhance efficiency and reliability on a systemic level. By harnessing the collective power of these research initiatives, we can drive significant disruptions in the energy industry, welcoming a new era of sustainability, reliability, and innovation.

I hope you enjoyed this blog! Check back for similar tech-related content and don’t hesitate to reach out with any questions in the comments or at [email protected]!

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI