The Nexus of AI and Power Consumption
The rapidly evolving field of artificial intelligence (AI) is continually pushing boundaries and uncovering new challenges. One such challenge is the overwhelming power consumption that AI processing demands, which is posing a significant problem for data center operators. As AI technology continues to evolve, the power grid’s capacity to keep up is becoming increasingly strained, causing operators to throttle back by as much as 30%.
Jensen Huang, CEO of Nvidia, highlighted this issue during his keynote address at the company’s annual GTC customer conference. He pointed out the wastefulness in AI factories, emphasizing that each unused watt of energy equates to a loss of sales. [Source]
Emerging Solutions: Niv-AI
In response to this pressing problem, startup Niv-AI has emerged from stealth mode with $12 million in seed funding. The company’s mission is to tackle this issue head-on by measuring GPU power consumption with new sensors and developing tools to manage it more efficiently. Based in Tel Aviv, Niv-AI was founded last year by CEO Tomer Timor and CTO Edward Kizis.
Notably, the startup is backed by several high-profile investors, including Glilot Capital, Grove Ventures, Arc VC, Encoded VC, Leap Forward, and Aurora Capital Partners. The company, however, has chosen to keep its valuation confidential.
The Challenge with Power Spikes
Data centers that operate thousands of GPUs to train and run advanced AI models often face power consumption spikes. These occur millisecond as the processors switch between computing tasks and communicating with other GPUs. These surges create difficulty for data centers to manage their power draw from the grid effectively. To avoid power shortages, data centers resort to paying for temporary energy storage to cover surges or throttle their GPU usage. Both scenarios lead to expensive chips reducing the return on investment.
“We simply can’t continue to build data centers the way we’re building them now,” commented Lior Handlesman, a partner at Grove Ventures who also sits on Niv’s board. [Source]
Niv-AI’s Roadmap
The first step in Niv’s roadmap is understanding the problem. The company is currently deploying rack-level sensors that detect power consumption at the millisecond level on its own GPUs and in collaboration with design partners. The goal is to understand the specific performance profiles of different deep learning tasks and develop mitigations that allow data centers to unlock more of their existing capacity.
Engineers at Niv-AI plan to build an AI model on the data they collect to predict and synchronize power loads across the data center effectively. This model will serve as a “copilot” for data center engineers, assisting them in managing power consumption more efficiently.
Niv expects to have a system up and running in several U.S. data centers within the next six to eight months. This is a compelling proposition, particularly for hyperscalers looking to build new data centers but facing challenging land-use and supply chain issues. The founders envision their solution as a missing “intelligence layer” between data centers and the power grid. [Source]
Implications and Outlook
“The power grid is actually afraid that the data center will use too much power at any given time,” Timor told TechCrunch. “We’re dealing with a problem on two fronts. One is helping data centers use more GPUs and hopefully get more out of the power they’re already paying for. On the other hand, we aim to create much more responsible power profiles between the data centers and the grid.”
As AI continues its rapid development, the challenge of managing power consumption will only become more critical. Innovative solutions like Niv-AI’s approach hold the promise of making AI more energy-efficient and sustainable. This is not only beneficial for the AI industry but also vital for the broader goal of responsible energy use and environmental sustainability. [Source]
