Nvidia’s data center hardware is shifting power in the AI industry

In short: AI data centers are increasingly being built around Nvidia hardware, and that is shifting influence in tech toward the companies that control chips and electricity.

What's going on

AI data centers are the giant buildings that run today’s most demanding AI systems. They are packed with specialized chips called GPUs (graphics processing units, but here they act like high speed engines for AI). In many of these new AI focused sites, Nvidia is supplying a large share of the key parts.

Nvidia is no longer just selling individual chips. It also sells the networking gear that links thousands of chips together, plus full rack sized systems that data center operators can install more quickly. Think of it like buying not just the car engine, but also the transmission and the wiring that lets many cars work together as one fleet.

The New York Times reports that in Meta’s planned Hyperion AI data center, each rack is expected to include 36 Nvidia “superchips” (a combined CPU and GPU package). Nvidia’s silicon is estimated to be about half of the total cost of that site, which could put the chip bill in the tens of billions of dollars.

Other companies still matter. Server makers like Dell and HPE assemble complete machines using chips from Nvidia and others. Power and cooling suppliers like Vertiv and Schneider Electric are also crucial, because these buildings need enormous electricity and heavy duty cooling.

What to watch

Big tech companies are trying to reduce their dependence on Nvidia by building their own AI chips, like Google’s TPUs and Amazon’s Trainium. Even so, the biggest AI training projects still rely heavily on Nvidia hardware, and access to power, land, and grid connections may become just as important as the software.

Source: NYTimes