OpenAI introduces its first custom AI chip built with Broadcom

In short: OpenAI has revealed its first custom chip, made with Broadcom, to help run OpenAI’s AI models using less power.

What happened

OpenAI said it has built its first custom “inference processor” in partnership with Broadcom. The chip is named Jalapeño. A processor is the main computing part of a device (like the engine in a car), and this one is designed for a specific job.

That job is “inference,” which means running an already trained AI model when you ask it a question. Inference is what happens when a chatbot writes a reply, a coding assistant suggests code, or an AI tool summarizes a document. OpenAI said its own AI models helped with parts of the chip’s development.

The company said Jalapeño is still being tested. It also said early results show better “performance per watt,” meaning it can do more work for the same amount of electricity. OpenAI also highlighted lower operating costs for real-time coding models.

OpenAI and Broadcom first announced their collaboration in October. TechCrunch notes that building a custom chip could reduce OpenAI’s reliance on Nvidia’s GPUs, which are widely used for AI work. Other large tech companies, including Google and Amazon, have also built their own AI-focused chips.

Why it matters

For most people, the cost to run AI shows up indirectly, in prices, limits, and speed. If OpenAI can run responses more efficiently, it could make its services cheaper to operate and potentially more affordable and reliable for users over time. It also signals that OpenAI wants more control over the behind-the-scenes systems that power its products.

Source: TechCrunch AI