Benchmarks suggest OpenAI Jalapeño chip can run AI faster and use less power

In short: OpenAI presented benchmark test results for its Jalapeño chip, which the company says can run AI responses faster while using less electricity.

What happened

OpenAI shared new details about a computer chip system it calls Jalapeño at the Hot Chips conference. The company also showed the first set of benchmark results, which are standardized tests used to compare speed and efficiency.

The results came from Semianalysis’ InferenceX benchmark. “Inference” is the step where an AI model produces an answer after it has already been trained (like a cashier ringing up purchases, not building the store). OpenAI said Jalapeño delivered more “tokens per user” and more “throughput per kilowatt” than current top inference processors. Tokens are small chunks of text that an AI reads and writes, so more tokens per user can mean the system can handle bigger or more frequent requests.

OpenAI hardware lead Richard Ho said the chip can “serve more AI work per unit of power” and return responses more quickly. He also said it is designed for low latency, meaning less waiting time between asking a question and getting an answer.

OpenAI said Jalapeño was developed with Broadcom. The company said it reduced slowdowns in two places, prefill (loading the prompt and context) and communication (moving data between parts of the system). It also highlighted keeping the “KV cache” local, which is a kind of short-term memory AI uses while writing a response (like keeping notes on your desk instead of walking to a filing cabinet).

OpenAI expects small deployments by the end of 2026, with wider rollout in 2027.

Why it matters

If these results hold up in real-world use, chips like Jalapeño could make popular AI services cheaper to operate and faster to respond, especially when many people use them at the same time.

Source: TechCrunch AI