On August 25, OpenAI published a set of hard numbers for Jalapeño, its first custom inference chip: on SemiAnalysis’ public InferenceX benchmark, against leading commercially available accelerators, the chip pushed both “useful work per watt” and “how long a user waits” onto the same Pareto frontier.
Official facts first. The tests covered three public-weight models—GPT‑OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. At peak throughput, Jalapeño delivered about 1.5× to 1.9× more useful AI work per watt; end-to-end latency was roughly 1.7× to 3.6× lower; on highly interactive workloads, the performance edge stretched to about 2.1× to 4.1×. On Kimi, the largest public model in the set, peak performance per watt rose about 1.5× while end-to-end latency fell about 3.4×. The package is rated at 700 W, with measured sustained power at or below 550 W, against comparison systems at 1,200 W (GB200) and 1,400 W (GB300). The appendix makes the contrast visceral: on GPT‑OSS, 0.69 ms versus 1.87 ms between tokens and 1,459 versus 535 tokens per second per user; on DeepSeek, 12,258 versus 118 mixed throughput per kilowatt on the same scoreboard.
This is not another “we built a faster chip” press release. It reframes the question: if the primary job of the silicon is serving modern language models—especially interactive agents that chain many steps—what should the hardware look like? In OpenAI’s account, Jalapeño was drawn to cut data movement, pin the KV cache nearby, and schedule compute, memory, and networking by inference phase. It is a blank-slate LLM-inference design, not a renamed general-purpose accelerator.
The sharper story is the flywheel. Earlier OpenAI models helped the team move from initial design to tape-out in nine months. After that, engineers used Codex with GPT‑Astra to bring three open-weight models that were never in the original production plan to high performance within two months. Chips feed models; models rewrite kernels, schedules, and verification. That is more than “custom silicon.” It means the inference stack can start iterating on a software release cadence rather than an ASIC’s multi-year clock.
The industry sting sits between the lines. NVIDIA is not written out: OpenAI says it will keep deploying accelerators from NVIDIA and other partners for training and inference at scale. Jalapeño is slated to enter OpenAI’s own infrastructure by year-end, with Gen 2 deep in development and Gen 3 taking shape. For clouds and chip vendors, the bruise is not a single 700 W part—it is a customer starting to measure shelves with “useful work per watt at a matched user experience,” where latency and throughput no longer force a binary tradeoff.
The caveats are equally concrete. The numbers are OpenAI’s own InferenceX runs, normalized to published chip power ratings. Independent full-system replications, rack-level overhead, and behavior on non-OpenAI workloads are not yet nailed down by a third-party lab. The appendix compares against “leading commercial systems,” not a public SKU’s full-rack TCO. Author’s read: calling Jalapeño “NVIDIA’s end” is lazy; reading it as “inference becoming a physical product differentiator” tracks the text. In the agent era, every extra tool hop compounds delay down the task chain—whoever can emit more steps inside the same watt budget, and make humans wait fewer seconds, gets closer to selling agents as daily infrastructure instead of demo reels.
The June Broadcom–Celestica announcement was the groundbreaking. August’s first results turned the ceremony into a scoreboard. What matters next is whether year-end internal deployment can turn those appendix milliseconds and per-kilowatt figures into shorter queues inside ChatGPT and Codex.