Rohan Paul

Satya Nadella officially announced that Microsoft will make its in-house Maia chips the primary inference system for its products, scaling to multiple gigawatts. This will make it less reliant on NVIDIA hardware, and Maia 300 is meant to replace AMD chips too, not only Nvidia. The key claim is 40% more tokens per megawatt than leading GPU-based systems A full rack packs 96 accelerators into 167kW with 27.6TB of HBM, and Microsoft claims it delivers up to 7x the token throughput of a Maia 200 rack. The shift will start in 2027 with Maia 300, a third-generation accelerator built for long-context reasoning, low-latency chat and multi-step agent tasks. Its predecessor, Maia 200, already serves production traffic including Microsoft Copilot and Microsoft claims Maia 200 produces up to 1.7x more tokens per watt than leading GPU infrastructure. Maia 300 goes after the cost of moving data, which Microsoft says now limits large inference systems more than raw arithmetic does. Maia 300 uses a “multi-chiplet” design, A traditional chip is one large slab of silicon that does everything: the math (compute), managing the memory (memory controller) and talking to other chips (I/O). Maia 300 instead splits those jobs into several smaller pieces of silicon, called chiplets, packaged together so they act as one chip. That leaves more room on the compute chiplets for the parts that actually run the AI math, and each chiplet can be made with whatever manufacturing process suits its job best. Each chip pairs 8 HBM4 stacks delivering 18.0 TB/s of peak bandwidth with a 950W TDP, and uses on-die SRAM as a scratchpad to relieve that memory. Support for low-precision formats such as an enhanced MXFP4 shrinks the KV cache, the stored attention state that grows with every token, so longer contexts fit within the same power budget. Supply is the harder constraint, since Microsoft has reportedly been negotiating with TSMC for more than 300,000 Maia 300 units for 2027 delivery, a figure still under negotiation rather than a signed purchase order.