Back to the stories

Microsoft announces Maia 300 inference accelerator and multi-gigawatt Azure deployment plan

Score 10.0,

AI Infrastructure

AI inference is entering a hardware-first phase: cloud providers are moving from general-purpose GPUs to purpose-built, rack-scale accelerators tuned for long, memory-heavy conversations.

Microsoft is leading that shift with Maia 300, its next-generation inference accelerator, and a plan to expand Azure Maia to multi-gigawatt scale beginning in 2027. Microsoft says Maia 300 will become the primary inference system for its products and services and that full racks of the new chips will be used across Azure.

Why that matters: Microsoft is selling more than a chip. It is selling a new unit of compute economics for decode-heavy workloads. Microsoft expects Maia 300 to deliver about 40% more tokens per megawatt than leading GPU systems, which changes the cost picture for services that must generate lots of text or keep very long context in memory.

How it works, in plain terms: Maia 300 is built for serving, not training. Think of a massive library that only fetches the handful of shelves needed to answer each question, instead of scanning every book. That design trades flexible generality for much greater efficiency when the task is long-context reasoning and lots of output. Microsoft also bundles those chips into large racks so a single deployment can handle far more concurrent decoding than older racks.

What changes now: Azure customers that run agentic assistants, long-context search, or continuous-memory workloads could get dramatically cheaper, denser inference. Microsoft also raises the bar for competitors who rely on GPU-only offerings; a single rack, Microsoft says, can deliver up to seven times the token throughput of the previous Maia 200.

What to watch next: will real-world benchmarks and customer rollouts confirm Microsoft's efficiency claims, and how quickly will this shift cloud pricing for businesses that actually need long, memory-intensive AI?