Microsoft proposes “useful yield” as the real measure of AI infrastructure
Microsoft is arguing that the next phase of AI infrastructure should be measured by “useful yield”: how effectively silicon, memory, networking, power and software combine to produce useful work.
Rani Borkar, president of Azure Hardware Systems and Infrastructure, presented the idea at SEMICON Taiwan 2026. The company points to measures such as tokens per dollar, tokens per watt, throughput and latency instead of relying only on a processor’s theoretical peak performance.
Why system-wide efficiency matters
An accelerator can look impressive in isolation while spending time waiting for memory, network transfers or software scheduling. Cooling, power delivery and failure rates also affect the cost and environmental footprint of an AI service. Microsoft’s approach treats the data centre as one co-designed system, from chips and racks to compilers and workloads.
The concept is particularly relevant as model use shifts from training runs to continuous inference. A small gain repeated across billions of queries can change both operating cost and electricity demand.
What buyers should ask for
“Useful” depends on the workload. A system optimised for short chatbot replies may not lead on scientific inference, image generation or long-context analysis. Comparisons should therefore state model quality, response length, latency target, power boundary, hardware utilisation and error rate.
Useful yield is Microsoft’s framing rather than a universal industry standard. It can improve discussion only if vendors publish enough methodology for customers and researchers to compare equivalent tasks.
Source: Microsoft official announcement and keynote summary.
Screenshot of Microsoft’s official announcement. Source: Microsoft.
