From Microcontroller to Training Cluster: What Investors Miss About AI Compute

The AI compute boom is the largest infrastructure build-out of the decade, and most of the capital flowing into it is being allocated using models written by people who have never deployed a model. Having built AI systems at every scale, from milliwatt microcontrollers in the field to cluster workloads, here is what that engineering perspective changes about the investment picture.

Compute is not a commodity: workloads are profiles

"AI demand" is treated as one number in most models. In reality, training and inference are different businesses. Training wants the newest accelerators, dense interconnects, extreme power density and tolerates location flexibility. Inference wants proximity to users, high utilization, and runs profitably on older hardware. A facility designed for one does not automatically serve the other: power density, cooling design, network topology and even floor loading differ. When a pitch deck says "AI-ready", the first engineering question is: ready for which workload profile?

GPU depreciation is an economic event, not a calendar one

Accelerators rarely die; they get outcompeted. Each hardware generation improves performance-per-watt enough that, at data-center electricity prices, older chips lose training work not because they're slow but because they're expensive to feed. The economic consequence: depreciation should be modeled as a cascade (flagship training, then fine-tuning, then inference, then niche workloads) with revenue per tier, rather than as a straight line to zero. The cascade's length depends on the facility's power price, which is why the same GPU fleet is worth different amounts in different buildings.

The edge is quietly absorbing inference

A structural trend investors underweight: workloads that once required a server round-trip now run on-device. Modern microcontrollers and NPUs execute vision, audio and anomaly-detection models locally, we ship such systems in industrial products today. Every model that moves to the edge is inference demand that never reaches a data center. This doesn't dent the training boom, but it caps the long-tail inference growth that many DC models quietly assume in their out-years. The result is a demand curve with a fatter near-term and a more uncertain tail than consensus models show.

Technology risk hides in the interconnect and the cooling

Investors scrutinize chip roadmaps but rarely the systems around them. Two examples with direct CapEx consequences: rack power density has multiplied within a few hardware generations, breaking air-cooling assumptions and forcing liquid retrofits mid-depreciation-cycle; and training clusters live or die by interconnect bandwidth, which dictates hall layout and cable infrastructure. A facility that cannot physically follow these curves doesn't become worthless, it becomes an inference facility, with inference economics. That transition should exist in the model as a scenario, not come as a surprise.

What defensible assumptions look like

None of this argues against data-center investment, the build-out is real and durable. It argues for models whose technical assumptions would survive a conversation with an engineer: workload-specific demand, cascade depreciation tied to power price, edge absorption in the inference tail, and retrofit scenarios for density and cooling. That is the standard we hold our own advisory work to, because we've been on both ends: writing the models and writing the firmware.