Microsoft Wants USB4-Linked PCs to Run Bigger Local AI Models

By Lounes Hareb11 views
  • Windows
Microsoft Wants USB4-Linked PCs to Run Bigger Local AI Models

A large local AI model has a fairly blunt entry requirement: enough memory to hold it. Microsoft is working on another answer besides buying one much bigger computer. Its RDMA over USB4 announcement describes connecting several Windows PCs so a compatible inference framework can split one model across their memory and GPU compute.

The appeal is easy to see. A model that will not fit on one machine might fit across several. The important catch is the calendar: Microsoft announced this on November 19, but the required Windows Insider build is still listed as coming soon. This is a developer-preview announcement, not something to switch on in an ordinary Windows installation today.

USB cables, with rather more going on behind them

RDMA means Remote Direct Memory Access. Here, it provides a lower-overhead way for connected PCs to exchange data than the usual networking path. Applications use Windows' NetworkDirect interface, with a new USB4 provider doing the transport work. Microsoft has published the provider and supporting tools under the MIT license, alongside an open-source, llama.cpp-based inference testbed.

That does not turn several PCs into one giant computer for every application. The inference software has to support the arrangement. Nor does plugging in a spare laptop automatically make a model faster: memory capacity, memory bandwidth, GPUs, model settings and communication between machines all matter.

The cabling is also less casual than a chain of external drives. Microsoft calls for a direct connection between every pair of PCs. Three computers need two suitable ports each and three cables in total. Daisy-chaining performs poorly, according to the preview notes. The machines also need the same supported Windows build, preview drivers and provider version.

Promising numbers, attached to specific hardware

Microsoft reports roughly 18.5 generated tokens per second for a quantized Qwen3-235B-A22B model across three AMD Ryzen AI Max mini-PCs, each with 128 GB of unified memory. AMD's own measurement on its setup is about 22 tokens per second. Those are vendor results for specified configurations, not a promise for a collection of old office PCs.

The useful part is the capacity demonstration: an open-weight model too large for a single machine can be run locally across a small cluster. For people experimenting with local models, that is a more interesting direction than another cloud subscription. It is not free hardware, and the preview does not remove each model's own license conditions.

Keep this off the everyday PC for now

The setup described by Microsoft requires test signing and turning off Secure Boot. Microsoft explicitly recommends dedicated test PCs rather than an everyday device. The transport also does not automatically retry a dropped connection; the application has to re-establish it.

My takeaway is to watch the project, rather than start rearranging the desk today. The open-source pieces are available for developers, while the required Windows preview build is still pending as of November 21. If the experiment matures, it could give existing high-memory PCs a useful second role together. For now, it is a promising local-AI experiment with a fairly serious setup checklist.

Announcement and repository checked November 21, 2026. No multi-PC performance test was conducted. Illustration: our capture of Microsoft's announcement, credited to Microsoft Community Hub.

Share this article