Apple's MLX adds RDMA-distributed inference across Mac Studio clusters
Apple's machine learning research team published benchmarks showing how the latest MLX release, combined with the low-latency RDMA over Thunderbolt 5 driver added in macOS 26.2, can run frontier-scale open-weight models across multiple Mac Studio computers. On four M5 Ultra Mac Studios, the 816B-parameter Muse Spark Open mixture-of-experts model (42B active, NVFP4) generates over 36 tokens per second at an 8,192-token context, while a single 512GB Mac Studio runs Qwen 3.8 27B and Mistral Medium 3.5 128B. Apple reports the M5 Ultra improves time to first token by up to 4.6x and decode speed by 1.6x versus M3 Ultra, and says pipeline parallelization lifted prefill by up to about 1.8x at 65,536-token contexts.
It shows frontier-scale open-weight models can now run privately on local desktop clusters, though multi-machine setups and new MLX tooling are required.