Sand.ai MAGI-2 Multi-GPU Deployment: Why More GPUs Did Not Help

Official setup versus this experiment

This record describes experiments completed by August 13, 2026. The official MAGI-2 Preview repository provides audio-video inference code. Our consumer-GPU route involved experimental adaptations; it is not an official, ready-to-run RTX 5090 installation recipe.

Most speed checks used reduced resolutions and 16 steps. They do not represent the then-documented 100 preview steps plus 5 refiner steps, or final 1080p production quality.

Sparse activation does not eliminate weight storage

The local checkpoint inventory recorded about 212.442 GiB for preview weights. Activating only some experts at a time does not make the remaining weights disappear: they must stay resident somewhere or be transferred when needed.

With four-way expert parallelism, EP4, the tested partition held approximately 60.487 GiB of local weights per GPU. That exceeded a 32GB card, requiring CPU offload. This is accounting for the tested partition, not a fixed requirement for every implementation.

The observed bottlenecks included offloaded weights, PCIe and NUMA transfers, and first-time compilation. GPU utilization or allocated VRAM alone could not explain the elapsed time.

Measured single-stream EP4 timing

These are total times for 10-second outputs at 16 steps. Warm requests reuse the process and relevant caches. Changing resolution also changes the task, so the rows are not equal-quality comparisons.

Two ways to use eight cards, two different constraints

Splitting eight cards into two EP4 groups permits two model instances. On the tested host, however, the extra copies increased host-memory and transfer pressure and could not retain the same optimized offload conditions. The report’s projected warm throughput did not beat the tuned single stream. That projection is not a universal claim about eight-GPU systems.

A single EP8 model encountered padding in the tested implementation: a 12-way structure was laid out across 16 slots, adding redundant work. Observed EP8 timings did not demonstrate a speedup, but cold-start, encoding and configuration differences prevent a clean causal comparison based only on card count.

What to record before scaling up

For an officially supported installation, start with the linked upstream repository and pin code and container versions. The internal consumer-GPU patches are not released in this article, so the upstream commands alone should not be expected to reproduce our numbers.

1. Budget host RAM and GPU interconnect topology alongside card memory.
2. Run a minimal valid request, then repeat the same shape in the same process to separate startup from warm latency.
3. Time encoding, sampling, decoding and file delivery independently.
4. Measure both request latency and completed outputs per unit time. Starting two requests together does not establish twice the throughput.
5. Hold duration, step count and quality checks fixed before comparing parallel layouts.

How to use the result

This record helps explain why a loaded model can still be slow and why extra cards can disappoint. It does not establish full-quality performance, results on another GPU family or the behavior of later releases.

The public evidence is a measurement summary and methodology, not the original inputs, server configuration or internal patches. A throughput experiment also establishes no claim about unrestricted content generation.

Official references

Use these references for model identity and the official setup. The measurements above come from our dated experiments, not a vendor performance guarantee.

• MAGI-2 Preview official inference repository