'128GB unified memory, runs 100B-scale models locally' — the hottest tagline for this year's AI mini-PCs, and technically true: unified memory pools RAM for CPU and GPU, so a model can take as much as it wants. But many buyers discover the hard way that fitting it ≠ running it fast; dense LLMs crawl. The problem isn't capacity, it's memory bandwidth.
64GB answers 'does it fit?'; bandwidth and the software ecosystem answer 'how fast does it run once loaded?'.
The bandwidth ceiling: why decode ≈ bandwidth ÷ model size
Generating each token requires reading the full model weights once (weight movement dominates decode). So decode has a hard ceiling, roughly: decode tps ≈ memory bandwidth ÷ model size. Run the same 70B 4-bit dense model on these machines and the gap widens instantly:
| Platform | Bandwidth | 70B 4-bit decode ceiling |
|---|
| RTX PRO 6000 | 1792 GB/s | ~45 tok/s |
| Mac M5 Max | 614 GB/s | ~15 tok/s |
| DGX Spark | 273 GB/s | ~7 tok/s |
Bandwidth (GB/s)
RTX 5090 / PRO 60001792
RTX 40901008
Mac M5 Max614
M5 Pro307
DGX Spark273
AMD AI Max+ 395256
An RTX 5090 has 7× the bandwidth of the AMD 395 and 6.5× the DGX Spark. That's why, at the same 128GB unified memory, the Mac hits ~27 tps on 27B while the AMD 395 manages ~15 and the DGX Spark ~13 — same capacity, wildly different bandwidth.
Why the '32 tok/s' hype is misleading: MoE activates only part of the model
Someone ran DeepSeek V4 Flash (284B, MoE) on an AMD 395 and posted 32 tok/s, sparking 'APU token freedom' hype. The truth: mixture-of-experts (MoE) activates only a fraction of parameters per token, so far fewer weights actually move. Qwen3.6-35B-A3B is labeled 35B, but each token touches roughly 3B — the speed numbers look inflated.
When reading speed numbers:① Model type: dense vs MoE aren't comparable (MoE looks inflated); ② prefill vs decode: prefill is how fast it 'understands', decode is the readable output speed.
Switch to a dense model and the truth shows: the 64GB M5 Pro hits only 33 tok/s on Qwen 3.8 27B 4-bit, while a 4090 does 117 and a 5090 does 138 on the same task. A DGX Spark on some dense LLM manages ~6.7 tok/s — a ¥30k machine squeezing out a few tokens per second is the bandwidth wall in action.
Quantization: lower bits = faster (less bandwidth pressure)
Lower weight precision = smaller files = fewer bytes moved per token = faster decode. On DGX Spark with llama.cpp, Q6 runs faster than Q8 precisely because it relieves bandwidth pressure — at a small accuracy cost. So low-bandwidth machines should lean on lower-bit quantization for big models, which is exactly why Q4/Q6 dominates on 128GB unified-memory boxes.
2026 DRAM shortage: why the 64GB version is a trap
Unified memory is soldered and can't be upgraded — the wrong size means a whole new machine. And unified memory needs binned quad-channel modules; this year's DDR5 surge took a 96GB kit from ~¥2,000 to ~¥7,000, lifting entire machines (AMD 128GB from ¥13-15k to ¥19-23k; DGX Spark +17.5% official; Apple even dropped the 256G/512G M3 Ultra options).
- Buy the 128GB from day one: the 64GB version can neither fit dense LLMs well nor run them fast — the known trap.
- Know why you're buying: if the real need is ComfyUI image/video generation or CUDA workflows, bandwidth and ecosystem decide the experience, not the 128GB.
- If you can wait, wait: DRAM-driven price hikes may ease once supply recovers; don't chase the spike.
Capacity asks 'does it fit?'; bandwidth asks 'how fast once loaded?'. Don't be lured by 128GB and find it crawling — do the bandwidth math before you buy.
Data notes: bandwidth and speed figures come from public reviews (SMZDM, Zhihu, Digital Applied, Presenc AI, 2026), vary by environment, for reference only.
