On model sites you'll see names like Qwen3-30B-A3B or Gemma-4-26B-A4B — a big number and a small one after 'A'. That's MoE (Mixture of Experts). This post explains: why such models have huge total params yet run faster, and why they're a gift for iGPU/CPU users.

The dense problem: every token moves all weights

A traditional dense model (e.g. Qwen3-14B) reads all 14B parameters for every single token. LLM generation is essentially memory-bandwidth-bound: compute speed matters less than how fast weights stream from RAM to the compute unit. Bigger dense model → slower per token.

The MoE idea: move only a slice

MoE splits the model into a router + a pool of experts. For each token, the router picks only a subset of experts to run. Hence active params = the experts actually computing (e.g. 4B) while total params = all experts combined (e.g. 26B: 30 layers × 128 routed + 1 shared).

Decode bandwidth only pays for active params, so MoE decodes much faster. Measured on pure CPU (see CPU-only tests):

MoE with 2-4B active beats dense 14B by 2-4×. That's the headline for iGPU/CPU users — MoE is the no-dGPU answer.

Three counter-intuitive truths

1. Big total params ≠ slow, but loading/prefill reads all of them

Decode only counts active params, but the weight file is still all experts combined (a 26B Q4 is ~15.6GB). Loading the model, and prefill (parsing a long prompt), read all weights. So don't try a 30B-A3B on an 8GB machine — the file alone won't fit.

2. iGPU-friendly: small active = small data movement

An iGPU shares system RAM with limited bandwidth. MoE's small active set means less data moved per token — a perfect fit. Measured: Gemma-4-E4B hits 20.3 t/s on an Intel iGPU (Vulkan), nearly tied with the 26B (21.4) — comparable active params give comparable speed (see E4B vs 26B).

3. llama.cpp assigns by layer; experts can't be split

A common myth: 'put active experts on GPU, idle ones on CPU'. Measurements with --n-cpu-moe show llama.cpp assigns weights per layer; expert tensor placement is fixed at load. Enabling --n-cpu-moe also adds a fixed sync cost, cutting decode ~30% — worse on Intel iGPUs.

The cost of MoE

• Heavy memory footprint: total params mean files 2-3× a dense model of the same class
• No prefill advantage: reads all weights; long prompts are actually slower
• Quality can be rougher at small scale: routing gets unstable under a few hundred MB (see small-model tool-calling)

How to read the naming

Rule of thumb: active params → speed, total params → memory. The number after 'A' sets decode class; the total decides whether your machine can hold it.