llama.cpp vs MLX on an M5 Ultra Mac Studio: same model, same quantization, different engine
Oct 25, 2026 · Reclaimed Node · 6 min read
Update, Oct 27, 2026
I re-ran this on the orcarouter 8-bit build of Qwen3.8 27B and got the same picture: MLX reads prompts about 1.6 times as fast as llama.cpp, and llama.cpp still serves two users better on short prompts.
The two-user slowdown on MLX now has an explanation. When mlx-vlm 0.7.10 batches speculative drafting for Qwen3.x, every request advances by the smallest number of drafted tokens any of them accepted, so one request holds back the other. A draft block size of 3 also looked faster than 4 in a quick test, but that's provisional. Details, and a comparison with vLLM on a CMP 170HX, are in Same Model, Three Engines: vLLM Puts the CMP 170HX Ahead of the Mac Studio.
The short version
I run local models on a Mac Studio (M5 Ultra, 256 GB), and until now that has meant llama.cpp. MLX, Apple's own framework, keeps getting faster, so I wanted to know what I'd gain or lose by switching, with nothing else changed.
Short version: MLX reads long prompts about 55 to 63% faster, generation speed is close to a tie, and the two engines handle two simultaneous users very differently. The details matter, so here's how I tested it and where the numbers can mislead.
What I tested
- Machine: Mac Studio, Apple M5 Ultra, 256 GB.
- Model: Huihui-Qwen3.8-27B-abliterated, a 27.3-billion-parameter model.
- llama.cpp: build b11604, the Q8_0 GGUF, flash attention on, 2 slots, MTP speculative drafting with a draft length of 2.
- MLX: mlx-vlm server, the 8-bit MLX conversion of the same model, with the MTP drafter (
Qwen3.8-27B-MTP-bf16). I left the draft length at the server's default, which for this drafter drafts 2 tokens per step (a block size of 3 in its config file), the same as llama.cpp. - Harness: LLM-Super-Bench, a sweep of prompt sizes (Short, 4K, 16K, 32K tokens) at 1 and 2 simultaneous streams, 3 repeats each, with a warm-up race thrown away. Every number below is the median of 3 runs. Replies were capped at 512 tokens, and thinking was switched off on both servers.
Q8_0 and MLX 8-bit are not bit-for-bit identical formats, but they are the closest like-for-like pair (the GGUF is about 29 GB).
Two things I got wrong first
My first comparison made MLX look about 40% slower. Two setup mismatches turned up along the way.
Speculative decoding was on for only one side. llama.cpp was launched with MTP drafting. The MLX server wasn't. Once I enabled MTP on MLX too, its generation speed on my test prompt went from about 32 to about 51 tokens per second. A lot of "engine A beats engine B" results are really "one engine had a speed-up turned on".
The thinking switch isn't spelled the same on both servers. llama.cpp reads enable_thinking inside chat_template_kwargs. The MLX server reads it as a top-level field and ignored the llama.cpp form, so my first attempt to turn thinking off only reached one of the two. Each server now gets the setting in the format it understands. If you benchmark two servers, send a trivial question to each with the switch on and off and compare the token counts.
I also ran the two sweeps one after the other, not as a head-to-head race. Both engines share one GPU, so racing them at the same instant would have measured them fighting each other.
Results
Reading the prompt (prefill)
| Prompt | llama.cpp | MLX | MLX vs llama.cpp |
|---|---|---|---|
| 4K tokens | 1,081 tok/s | 1,680 tok/s | 1.55× |
| 16K tokens | 1,052 tok/s | 1,670 tok/s | 1.59× |
| 32K tokens | 975 tok/s | 1,586 tok/s | 1.63× |
This is where MLX wins clearly, and the lead grows with prompt length. llama.cpp slows down as the prompt gets longer. MLX holds up better.
Time to first word
With a single user, MLX starts answering 36 to 39% sooner than llama.cpp: 18.2 seconds against 29.5 seconds at 32K tokens. With two users the gap is about the same (35 to 37%).
Generation speed
| Prompt | llama.cpp | MLX |
|---|---|---|
| Short | 51.2 tok/s | 51.8 tok/s |
| 4K | 55.7 tok/s | 48.4 tok/s |
| 16K | 48.9 tok/s | 48.4 tok/s |
| 32K | 44.2 tok/s | 44.3 tok/s |
Once the model starts writing, the engines are close to level. The one exception is 4K, where llama.cpp is about 15% ahead. With MTP drafting, speed depends on how predictable the text is, so I wouldn't read much into a single row.
Two users at once
This is the most interesting result. Per-stream speed with 2 simultaneous requests:
| Prompt | llama.cpp | MLX |
|---|---|---|
| Short | 46.6 tok/s | 28.4 tok/s |
| 4K | 45.6 tok/s | 29.2 tok/s |
| 16K | 26.9 tok/s | 41.8 tok/s |
| 32K | 22.0 tok/s | 39.2 tok/s |
On short prompts llama.cpp serves two users with a modest slowdown (about 9% at Short, 18% at 4K), while MLX's per-stream speed drops by about 45%. With long prompts it flips: llama.cpp falls to roughly half its single-stream speed, and MLX stays near 40 tokens per second. I haven't investigated why. It could be how each engine schedules one request's prompt processing against another's generation, but that's a guess and I haven't tested it.
Start to finish
Prefill dominates long prompts, so MLX finishes first: about 30% sooner at 16K and 35% sooner at 32K with one stream, and 25 to 31% sooner with two. At 4K the gap is only about 11%.
What this means
- Long documents, agents, big contexts: MLX, because of prefill speed.
- Several short requests at once, such as a voice assistant or chat for a few people: llama.cpp, at least with these settings.
- One user writing long answers from a short prompt: it doesn't matter, so pick on convenience.
Caveats
- One model, one machine, 3 runs per cell. The runs within a cell agreed closely, but this is not a survey of models.
- The two servers count prompt tokens slightly differently (for example 28,804 against 28,802 at 32K), so prompts are the same text, not byte-identical token streams.
- llama.cpp ran 2 slots with 131,072 tokens each. MLX uses continuous batching with no fixed slot count. I swept 1 and 2 simultaneous streams on both, so they got the same load.
- I measured speed only. I didn't check whether the two engines produce equally good answers, and speculative drafting can change output. The harness has a quality mode (IFEval and others) for that, and it's the obvious next test.
- I didn't measure power or memory use, so I'm not claiming anything about efficiency.
- I didn't test thinking mode here. That's a separate sweep.
Reproduce it
Start each server one at a time, add both as machines in LLM-Super-Bench, then run the identical sweep against each:
bun run sweep --machines "Studio llama.cpp" --sizes short,4k,16k,32k --streams 1,2 --repeats 3
bun run sweep --machines "Studio MLX" --sizes short,4k,16k,32k --streams 1,2 --repeats 3The harness saves each result with the setup the server reports (llama.cpp reports a lot, MLX reports little) plus the notes you type in, so write down how each server was launched. The charts above were drawn from those two sweep files.
Related: Same Model, Two Machines: Do the Answers Match? · Race Your Local AI Machines and Trust the Numbers · Same Model, Three Engines: vLLM Puts the CMP 170HX Ahead.