DeepSeek V4.1 Flash on a Mac Studio: DSpark at 8 Users and a One-Line llama.cpp Fix

Nov 11, 2026 · Reclaimed Node · 9 min read

The short version

In our DeepSeek V4.1 Flash Head to Head: 4x CMP 170HX vs Mac Studio M5 Ultra post, four CMP 170HX cards beat a Mac Studio M5 Ultra at every test. The Mac ran Unsloth Studio's default setup, so the obvious question was whether matching its settings to the CMP box would close the gap. We matched everything a 256 GB Mac can: 8 requests at once, a 262,144-token context, DSpark speculative decoding and a BF16 cache.

It did not close the gap. One user still gets 41 to 43 tokens/sec, 2.8 to 3.0 times slower than the CMP box, and with several users the matched Mac was slower than Unsloth Studio's defaults: 46 tokens/sec in total at 4 users with short prompts, down from 82. Getting there took dropping Unsloth Studio's wrapper and a one-line fix to llama.cpp, without which the server crashed as soon as 8 users ran together.

Setup

Test settings, the same as the first run: LLM-Super-Bench from a second Mac on the network, thinking off, temperature 0, replies capped at 512 tokens, prompts from Mill's On Liberty tagged so no cache could help. Prompt sizes were short (about 40 tokens), 4K and 16K, with 1, 2, 4 and 8 users and 3 runs each: 36 races, none failed. The Mac ran alone; the CMP comparisons use the head-to-head numbers, which matched the CMP box's solo run within 4%.

What it took to match the settings

1. A closer build would not fit

The obvious first step was a higher-precision build than 2-bit. The 3-bit UD-IQ3_XXS needs about 233 GB of weights in memory, plus about 9 GB for the DSpark drafter. Even with the Mac's GPU memory limit raised to 240,000 MB, Unsloth refused to load it: it estimated 236 GiB against the 221 GiB it allows, to keep macOS from running out of memory. Moving the Engram tables into RAM was further out of reach, at about 323 GiB with the 2-bit build. So the Mac stayed on 2-bit, with Engram on the SSD.

2. Eight users crashed the server

With 8 slots and DSpark on, the server crashed the first time 8 requests ran together. llama.cpp has a fast batched kernel for mixture-of-experts layers that it switches to at 32 tokens or more, built in versions for 1, 2, 4, 5, 6, 8, 9, 10, 16 and 22 experts per token. The DSpark drafter uses 3. Eight users times a 5-token draft is 40 tokens, so the drafter asked for a kernel that did not exist. Unsloth Studio's default of 4 users times 3 drafts stayed under 32, which is why the first run never crashed.

The fix is one line in ggml/src/ggml-metal/kernels/mul_mm.metal, which adds the missing version. The kernel itself is generic, so nothing else changes:

template [[host_name("kernel_mul_mm_id_map0_ne20_3" )]] kernel kernel_mul_mm_id_map0_t kernel_mul_mm_id_map0<3>;

3. We dropped the Unsloth wrapper

Unsloth Studio launches llama-server behind its own Python proxy, which adds a hop to every request and pins some sampling settings over the client's. We rebuilt Unsloth's llama.cpp source with the patch and ran llama-server directly on the same port, with the same model files:

llama-server -m UD-Q2_K_XL/DeepSeek-V4.1-Flash-1017-UD-Q2_K_XL-00001-of-00007.gguf \
  --mmproj mmproj-F16.gguf \
  --model-draft dspark-DeepSeek-V4.1-Flash-1017-Q8_0.gguf \
  --spec-type draft-dspark --spec-draft-n-max 7 \
  --parallel 8 --kv-unified -c 262144 \
  -b 2048 -ub 512 -ctk bf16 -ctv bf16 \
  -ngl -1 --flash-attn on --no-context-shift \
  --lazy-mode on --load-mode mmap+mlock \
  --jinja --reasoning on \
  --temp 1.0 --top-p 1.0 --min-p 0.0 --top-k 0 \
  --host 0.0.0.0 --port 8001

4. Checking the patch

At temperature 0 the drafter only proposes tokens, and the main model keeps only what it would have picked anyway, so a broken drafter kernel can lower the share of drafts accepted but cannot change the text. That share matched: 24 to 27% with prompts run one at a time on the old kernel, 23 to 26% with the same prompts batched on the patched one. Then 8 prompts of 23,000 tokens each ran at once: all finished, all answers were correct, with no errors and no swapping.

Single-user speed

Once an answer starts, every setup holds its speed as prompts grow: the CMP box at 120 to 128 tokens/sec, the Mac at 42 to 49 with Unsloth Studio's defaults and 41 to 43 with matched settings. Matching made one user slightly slower, not faster.

Prompt reading and waits

The CMP box reads a prompt 6 to 7 times as fast, and nothing in the matched settings changes how the Mac reads prompts, so those numbers barely moved.

Several users at once

With short prompts the CMP box climbs to 294 tokens/sec at 8 users. The Mac's first run peaked at 82 with 4 users; with twice the slots, this run reached only 55 at 8. With 4K and 16K prompts every setup spends most of its time reading prompts, so adding users adds waiting instead of output, and the two Mac runs end up close.

Why more users do not help the Mac

The Mac reads prompts at about 550 to 580 tokens/sec, and all users share that one reader. Eight 16K prompts are about 116,000 tokens, roughly three and a half minutes of reading, and llama.cpp mixes those prompt chunks into the same steps as the answers already being written. So every user's writing slows down while the others' prompts load: at 8 users with 16K prompts, each user gets 2.4 tokens/sec. The CMP box follows the same pattern, but reads 6 to 7 times as fast.

The drop from the first run at 2 to 8 users is the open question. Several things changed at once (8 slots instead of 4, a BF16 cache, a 4x larger context and the direct server), but the most likely cause is DSpark's draft length. This run checks up to 6 tokens per user per step instead of 4; with 8 users that is 48 tokens a step, and only about a quarter of drafts are accepted. The CMP box's vLLM shortens its drafts as load grows; llama.cpp always drafts the full length. We have not tested this yet, so treat it as the leading suspect, not a finding.

What this shows, and what it does not

• The gap is not about settings. With every setting the Mac can match now matched, one user is still 2.8 to 3.0 times faster on the CMP box, and prompt reading is still 6 to 7 times faster.
• Two builds, not one file. The Mac's 2-bit build is smaller and lossier than the CMP box's EXL3 + FP8 build, and its Engram tables are read from the SSD instead of RAM. A closer build needs more memory than a 256 GB Mac has.
• Matched is not the same as best. Copying the CMP box's settings made the Mac slower under load. A fair race between machines can still give each one the settings that suit it, as long as the post says which.
• The patch is small and safe. It adds a missing version of an existing kernel. Anyone running DSpark with many users on a Mac will need it until llama.cpp adds it.
• Speed only, so far. This covers prompts up to 16K, with no quality tests on the Mac yet.

Next

• DSpark draft length. Rerun with up to 3 draft tokens and with DSpark off, at 1 and 8 users, to confirm what cost the speed.
• Bigger prompt chunks. Prompt reading is the Mac's bottleneck. Raising llama.cpp's micro-batch from 512 to 4,096 tokens read Qwen prompts much faster on this Mac; it would be a best-Mac setup, listed separately from the matched one.
• A model that suits the Mac. We ran it: Qwen3.8-Flash-Next on a Mac Studio M5 Ultra uses about 6B parameters per token against DeepSeek's 16B, fits at 4-bit and runs about twice as fast on the same Mac and the same llama.cpp build.
• Long prompts and quality. The 32K and 64K races and the quality tests on the Mac.

Related: DeepSeek V4.1 Flash Head to Head: 4x CMP 170HX vs Mac Studio M5 Ultra · Qwen3.8-Flash-Next on a Mac Studio M5 Ultra · DeepSeek V4.1 Flash on 4x CMP 170HX: Benchmark Results · Race Your Local AI Machines and Trust the Numbers: LLM-Super-Bench