DeepSeek V4.1 Flash on 4x CMP 170HX: Benchmark Results

Nov 9, 2026 · Reclaimed Node · 5 min read

The short version

One user gets 111 to 129 tokens/sec from DeepSeek V4.1 Flash on four CMP 170HX cards, and that speed barely moves from a one-line prompt to a 115,000-token prompt. Prompts are read at about 3,600 tokens/sec, so a 128K prompt takes 34 seconds before the first word. The weak spot is several users with long prompts: they share that one prompt-reading rate, so waits grow in step with the number of users.

Measure	Result
Single-user generation speed	111–129 tokens/sec (short to 128K prompt)
Prompt reading (prefill)	3,387–3,815 tokens/sec (4K to 128K)
Best total throughput	296 tokens/sec (8 users, short prompts)
GSM8K math	96% (192/200)
MMLU-Pro knowledge	86% (172/200)
IFEval instructions	85% of prompts, 90.5% of rules
Long-context fact finding to 128K	21/21 correct (16/21 in the strict format)

Every race finished: 59 speed races and 521 quality questions, no errors.

Setup

One machine, run alone from NS-STUDIO with LLM-Super-Bench scripts/head-to-head.sh on 9 November 2026 (about 55 minutes). The Mac Studio comparison is its own post: DeepSeek V4.1 Flash Head to Head: 4x CMP 170HX vs Mac Studio M5 Ultra.

Item	Value
Machine	thebeast: bare-metal Ubuntu, 4x CMP 170HX (64 GB each, unlocked), PCIe Gen2 x16, 247 GB system RAM
Engine	vLLM 0.32.1rc1.dev515+ge05403fdc (Docker), tensor parallel 4, GPU memory use 95%
Model file	DeepSeek-V4.1-Flash-1017-hybrid-exl3-fp8, loaded as EXL3 (the tool labels it FP8; it is a mixed EXL3/FP8 build)
Engram tables	System RAM, not SSD: about 189 GiB of pinned host memory (23.6 GiB per card for each of the 2 Engram layers), read by the cards directly over PCIe
Memory in use	Each card 64.5 of 65.5 GB; host 214 of 247 GB, 32 GB free
Batching	Up to 8 requests at once (`--max-num-seqs 8`), 2,048 tokens per step (`--max-num-batched-tokens 2048`), 512-token prefill chunks beside decodes
Speculative decoding	DSpark, up to 7 draft tokens, adjusted by load
KV cache / context	BF16, 262,144 tokens per request
Test settings	Thinking off, temperature 0 (sent with each request, overriding the server's default of 1.0), reply cap 512 tokens (2,048 for quality)
Prompt text	Mill's On Liberty, each prompt tagged so no cache can help

The race config listed 16 slots, but the server runs 8 requests at once, so in the 16-user runs half the requests waited in a queue.

Single-user speed

Bar chart: one-user generation speed for DeepSeek V4.1 Flash on 4x CMP 170HX stays between 111 and 129 tokens per second from a 37-token prompt to a 115,059-token prompt

A long prompt costs time up front but no speed afterwards. The model uses about 16B of its 552B parameters per output token (vLLM recipe), which is why four 64 GB mining cards keep up.

Prompt reading

Line chart: seconds to first word for one user grows in a straight line with prompt size, 1.0 s at 4K, 3.8 s at 16K, 7.8 s at 32K, 15.8 s at 64K and 34.0 s at 128K

The wait grows in a straight line, about 0.28 seconds per 1,000 prompt tokens. Reading speed slips only at 128K (3,387 tokens/sec, against 3,600 to 3,800 below it).

Several users at once

Line chart: total tokens per second by users at once. Short prompts scale from 122 to 296 at 8 users, while 4K to 64K prompts fall to between about 10 and 90

Line chart, log scale: median seconds to first word rises with every added user, reaching 84 s for four 64K prompts and 91 s for sixteen 16K prompts

All users share one prompt reader of about 3,600 tokens/sec, fed 2,048 tokens per step. Sixteen 4K prompts are 57,000 tokens to read, so the cards spend most steps reading instead of writing, and total output stays near 80 tokens/sec. At 16 users, 8 requests also queue for a free slot. Short prompts avoid both: at 8 users the box writes 296 tokens/sec and each user still sees 46.

Quality

This quantized build answers well with thinking off: 96% on grade-school math and 86% on MMLU-Pro. Every test was zero-shot at temperature 0, so compare it with other machines run the same way, not with leaderboards.

Test	Questions	Strict	Loose	Cut off at 2,048 tokens
GSM8K math	200	96.0% (192)	96.0%	0
MMLU-Pro knowledge, 14 subjects	200	86.0% (172)	86.5%	5
IFEval following instructions	100	85.0% of prompts, 90.5% of rules	87.0% of prompts	1
Long context, 8K / 32K / 128K	21	76.2% (16)	100% (21)	0

Long context: needle and multi-key lookups were perfect at every size, 128K included. The five strict misses are all variable-chain answers with the right values, listed one per line or with commas instead of the format the checker wants.
MMLU-Pro by subject: weakest were engineering (10/16), psychology (8/13) and law (14/18); biology, history, philosophy and computer science were perfect. With 6 to 23 questions per subject, treat these as hints, not rankings.
Cut off: six answers hit the 2,048-token cap. Raising Q_TOKENS would recover a point or two.

Takeaways and next steps

Four mining cards run a 552B-parameter model at over 110 tokens/sec for one user, at any prompt length up to 128K. The limit is shared prompt reading: once a second user with a 4K+ prompt joins, each person waits longer and total output does not rise.

Try a tuning pass for long-prompt concurrency: raise --max-num-batched-tokens from 2,048 to 8,192 or 16,384, then re-run ./scripts/head-to-head.sh speed to compare. Watch host RAM: only 32 GB is free with the Engram tables loaded.
Mac Studio side: raced head to head on 11 November, written up as DeepSeek V4.1 Flash Head to Head: 4x CMP 170HX vs Mac Studio M5 Ultra.
More tests: a multi-turn agent test and a cache-reuse test, which LLM-Super-Bench does not have yet.