Local LLM on a Mac or a DGX Spark? In ds4's Own Benchmarks, the Reply Length Picks the Winner

ds4 is the DwarfStar 4 inference engine from Salvatore Sanfilippo (antirez). It now has a community site, dwarfstar.sh, whose pages carry a build date of 17 September and which pitches it as a way to run a local LLM on hardware you own. The site's hardware page gives one reference line for a 128 GB machine: an M5 Max at 32K context, generating 34.4 tokens per second with prefill at 557 per second. The site's benchmark table lists the DGX Spark at about 14 tokens per second generated at the same depth. Next to that, the Mac looks like the obvious buy.

For one workload it is. For another it is the slower machine, and the table cannot tell you which you have. The raw CSVs behind it can.

The claim, and what the table measures

The table comes from the repository's docs/PERFORMANCE.md, which the community site mirrors. It is DeepSeek V4 Flash at Q2 on both machines. The tool walks a fixed public-domain text, the novel I Promessi Sposi, out to 65,536 tokens in 2,048-token steps. At each step it times two things: prefilling the newest 2,048 tokens, and generating a 128-token greedy probe. The document calls these recorded baselines, not measurements of every later commit. The commit that last refreshed the M5 Max and DGX Spark files is dated 5 August. The repository has had more than 300 commits since, has no tags or releases at all, and its newest commit is from 20 September.

The rows read like this:

• M5 Max, 128 GB: 790 prefill and 39.4 generation at 2,048 tokens; 398.5 prefill and 27.6 generation at 65,536.
• DGX Spark, 128 GB: 826 prefill and 18.1 generation at 2,048; 823 prefill and 13.8 generation at 65,536.

The community benchmark page says it plainly: each row is the speed at that depth, not an average over the run. So a single row tells you nothing about how long a long prompt takes to read. You have to add up the steps, and neither the repository nor the community site publishes that sum.

Prefill vs decode: where it flips

Summing the 32 intervals in each CSV gives the cold-start cost of a prompt, meaning everything before the first output token:

• 16,384 tokens: 24.9 seconds on the M5 Max, 18.8 on the Spark.
• 32,768 tokens: 53.8 seconds against 37.8.
• 65,536 tokens: 127.2 seconds against 76.9.

The Mac's prefill rate keeps falling as context grows. The Spark's barely moves. Generation goes the other way: the Mac is 2.0 to 2.4 times as fast at every depth. So total turn time depends on one number: how many tokens the model writes before it stops. Set the two totals equal and solve for that number. At a cold 16K prompt, the Spark finishes first for any reply under about 160 tokens. At 32K the crossover is about 400 tokens. At 64K it is about 1,400 tokens.

That reply count includes reasoning. The README says thinking is on by default, so every thinking token counts toward the crossover, and on the Spark each one decodes at the slower rate.

What the disk KV cache changes

A coding-agent turn that continues an existing conversation is not cold. The serving guide says ds4-server with --kv-disk-dir first tries the live token prefix. It then tries matching rendered-text prefixes saved on disk, and prefills only the new suffix. When the cache holds the conversation, a turn is one 2,048-token interval plus the reply. At a 65,536-token depth with a 500-token reply, that is 23.2 seconds on the M5 Max and 38.6 on the Spark. In the CSVs, the Mac wins every resumed turn with a reply of 128 tokens or more, at every depth.

So the benchmark splits along workload lines, not hardware tiers. A long-running agent session that resumes from cache favors the Mac. A retrieval pipeline that builds a fresh 30K to 60K-token prompt each time, from documents the cache has never seen, favors the Spark up to the crossover. Our buy, build or fine-tune guide argues that you start from the question, not the technology. That applies here at the hardware level: a document-heavy RAG system and a coding agent can want opposite machines from the same engine and the same weights. The same disk-cache mechanism carries its own retention questions, and those are worth settling before you rely on it.

What would make this wrong

There are three limits, all in the data itself.

The generation probe is 128 tokens. A 1,400-token reply at 64K depth is beyond anything the CSV measured, so the crossover assumes decode speed holds that long.

The baseline is from August. A faster Metal or CUDA prefill path landing since then could move every number above. This post is wrong for current code if a fresh ds4-bench sweep on either machine changes the 64K cumulative prefill by more than a few seconds.

The build matters too. Entrpi/ds4-on-spark, a third-party fork for the Spark, claims 2.4 to 3.3 times upstream prefill and 1.33 to 1.47 times upstream decode on that box. If those multipliers hold, the 64K crossover moves from about 1,400 reply tokens to somewhere between 5,200 and 8,000. Even at 16K, the crossover would land between about 770 and 1,100 tokens. Those are the fork author's numbers, not upstream's, and nothing here reproduces them.

If you need help deciding which side of the line your workload sits on, that sizing is part of how we approach private LLM deployment.

Before you buy a local LLM box

1. Log one week of real requests. For each one, record the prompt tokens that were not served from cache and the output tokens, with reasoning included.
2. Compare each request against the crossover: about 160 reply tokens at 16K uncached, 400 at 32K, and 1,400 at 64K. If most uncached requests fall below their line, the Spark finishes first. If most turns resume from cache, the Mac does.
3. Rerun the sweep on the hardware you can borrow, with the documented command: ./ds4-bench -m ds4flash.gguf --prompt-file speed-bench/promessi_sposi.txt --ctx-start 2048 --ctx-max 65536 --step-incr 2048 --gen-tokens 128. Add up prefill_tokens / prefill_tps over the rows rather than reading one row.

A local LLM has no per-token bill, so the trade is all latency. Price it in seconds per request, not in the generation number on a table row.