The Fastest Local LLM on a Mac Ships From a Repository Two Days Old

If you want a local LLM that keeps up with a coding agent, the constraint has never been the model. It is that a general-purpose runtime is built to serve any checkpoint you hand it. Inco AI released Splash on 17 September on the opposite premise, which it states plainly: its kernels, draft model and memory plan are specialised for the model it serves, and there is no generic multi-model runtime and no fallback path. LM Studio added it as a downloadable backend the next day.

The numbers are good. The repository is two days old.

What it is, and what you would otherwise hand-roll

Splash is an Apache-2.0 inference engine for Apple silicon that serves a fixed set of models over OpenAI Chat Completions, OpenAI Responses and Anthropic Messages on one Mac. Two models are supported today: Qwen3.8-27B, dense, and Qwen3.6-35B-A3B, a mixture-of-experts model with roughly 3B active parameters per token.

What it replaces is the tuning pass. Each model ships with its own DFlash 2 draft, precompiled Metal kernels written for that model's exact dimensions, and a memory plan computed at startup from what Metal reports as available. There is no config file, and speculative decoding is not a flag you turn on, it is the decode path. The flags that exist are ceilings such as --max-memory and --max-context.

The numbers, and the setting they were taken at

Inco measured on one machine: a 48 GB M5 Pro with a 16-core GPU, running coding prompts from NVIDIA's SPEED-Bench up to 32K tokens with a 1,024-token output limit, single-request figures reported as medians.

On Qwen3.8-27B, Splash decodes at 74 tokens per second on short prompts and 54 at 32K. The next-fastest engine Inco measured, oMLX, does 38 and 28; Ollama does 24 and 19; uzu does 19 and 15. On the mixture-of-experts model the figures are 210 and 143 against oMLX at 126 and 83.

Two things in that table matter more than the ratios. Splash at 32K context is still faster than every other engine measured at a short prompt, which is the shape an agent session takes once its history has grown. And the 35B mixture-of-experts model beats the 27B dense model in every prompt bucket while being the larger download, 20.9 GB against 17.4 GB: on a machine that fits either, the bigger package is the fast one.

Now the part the announcement does not put next to the numbers. Inco states the 27B was benchmarked with reasoning on at its medium level, and that decode figures include reasoning tokens. The model card for that same package states reasoning is on by default at xhigh. Those are not the same configuration, so a team reproducing 74 tokens per second has to set reasoning_effort explicitly rather than accept what the package gives them. Inco is also direct that each engine ran at its own recommended settings, which makes these end-to-end comparisons rather than a measurement of any single design choice.

What the commit history says and the README does not

This is where a two-day-old project stops being a detail. As of 20 September the incoai/splash repository holds twelve commits by two people, eleven of them by one. The first is titled Splash 1.0 and is dated 18 September, the day the engine appeared in LM Studio. The other eleven landed on 20 September and became release 1.0.1.

Read their subjects and the shape of the risk is legible: preserving prefill progress when memory pressure reduces concurrency, scheduling prefill admission within a work budget, reusing shared prefixes during concurrent prefill, budgeting HTTP uploads. The memory plan the design rests on is the thing being repaired, two days in, under exactly the concurrent load the headline advertises.

The open issues agree. One asks for a backend for Apple GPU family 8 so M2 Ultra machines can run it. Another reports that the engine's source capability query fails against the macOS 26.2 SDK. A third asks for a way to say where models live on disk. These are week-one issues, not the issues a project has when you put a team's agent traffic through it.

The packages only load in one place

The model repositories are not checkpoints in the usual sense. Each holds a manifest.json, a target/ directory of per-layer .bin files and a draft/ directory, with library_name set to splash. The card for the 27B package states plainly that it is not a Transformers or MLX checkpoint and loads nowhere but Splash. Its recorded base models are mlx-community/Qwen3.8-27B-4bit and Inco's own DFlash 2 draft, with the relation given as quantised.

So adopting Splash means 17.4 GB of weights that no other tool on the machine can read, declared as derived from a 4-bit quantisation someone else published. The speed is in the engine rather than in the weights, and the engine is where the lock sits. That is a buy, build or fine-tune question that a throughput table does not answer, and it is the kind of trade a private LLM deployment has to price before anyone runs brew install.

Who should not use it

Four groups, from the requirements rather than from opinion. Anyone on an M1 or M2 Mac: the floor is M3 or newer. Anyone below 36 GB of unified memory, since the 27B alone needs 15 GiB of weights and a 1.2 GiB draft before a single KV page exists, 16.2 GiB of a 36 GB machine. Anyone who cannot move to macOS 26.4. And anyone who needs a third model, because there is no fallback path by design.

Note also that the server binds 127.0.0.1:8000 with authentication off by default, and that SSD cache offload exists only on an unmerged pull request you would build from source.

What to do this week

If you have a 48 GB M3-or-newer Mac and an agent that talks to an OpenAI or Anthropic endpoint, this is a two-command evaluation and worth an afternoon. Run it against the 35B mixture-of-experts package, not the 27B. Set reasoning_effort in your request body rather than trusting the default, or your numbers will not resemble the published ones. Set SPLASH_API_KEY before anything but your own shell can reach port 8000.

What would change this assessment: a third supported model, an M2 Ultra backend, and a month of releases from more than one committer. Until those exist, treat Splash as the fastest way to demo a local LLM on a Mac and not yet as infrastructure — the gap between those two is the same one that keeps proofs of concept from reaching production.