Clef as a Private LLM: The Open Weights Read 16,384 Tokens, Not the 65,536 Cloudflare Hosts

A private LLM deployment usually starts with a question about weights: can we download it, may we run it, and what does it take? Cloudflare's Clef, released on 1 October, answers the first two cleanly. Clef and Clef-flash are decision models: you send a state and a schema of typed questions, and one forward pass returns a probability for every allowed option, with no generated text to parse. Both are on Hugging Face under Apache 2.0, the same license as the Qwen backbones they were post-trained from.

The third answer is where the repository and the announcement part ways. The announcement states a 64k context window, and the Workers AI model page lists 65,536 tokens. The inference code in the repository defaults to 16,384.

What the open release actually does with a long input

The release ships one Python file, joint_schema_model.py, which the model card tells you to import from the downloaded snapshot. Its encode_record function takes a max_length argument that defaults to 16,384 tokens, and the Jev-compatible systemone wrapper uses the same default. The function builds the prompt prefix, the schema of questions and options, and the suffix first. It raises an error only if those alone exceed the limit. The state, which is the thing you are asking the model to decide about, is then trimmed to whatever budget is left, keeping its beginning and discarding its end.

Nothing is logged and nothing is raised when that happens. The response's usage.input_tokens reports the length after trimming, so it looks normal. A support thread, a contract, or a log excerpt past roughly 16,000 tokens gets decided on its first part only.

The hosted endpoint truncates too, and its docs say so: long text state is cut to fit the token limit. The difference is the limit. Hosted Clef cuts at its 65,536-token window. Self-hosted Clef, run the way the model card shows, cuts at a quarter of that. The backbone is not what sets the limit: the Qwen3.8-27B card states 262,144 tokens natively. The 16,384 is a default in Cloudflare's wrapper, and you can raise it with one argument. What the model card does not report is any accuracy figure at longer states, so raising it is a test you run, not a setting you trust.

This is the failure mode our PoC-to-production checklist puts first under monitoring: silent degradation. A classifier that loses the end of its input still returns calibrated-looking probabilities, and nothing about a wrong answer at 0.91 confidence tells you it never saw the evidence.

Local LLM hardware requirements, from the files

The Clef repository holds 27,356,728,560 BF16 parameters in twelve backbone shards that add up to 54.7 GB, plus a 256 MB joint schema head. Clef-flash has 9,409,813,744 parameters in four shards, about 18.8 GB. Both model cards say the code was tested with torch 2.11 and transformers 5.10.2 on a single H200. That is the only configuration either card names. The documented path is plain transformers in BF16, with no quantized variant and no serving-engine recipe.

The announcement describes training as a frozen Qwen backbone with rank-256 low-rank adapters. The loader's docstring calls the released backbone merged, and the shards are full-size, so you download and serve the whole model, not a small adapter over a stock Qwen checkpoint you already host.

Maturity: one day old, two authors

At the time of this run each repository was under a day old. Clef had ten commits from two Hugging Face accounts between 30 September and 1 October, most of them editing the model card. Each repository had one discussion, opened by a member of Hugging Face's open-source team offering a demo Space, and no git tags. Clef showed 390 likes and 18 downloads. There is no track record yet. Whatever you deploy, you deploy at a specific commit.

Read the full table, not the blog's ten rows

The announcement shows ten benchmark rows. The model card shows 41 quality rows from the same internal run. Across all 41, Clef beats Typesafe's Jev on 26, and Jev beats both Clef models on 11. Those 11 include the three largest gaps in Jev's favour: GPQA Diamond, 78.3 for Jev against 48.0 for Clef; MMLU-Pro, 82.7 against 65.9; BBH, 92.9 against 73.7. All of these numbers are Cloudflare's own run.

Clef-flash is the more interesting private deployment. It holds the best score on 16 rows to Clef's 11, at a median 38.8 ms against 209.3 ms. It also collapses on two rows that matter for guarding a pipeline: out-of-scope intent detection on CLINC150+OOS (66.8 against 97.4) and hallucination detection on RAGTruth (35.6 against 79.4).

Who should not use it

• Teams whose states regularly run past 16,000 tokens, unless they raise max_length and measure accuracy at that length themselves first.
• Teams without one GPU that holds Clef's 54.7 GB of weights plus activations. The only tested setup is an H200. Clef-flash's 18.8 GB is the smaller option.
• Anyone using Clef-flash to catch out-of-scope requests or ungrounded answers.
• Anyone choosing it on hosted price. Hosted Clef lists 0.24 USD per million input tokens and Clef-flash 0.09 USD. Typesafe lists Jev at 0.042 USD.

The claim here is falsifiable: the open release's default will decide on a truncated state for any input past its 16,384-token budget. A commit that raises the default, or adds a warning to encode_record, makes it wrong.

Before you deploy it

1. Pin the revision: snapshot_download('Cloudflare/clef', revision='2f3de3dd85f379784083b0814d997ab627200f0c'), or 17f0b0ad64efb65d273590632833508766b2aae6 for Clef-flash. You are importing that repository's Python, so review the file at that commit.
2. Pass max_length explicitly to every encode_record and systemone call.
3. Treat len(encoded.input_ids) >= max_length as truncated. The trim fills the budget exactly, so a request at the cap lost part of its state. Send it to a human queue or chunk it.
4. Before you raise the limit, rerun 50 of your own labelled long cases at 16,384 and at the new value. If accuracy drops more than 2 points, keep the cap and chunk.

If what you need is the workflow around the weights, not just the weights, that is what our private LLM deployment work covers. The model is the easy part. For the same lesson from the Transformers side, see what running an LLM locally in Transformers depends on.