Seven Times the Weights, 1.2 Times the Compute

Mistral published Shieldstral 1.0 on August 4: a 3B multimodal moderation model under Apache 2.0, with weights on Hugging Face and a paper submitted to arXiv on July 28. The claim in both is that it matches or outperforms open guard models nearly seven times its size on text safety benchmarks.

The ratio is accurate. It is a ratio of stored weights, and the model it is measured against does not spend all of them on any given token.

What the seven is counting

The largest text baseline in the model card's tables is GPT-OSS-Safeguard-20B. OpenAI's model card for that release lists 21B total parameters and 3.6B active per token. Set against Shieldstral's 3B, that is seven times the weights a serving host has to hold, and 1.2 times the parameters that run on each forward pass.

Those answer different questions. Seven times the weights is a residency question: how many guard replicas fit on an accelerator, whether the guard can share a device with the model it is filtering, what it costs to keep warm at low traffic. 1.2 times the active parameters is the throughput question, and there the two are close to parity on the numbers each card reports.

A team that reads the announcement as a seven-fold cut in guardrail compute has read a memory claim as a compute claim. If your constraint is accelerator memory — and it is whenever the guard shares a device with the model it filters — seven is the number that matters and Shieldstral is a straightforward win. If your constraint is tokens per second, the honest expectation is parity until measured.

The win rate flips between input and output

The model card publishes F1 scores against six baselines across three task groups. Counting Shieldstral against GPT-OSS-Safeguard-20B specifically:

• Prompt classification — four of five benchmarks. It leads on WildGuardTest (88.1 to 87.3), ToxicChat (84.1 to 79.8), Aegis v2 (86.2 to 84.4) and HarmBench (99.4 to 94.5), and trails on OpenAI Moderation (81.4 to 84.0).
• Response classification — two of six. It leads on Aegis v2 (87.2 to 75.2) and BeaverTails (85.0 to 83.8), and trails on WildGuardTest, HarmBench, XSTest Harm and Qwen3GuardTest.
• Refusal detection — none of three, trailing on WildGuardTest (90.3 to 93.9), XSTest (94.6 to 94.9) and PolyGuard (89.5 to 92.3).

That is the boundary condition, and it is not a footnote. Prompt classification is filtering what a user sends. Response classification and refusal detection are filtering what your own model produced. OpenAI's card describes the baseline's job as input-output filtering, both directions — so which placement your system uses decides which of those three groups is your column, and only the first one carries the headline result.

On refusal detection the announcement's framing, outperforming larger models, is supported by the 7B baselines in that table: Shieldstral beats both WildGuard-7B and PolyGuard-Qwen-7B on all three. It is not supported by the 21B one.

Multimodal is where the margins get wide rather than fractional. Shieldstral leads VLGuard at 97.7 against 88.5 for the next-best baseline, and UnsafeBench at 81.8 against 72.6, over a field of OmniGuard-7B, Nemotron-3.5-Safety-4B, ShieldGemma-2-4B and LlamaGuard-4-12B. It trails only LlavaGuard-7B, on the LlavaGuard benchmark (72.0 to 81.4). Two of three, and the two are not close calls.

The context number is three numbers

The announcement gives no context length. The model card says training used sequences up to 32k tokens with theoretical support for 256k. The config file says what that means: max_position_embeddings is 262144, reached by YaRN scaling with a factor of 16 over an original_max_position_embeddings of 16384.

So there are three windows, not one. 16k is the native rotary range, 32k is where training stopped, 256k is extrapolation. The card's own limitations name lengthy documents as a reliability weak point, alongside adversarial and obfuscated inputs and uneven coverage across languages. The multilingual table shows what uneven means: on RTP-LX Prompt, Shieldstral scores 70.3 against 86.1 for Nemotron-3.5-4B, trailing a 4B baseline by 15.8 points on the same axis where it leads PolyGuard Prompt at 84.6.

What to do this week

The interface is a single yes/no token that resolves to a score between 0 and 1, with 0.5 as the default flagging threshold. That is the useful part, and also where the published benchmark stops being your benchmark.

1. Do not ship 0.5. Sweep the threshold per policy against your own labelled traffic and pick a point on your precision/recall curve, not the one that produced the published F1.
2. Test at your real payload length. If moderated documents exceed 32k tokens you are outside the training range and inside the limitation the card names.
3. Benchmark output-side before committing. If the guard sits after generation, the tables put the 21B baseline ahead on four of six response benchmarks and all three refusal ones.
4. Decide on memory, not throughput. Seven times fewer weights to hold is the real saving; assume parity on tokens per second until you have measured otherwise.

This is precisely the line item from PoC to production calls monitoring and guardrails — the place where a published F1 gets mistaken for a production threshold nobody re-derived. And picking a 3B open-weights guard over a hosted moderation API is the same buy, build, or fine-tune decision as any other: a small tuned model wins on cost and latency for a narrow high-volume task, which output filtering is, provided it wins on the half of the task you actually run.

What would show this reading wrong: benchmark both models at your batch size and find Shieldstral delivering materially higher throughput per accelerator rather than parity. If holding 21B of weights turns out to dominate serving cost in practice, the seven is a serving win after all and the memory-versus-compute distinction was too clever by half.