The 24 GB Card Fits the Weights, Not the Features

Meta published Muse Glimmer on 10 August 2026 under Apache 2.0, weights at meta-models/Muse-Glimmer-30B. The model card gives roughly 29.6B parameters including a ~1.8B vision encoder; the Hugging Face writeup describes a 2B perception encoder feeding a 28B decoder. Text and images in, text out — the model card lists video as not optimized, with frames processed individually, and the ExecuTorch release does not accept video input at all. Knowledge cutoff is January 4, 2026. Recommended sampling is temperature 1.0, top-p 0.95, top-k 64, with a reasoning-strength dial offering low, medium, high, and xhigh.

The framing is local agentic workflows. Meta's announcement puts the quantized model under 20 GB and names consumer GPUs with 24 to 32 GB as the target, reporting DFlash speculative decoding at 3.1x on an RTX 5090, 1.8x on an M5-Max, and 1.5x on an M4-Max. The ExecuTorch team measured 21.6 tok/s solo against 33.0 tok/s with DFlash on a 64 GiB M5 Pro, a 52.8% improvement.

If you have been waiting for a local model that can hold a real agent loop, the interesting question is not whether the weights fit. It is what else has to be resident at the same time.

The context window is the cheap part

The model card advertises 131,072+ tokens, and config.json confirms max_position_embeddings of 131072. What it also shows is why that number is affordable. The decoder runs 52 layers with num_attention_heads of 32 and num_key_value_heads of 2 — the 16:1 sharing the writeup describes — at a head dimension of 128. The attention pattern alternates three sliding-window layers of 2,048 tokens with one full-attention layer, repeated thirteen times. The ExecuTorch post states the same split from the runtime side: 13 of 52 layers are global, 39 are sliding-window.

That makes the KV cache computable. Each layer stores a key and a value vector of 2 heads by 128 dimensions, at two bytes each in bf16: exactly 1 KiB per token per layer. The 13 global layers at 131,072 tokens come to 1.625 GiB. The 39 sliding-window layers cap at 2,048 tokens each, adding 78 MiB. Full context costs about 1.7 GiB of KV cache. Neither Meta's announcement nor the model card prints that figure.

For scale, Qwen3.6-27B — the model Meta's own benchmark table compares against — declares 64 layers with 4 key-value heads at a head dimension of 256, and a full_attention_interval of 4, giving 16 full-attention layers. On the same axis at the same 131,072 tokens, its growing cache is 8 GiB, roughly five times Muse Glimmer's. Qwen's remaining 48 layers use linear attention, whose state size is not something this run measured, so the comparison covers only the part of the cache that grows with the conversation.

The expensive part is the two optional files

The GGUF repository lists four artifacts. Two are the model: muse-glimmer-30B-kquant-17gb.gguf at 16.8 GB and muse-glimmer-30B-kquant-dynamic.gguf at 19.7 GB. The other two are mmproj-kquant.gguf at 1.4 GB, which is what makes image input work, and dflash-kquant.gguf at 1.6 GB, which is what produces the 3.1x.

Add them up before any conversation exists. Smaller build, multimodal, with the drafter: 19.8 GB of resident files, plus roughly 1.7 GiB of KV at full context, lands just over 21 GB — inside a 24 GB card, but with only a couple of gigabytes left for activations and runtime overhead. The dynamic build in the same configuration reaches 22.7 GB before a single token, and adding full-context KV puts it past 24 GB outright.

So the central claim: on a 24 GB consumer GPU, the binding constraint is the vision projector and the drafter, not the context window. The two files that cost three gigabytes are precisely what separates this deployment from a text-only, single-model one. What would show this wrong is straightforward — someone loading muse-glimmer-30B-kquant-dynamic.gguf with both mmproj-kquant.gguf and dflash-kquant.gguf resident on a 24 GB card at 131,072 tokens and reporting it stable.

Two more details from the artifacts that matter for planning. The GGUF card notes that -c is split across slots, so an agent server configured for parallel sessions divides the declared context between them. And the reference agent configuration on the announcement page sets a context window of 32,768 with a generation cap of 8,192 — a quarter of the advertised maximum. The pages do not explain the gap between the two numbers.

What breaks if you deploy this as an always-on agent

The ExecuTorch post is candid about its own limits: no cross-session prefix sharing, no checkpointing, and no continuous batching in this release. For a model aimed at local agent workflows, that means every session re-prefills from scratch, and concurrent sessions do not share work. Prefill cost is the number to instrument first, not decode throughput.

The GGUF card also pins a minimum runtime: llama.cpp build b10353 or newer, with older builds unable to load the files. The release notes for b10353 list one change — a contiguity requirement for the ROLL operation on the CUDA and Metal backends, fixing silently wrong results on non-contiguous inputs. The notes make no mention of Muse Glimmer or DFlash. Pin the build anyway; the floor is a numerics fix, not a feature flag.

Verdict

Adopt for evaluation, not yet for a persistent service. This is the PoC-to-production gap in its most physical form: the demo runs because a demo holds one session, one modality, and a short conversation. A production agent holds several, and the memory line items that were rounding errors in the demo are the ones that decide whether the process survives.

The checklist before you commit hardware:

1. Budget 16.8 GB (or 19.7 GB) for weights, plus 1.4 GB if you need images, plus 1.6 GB if you want the 3.1x.
2. Compute KV as 1 KiB per token per layer across 13 global layers, plus 2,048 tokens each across 39 sliding-window layers. At 131,072 tokens that is about 1.7 GiB.
3. Divide your -c value by the number of parallel slots to get real per-session context.
4. Pin llama.cpp to b10353 or newer.
5. Measure prefill, not decode, because nothing is cached across sessions yet.

If the total exceeds your card, the first thing to drop is the drafter, not the context. You lose speed. Dropping the projector loses the reason you chose this model.