AI Engineering — page 2 of 3
- AI Engineering6 min
Confidential AI Moved the Trust Boundary. It Did Not Close It.
By Petru Popa ·Google DeepMind, MLCommons, AVERI, OpenMined and Singapore AISI ran a proprietary model against a private benchmark inside a hardware-encrypted enclave, so neither side saw the other's asset. The announcement page stops there. The technical report discloses that not all inference code was allowlisted, the enclave builds are not independently reproducible, and Google signs and verifies its own attestation.
Read - AI Engineering6 min
MXFP4 Quantization Did Not Improve the Model. A Second Teacher Did.
By Petru Popa ·A Hugging Face write-up reports a 4-bit MXFP4 model beating its own full-precision original on 7 of 9 benchmarks. The paper underneath it says the teacher was already MXFP4, that every number is a single run, and that the baseline which would settle the claim was never run.
Read - AI Engineering6 min
Agentic Memory Helps Until You Cap the Retrieval
By Petru Popa ·IBM Research reports gains of up to 16.1 points from feeding an agent guidelines mined from its own past runs. The technical report underneath contains a configuration of the same system that scores below the no-memory baseline, and the library that ships defaults to a retrieval backend no reported configuration used.
Read - AI Engineering6 min
Word Error Rate Fell 3.5 Points When Someone Fixed the Transcript
By Petru Popa ·Artificial Analysis hand-corrected 628 VoxPopuli clips and average model word error rate dropped 3.5 points with no retraining. A new set of probes explains the other half of that result: the models scoring best had learned to reproduce the errors.
Read - AI Engineering5 min
One NDCG Point for Twelve Times the Index
By Petru Popa ·Late interaction retrieval lands in Sentence Transformers 6.0 with a controlled benchmark behind it: same backbone, same data, one vector per token instead of one per document. The margin is about one NDCG point. The release notes show what that point costs in storage, and the model table shows which knob actually moves it.
Read - AI Engineering6 min
LLM Evaluation Is Three Different Jobs
By Petru Popa ·Picking a model, catching a regression, and watching production are three different measurement problems that share one name. Teams get into trouble when the artifact from one gets used to answer another — and the LLM-as-judge shortcut has failure modes with numbers attached.
Read - AI Engineering6 min
LLM Observability Is Mostly a Schema Problem
By Petru Popa ·Buying a dashboard is the easy half. The hard half is whether your traces carry the fields you will need at two in the morning — and the standard that defines those fields is still marked as in development.
Read - AI Engineering6 min
Pick LLM Evaluation Metrics by What You Can Supply
By Petru Popa ·The usual taxonomy sorts metrics by how they work — n-gram, embedding, model judge. That axis is useless for choosing. Sort them instead by what reference data each one demands from you, because that is the constraint you actually have.
Read - AI Engineering6 min
LLM Testing Already Had a Playbook in 2020
By Petru Popa ·Teams treat testing a language model as a new discipline with no precedent. The methodology that fits it best was published in 2020, and the test type it names as most valuable is the one almost nobody writes.
Read - AI Engineering6 min
Fine Tuning vs RAG Is the Wrong Question
By Petru Popa ·Two papers settle the versus framing between them. One finds retrieval beats unsupervised fine-tuning at teaching a model facts. The other finds the gains from both are cumulative — and things that add are not substitutes.
Read - AI Engineering6 min
Verified Is Not Reproduced
By Petru Popa ·A community effort reproduced claims from 2,226 ICML 2026 papers and reported 51% with at least one verified claim. Opening the frozen verdicts dataset shows what that percentage covers, what a $20 compute cap excludes, and why there is no way to look your paper up.
Read - AI Engineering5 min
The 3.3x Belongs to the Model, Not the Flag
By Petru Popa ·vLLM published a 3.3x throughput gain for decode context parallelism on Kimi K2.6. The config source shows the ceiling for grouped-query models is set by KV head count — and for Qwen3-235B-A22B on one node, that ceiling is two.
Read