AI Engineering — page 2 of 4
- AI Engineering6 min
Kumo Tabular Is a Tabular Foundation Model You Can Ship Commercially. Its First Place Is Self-Reported.
By Petru Popa ·NVIDIA released Kumo Tabular on 29 September, an openly licensed tabular foundation model that claims first place on TabArena and BeyondArena. Neither leaderboard lists it yet, its benchmark runs used ensemble and precision settings a plain library call does not apply, and the model card's pip command installs an empty placeholder. The license is the real news.
Read - AI Engineering6 min
Zero-Shot Classification on a Local LLM: Jeff Accepts 255 Options and Picks From 26
By Petru Popa ·Jeff is a day-old MIT repository that fine-tunes Qwen3.5 and Gemma 4 into single-pass zero-shot classifiers you run locally. The server accepts up to 255 options. A third-party issue shows the Qwen models scoring zero whenever the right answer sits past option 26, and no builder in the published training recipe that we read produces a question that wide.
Read - AI Engineering6 min
AI Watermarking in vLLM Costs No Throughput. Four Settings Decide Whether Your Output Carries It.
By Petru Popa ·vLLM 0.30.0 ships distortion-free text watermarking, and its blog shows no throughput cost on an H100. The shipped code tells a narrower story: greedy requests are never marked, any caller can opt out per request, speculative decoding skips the repetition guard, and the dual-key detector defaults disagree with the generator.
Read - AI Engineering6 min
vLLM vs llama.cpp on a Mac: vllm-metal Wins the First Token, Not the Finish
By Petru Popa ·vllm-metal brings vLLM's scheduler and paged KV cache to Apple Silicon, and its launch post shows it beating llama.cpp and oMLX under concurrent agent load. The benchmark repository behind that post holds a different, footprint-matched run. It agrees on first-token latency and disagrees on how fast requests finish.
Read - AI Engineering6 min
To Run an LLM Locally in Transformers, the Metal Kernel Has to Be There
By Petru Popa ·Hugging Face shipped a path that runs llama.cpp quants inside transformers without dequantizing them. The packed path needs a Hub kernel whose only published driver family is metal, it forces float32, it covers Qwen3.5 and nothing else, and the fallback when the kernel is absent costs you the whole quantization.
Read - AI Engineering6 min
DSpark Speculative Decoding: The 4x Was Measured on Math, and Tool Calls Accept About Half as Many Draft Tokens
By Petru Popa ·The vLLM team's DSpark drafter for Kimi K3 quadruples single-stream decode speed on math reasoning. The Hugging Face cards for the four published DSpark drafters show tool-calling prompts accept 43 to 59 percent as many extra tokens per step as math does, every published speed chart is a math chart, and vLLM's confidence-scheduled verification, the half of DSpark built for high concurrency, defaults to off.
Read - AI Engineering6 min
The Fastest Local LLM on a Mac Ships From a Repository Two Days Old
By Petru Popa ·Splash is an Apache-2.0 inference engine for Apple silicon that beats every engine Inco AI measured against it, on one machine, on two models. The repository behind it was created on 18 September and has twelve commits. Here is what the issue tracker and the model packages say about adopting it.
Read - AI Engineering5 min
Chord's INT4 MoE Kernel: Up to 2.15x per Layer, 4 to 8 Percent for a Self-Hosted LLM
By Petru Popa ·vLLM and Novita AI announced Chord, an INT4 mixture-of-experts kernel for Kimi K2.x, with per-layer speedups up to 2.15x. The end-to-end run in the same post shows 4.1 to 8.0 percent more decode throughput on eight H200s, the baseline is a Humming release from July, and the install gives vLLM's humming dependency a second owner.
Read - AI Engineering6 min
WeKnora Is Enterprise RAG. Seven of Its Ten Security Advisories Hit the Agent.
By Petru Popa ·Tencent's WeKnora is an open-source enterprise RAG platform trending this week. Seven of its ten security advisories sit in agent tools and none in retrieval, while the quick-start .env file still ships fixed encryption keys with registration open.
Read - AI Engineering6 min
Self-Hosted AI Now Keeps Its KV Cache on Disk, and vLLM Never Expires It
By Petru Popa ·vLLM's tiered KV cache offloading writes prompt-derived blocks to a filesystem or an S3 bucket. The v0.29.0 tier code has no age or capacity expiry, creates block files mode 0644, and names them with a hash any instance can recompute.
Read - AI Engineering6 min
PatchTST-FM-r2 Beats r1 on Average and Loses on 24 of 97 GIFT-Eval Configurations
By Petru Popa ·IBM's Granite PatchTST-FM-r2 ranks second among zero-shot, replicable models on GIFT-Eval, and a recompute from the benchmark's own result files confirms it. Those files also show r2 scoring worse than r1 on a quarter of the benchmark, and the Hub now tags the model OpenMDW rather than Apache.
Read - AI Engineering6 min
AI Agent Governance Does Not Survive The Approval Prompt
By Petru Popa ·GitHub made enterprise managed permissions for Copilot agent operations generally available on 9 September, and the reference documentation says more than the announcement did: the permission keys do not reach the cloud agent at all, and one allow entry turns every unlisted operation into a prompt.
Read