launch80:~$ ls insights/
Notes from the lab.
# research, benchmarks, and field notes on running local models.
/6 min read
We Shaped the Report Template Around One Benchmark
Publishing sends data, not markup — so the template on our end decides what a result can say. We tuned ours for BetterBench output. This is what that buys, what it costs every other kind of result, and why the limitation is temporary.
#engineering#benchmark#tooling
/11 min read
One Model, Every Local Stack
We pointed every local-AI server we could run — ctrl8d, mlx-vlm, mlx_lm, vMLX, LM Studio, llama.cpp, ollama, vLLM — plus four coding agents at the same 35B model on one Mac and measured them. Several couldn’t even load it.
#engineering#ai-infrastructure#benchmark
/7 min read
Two Optimizations That Were Never Allowed to Run Together
Our local inference server made us choose between fast responses and fast typing. We forked it, removed the choice — then found our own benchmark had been flattering us, and re-ran it properly.
#engineering#ai-infrastructure#performance