Benchmark an endpoint
L80 betterbench runs BetterBench against any OpenAI-compatible endpoint and publishes its results.json as written. The launch80 page computes every table and figure from the raw samples, so it matches BetterBench's own HTML report.
Run and publish
$ L80 betterbench --endpoint http://host:8080/v1 --model Qwen3-8B --note engine="vLLM 0.9.3" --note hardware="1x RTX 4090"$ L80 betterbench --endpoint http://host:8080/v1 --model Qwen3-8B --quick$ L80 betterbench --results results.json$ L80 betterbench --results results.json --dry-run --payload-out payload.jsonA full run takes a while. The results file stays in the working directory; the share URL is on the last line of output. Publishing the same file twice makes two pages.
Flags
| flag | does |
|---|---|
| --endpoint <url> | OpenAI-compatible base URL to benchmark, e.g. http://host:8080/v1. |
| --model <name> | Model name to request. |
| --results <file> | Publish this existing results.json instead of running. |
| --quick | Short smoke run (5 passes). The page says so; it can be verified only as a quick run. |
| --passes N, --warmup N | Measured and warmup passes per category. Defaults are BetterBench's: 20 measured. |
| --decode, --prefill, --concurrency | Run only the named phases. Combinable. A verifiable run needs all three. |
| --no-prefill, --no-concurrency | Skip a sweep. |
| --note key=value | A fact about the run, shown as a chip. Repeatable. Use engine= and hardware=. |
| --api-key <key> | Key for an endpoint behind a gateway. Or L80_BETTERBENCH_API_KEY. Must come before any --. |
| --out <path> | Where results.json is kept. Default ./betterbench-<model>-<time>.json. |
| --title, --summary | Page heading and the paragraph under it. |
| --section "Heading=Body" | An extra prose section. Repeatable. |
| --template <id> | bench.betterbench.v1 (default) or bench.report.v1, the summary layout. |
| --engine, --hardware | Labels for bench.report.v1 only. |
| --dry-run | Run or read, prepare the page, do not publish. |
| --payload-out <path> | Also write the exact payload to this path. |
| --json | Machine-readable output. |
| --api-base <url> | Server override, same as L80_API_BASE. |
| -- <args> | Everything after -- goes to betterbench run unchanged. |
Which BetterBench runs
Each L80 release pins the BetterBench it was validated against: 0.6.1 in 1.0.0 (L80 tools list shows it). A betterbench on PATH is used when it is at least that version; otherwise L80 bootstraps the pinned commit with uvx. L80_BETTERBENCH=/path/to/betterbench overrides both, unchecked. L80 switches off BetterBench's own update check, because the pinned version is the one the launch80 page renders. Pinned tools has the details.
Before publishing, L80 warns about:
- a file from an older or newer BetterBench than the pin;
- requests the server answered from its prefix cache on a run that asked for a cold one. The page shows the same thing as a “⚠ N from server prefix cache — not cold” chip.
What leaves your machine
BetterBench records the endpoint URL it hit (env.endpoint) and the machine's hostname (env.host). L80 removes both before upload, and the server refuses a file that still carries them. An endpoint key reaches BetterBench by environment, so it never appears in the echoed command or in results.json; a key after -- is refused.
Make it verifiable
Only a BetterBench page can be verified by Launch80, and only when the run is complete and described well enough to reproduce:
- a full run (at least 20 passes and one warmup) for a plain “Verified” badge. A
--quickrun can be verified too, and its badge says “quick run”; - all three phases: single-stream, prefill and concurrency;
- BetterBench 0.6.1 or later, with no request served from the prefix cache;
--note engine="…"and--note hardware="…"recorded.
$ L80 verify check results.jsonWhat the page shows
The header carries the model, corpus version, passes, sampling, cache mode, any notes and the detected GPU, then the headline tiles (combined decode, ITL 1% low or update p99, TTFT p50, peak aggregate throughput, deepest prefill). Caveats sit above the numbers: quick runs, under-sampled percentiles, truncation, failed requests and batched decoding. The charts and tables draw only the phases and concurrency levels the run contains. See an example.
Publishing the summary layout instead
--template bench.report.v1 maps the same file to the hand-written benchmark layout, with--engine and --hardware as labels. It keeps less of the run and cannot be verified.