whichllm: pick a local model that is actually fast

Running local models gives you control over your destiny at local electrity prices. These are the best models to run on this machine:

uvx whichllm@latest --speed fast --top 3

GPU 0: Apple M4 Max — 64.0 GB shared (budget 62.0 GB) — BW: 546 GB/s
CPU: Apple M4 Max — 16 cores
RAM: 64.0 GB
Disk free: 103.5 GB
OS: darwin
VRAM headroom: 2.0 GB reserved per GPU

#	Model	Quant	Fit / VRAM	Speed	Published	Score
1	google/gemma-4-26B-A4B-it (26.5B, 3.8Ba)	Q8_0	Full GPU (27.7 GB)	49.9 tok/s ?	2026-03-11	85.3
2	Qwen/Qwen3-30B-A3B (30.5B, 3.0Ba)	Q8_0	Full GPU (31.6 GB)	58.2 tok/s ?	2025-04-27	84.3
3	openai/gpt-oss-120b (116.8B, 5.1Ba)	Q3_K_M	Full GPU (49.1 GB)	41.1 tok/s ?	2025-08-04	84.3

Qwen at #2 is estimated at 58 tok/s — that looks great — but it’s an April 2025 model. Gemma 4 is newer (March 2026) and still takes #1 on score, even at ~50 tok/s. The gap is only +0.9, so this isn’t a landslide; the ? on every speed means treat the decimals as planning numbers, not a live bench of your stack.

Best for what?

--profile is how tell what tradeoffs you want to make. (and pulls VLMs when vision/any is on):

Profile	What it keeps
general (default)	Everyday chat. Drops coding, vision, and math specialists.
coding	Coding-specialized names. Aider-weighted.
vision	Vision / multimodal names. Includes VLM candidates.
math	Math-specialized names.
any	Everything recognized, including VLMs.

uvx whichllm@latest --profile coding --speed fast --top 3
uvx whichllm@latest --profile vision --speed fast --top 3
uvx whichllm@latest --profile math --speed fast --top 3
uvx whichllm@latest --profile any --speed fast --top 3

On this snapshot, --profile vision --speed fast returned nothing that cleared the floor on this machine. Math landed on tiny 2024 Qwen2.5-Math 1.5B rows with scores in the 20s.

Best for coding

Same machine, coding profile:

uvx whichllm@latest --profile coding --speed fast --top 3

#	Model	Quant	Fit / VRAM	Speed	Published	Score
1	Qwen/Qwen3-Coder-30B-A3B-Instruct (30.5B, 3.0Ba)	Q8_0	Full GPU (31.6 GB)	58.2 tok/s ?	2025-07-31	84.3
2	ManniX-ITA/Qwen3.6-27B-A3B-Coder-MTP-GGUF (26.2B, 3.0Ba)	Q8_0	Full GPU (27.3 GB)	63.2 tok/s ?	2026-07-15	62.7 ~
3	yuxinlu1/gemma-4-12B-coder-fable5-composer2.5-v1-GGUF (11.9B)	Q4_K_M	Full GPU (8.3 GB)	36.8 tok/s ~	2026-06-10	53.4 ~

~ on the score means inherited / interpolated, not a direct bench of that exact repo. Qwen3-Coder takes the slot the general ranking gave to Qwen3-30B-A3B — same size class, coding-specialized. #2 and #3 fall off a cliff once you look at the evidence tag. That’s the point of the table: don’t grab the second name because it says Coder.

Run the top pick

The ranking is only useful if you actually load the thing. whichllm run creates an isolated env via uv, installs the runtime, downloads the weights, and starts a chat. GGUF, AWQ, GPTQ, transformers — no separate LM Studio step.

Bare whichllm run auto-picks from the default ranking (quality-first, any speed). That’s not the fast pick. The scriptable version is: ask for #1 at --speed fast, pull the Hugging Face id, hand it to run.

whichllm --top 1 --speed fast --json | jq -r '.models[0].model_id'

google/gemma-4-26B-A4B-it

Then download it and chat:

whichllm run "$(whichllm --top 1 --speed fast --json | jq -r '.models[0].model_id')"

Best coding model, same shape:

whichllm run "$(whichllm --profile coding --top 1 --speed fast --json | jq -r '.models[0].model_id')"

Or skip the pipeline and name something fuzzy:

whichllm run
whichllm run "qwen 2.5 1.5b gguf"
whichllm snippet "qwen 7b"   # copy-paste Python instead of chatting

First run is slow: deps and weights. After that it’s a chat loop; exit / quit / q to stop. Apple Silicon prefers GGUF.

What --speed actually does

Speed is a filter, not a re-rank primary. whichllm still scores models on benchmarks (LiveBench, Artificial Analysis, Aider, Arena ELO, Open LLM Leaderboard, with confidence tags). After scoring, --speed drops rows below a tok/s floor:

Flag	Floor	Meaning
--speed any (default)	none	Everything runnable, including crawl-speed edge fits
--speed usable	10 tok/s	Interactive-ish floor — chat won’t feel broken
--speed fast	30 tok/s	Snappy local gen — the “I refuse to wait” bar
--min-speed N	exact N	Custom floor; overrides the named presets

Display colors match those bands: red under 4, yellow 4–10, green 10–30, bright green 30+. Estimates are marked ~ when there’s a range and ? when confidence is low.

Simulating a 4090 on whichllm 0.5.16, the default #1 was Qwen3.6-27B at Q5_K_M and 27.5 tok/s. Add --speed fast and the same family drops to Q4_K_M at 35.5 tok/s — slightly lower quality score, but it clears 30. That is the whole point of the flag: don’t recommend a “best” model that will feel like molasses.

uvx whichllm@latest --gpu "RTX 4090" --speed fast --top 5

Default whichllm is ambitious: partial offload, tight VRAM, “it might work.” For something you will actually live with:

whichllm --gpu-only --speed fast --vram-headroom 1.5GB

Scores are evidence-graded (direct / variant / base_model / line_interp / self_reported), with recency demotion so a 2024 score can’t beat a current-generation model on a stale leaderboard. Speed is a usability gate on top of that quality core — not a substitute for it.

Source

• Repo: Andyyyy64/whichllm (MIT, PyPI whichllm, Python 3.11+)
• CLI reference and scoring docs live under docs/ in that repo
• Version checked for this write-up: 0.5.16 (2026-08-14). General table from a live M4 Max 64GB run; coding table from the same hardware class via --gpu "Apple M4 Max".

If you’ve been collecting Hugging Face tabs and half-remembered VRAM rules of thumb, replace that ritual with one flag and a run. Start with --speed fast when you care about feel; --profile coding when the job is code; pipe --json when you want the top id in a script.