#quantization
POSTS
- A 27B Coding Agent That Fits in 5.9GB Changes What I Self-Host
The number I care about isn't 83.9, it's 5.9GB. That's a 27B-class multimodal model with a 262K context, Apache 2.0, running coding-agent…
- The 17 GB version is the one I'd actually run
I have been talking myself out of a bigger GPU for months, and this settles it. Q4KM of Qwen3.8 27B is 17 GB, matches full BF16 on…
- Cloudflare's quantization math is the same math I run on the boat
The part of this I keep rereading is the throughput table for the KV cache. At one concurrent request, BF16 beats FP8 — 137 tokens per…