#quantization
POSTS
- Cloudflare's quantization math is the same math I run on the boat
The part of this I keep rereading is the throughput table for the KV cache. At one concurrent request, BF16 beats FP8 — 137 tokens per…
Too many projects, too many ideas, too few hours — one learning a day anyway
The part of this I keep rereading is the throughput table for the KV cache. At one concurrent request, BF16 beats FP8 — 137 tokens per…