At every generated token, attention uses keys and values derived from earlier tokens. KV quantization changes those stored numbers and sometimes the attention arithmetic too. The model weights can be identical while the conditional token distribution changes. It is not enough to ask whether the cache occupies roughly half as many bytes. The total GPU footprint still includes weights, workspace, fragmentation, and other state, and the extra concurrency changes the scheduler's load.

First isolate the claim. Run the same model revision, prompt construction, source passages, tokenizer, decoding settings, and hardware path with BF16 versus FP8 KV. Check the exact source token positions and whether the correct exception was present before inference. Compare the model's answer and claim support across context lengths, not only a general benchmark average. If the exception was cut before prefill, KV precision is innocent. If it was present and the failure appears only with quantized KV, inspect calibration, scaling granularity, attention backend, and whether a layer type is especially sensitive. Current vLLM documentation describes per-tensor and supported per-head scales, recommends dataset calibration for accuracy, and notes that some layers can be left at native precision. Those are implementation options, not a guarantee that this workload will pass.

I would test two things separately. At fixed load, did answer quality move? At fixed quality gate, did SLO goodput improve? A fleet can fit more sequences and still complete fewer useful answers if long-context errors or p99 rise. Use paired cases with short and long context, distant exceptions, exact numbers, multiple document versions and tasks that require abstaining. Have reviewers compare grounded claims rather than prose similarity. Include the cost of calibration and of maintaining separate cache configurations in the rollout.

If only one sensitive slice fails, try calibrated scales or leaving sensitive layers at higher precision and rerun the same test. Routing that slice to BF16 is also possible if the route is predictable before generation and capacity exists. If it is a high-consequence policy question and the evidence shows a real regression, keep the old path until fixed. The answer changes if the old-manual failures are actually from stale retrieval. Prove the first failing stage before undoing a memory optimization.