The same model runs differently on different hardware and software, quietly drifting off-course and even breaking tool calls.

Forums are full of people raving 'this model is amazing,' and then you download it and think 'that's it?' Often the model is fine — the local build just isn't the same thing the lab ran.

Same weights, different GPU, different software stack, different CUDA kernels: the next token comes out different. The author calls it 'implementation divergence.'

1. Swap the attention kernel, break the toolbox

Testing Qwen3.6-27B, the author changed only one thing — the vLLM attention backend, from FlashAttention 2 to Flash Inference. Same GPU, same everything else. Result: a Cisco command that should target `GigabitEthernet0/0/1.201` instead targets `GigabitEthernet0/1/4`, and from there it goes downhill — a `show mac address-table` request becomes a `show run`. One kernel change, and tool calling falls apart.

2. Shrink the KV cache, lose long-term memory

Dropping the KV cache from BF16 to INT8 is recoverable; drop it to INT4 and past 40k tokens tool calls break down. The longer the context, the more errors pile up, and the dumber the model gets.

3. Quantization saves VRAM, but costs IQ

Comparing the official BF16, FP8, INT8, NVFP4, and AWQ quantized variants, NVIDIA's NVFP4 comes in last — by 88k context, half of its next-token choices have changed. Both NVFP4 and AWQ mangled a Cisco command, running `show run` instead of `show arp`, while INT8 and FP8 got it right.

Bottom line: the gap between your local model and the lab's is on both hardware and software. Check whose implementation you're running before trusting any benchmark.