Quantisation is a free lunch — until it isn't. A practical walk through the datapath choices that decide whether INT8 inference is production-ready or just a benchmark trick.
The two failure modes
Most INT8 implementations fail in one of two ways: accuracy collapse from poor calibration, or throughput collapse from naive datapath layout. We walk through how we designed around both.