Quantization has an error budget

15-minute session: read for 3 minutes, trace for 3, experiment for 7, and complete the check for 2. Spend longer whenever you need.

A simple symmetric quantizer divides by a scale, rounds, and clamps to an integer range. Dequantization multiplies by the scale. Clipping and rounding lose information. Measure both numerical error and the task’s accuracy before calling a smaller representation acceptable. This scalar exercise is not an implementation of every production quantization format.

Practice and evidence: Predict the example or draw its state transitions. Change one assumption, test the boundary, and record the actual result. Explain what this small model does not establish about a production system.

value=0.36
scale=0.1
q=max(-127,min(127,round(value/scale)))
restored=q*scale
print(q)
print(round(abs(value-restored),3))

Expected result: 4 0.04

Practice and evidence

Keep three notes: your prediction, the actual result, and why they match or differ. The completion check grades understanding; the practice task builds skill.

Language reference: git.agilaos.com

Language reference: onnx.ai