8 lines
1.5 KiB
HTML
8 lines
1.5 KiB
HTML
<!doctype html><html lang="en"><meta charset="utf-8"><title>Quantization has an error budget</title><h1>Quantization has an error budget</h1><p>15-minute session: read for 3 minutes, trace for 3, experiment for 7, and complete the check for 2. Spend longer whenever you need.</p><p>A simple symmetric quantizer divides by a scale, rounds, and clamps to an integer range. Dequantization multiplies by the scale. Clipping and rounding lose information. Measure both numerical error and the task’s accuracy before calling a smaller representation acceptable. This scalar exercise is not an implementation of every production quantization format.</p><p><strong>Practice and evidence:</strong> Predict the example or draw its state transitions. Change one assumption, test the boundary, and record the actual result. Explain what this small model does not establish about a production system.</p><pre><code class="language-python">value=0.36
|
||
scale=0.1
|
||
q=max(-127,min(127,round(value/scale)))
|
||
restored=q*scale
|
||
print(q)
|
||
print(round(abs(value-restored),3))
|
||
</code></pre><p><strong>Expected result:</strong> <code>4
|
||
0.04</code></p><h3>Practice and evidence</h3><p>Keep three notes: your prediction, the actual result, and why they match or differ. The completion check grades understanding; the practice task builds skill.</p><p><a href="https://git.agilaos.com/matt/open_agx">Language reference: git.agilaos.com</a></p><p><a href="https://onnx.ai/onnx/intro/">Language reference: onnx.ai</a></p></html> |