4 lines
1.4 KiB
HTML
4 lines
1.4 KiB
HTML
<!doctype html><html lang="en"><meta charset="utf-8"><title>Numerical tolerance and a reference</title><h1>Numerical tolerance and a reference</h1><p>15-minute session: read for 3 minutes, trace for 3, experiment for 7, and complete the check for 2. Spend longer whenever you need.</p><p>Floating-point calculations can differ slightly after reordering. Compare an optimized implementation with a trusted reference using documented absolute and relative tolerances, and reject NaN when it is not expected. Never choose an enormous tolerance merely to make a test pass. Test zero, negative values, large magnitudes, and the application’s normal range.</p><p><strong>Practice and evidence:</strong> Predict the example or draw its state transitions. Change one assumption, test the boundary, and record the actual result. Explain what this small model does not establish about a production system.</p><pre><code class="language-python">import math
|
||
actual=0.1+0.2
|
||
print(math.isclose(actual,0.3,rel_tol=1e-9,abs_tol=1e-12))
|
||
</code></pre><p><strong>Expected result:</strong> <code>True</code></p><h3>Practice and evidence</h3><p>Keep three notes: your prediction, the actual result, and why they match or differ. The completion check grades understanding; the practice task builds skill.</p><p><a href="https://git.agilaos.com/matt/open_agx">Language reference: git.agilaos.com</a></p><p><a href="https://onnx.ai/onnx/intro/">Language reference: onnx.ai</a></p></html> |