Numerical tolerance and a reference

15-minute session: read for 3 minutes, trace for 3, experiment for 7, and complete the check for 2. Spend longer whenever you need.

Floating-point calculations can differ slightly after reordering. Compare an optimized implementation with a trusted reference using documented absolute and relative tolerances, and reject NaN when it is not expected. Never choose an enormous tolerance merely to make a test pass. Test zero, negative values, large magnitudes, and the application’s normal range.

Practice and evidence: Predict the example or draw its state transitions. Change one assumption, test the boundary, and record the actual result. Explain what this small model does not establish about a production system.

import math
actual=0.1+0.2
print(math.isclose(actual,0.3,rel_tol=1e-9,abs_tol=1e-12))

Expected result: True

Practice and evidence

Keep three notes: your prediction, the actual result, and why they match or differ. The completion check grades understanding; the practice task builds skill.

Language reference: git.agilaos.com

Language reference: onnx.ai