Skill Tracer
Skills › Scientific Agent Skills › uncertainty-and-units › uncertainties-recipes.md

uncertainties-recipes.md

184 lines · skills/uncertainty-and-units/references/uncertainties-recipes.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# uncertainties recipes
2
3Verified against uncertainties 3.2.3 with NumPy 2.5.3 and SciPy 1.18.1. The package
4performs first-order (linear) propagation with automatic differentiation (mostly analytic elementary derivatives), tracking
5correlations automatically through every operation. This implements first-order variance propagation, a part of the GUM
6uncertainty framework; it does not perform Monte Carlo propagation and does not check
7whether linearization was appropriate.
8

uncertainties recipes

Verified against uncertainties 3.2.3 with NumPy 2.5.3 and SciPy 1.18.1. The package performs first-order (linear) propagation with automatic differentiation (mostly analytic elementary derivatives), tracking correlations automatically through every operation. This implements first-order variance propagation, a part of the GUM uncertainty framework; it does not perform Monte Carlo propagation and does not check whether linearization was appropriate.

Raw lines 1 to 8
9## Variables and the correlation they carry
11```python
12from uncertainties import ufloat
14x = ufloat(1.0, 0.1)
15x - x # 0.0+/-0 the same variable, perfectly correlated
16x - ufloat(1.0, 0.1) # 0.00+/-0.14 two independent variables
17```
19That pair of lines is the whole design. A `ufloat` is an *identity*, not a number with
20an attached error bar, and the difference of a variable with itself is exactly zero.
21This is the correct answer, and it is why arithmetic on ufloats beats manual quadrature
22in any expression where a quantity appears more than once.
24The corollary is the most common defect:
26```python
27copy = ufloat(x.nominal_value, x.std_dev) # a NEW, independent variable
28x - copy # 0.00+/-0.14, not 0
29```
31Rebuilding a variable from `.nominal_value` and `.std_dev` throws away every correlation
32it carried. So does serializing to JSON and back, storing in a DataFrame column of
33floats, or passing through any interface that speaks in pairs of numbers.
34`scripts/audit_units.py` flags this construction as `UNC004`.

Variables and the correlation they carry

from uncertainties import ufloat

x = ufloat(1.0, 0.1)
x - x                    # 0.0+/-0        the same variable, perfectly correlated
x - ufloat(1.0, 0.1)     # 0.00+/-0.14    two independent variables

That pair of lines is the whole design. A ufloat is an identity, not a number with an attached error bar, and the difference of a variable with itself is exactly zero. This is the correct answer, and it is why arithmetic on ufloats beats manual quadrature in any expression where a quantity appears more than once.

The corollary is the most common defect:

copy = ufloat(x.nominal_value, x.std_dev)   # a NEW, independent variable
x - copy                                    # 0.00+/-0.14, not 0

Rebuilding a variable from .nominal_value and .std_dev throws away every correlation it carried. So does serializing to JSON and back, storing in a DataFrame column of floats, or passing through any interface that speaks in pairs of numbers. scripts/audit_units.py flags this construction as UNC004.

Raw lines 9 to 35
Runs code scripts/audit_units.py :34
36## Correlated inputs
38```python
39import numpy as np
40from uncertainties import correlated_values, correlation_matrix, covariance_matrix
42cov = np.array([[0.04, 0.012],
43 [0.012, 0.09]])
44a, b = correlated_values([1.0, 2.0], cov)
46a + b # 3.0+/-0.4
47correlation_matrix([a, b])[0][1] # 0.19999999999999987
48```
50`correlated_values` is how a fit's covariance matrix enters the calculation intact.
51`correlation_matrix` returns a NumPy array; `covariance_matrix` returns a nested list —
52index it as `[i][j]`, not `[i, j]`.
54`correlated_values_norm` takes `[(value, std_dev), ...]` plus a *correlation* matrix
55instead of a covariance matrix, which is usually what a paper reports.

Correlated inputs

import numpy as np
from uncertainties import correlated_values, correlation_matrix, covariance_matrix

cov = np.array([[0.04, 0.012],
                [0.012, 0.09]])
a, b = correlated_values([1.0, 2.0], cov)

a + b                            # 3.0+/-0.4
correlation_matrix([a, b])[0][1] # 0.19999999999999987

correlated_values is how a fit's covariance matrix enters the calculation intact. correlation_matrix returns a NumPy array; covariance_matrix returns a nested list — index it as [i][j], not [i, j].

correlated_values_norm takes [(value, std_dev), ...] plus a correlation matrix instead of a covariance matrix, which is usually what a paper reports.

Raw lines 36 to 56
57## Functions
59Ordinary `math` and `numpy` functions have no derivative rule for these objects and
60fail, loudly on scalars and inside the ufunc loop on arrays:
62```python
63import math, numpy as np
64from uncertainties import umath, unumpy
66math.sqrt(a) # TypeError: can't convert an affine function ... to float
67np.sqrt(a) # TypeError: loop of ufunc does not support argument 0 of type AffineScalarFunc
68umath.sqrt(a) # 1.00+/-0.10
69```
71`umath` mirrors `math`: sqrt, exp, log, log10, log1p, expm1, the trigonometric and
72hyperbolic functions and their inverses, atan2, hypot, degrees, radians, and erf. `abs(UFloat)` and `umath.fabs` are deprecated
73in 3.2.3; the bundled evaluator uses `uncertainties.wrap` with an explicit derivative
74for absolute value and rejects the undefined derivative at zero.
76For arrays, `unumpy` provides the wrapped versions plus constructors and accessors:
78```python
79arr = unumpy.uarray([1.0, 2.0], [0.1, 0.2])
80unumpy.sqrt(arr) # [1.0+/-0.05 1.4142135623730951+/-0.07071067811865475]
81unumpy.nominal_values(arr) # [1. 2.]
82unumpy.std_devs(arr) # [0.1 0.2]
83np.mean(arr) # 1.50+/-0.11 -- works: object-array reduction
84np.sqrt(arr) # TypeError -- fails: ufunc loop
85```
87The split is worth internalizing: reductions written in terms of Python arithmetic
88(`mean`, `sum`, `dot`) work on object arrays, while ufuncs do not. When in doubt use
89`unumpy`.

Functions

Ordinary math and numpy functions have no derivative rule for these objects and fail, loudly on scalars and inside the ufunc loop on arrays:

import math, numpy as np
from uncertainties import umath, unumpy

math.sqrt(a)   # TypeError: can't convert an affine function ... to float
np.sqrt(a)     # TypeError: loop of ufunc does not support argument 0 of type AffineScalarFunc
umath.sqrt(a)  # 1.00+/-0.10

umath mirrors math: sqrt, exp, log, log10, log1p, expm1, the trigonometric and hyperbolic functions and their inverses, atan2, hypot, degrees, radians, and erf. abs(UFloat) and umath.fabs are deprecated in 3.2.3; the bundled evaluator uses uncertainties.wrap with an explicit derivative for absolute value and rejects the undefined derivative at zero.

For arrays, unumpy provides the wrapped versions plus constructors and accessors:

arr = unumpy.uarray([1.0, 2.0], [0.1, 0.2])
unumpy.sqrt(arr)             # [1.0+/-0.05 1.4142135623730951+/-0.07071067811865475]
unumpy.nominal_values(arr)   # [1. 2.]
unumpy.std_devs(arr)         # [0.1 0.2]
np.mean(arr)                 # 1.50+/-0.11    -- works: object-array reduction
np.sqrt(arr)                 # TypeError      -- fails: ufunc loop

The split is worth internalizing: reductions written in terms of Python arithmetic (mean, sum, dot) work on object arrays, while ufuncs do not. When in doubt use unumpy.

Raw lines 57 to 90
91## Formatting
93The format spec extends the standard one. `u` counts significant digits *in the
94uncertainty*, and the value is rounded to match:
96```python
97from uncertainties import ufloat
98v = ufloat(12.3456, 0.0234)
100f"{v:.2u}" # 12.346+/-0.023
101f"{v:.2uS}" # 12.346(23) concise notation
102f"{v:.1uP}" # 12.35±0.02 pretty Unicode
103f"{v:.2uL}" # 12.346 \pm 0.023 LaTeX
104f"{ufloat(0.00012345, 0.0000023):.2ue}" # (1.234+/-0.023)e-04
105```
107This handles the GUM rounding rule for you. `scripts/format_result.py` covers the cases
108this does not: an expanded uncertainty with a stated k, the accompanying sentence, and
109the warnings about reporting one digit when the leading digit is 1 or 2.

Formatting

The format spec extends the standard one. u counts significant digits in the uncertainty, and the value is rounded to match:

from uncertainties import ufloat
v = ufloat(12.3456, 0.0234)

f"{v:.2u}"    # 12.346+/-0.023
f"{v:.2uS}"   # 12.346(23)          concise notation
f"{v:.1uP}"   # 12.35±0.02          pretty Unicode
f"{v:.2uL}"   # 12.346 \pm 0.023    LaTeX
f"{ufloat(0.00012345, 0.0000023):.2ue}"   # (1.234+/-0.023)e-04

This handles the GUM rounding rule for you. scripts/format_result.py covers the cases this does not: an expanded uncertainty with a stated k, the accompanying sentence, and the warnings about reporting one digit when the leading digit is 1 or 2.

Raw lines 91 to 110
Runs code scripts/format_result.py :107
111## Where the uncertainty came from
113```python
114x = ufloat(1.0, 0.2)
115y = ufloat(2.0, 0.3)
116result = x**2 + y
117result.derivatives[x] # 2.0 -- partial with respect to this independent input
118result.error_components() # contributions keyed by independent Variable objects
119```
121For independent original `ufloat` inputs this is a useful budget. For
122`correlated_values`, derivatives and components are keyed by internally constructed
123independent variables, not by the correlated derived objects `a` and `b` above.
124`result.derivatives[a]` therefore does not return a physical partial derivative.
125Compute sensitivities using independent placeholders and combine them with the full
126original covariance matrix; report covariance terms separately.

Where the uncertainty came from

x = ufloat(1.0, 0.2)
y = ufloat(2.0, 0.3)
result = x**2 + y
result.derivatives[x]        # 2.0 -- partial with respect to this independent input
result.error_components()    # contributions keyed by independent Variable objects

For independent original ufloat inputs this is a useful budget. For correlated_values, derivatives and components are keyed by internally constructed independent variables, not by the correlated derived objects a and b above. result.derivatives[a] therefore does not return a physical partial derivative. Compute sensitivities using independent placeholders and combine them with the full original covariance matrix; report covariance terms separately.

Raw lines 111 to 127
128## Fitted parameters
130The covariance matrix from a fit contains parameter variances and covariances, and discarding
131it is a routine error:
133```python
134import numpy as np
135from scipy.optimize import curve_fit
136from uncertainties import correlated_values
138# Illustrative integration: define model, x, y and standard uncertainties sigma.
139popt, pcov = curve_fit(model, x, y, sigma=sigma, absolute_sigma=True)
140slope, intercept = correlated_values(popt, pcov) # keeps the correlation
141```
143Taking `np.sqrt(np.diag(pcov))` and building independent ufloats discards it, and for a
144straight-line fit the slope-intercept correlation is strongly negative — predictions
145near the centroid of the data come out far too uncertain.
147`absolute_sigma` decides what `pcov` means, and the default is not what most users
148assume:
150```python
151popt, pcov = curve_fit(f, x, y, sigma=sigma, absolute_sigma=False) # default
152# pcov is rescaled by the reduced chi-square: parameter uncertainties absorb the
153# goodness of fit. Relative sigma weights remain; this is not generally an unweighted fit.
155popt, pcov = curve_fit(f, x, y, sigma=sigma, absolute_sigma=True)
156# pcov reflects the standard uncertainties you supplied.
157```
159On one synthetic straight-line fit the two give parameter standard deviations of
160`[0.0364, 0.2154]` and `[0.0477, 0.2820]` — a 31% difference, with no warning. Pass
161`absolute_sigma=True` whenever `sigma` holds real standard uncertainties;
162`scripts/audit_units.py` flags the omission as `UNC001`.

Fitted parameters

The covariance matrix from a fit contains parameter variances and covariances, and discarding it is a routine error:

import numpy as np
from scipy.optimize import curve_fit
from uncertainties import correlated_values

# Illustrative integration: define model, x, y and standard uncertainties sigma.
popt, pcov = curve_fit(model, x, y, sigma=sigma, absolute_sigma=True)
slope, intercept = correlated_values(popt, pcov)   # keeps the correlation

Taking np.sqrt(np.diag(pcov)) and building independent ufloats discards it, and for a straight-line fit the slope-intercept correlation is strongly negative — predictions near the centroid of the data come out far too uncertain.

absolute_sigma decides what pcov means, and the default is not what most users assume:

popt, pcov = curve_fit(f, x, y, sigma=sigma, absolute_sigma=False)  # default
# pcov is rescaled by the reduced chi-square: parameter uncertainties absorb the
# goodness of fit. Relative sigma weights remain; this is not generally an unweighted fit.

popt, pcov = curve_fit(f, x, y, sigma=sigma, absolute_sigma=True)
# pcov reflects the standard uncertainties you supplied.

On one synthetic straight-line fit the two give parameter standard deviations of [0.0364, 0.2154] and [0.0477, 0.2820] — a 31% difference, with no warning. Pass absolute_sigma=True whenever sigma holds real standard uncertainties; scripts/audit_units.py flags the omission as UNC001.

Raw lines 128 to 163
Runs code scripts/audit_units.py :162
164## Limits
166- **First order only.** For a strongly nonlinear model the reported std_dev is the
167 linearized one, and the package cannot tell you that. Cross-check with
168 `scripts/propagate_uncertainty.py`, which runs Monte Carlo alongside.
169- **No distribution.** A ufloat carries a standard deviation, not a shape. Rectangular
170 and normal inputs with the same u are indistinguishable to it.
171- **No degrees of freedom.** Coverage factors are your problem; use
172 `scripts/uncertainty_budget.py`.
173- **`float()` fails**, deliberately, on anything with an uncertainty. Ordering comparisons are deprecated in 3.2.3; explicitly compare `.nominal_value`
174 only when that is intended. Equality also depends on uncertainty/correlation and
175 is not a statistical hypothesis test.
176- **Object arrays are slow.** For large arrays, propagate analytically or by Monte Carlo
177 rather than element-wise.
179Current references: [user guide](https://uncertainties.readthedocs.io/en/latest/user_guide.html),
180[technical guide](https://uncertainties.readthedocs.io/en/latest/tech_guide.html), and
181[SciPy curve_fit](https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.curve_fit.html).
182`absolute_sigma=True` does not repair a wrong model, dependence ignored in sigma, poor
183conditioning, or the local linear approximation behind `pcov`. A 2-D `sigma` is the
184observation covariance matrix; a 1-D `sigma` is observation standard deviations.

Limits

  • First order only. For a strongly nonlinear model the reported std_dev is the linearized one, and the package cannot tell you that. Cross-check with scripts/propagate_uncertainty.py, which runs Monte Carlo alongside.
  • No distribution. A ufloat carries a standard deviation, not a shape. Rectangular and normal inputs with the same u are indistinguishable to it.
  • No degrees of freedom. Coverage factors are your problem; use scripts/uncertainty_budget.py.
  • **float() fails**, deliberately, on anything with an uncertainty. Ordering comparisons are deprecated in 3.2.3; explicitly compare .nominal_value only when that is intended. Equality also depends on uncertainty/correlation and is not a statistical hypothesis test.
  • Object arrays are slow. For large arrays, propagate analytically or by Monte Carlo rather than element-wise.

Current references: user guide, technical guide, and SciPy curve_fit. absolute_sigma=True does not repair a wrong model, dependence ignored in sigma, poor conditioning, or the local linear approximation behind pcov. A 2-D sigma is the observation covariance matrix; a 1-D sigma is observation standard deviations.

Raw lines 164 to 184
Runs code scripts/propagate_uncertainty.py :168
Runs code scripts/uncertainty_budget.py :172