Skill Tracer
Skills › Scientific Agent Skills › timesfm-forecasting › examples_and_validation.md

examples_and_validation.md

58 lines · skills/timesfm-forecasting/references/examples_and_validation.md at 5f2bc03

Open on GitHub

Most marks come from matching words in the file, so treat them as pointers.

1# Examples and validation scope
2
3Run all commands from the skill root in the isolated TimesFM environment.
4Choose output directories outside the shipped skill tree. Plots require
5Matplotlib; GIF generation also requires Pillow.
6
7```bash
8python examples/global-temperature/run_forecast.py --output-dir /tmp/timesfm-temperature
9python examples/global-temperature/visualize_forecast.py --output-dir /tmp/timesfm-temperature
10python examples/global-temperature/generate_animation_data.py --output-dir /tmp/timesfm-temperature
11python examples/global-temperature/generate_gif.py --output-dir /tmp/timesfm-temperature
12python examples/global-temperature/generate_html.py --output-dir /tmp/timesfm-temperature
13python examples/anomaly-detection/detect_anomalies.py --output-dir /tmp/timesfm-anomalies
14python examples/covariates-forecasting/demo_covariates.py --output-dir /tmp/timesfm-covariates
15```
17Global-temperature and anomaly scripts download/use the pinned 2.5 checkpoint
18and run preflight. The bundled temperature-shaped values have unverified
19provenance; they are illustrative input, not a current NASA/NOAA data release.
20Retired generated results were removed instead of relabeling incorrect bands.
21The corrected schema rejects legacy results in plot/animation tools.
23The covariate demo generates 108 synthetic rows (3 stores x 36 weeks) and plots
24known generator components. It does not execute pretrained forecasting or
25establish that adding covariates improves a model. Its 24-point context supports
26the `xreg + timesfm` example; `timesfm + xreg` requires a longer history (>32).

Examples and validation scope

Run all commands from the skill root in the isolated TimesFM environment. Choose output directories outside the shipped skill tree. Plots require Matplotlib; GIF generation also requires Pillow.

python examples/global-temperature/run_forecast.py --output-dir /tmp/timesfm-temperature
python examples/global-temperature/visualize_forecast.py --output-dir /tmp/timesfm-temperature
python examples/global-temperature/generate_animation_data.py --output-dir /tmp/timesfm-temperature
python examples/global-temperature/generate_gif.py --output-dir /tmp/timesfm-temperature
python examples/global-temperature/generate_html.py --output-dir /tmp/timesfm-temperature
python examples/anomaly-detection/detect_anomalies.py --output-dir /tmp/timesfm-anomalies
python examples/covariates-forecasting/demo_covariates.py --output-dir /tmp/timesfm-covariates

Global-temperature and anomaly scripts download/use the pinned 2.5 checkpoint and run preflight. The bundled temperature-shaped values have unverified provenance; they are illustrative input, not a current NASA/NOAA data release. Retired generated results were removed instead of relabeling incorrect bands. The corrected schema rejects legacy results in plot/animation tools.

The covariate demo generates 108 synthetic rows (3 stores x 36 weeks) and plots known generator components. It does not execute pretrained forecasting or establish that adding covariates improves a model. Its 24-point context supports the xreg + timesfm example; timesfm + xreg requires a longer history (>32).

Raw lines 1 to 27
Runs code python examples/global-temperature/run_forecast.py :8
Runs code python examples/global-temperature/visualize_forecast.py :9
Runs code python examples/global-temperature/generate_animation_data.py :10
Runs code python examples/global-temperature/generate_gif.py :11
Runs code python examples/global-temperature/generate_html.py :12
Runs code python examples/anomaly-detection/detect_anomalies.py :13
Runs code python examples/covariates-forecasting/demo_covariates.py :14
28## Checks before reporting a result
30- Check model/package/revision, cadence, target units and forecast origins.
31- Retain positions for missing timestamps/values; fit transformations per origin.
32- Check all forecast values are finite, output dimensions match, and quantiles
33 are ordered. For 2.5 point=q[...,5], while 3.0 point=q[...,4].
34- Report nominal interval level correctly: q10-q90 is 80%, q20-q80 is 60%.
35- Evaluate against held-out actuals and naive baselines; report coverage **and**
36 width by horizon, not only mean accuracy.
37- Qualify illustrative retrospective screens and synthetic injected events;
38 observed flags do not validate alarm severity or clinical/climate meaning.
39- Plot every band's extent and align each successive forecast with its own
40 historical cutoff. A final forecast is only a retrospective reference layer.

Checks before reporting a result

  • Check model/package/revision, cadence, target units and forecast origins.
  • Retain positions for missing timestamps/values; fit transformations per origin.
  • Check all forecast values are finite, output dimensions match, and quantiles are ordered. For 2.5 point=q[...,5], while 3.0 point=q[...,4].
  • Report nominal interval level correctly: q10-q90 is 80%, q20-q80 is 60%.
  • Evaluate against held-out actuals and naive baselines; report coverage and width by horizon, not only mean accuracy.
  • Qualify illustrative retrospective screens and synthetic injected events; observed flags do not validate alarm severity or clinical/climate meaning.
  • Plot every band's extent and align each successive forecast with its own historical cutoff. A final forecast is only a retrospective reference layer.
Raw lines 28 to 41
42## Executed checks for this refresh
44The repository suite covers strict CSV/date handling, missingness, quantile
45indices/shapes, system checks, native 2.5 preprocessing/compile/XReg with a
46controlled lightweight decoder, tiny random 2.5 native transformer/decode, and
47tiny random 3.0 native save/load/inference.
48Example plots and animations are tested with clearly synthetic model outputs.
49These verify API mechanics and file/visual alignment, not pretrained accuracy,
50calibration, cloud availability or GPU/MLX performance. Checkpoint-load snippets
51remain illustrative until run against the real weights for the user's data.
53From the repository root:
55```bash
56uv run skills-ref validate skills/timesfm-forecasting
57python tests/run_all.py --isolated timesfm-forecasting
58```

Executed checks for this refresh

The repository suite covers strict CSV/date handling, missingness, quantile indices/shapes, system checks, native 2.5 preprocessing/compile/XReg with a controlled lightweight decoder, tiny random 2.5 native transformer/decode, and tiny random 3.0 native save/load/inference. Example plots and animations are tested with clearly synthetic model outputs. These verify API mechanics and file/visual alignment, not pretrained accuracy, calibration, cloud availability or GPU/MLX performance. Checkpoint-load snippets remain illustrative until run against the real weights for the user's data.

From the repository root:

uv run skills-ref validate skills/timesfm-forecasting
python tests/run_all.py --isolated timesfm-forecasting
Raw lines 42 to 58
Runs code python tests/run_all.py :57