Skill Tracer
Skills › Scientific Agent Skills › timesfm-forecasting › data_preparation.md

data_preparation.md

66 lines · skills/timesfm-forecasting/references/data_preparation.md at 5f2bc03

Open on GitHub
1# Prepare a forecast without changing its time axis
2
3TimesFM 2.5 sees positions, not timestamps. Give a list of finite one-dimensional
4float arrays, one regular univariate history per array. Variable lengths are
5allowed; each series needs its own known cadence and forecast origin. A batch
6horizon of 12 means 12 steps, not a shared wall-clock duration across cadences.
7

Prepare a forecast without changing its time axis

TimesFM 2.5 sees positions, not timestamps. Give a list of finite one-dimensional float arrays, one regular univariate history per array. Variable lengths are allowed; each series needs its own known cadence and forecast origin. A batch horizon of 12 means 12 steps, not a shared wall-clock duration across cadences.

Raw lines 1 to 7
8## CSV/DataFrame recipe
9
10The bundled helper sorts dates, validates uniqueness and checks the full grid.
11Use it directly for CSVs; when adapting DataFrames apply the same checks.
13```python
14# Run from the skill scripts directory, or put that directory on sys.path.
15from forecast_csv import load_csv, prepare_inputs
16frame, columns, date_column = load_csv(
17 "monthly_sales.csv", date_col="date", value_cols=["sales"], freq="MS"
19inputs = prepare_inputs(frame, columns, missing="error")
20```
22Select target columns explicitly when numeric IDs/metadata exist. Two or fewer
23dates cannot establish a unique frequency, so declare it. Do not silently
24resample: a sum for counts, mean for intensities and last value for stock levels
25represent different targets. Record timezone and daylight-saving decisions.
26For long-format data, group by series ID, validate each grid separately and
27retain a parallel ordered ID list. Do not use `.dropna()` to build histories.
29Pandas input adapters: `read_excel(..., engine="openpyxl")` needs openpyxl;
30`read_parquet(..., engine="pyarrow")` needs pyarrow. These optional adapters were
31source-checked, not exercised in this refresh. JSON/NumPy input must still be
32checked for finite values, shape, cadence and origin.

CSV/DataFrame recipe

The bundled helper sorts dates, validates uniqueness and checks the full grid. Use it directly for CSVs; when adapting DataFrames apply the same checks.

# Run from the skill scripts directory, or put that directory on sys.path.
from forecast_csv import load_csv, prepare_inputs
frame, columns, date_column = load_csv(
    "monthly_sales.csv", date_col="date", value_cols=["sales"], freq="MS"
)
inputs = prepare_inputs(frame, columns, missing="error")

Select target columns explicitly when numeric IDs/metadata exist. Two or fewer dates cannot establish a unique frequency, so declare it. Do not silently resample: a sum for counts, mean for intensities and last value for stock levels represent different targets. Record timezone and daylight-saving decisions. For long-format data, group by series ID, validate each grid separately and retain a parallel ordered ID list. Do not use .dropna() to build histories.

Pandas input adapters: read_excel(..., engine="openpyxl") needs openpyxl; read_parquet(..., engine="pyarrow") needs pyarrow. These optional adapters were source-checked, not exercised in this refresh. JSON/NumPy input must still be checked for finite values, shape, cadence and origin.

Raw lines 8 to 33
34## Missing values and transformations
36The release's base forecast strips leading NaNs and interpolates later NaNs,
37including trailing last-value extension. This is computational convenience,
38not a missing-data model. Reject all-missing, empty, infinite or overflowed
39float32 inputs. Choose a documented missingness policy before forecasting.
41The CLI rejects missing data by default. `--missing interpolate` fills internal
42gaps only and retains the same array length; leading/trailing gaps fail because
43they make the forecast origin or observed history ambiguous. Impute separately
44inside each training cutoff. Interpolation across a held-out target leaks future
45information. Imputed observations should not become “actuals” for coverage.
47Validate clipping/outlier handling scientifically; do not erase real events to
48improve a score. Fit scalers, detrending and feature selection on training data
49only, reverse transformations for evaluation, and retain units. Constant or short
50series deserve a naive baseline; there is no universal 32-point model minimum
51or guarantee that more history improves accuracy.

Missing values and transformations

The release's base forecast strips leading NaNs and interpolates later NaNs, including trailing last-value extension. This is computational convenience, not a missing-data model. Reject all-missing, empty, infinite or overflowed float32 inputs. Choose a documented missingness policy before forecasting.

The CLI rejects missing data by default. --missing interpolate fills internal gaps only and retains the same array length; leading/trailing gaps fail because they make the forecast origin or observed history ambiguous. Impute separately inside each training cutoff. Interpolation across a held-out target leaks future information. Imputed observations should not become “actuals” for coverage.

Validate clipping/outlier handling scientifically; do not erase real events to improve a score. Fit scalers, detrending and feature selection on training data only, reverse transformations for evaluation, and retain units. Constant or short series deserve a naive baseline; there is no universal 32-point model minimum or guarantee that more history improves accuracy.

Raw lines 34 to 52
53## XReg inputs
55For each series i with context length Li and requested horizon H:
57- target is `(Li,)`, containing only observations available at origin;
58- each dynamic feature is `(Li+H,)`, with the future segment known at origin;
59- each static feature supplies one value per series;
60- feature names, series order, time positions and inferred horizons agree.
62A weather forecast issued at origin can be a future covariate; the weather later
63observed at that date cannot be used for a realistic backtest. Planned prices
64and holiday schedules are conditional assumptions, not causal effects. Numeric
65and categorical covariates are processed by pooled linear XReg, not native
66multivariate target attention. The distinct 3.0 API supports past-only features.

XReg inputs

For each series i with context length Li and requested horizon H:

  • target is (Li,), containing only observations available at origin;
  • each dynamic feature is (Li+H,), with the future segment known at origin;
  • each static feature supplies one value per series;
  • feature names, series order, time positions and inferred horizons agree.

A weather forecast issued at origin can be a future covariate; the weather later observed at that date cannot be used for a realistic backtest. Planned prices and holiday schedules are conditional assumptions, not causal effects. Numeric and categorical covariates are processed by pooled linear XReg, not native multivariate target attention. The distinct 3.0 API supports past-only features.

Raw lines 53 to 66