Skill Tracer
Skills › Scientific Agent Skills › scikit-survival › ensemble-models.md

ensemble-models.md

302 lines · skills/scikit-survival/references/ensemble-models.md at 5f2bc03

Open on GitHub
1# Survival trees, forests, and boosting
2
3Verified for scikit-survival 0.28.0 on 2026-10-01.
4

Survival trees, forests, and boosting

Verified for scikit-survival 0.28.0 on 2026-10-01.

Raw lines 1 to 4
5## Model families
6
7- `SurvivalTree`: one log-rank survival tree.
8- `RandomSurvivalForest`: bootstrap-aggregated survival trees with random feature
9 subsets.
10- `ExtraSurvivalTrees`: additional randomization of candidate split thresholds.
11- `GradientBoostingSurvivalAnalysis`: regression-tree gradient boosting.
12- `ComponentwiseGradientBoostingSurvivalAnalysis`: linear componentwise base
13 learners and implicit sparse selection.
15Model family does not determine quality in advance. Compare prespecified candidates
16with identical outer resamples, censoring assumptions, time grids, and preprocessing.

Model families

  • SurvivalTree: one log-rank survival tree.
  • RandomSurvivalForest: bootstrap-aggregated survival trees with random feature subsets.
  • ExtraSurvivalTrees: additional randomization of candidate split thresholds.
  • GradientBoostingSurvivalAnalysis: regression-tree gradient boosting.
  • ComponentwiseGradientBoostingSurvivalAnalysis: linear componentwise base learners and implicit sparse selection.

Model family does not determine quality in advance. Compare prespecified candidates with identical outer resamples, censoring assumptions, time grids, and preprocessing.

Raw lines 5 to 17
18## Random survival forest
20```python
21from sksurv.ensemble import RandomSurvivalForest
23model = RandomSurvivalForest(
24 n_estimators=500,
25 min_samples_split=10,
26 min_samples_leaf=8,
27 max_features="sqrt",
28 n_jobs=1,
29 random_state=20260723,
31model.fit(X_train, y_train)
32```
34Current defaults include `n_estimators=100`, `min_samples_split=6`,
35`min_samples_leaf=3`, `max_features="sqrt"`, `bootstrap=True`, and
36`low_memory=False`.
38Each terminal node estimates:
40- a survival function using Kaplan-Meier;
41- a cumulative hazard function using Nelson-Aalen;
42- a risk score summing the estimated cumulative hazard over the training event
43 times; its magnitude is neither a probability nor an expected count at a named horizon.
45Forest predictions average tree predictions:
47```python
48risk = model.predict(X_test) # (n_test,), higher is riskier
49survival = model.predict_survival_function(X_test, return_array=True)
50cumulative_hazard = model.predict_cumulative_hazard_function(
51 X_test, return_array=True
53times = model.unique_times_
54```
56Array predictions use the model's `unique_times_`. For an evaluation grid, returned
57step functions are often more convenient:
59```python
60import numpy as np
62functions = model.predict_survival_function(X_test, return_array=False)
63survival_on_grid = np.vstack([fn(evaluation_times) for fn in functions])
64```
66Do not treat the numeric magnitude of `predict()` as an event probability.

Random survival forest

from sksurv.ensemble import RandomSurvivalForest

model = RandomSurvivalForest(
    n_estimators=500,
    min_samples_split=10,
    min_samples_leaf=8,
    max_features="sqrt",
    n_jobs=1,
    random_state=20260723,
)
model.fit(X_train, y_train)

Current defaults include n_estimators=100, min_samples_split=6, min_samples_leaf=3, max_features="sqrt", bootstrap=True, and low_memory=False.

Each terminal node estimates:

  • a survival function using Kaplan-Meier;
  • a cumulative hazard function using Nelson-Aalen;
  • a risk score summing the estimated cumulative hazard over the training event times; its magnitude is neither a probability nor an expected count at a named horizon.

Forest predictions average tree predictions:

risk = model.predict(X_test)  # (n_test,), higher is riskier
survival = model.predict_survival_function(X_test, return_array=True)
cumulative_hazard = model.predict_cumulative_hazard_function(
    X_test, return_array=True
)
times = model.unique_times_

Array predictions use the model's unique_times_. For an evaluation grid, returned step functions are often more convenient:

import numpy as np

functions = model.predict_survival_function(X_test, return_array=False)
survival_on_grid = np.vstack([fn(evaluation_times) for fn in functions])

Do not treat the numeric magnitude of predict() as an event probability.

Raw lines 18 to 67
68### Missing values and memory
70Current survival trees and forest split logic supports missing values, with fixes
71aligned to scikit-learn 1.8 in scikit-survival 0.27. This does not remove the need
72to:
74- verify which columns and missingness patterns are supported;
75- encode categorical columns consistently;
76- keep all learned preprocessing within training folds;
77- assess whether missingness itself changes across deployment settings.
79`low_memory=True` reduces stored prediction state but disables survival-function
80and cumulative-hazard prediction. It is incompatible with Brier-score workflows
81that require survival probabilities.

Missing values and memory

Current survival trees and forest split logic supports missing values, with fixes aligned to scikit-learn 1.8 in scikit-survival 0.27. This does not remove the need to:

  • verify which columns and missingness patterns are supported;
  • encode categorical columns consistently;
  • keep all learned preprocessing within training folds;
  • assess whether missingness itself changes across deployment settings.

low_memory=True reduces stored prediction state but disables survival-function and cumulative-hazard prediction. It is incompatible with Brier-score workflows that require survival probabilities.

Raw lines 68 to 82
83### OOB estimates
85With `oob_score=True` and bootstrap sampling, `oob_score_` provides an internal
86out-of-bag concordance estimate. It is not a substitute for:
88- nested tuning when parameters were selected using OOB results;
89- an independent test set;
90- censoring-aware probability metrics;
91- external validation.

OOB estimates

With oob_score=True and bootstrap sampling, oob_score_ provides an internal out-of-bag concordance estimate. It is not a substitute for:

  • nested tuning when parameters were selected using OOB results;
  • an independent test set;
  • censoring-aware probability metrics;
  • external validation.
Raw lines 83 to 92
93## Extra survival trees
95```python
96from sksurv.ensemble import ExtraSurvivalTrees
98model = ExtraSurvivalTrees(
99 n_estimators=500,
100 min_samples_leaf=8,
101 max_features="sqrt",
102 n_jobs=1,
103 random_state=20260723,
105model.fit(X_train, y_train)
106```
108Extra trees randomize split thresholds in addition to feature selection. They are
109not guaranteed to be faster, better regularized, or better calibrated for a given
110dataset. Tune and evaluate them as a distinct candidate under the same protocol.

Extra survival trees

from sksurv.ensemble import ExtraSurvivalTrees

model = ExtraSurvivalTrees(
    n_estimators=500,
    min_samples_leaf=8,
    max_features="sqrt",
    n_jobs=1,
    random_state=20260723,
)
model.fit(X_train, y_train)

Extra trees randomize split thresholds in addition to feature selection. They are not guaranteed to be faster, better regularized, or better calibrated for a given dataset. Tune and evaluate them as a distinct candidate under the same protocol.

Raw lines 93 to 111
112## Permutation importance
114Survival forest impurity importance is not implemented as a valid
115`feature_importances_` measure. Use held-out permutation importance with an
116explicit score:
118```python
119from sklearn.inspection import permutation_importance
121result = permutation_importance(
122 fitted_pipeline,
123 X_test,
124 y_test,
125 scoring=None, # estimator.score: Harrell concordance
126 n_repeats=20,
127 random_state=20260723,
128 n_jobs=1,
130```
132If Harrell C is not the target, wrap the estimator with the appropriate
133scikit-survival scorer class before permutation or write a scorer that fits no
134state on the test set. Importance depends on:
136- the metric and horizon;
137- correlated features;
138- the held-out population;
139- preprocessing and random seed.
141It is predictive sensitivity, not causal importance.

Permutation importance

Survival forest impurity importance is not implemented as a valid feature_importances_ measure. Use held-out permutation importance with an explicit score:

from sklearn.inspection import permutation_importance

result = permutation_importance(
    fitted_pipeline,
    X_test,
    y_test,
    scoring=None,  # estimator.score: Harrell concordance
    n_repeats=20,
    random_state=20260723,
    n_jobs=1,
)

If Harrell C is not the target, wrap the estimator with the appropriate scikit-survival scorer class before permutation or write a scorer that fits no state on the test set. Importance depends on:

  • the metric and horizon;
  • correlated features;
  • the held-out population;
  • preprocessing and random seed.

It is predictive sensitivity, not causal importance.

Raw lines 112 to 142
143## Tree gradient boosting
145```python
146from sksurv.ensemble import GradientBoostingSurvivalAnalysis
148model = GradientBoostingSurvivalAnalysis(
149 loss="coxph",
150 learning_rate=0.05,
151 n_estimators=300,
152 max_depth=2,
153 subsample=0.8,
154 random_state=20260723,
156model.fit(X_train, y_train)
157```
159Current losses:
161- `"coxph"`: Cox partial-likelihood objective; `predict()` is a higher-is-riskier
162 score, and baseline-based survival/cumulative-hazard methods are available.
163- `"ipcwls"`: intended IPC-weighted least-squares AFT objective; see the release
164 defects below before relying on it.
165- `"squared"`: squared-error time-oriented objective.
167Time-oriented losses do not make `predict()` a Cox risk score and do not provide
168the same baseline survival-function interface. Both non-Cox losses return exponentiated predictions on the original-time scale.
169Confirm prediction direction before
170using concordance or dynamic AUC; negate a predicted-time output only when that
171conversion is explicitly intended and reported.
173Released 0.28.0 `ipcwls` has two native/source-confirmed limitations in both
174boosting classes: positive times below 1 become negative after the internal log
175transform and the censoring-weight estimator rejects them; and the gradient
176omits IPC weights even though the reported loss uses them. With fixed iterations,
177full subsampling, and default sample weights, changing event/censor labels while
178keeping observed times fixed produced identical predictions on the synthetic
179fixture. Do not treat a successful `ipcwls` fit as validated IPC-weighted training
180or change time units merely to hide the error. Prefer a verified alternative such
181as `IPCRidge` when an IPC-weighted AFT fit is required.
183Current regularization controls:
185- `learning_rate`
186- `n_estimators`
187- `subsample`
188- `dropout_rate`
189- tree depth/leaf controls
190- `ccp_alpha`
191- `validation_fraction`, `n_iter_no_change`, and `tol`
192- a custom `monitor` passed to `fit()` for controlled early stopping
194The old `criterion` parameter was removed in 0.28. Do not copy it from older
195examples.
197Use only training data for internal early stopping. The final test set must not be
198the monitor or validation fraction.

Tree gradient boosting

from sksurv.ensemble import GradientBoostingSurvivalAnalysis

model = GradientBoostingSurvivalAnalysis(
    loss="coxph",
    learning_rate=0.05,
    n_estimators=300,
    max_depth=2,
    subsample=0.8,
    random_state=20260723,
)
model.fit(X_train, y_train)

Current losses:

  • "coxph": Cox partial-likelihood objective; predict() is a higher-is-riskier score, and baseline-based survival/cumulative-hazard methods are available.
  • "ipcwls": intended IPC-weighted least-squares AFT objective; see the release defects below before relying on it.
  • "squared": squared-error time-oriented objective.

Time-oriented losses do not make predict() a Cox risk score and do not provide the same baseline survival-function interface. Both non-Cox losses return exponentiated predictions on the original-time scale. Confirm prediction direction before using concordance or dynamic AUC; negate a predicted-time output only when that conversion is explicitly intended and reported.

Released 0.28.0 ipcwls has two native/source-confirmed limitations in both boosting classes: positive times below 1 become negative after the internal log transform and the censoring-weight estimator rejects them; and the gradient omits IPC weights even though the reported loss uses them. With fixed iterations, full subsampling, and default sample weights, changing event/censor labels while keeping observed times fixed produced identical predictions on the synthetic fixture. Do not treat a successful ipcwls fit as validated IPC-weighted training or change time units merely to hide the error. Prefer a verified alternative such as IPCRidge when an IPC-weighted AFT fit is required.

Current regularization controls:

  • learning_rate
  • n_estimators
  • subsample
  • dropout_rate
  • tree depth/leaf controls
  • ccp_alpha
  • validation_fraction, n_iter_no_change, and tol
  • a custom monitor passed to fit() for controlled early stopping

The old criterion parameter was removed in 0.28. Do not copy it from older examples.

Use only training data for internal early stopping. The final test set must not be the monitor or validation fraction.

Raw lines 143 to 199
200## Componentwise boosting
202```python
203from sksurv.ensemble import ComponentwiseGradientBoostingSurvivalAnalysis
205model = ComponentwiseGradientBoostingSurvivalAnalysis(
206 loss="coxph",
207 learning_rate=0.1,
208 n_estimators=300,
209 subsample=0.8,
210 random_state=20260723,
212model.fit(X_train_scaled, y_train)
213```
215At each iteration, a componentwise learner updates one encoded feature. The final
216model is linear and often sparse. Its `coef_` includes the fitted intercept entry
217used by the implementation; align coefficients with transformed feature names
218carefully.
220Iteration count is a selection parameter. Repeatedly checking test performance
221while increasing `n_estimators` leaks the test set. Tune it inside inner CV, and
222assess selected-feature stability across outer resamples.

Componentwise boosting

from sksurv.ensemble import ComponentwiseGradientBoostingSurvivalAnalysis

model = ComponentwiseGradientBoostingSurvivalAnalysis(
    loss="coxph",
    learning_rate=0.1,
    n_estimators=300,
    subsample=0.8,
    random_state=20260723,
)
model.fit(X_train_scaled, y_train)

At each iteration, a componentwise learner updates one encoded feature. The final model is linear and often sparse. Its coef_ includes the fitted intercept entry used by the implementation; align coefficients with transformed feature names carefully.

Iteration count is a selection parameter. Repeatedly checking test performance while increasing n_estimators leaks the test set. Tune it inside inner CV, and assess selected-feature stability across outer resamples.

Raw lines 200 to 223
224## Leakage-safe nested tuning
226```python
227from sklearn.model_selection import GridSearchCV
229inner_search = GridSearchCV(
230 pipeline,
231 {
232 "model__min_samples_leaf": [3, 8, 16],
233 "model__max_features": ["sqrt", 0.5, 1.0],
234 },
235 cv=inner_splits,
236 error_score="raise",
237 n_jobs=1,
239inner_search.fit(X_outer_train, y_outer_train)
240risk_outer = inner_search.predict(X_outer_valid)
241```
243Run this search inside each outer fold when reporting cross-validated tuned
244performance. Every outer score must use:
246- outer-training preprocessing only;
247- outer-training censoring distribution for IPCW metrics;
248- an evaluation grid supported by that outer-training fold;
249- outer-validation predictions never used in parameter selection.
251After protocol assessment, tune on all development data and evaluate once on the
252reserved test set.

Leakage-safe nested tuning

from sklearn.model_selection import GridSearchCV

inner_search = GridSearchCV(
    pipeline,
    {
        "model__min_samples_leaf": [3, 8, 16],
        "model__max_features": ["sqrt", 0.5, 1.0],
    },
    cv=inner_splits,
    error_score="raise",
    n_jobs=1,
)
inner_search.fit(X_outer_train, y_outer_train)
risk_outer = inner_search.predict(X_outer_valid)

Run this search inside each outer fold when reporting cross-validated tuned performance. Every outer score must use:

  • outer-training preprocessing only;
  • outer-training censoring distribution for IPCW metrics;
  • an evaluation grid supported by that outer-training fold;
  • outer-validation predictions never used in parameter selection.

After protocol assessment, tune on all development data and evaluate once on the reserved test set.

Raw lines 224 to 253
254## Probability prediction and calibration
256Forests and Cox-loss boosting can produce survival probabilities. To use Brier
257metrics:
259```python
260functions = fitted_pipeline.predict_survival_function(X_test)
261survival_probability = np.vstack([fn(times) for fn in functions])
262```
264Then verify:
266- shape is `(n_test, n_times)`;
267- values are within `[0, 1]`;
268- each row is non-increasing over time;
269- `times` is strictly increasing and train-supported;
270- censoring weights are learned from training outcomes.
272A lower Brier score does not prove good calibration in every subgroup or horizon.
273Use horizon-specific calibration assessment on independent data. Do not claim
274clinical utility from concordance, AUC, or Brier score alone.

Probability prediction and calibration

Forests and Cox-loss boosting can produce survival probabilities. To use Brier metrics:

functions = fitted_pipeline.predict_survival_function(X_test)
survival_probability = np.vstack([fn(times) for fn in functions])

Then verify:

  • shape is (n_test, n_times);
  • values are within [0, 1];
  • each row is non-increasing over time;
  • times is strictly increasing and train-supported;
  • censoring weights are learned from training outcomes.

A lower Brier score does not prove good calibration in every subgroup or horizon. Use horizon-specific calibration assessment on independent data. Do not claim clinical utility from concordance, AUC, or Brier score alone.

Raw lines 254 to 275
276## Practical selection questions
278- Need coefficient-level PH interpretation? Start with a prespecified Cox model.
279- Need nonlinear interactions and probability curves? Compare forest and Cox-loss
280 boosting.
281- Need sparse linear prediction? Compare Coxnet and componentwise boosting.
282- Need time-oriented AFT prediction? Consider IPC-weighted losses and state the
283 censoring assumptions.
284- Need very large data? Benchmark memory and run time; kernel SVM and large
285 survival forests can be expensive.
287These are candidate-selection prompts, not performance guarantees.

Practical selection questions

  • Need coefficient-level PH interpretation? Start with a prespecified Cox model.
  • Need nonlinear interactions and probability curves? Compare forest and Cox-loss boosting.
  • Need sparse linear prediction? Compare Coxnet and componentwise boosting.
  • Need time-oriented AFT prediction? Consider IPC-weighted losses and state the censoring assumptions.
  • Need very large data? Benchmark memory and run time; kernel SVM and large survival forests can be expensive.

These are candidate-selection prompts, not performance guarantees.

Raw lines 276 to 288
289## Sources
291Official sources checked 2026-10-01:
293- [Released survival losses](https://github.com/sebp/scikit-survival/blob/v0.28.0/sksurv/ensemble/survival_loss.py) and [fit loops](https://github.com/sebp/scikit-survival/blob/v0.28.0/sksurv/ensemble/boosting.py) — IPCW caveats reproduced natively.
295- [Random survival forest user guide](https://scikit-survival.readthedocs.io/en/stable/user_guide/random-survival-forest.html)
296- [Gradient boosting user guide](https://scikit-survival.readthedocs.io/en/stable/user_guide/boosting.html)
297- [RandomSurvivalForest API](https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.ensemble.RandomSurvivalForest.html)
298- [ExtraSurvivalTrees API](https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.ensemble.ExtraSurvivalTrees.html)
299- [GradientBoostingSurvivalAnalysis API](https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.ensemble.GradientBoostingSurvivalAnalysis.html)
300- [ComponentwiseGradientBoostingSurvivalAnalysis API](https://scikit-survival.readthedocs.io/en/stable/api/generated/sksurv.ensemble.ComponentwiseGradientBoostingSurvivalAnalysis.html)
301- [0.28 release notes](https://scikit-survival.readthedocs.io/en/stable/release_notes/v0.28.html)
302- [0.27 release notes](https://scikit-survival.readthedocs.io/en/stable/release_notes/v0.27.html)