Skip to content

sklearn/refit-per-n-estimators

Rule refit-per-n-estimators
Pack scikit-learn
Severity warning
Fix none, edit by hand
Library versions every version

Fitting a new forest for each tree count in a loop grows every tree again. Build the model once with warm_start=True and raise n_estimators before each fit.

forest = RandomForestClassifier(warm_start=True, random_state=0)
for n in [10, 50, 100]:
forest = RandomForestClassifier(n_estimators=n, random_state=0)
forest.set_params(n_estimators=n)
forest.fit(X_train, y_train)
scores.append(forest.score(X_test, y_test))

The rule reports a RandomForest, ExtraTrees, GradientBoosting or Bagging model or an IsolationForest built with n_estimators=n in a loop over n and fitted there.

The rule leaves the following loops alone:

xpd has no automatic fix, so make the change by hand.

The new form relies on warm_start=True, which keeps the trees already grown and draws every seed from random_state in the same order. The grown models were bit equal to models fitted from scratch, and warm start was faster:

Trees Classes Warm start was faster by
n_estimators 10, 20, 50 and 100, at 1,000 and 5,000 rows RandomForestClassifier, RandomForestRegressor, ExtraTreesClassifier, GradientBoostingClassifier, BaggingClassifier, IsolationForest 1.64 to 2.75 times
10 to 100 in steps of 10 RandomForestClassifier 6.82 and 3.59 times
  • a loop whose model is passed to a cross-validation function or to clone, which clones it, so warm start can't help
  • a loop with another loop inside it, where another setting changes between fits
  • a loop that reads a clock, because it times each fit

The rule as the readers saw it made 1 report on the held-back files, on a loop that timed each fit, and both readers called it wrong. Recall was 30 of 30.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

Each pass of this loop fits $C again from scratch. Create it once before the loop with warm_start=True and call set_params(n_estimators=$N) before fit, which keeps the trees already grown, gives the same model while $N grows, and measured at least 1.6 times faster for 10, 20, 50 and 100 trees and at least 3.5 times faster for 10 to 100 trees in steps of 10.

Browse the other rules of the scikit-learn pack.