Skip to content

sklego/outlier-remover-before-model

Rule outlier-remover-before-model
Pack scikit-lego
Severity error
Fix none, edit by hand
Library versions every version

OutlierRemover drops rows of X and leaves y whole, so the model after it raises ValueError. Drop the outliers from X and y before the pipeline.

pipeline = make_pipeline(OutlierRemover(IsolationForest(), refit=True), LinearRegression())
pipeline.fit(X, y)
kept = IsolationForest().fit_predict(X) != -1
model = LinearRegression().fit(X[kept], y[kept])

The rule reports OutlierRemover in a pipeline that also holds a step named like a regressor or a classifier. The remover drops rows of X while the pipeline is fitted, and a scikit-learn pipeline can't drop the same rows of y.

xpd has no automatic fix, so make the change by hand.

fit raised ValueError "Found input variables with inconsistent numbers of samples: [328, 400]". A pipeline that ended in KMeans, which reads no y, fitted on the 328 kept rows, and the rule leaves it alone.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

OutlierRemover drops rows of X and leaves y whole, so the model after it raises ValueError "Found input variables with inconsistent numbers of samples: [328, 400]". Drop the outliers from X and y before the pipeline, with the detector's fit_predict as a mask.

Browse the other rules of the scikit-lego pack.