Skip to content

sklego/regression-outlier-column-name

Rule regression-outlier-column-name
Pack scikit-lego
Severity error
Fix none, edit by hand
Library versions every version

RegressionOutlierDetector with a column name keeps the target among the features, so the inner model predicts the target from itself. Pass the column's position on an array instead.

detector = RegressionOutlierDetector(LinearRegression(), column="y").fit(frame)
position = frame.columns.get_loc("y")
detector = RegressionOutlierDetector(LinearRegression(), column=position).fit(frame.to_numpy())

The rule reports RegressionOutlierDetector(model, column="y"), where the column is a name. The detector finds the target of a frame by name, and then drops the column whose position equals column from the features. A name equals no position, so the target stays among the features and the inner model predicts it from itself.

xpd has no automatic fix, so make the change by hand.

On 500 rows with 10 planted outliers, the two forms gave the following fits:

Form Features Coefficients Spread of the residuals Rows flagged
column="y" on the frame 3 0, 0 and 1 9.50e-16 11
column=2 on frame.to_numpy() 2 not given 3.25 the 10 planted rows and no other

A position on a frame is wrong in another way, which a pattern can't see. The detector compares the frame's column names with the number and finds no match, so the first column becomes the target. scikit-lego's own test of the name form passes because the planted rows have the largest rounding errors.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

RegressionOutlierDetector with a column name keeps the target among the features, so the inner model predicts the target from itself. With 10 planted outliers it fitted the coefficients 0, 0 and 1 and flagged 11 rows, where column=2 on df.to_numpy() flagged the 10 planted rows. Pass the position of the column and fit on a NumPy array.

Browse the other rules of the scikit-lego pack.