Skip to content

sklego/decay-on-shuffled-rows

Rule decay-on-shuffled-rows
Pack scikit-lego
Severity warning
Fix none, edit by hand
Library versions every version

DecayEstimator gives the last rows the most weight, and train_test_split shuffles the rows, so the weights no longer follow time. Pass shuffle=False to the split.

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0)
X_train, X_test, y_train, y_test = train_test_split(X, y, shuffle=False)
model = DecayEstimator(Ridge(), decay_kwargs={"decay_rate": 0.99}).fit(X_train, y_train)

The rule reports train_test_split without shuffle=False in a file that builds a DecayEstimator. The estimator weights each row by its position, the last row most, and never reads a date. It is therefore right only on rows in time order.

xpd has no automatic fix, so make the change by hand.

On 2,000 rows whose slope changed from 1 to 3 at row 1,000, the model was fitted on the first 1,500 rows and scored on the last 500:

Order of the 1,500 training rows Slope Mean absolute error
in time order 3.028 0.078
shuffled, as the mean of 30 shuffles 1.621 0.648

A pattern can't see which rows the estimator is fitted on, so the rule is a warning.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

DecayEstimator weights rows by position, the last row most, and train_test_split shuffles the rows. On data whose slope changed from 1 to 3, the model fitted on shuffled rows had the slope 1.62 as the mean of 30 shuffles, where the same rows in time order gave 3.03. Pass shuffle=False, or sort the training rows by time before the fit.

Browse the other rules of the scikit-lego pack.