Skip to content

pandas/groupby-lambda-window

Rule groupby-lambda-window
Pack pandas
Severity warning
Fix unsafe, with xpd check --fix --unsafe-fixes
Library versions every version

A lambda in transform that calls shift, diff or a similar method runs Python once for every group. Call the method on the groupby directly.

previous = df.groupby("region")["load"].transform(lambda group: group.shift(1))
previous = df.groupby("region")["load"].shift(1)

The rule reports g["b"].transform(lambda v: v.shift(1)), which runs Python once per group where the groupby method runs on all groups at once.

xpd check --fix --unsafe-fixes applies the fixes of this rule. Check the result, because an unsafe fix can change what the code does.

The unsafe fix writes g["b"].shift(1), g["b"].cumsum(). The method can give another dtype or index type on an empty or all-missing group.

The fixes for cumsum and cumprod are unsafe, because the groupby methods add up a running total with compensated summation, which keeps track of the rounding error of each addition. A total over values far apart in size can therefore change. The rule leaves shift(freq=) alone, because it moves the index.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

Call $G.$W(...) directly. A lambda in transform runs in Python once per group.

Browse the other rules of the pandas pack.