pandas/groupby-lambda-window
| Rule | groupby-lambda-window |
|---|---|
| Pack | pandas |
| Severity | warning |
| Fix | unsafe, with xpd check --fix --unsafe-fixes |
| Library versions | every version |
A lambda in transform that calls shift, diff or a similar method runs Python once for every group. Call the method on the groupby directly.
previous = df.groupby("region")["load"].transform(lambda group: group.shift(1))previous = df.groupby("region")["load"].shift(1)What the rule reports
Section titled “What the rule reports”The rule reports g["b"].transform(lambda v: v.shift(1)), which runs Python once per group where the groupby method runs on all groups at once.
How to fix it
Section titled “How to fix it”xpd check --fix --unsafe-fixes applies the fixes of this rule. Check the result, because an unsafe fix can change what the code does.
The unsafe fix writes g["b"].shift(1), g["b"].cumsum(). The method can give another dtype or index type on an empty or all-missing group.
The fixes for cumsum and cumprod are unsafe, because the groupby methods add up a running total with compensated summation, which keeps track of the rounding error of each addition. A total over values far apart in size can therefore change. The rule leaves shift(freq=) alone, because it moves the index.
Message
Section titled “Message”xpd prints this message with the issue. A name that starts with $ stands for code from your file.
Call $G.$W(...) directly. A lambda in transform runs in Python once per group.Browse the other rules of the pandas pack.