pandas/apply-recomputes-aggregate
| Rule | apply-recomputes-aggregate |
|---|---|
| Pack | pandas |
| Severity | warning |
| Fix | none, edit by hand |
| Library versions | every version |
A lambda that computes df["b"].mean() runs that mean again for every row, so the time grows with the square of the row count. Compute the value once and use it on the whole column.
centered = df["load"].apply(lambda value: value - df["load"].mean())centered = df["load"] - df["load"].mean()What the rule reports
Section titled “What the rule reports”The rule reports df["b"].apply(lambda v: v - df["b"].mean()), which computes the mean of the whole column again for every row. The time grows with the square of the row count. On 20,000 rows the lambda took 0.34 seconds and the column form took 0.2 milliseconds. The rule reports a reduction of a column taken by a text label. It leaves lambda v: v["scores"].mean() alone, because the lambda reads the row's own value.
How to fix it
Section titled “How to fix it”xpd has no automatic fix, so make the change by hand.
Message
Section titled “Message”xpd prints this message with the issue. A name that starts with $ stands for code from your file.
This lambda computes $O.$A() again for every row. Compute it once and use it on the whole column, as in $S - $O.$A().Browse the other rules of the pandas pack.