pandas/column-loop-to-long
| Rule | column-loop-to-long |
|---|---|
| Pack | pandas |
| Severity | info |
| Fix | none, edit by hand |
| Library versions | every version |
A loop that builds one frame per column and stacks them does what melt does in one call. Use df.melt with id_vars and value_vars, which gave the same rows faster.
parts = [df[["id", year]].rename(columns={year: "sales"}).assign(year=year) for year in years]long = pd.concat(parts, ignore_index=True)long = df.melt(id_vars="id", value_vars=years, var_name="year", value_name="sales")What the rule reports
Section titled “What the rule reports”The rule reports a loop over column names that selects one column at a time, renames it to a shared name and stacks the parts with pd.concat. The loop can be a list comprehension, passed to pd.concat directly or through a name that the same function passes to it, or a for statement that appends. For 100,000 rows and 20 columns, melt gave the same 2,000,000 rows in the same order and ran 1.4 times faster.
How to fix it
Section titled “How to fix it”xpd has no automatic fix, so make the change by hand.
Message
Section titled “Message”xpd prints this message with the issue. A name that starts with $ stands for code from your file.
This builds one frame per column and stacks them. melt does that in one call, as in df.melt(id_vars=["id"], value_vars=columns, var_name="year", value_name="sales"). For 20 columns it gave the same rows in the same order and ran 1.4 times faster.Browse the other rules of the pandas pack.