Skip to content

pandas/copy-in-chain

Rule copy-in-chain
Pack pandas
Severity warning
Fix unsafe, with xpd check --fix --unsafe-fixes
Library versions every version

Methods such as rename and assign return a new frame, so a .copy() next to them is thrown away. Drop the .copy().

clean = df.copy().rename(columns=names)
tidy = df.reset_index(drop=True).copy()
clean = df.rename(columns=names)
tidy = df.reset_index(drop=True)

The rule reports df.copy().rename(...) and df.rename(...).copy(), also with assign, reset_index, set_index, astype, fillna, drop(columns=) and pd.concat, and it reports drop(name, axis=1).copy(). Under copy-on-write each of the methods returns a new frame, so the copy does the work twice. Each copy cost 4 to 10 milliseconds on 6 columns, and dropping it was 3 to 118 times faster.

The rule leaves a copy next to a slow method such as sort_values or merge alone, because dropping the copy made the code less than 2 times faster. It leaves df.copy().pipe(f) alone, because f may write to its argument.

xpd check --fix --unsafe-fixes applies the fixes of this rule. Check the result, because an unsafe fix can change what the code does.

The unsafe fix drops .copy(). Without the copy, .values and .to_numpy() of the result give an array that is read-only or shares memory with the source.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

$M returns a new frame, so the copy before it is thrown away. Drop .copy().

Browse the other rules of the pandas pack.