Skip to content

pandas/read-csv-converters

Rule read-csv-converters
Pack pandas
Severity warning
Fix none, edit by hand
Library versions every version

converters in read_csv calls a Python function once for every value, which was 3.6 to 5.1 times slower. Pass dtype for a type, or change the column after the read.

df = pd.read_csv(path, converters={"price": float, "name": str.strip})
df = pd.read_csv(path, dtype={"price": float})
df["name"] = df["name"].str.strip()

The rule reports read_csv(..., converters=...), which calls a Python function once per value and was 3.6 to 5.1 times slower than dtype or a column method after the read, measured with the reader's default threads on a 32-core machine.

xpd has no automatic fix, so make the change by hand.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

converters calls a Python function once per value. Pass dtype for a type, or change the column after reading, as in df["s"].str.strip().

Browse the other rules of the pandas pack.