Skip to content

pandas/concat-then-category

Rule concat-then-category
Pack pandas
Severity warning
Fix unsafe, with xpd check --fix --unsafe-fixes
Library versions pandas 3.1 and later

Casting to category after concat makes a second pass over the data. On pandas 3.1 and later, pass union_categories=True to concat instead.

combined = pd.concat(parts).astype("category")
combined = pd.concat(parts, union_categories=True)

The rule reports pd.concat(parts).astype("category") on pandas 3.1 and later.

xpd check --fix --unsafe-fixes applies the fixes of this rule. Check the result, because an unsafe fix can change what the code does.

The unsafe fix writes pd.concat(parts, union_categories=True).

The fix is unsafe, because the order of the categories differs. Parts that are not categorical also stay as they are.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

concat(..., union_categories=True) keeps the categorical dtype without a second pass over the data.

Browse the other rules of the pandas pack.