Skip to content

category_encoders/hashing-default-components

Rule hashing-default-components
Pack Category Encoders
Severity warning
Fix none, edit by hand
Library versions every version

HashingEncoder hashes every value into 8 columns by default, shared by all encoded columns, so many values collide. Pass n_components large enough for your categories.

encoder = ce.HashingEncoder(cols=["city", "shop"])
encoder = ce.HashingEncoder(cols=["city", "shop"], n_components=1024)

The rule reports HashingEncoder(...) without n_components. The encoder hashes every value into n_components columns, 8 by default, and all encoded columns share them.

xpd has no automatic fix, so make the change by hand.

n_components Distinct encoded rows for a column of 100 distinct values
8, the default 8
64 50
1024 94

Two columns encoded by one HashingEncoder() gave 8 output columns in all, and the rows (x, y) and (y, x) got the same code. The encoder therefore also loses which column a value came from. The rule has no fix, because the right number depends on how many distinct values the columns hold. A call that passes the number by position, a call inside isinstance, and HashingEncoder().hashing_trick(...), which takes the number as N, are left alone.

Other advice about the class was measured and dropped:

  • With the default number of threads, max_process=1 was 23 times faster than the default at 1,000 rows and 2.2 times slower at 100,000 rows, so no setting is right for both.
  • FeatureHasher of scikit-learn was 23 times faster at 10,000 rows and 5.4 times faster at 100,000 rows with the default number of threads, but it returns a sparse matrix and takes no column names, so it can't take the place of the encoder.

xpd prints this message with the issue. A name that starts with $ stands for code from your file.

HashingEncoder hashes into 8 columns by default, shared by all encoded columns. A column of 100 distinct values gave 8 distinct encoded rows, against 94 with n_components=1024. Pass n_components.

Browse the other rules of the Category Encoders pack.