Category Encoders
The Category Encoders pack has 16 rules, and xpd can fix 6 of them.
The Category Encoders rules report code that leaks the target into the test rows, arguments that
give wrong codes without an error, code that raises on category_encoders 2.11.2, and forms that a
release removed or deprecated, in each file that imports category_encoders. Every number about the
Category Encoders rules was measured on category_encoders 2.11.2 with scikit-learn 1.9.1, pandas 3.1.1 and NumPy
2.5.5 on Python 3.13. The examples assume import category_encoders as ce and that the
scikit-learn names they use are imported. Names such as X_train, y_test and encoder stand for
values from your program.
OneHotEncoder, OrdinalEncoder and TargetEncoder exist in scikit-learn as well. A rule about
one of the three reports a call only with proof that the class comes from category_encoders. The
proof is the module in front of the name, as in ce.OneHotEncoder, or an import of the name from
category_encoders in the same file. A file that imports the same name from both libraries gives
no proof, so the rules about the three classes stay silent there. The other 20 encoder names
belong to category_encoders alone and need no proof.
A rule document whose advice needs a newer release carries since. xpd reads the installed
version from the project's venv and skips the document on an older release. When the venv has no
category_encoders, or there is no venv, every document runs.
The scikit-learn pack already reports a target encoder that is fitted before the split, fitted
before cross-validation, or refitted on the test rows, through
fit-before-split,
cross-val-on-fitted-data and fit-on-test. A file
that uses category_encoders with a split nearly always imports scikit-learn, so the Category
Encoders pack doesn't repeat the three rules. It adds the one form they leave alone, in
encode-before-split.
The proof each rule needs
Section titled “The proof each rule needs”Each error and warning document ships only with a proof that was run, and the message quotes
what the proof measured. A proof is one of the following kinds:
| Proof | What runs | When it passes |
|---|---|---|
| score | The old and the new form, over 30 seeds on data whose target is drawn apart from every feature, where the honest AUC is 0.5. | The old form's mean is more than three standard errors above 0.5, and the new form's mean is within 0.02 of it. |
| result | The old form, with the codes it gives printed next to the codes of the new form. | The old form gives a code that is wrong for every input of that shape, such as -1.0 for every row. |
| interpreter | The old form under python -W error. |
The old form raises an exception or a warning, and the new form runs clean. |
| speed | Both forms on one thread, as the median of 7 runs, at 10,000 and 100,000 rows. | The new form is at least 1.5 times faster at both sizes. |
category_encoders 2.11.2 itself sets a pandas option that pandas 3.1 deprecates, so the interpreter proofs ignored the pandas warning about the option.
A document is info when no proof passed for it, or when its advice depends on something the code
doesn't show, such as the dtype of a column. sklearn-ships-encoder passed its speed proof and is
still info, because the two libraries return different things and the swap is a choice.
nan-into-pipeline and ordinal-mapping-unknown are warning, because their proofs show a raise
and a wrong code for a category that the training rows lacked, and the code doesn't show whether
new data brings one.
The rules were run over two sets of code:
- the library's own code at its latest release, which is the 86 Python files of the repository
and 35 more blocks taken from its docstrings, its
.rstpages and its README - 55 files from public repositories that import
category_encoders
After the runs, the rules below were narrowed, and the table gives the reports that are left:
invalid-handle-optionreported three scikit-learnOneHotEncoder(handle_unknown="ignore")calls in files that import the name from both libraries. A second import now removes the proof.hashing-default-componentsreportedHashingEncoder().hashing_trick(df, N=3), which passes the number of columns itself.zero-regularizationreportedCountTargetEncoder(smoothing=0)insideassertRaises. Every rule about a form that raises now leaves alone code that expects the error.sklearn-ships-encoderreported 6 encoders handed toPolynomialWrapperorNestedCVWrapper, which take only the encoders of category_encoders.
| Rule | Severity | Proof | Reports on the library | Reports on 55 public files | Fix |
|---|---|---|---|---|---|
transform-with-test-target |
error, info | score: 0.691 against 0.517 AUC | 4 | 0 | unsafe |
encode-before-split |
error | score: 0.655 against 0.495 AUC | 0 | 0 | none |
ordinal-mapping-pairs |
error | result: every code is -1.0 | 0 | 0 | unsafe |
ordinal-mapping-unknown |
warning | result: -1.0 for a category the mapping lacks | 3 | 1 | none |
zero-regularization |
error | result: inf. Interpreter: ValueError | 0 | 0 | none |
hashing-default-components |
warning | result: 100 values give 8 distinct rows | 10 | 1 | none |
cols-left-to-dtype |
info | result: 1 column where 5 were meant | 1 | 0 | none |
nan-into-pipeline |
warning | interpreter: ValueError at predict | 0 | 0 | none |
supervised-encoder-without-target |
error | interpreter: TypeError, ValueError | 0 | 0 | none |
invalid-handle-option |
error | interpreter: ValueError, TypeError | 0 | 0 | none |
named-cols-after-array-step |
error | interpreter: ValueError | 0 | 0 | none |
wrapper-fit-returns-none |
error | interpreter: AttributeError | 0 | 0 | none |
removed-in-category-encoders |
error | interpreter: TypeError, AttributeError, KeyError | 0 | 0 | unsafe |
deprecated-in-category-encoders |
warning, info | interpreter: FutureWarning. Result: equal frames | 0 | 0 | safe |
sklearn-ships-encoder |
info | speed: 1.67 times at the lowest | 61 | 16 | none |
encoder-noise-without-seed |
info | result: two runs differ by up to 0.15 | 4 | 0 | none |
A score proof gives the old form's mean against the new form's mean. The speed proof gives the lowest ratio over the three classes and both sizes.
None of the 83 reports on the library names a line where the rule's claim is false. The advice still doesn't apply to most of them, because the library uses or tests its own classes:
- All 61 reports of
sklearn-ships-encoderare the library's tests, examples and documentation building its ownOneHotEncoder,OrdinalEncoderorTargetEncoder. - The 4 reports of
transform-with-test-targetare tests that pass a target totransformon purpose, to check what the encoder does with it. - The 10 reports of
hashing-default-componentsare calls that do run with 8 columns, of which 4 are in tests, 4 in benchmark examples and 2 in the documentation. - The report of
cols-left-to-dtypeis an example that hands aTargetEncoder()two text columns, which it encodes.
The 18 reports on the 55 public files all name a category_encoders class. Recall was not measured. Most rules have no report on the public files, so nothing here says how often the old forms occur.
Leakage
Section titled “Leakage”Each rule in the group reports a target encoder that sees the labels of the test rows, so the score on the test rows is too high. A target encoder codes a category by the labels of its rows, so the code of a test row then holds part of its own label. The score proofs fit a random forest of 50 trees on 600 rows with one text column of 100 levels and a target drawn apart from it, and give the mean test AUC over 30 seeds.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| transform-with-test-target | error | unsafe | Passing the test labels to transform makes a supervised encoder use each test row's own label, which leaks the answer into the features. |
| encode-before-split | error | none | A TargetEncoder fitted on every row before the split puts each test row's own label into its code, which scored 0.655 AUC on data with no signal. |
Codes that are wrong without an error
Section titled “Codes that are wrong without an error”Each rule in the group reports a form that runs and gives another number than the code asks for.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| ordinal-mapping-pairs | error | unsafe | An OrdinalEncoder mapping written as a list of pairs matches no category on category_encoders 2.11, so every row is coded -1.0. |
| ordinal-mapping-unknown | warning | none | With a mapping and no handle_unknown, a category the mapping does not name is coded -1 without a warning. |
| zero-regularization | error | none | A regularization or smoothing of 0 codes a category that holds one class alone as inf, without an error. |
| hashing-default-components | warning | none | HashingEncoder hashes every value into 8 columns by default, shared by all encoded columns, so many values collide. |
| cols-left-to-dtype | info | none | A category_encoders encoder without cols encodes only text and category columns, so a numeric column that a ColumnTransformer hands it passes through unchanged. |
| nan-into-pipeline | warning | unsafe | With "return_nan", the encoder hands NaN to the next step of the pipeline, and many models raise on NaN at predict. |
Code that raises
Section titled “Code that raises”Each rule in the group reports a form that raises on category_encoders 2.11.2. Code inside
assertRaises, pytest.raises or a try block with an except clause expects the error and is
left alone.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| supervised-encoder-without-target | error | none | A supervised encoder learns from the target, so fit(X) without it raises. |
| invalid-handle-option | error | unsafe | The encoder does not take this handle_unknown or handle_missing value, so fit raises ValueError on category_encoders 2.10 and later. |
| named-cols-after-array-step | error | none | An encoder that names its cols follows a step that returns a NumPy array, so the names are gone and fit raises ValueError. |
| wrapper-fit-returns-none | error | none | fit of PolynomialWrapper and NestedCVWrapper returns None, so calling a method on its result raises AttributeError. |
Removed and deprecated forms
Section titled “Removed and deprecated forms”Each rule in the group reports a form that a release of category_encoders removed or deprecated.
The release of each removal was found by reading the constructor of every class in the wheels of
1.3.0, 2.2.2, 2.5.1, 2.6.4, 2.7.0, 2.8.0, 2.8.1, 2.9.0, 2.10.0 and 2.11.2, and each exception was
raised on 2.11.2. Code inside assertRaises, pytest.raises or a try block with an except
clause expects the error or handles both releases, and is left alone.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| removed-in-category-encoders | error | unsafe | These forms were removed from category_encoders, so the call raises on 2.11.2. |
| deprecated-in-category-encoders | warning | safe | These names and arguments warn on category_encoders and will be removed. |
Encoders that scikit-learn ships
Section titled “Encoders that scikit-learn ships”Use scikit-learn's own encoder where it ships the same one, because its OneHotEncoder ran fit_transform 2.7 to 4.5 times faster.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| sklearn-ships-encoder | info | none | scikit-learn ships an encoder of the same name that was 2.7 to 4.5 times faster. |
Reproducible results
Section titled “Reproducible results”Pass random_state to an encoder that adds noise, so two runs give the same codes.
| Rule | Severity | Fix | What it finds |
|---|---|---|---|
| encoder-noise-without-seed | info | none | An encoder that adds random noise draws a new seed on every run, so the codes change from run to run. |
What the pack leaves out
Section titled “What the pack leaves out”The pack has no rule for the following habits, which were looked at:
- Habits that a pattern can't see in the code:
- A target of more than two classes given to
TargetEncoderas whole numbers is coded as the mean of the class numbers, which means nothing.PolynomialWrapperis the answer, but a pattern cannot see how many classes a target has.WOEEncoderraisesValueError: The target column y must be binaryin the same case. Xandywith indexes that do not match raiseValueErrorat fit, and a pattern cannot see an index.GLMMEncoderneeds statsmodels, which category_encoders stopped installing in 2.11, and raisesImportErrorwithout it. A package check runs for every project that has category_encoders and cannot ask whether a file uses that class, so the pack has none.
- A target of more than two classes given to
- Advice that the measurements didn't support:
encoder.fit(X_train, y_train)followed byencoder.transform(X_train)gives the training rows of aLeaveOneOutEncoderthe plain mean of their category. On 600 rows whose 100 levels each carry their own share of positives, with a random forest over 30 seeds, the form scored 0.637 AUC on the test rows against 0.580 with the target passed. The form the documentation asks for was the worse one here, so no rule reports the other.- On the same data
LeaveOneOutEncoder()withoutsigmascored 0.580 against 0.559 withsigma=0.05in a forest, and both scored 0.636 in a logistic regression, so there is no proof that the noise helps. - With the default number of threads, one encoder per column in a loop was 1.15 to 1.25 times
slower than one encoder with
colsforOrdinalEncoderandTargetEncoder, and 3.5 to 5.4 times slower forOneHotEncoder.