The NumPy pack has 63 rules, and xpd can fix 45 of them.
The NumPy rules find code that gives a wrong result or raises on NumPy 2.5, and code that has a
faster or more exact form. They run on files that import numpy, and examples assume
import numpy as np. Each claim in a message was measured on NumPy 2.5.5, and the table says what each claim means.
| Claim |
What it means on NumPy 2.5.5 |
| a form is removed |
it raises |
| a form is wrong |
it gives the wrong value |
| a form is faster |
the median of 7 runs on one core, at 1,000,000 values unless the rule says otherwise |
A fix that writes np. applies only in a file with import numpy as np, and a fix that writes
assert_allclose only in a file with from numpy.testing import assert_allclose. A safe fix returns a
value of the same dtype that is equal bit for bit, and the Fix: line of an unsafe fix names
what can differ.
xpd skips a rule, or one form of a rule, that asks for an API newer than the NumPy in the
project's virtual environment (venv). When the venv has no NumPy, or there is no venv, every rule runs.
Fix every report in the group, because the reported code gives a wrong value without an error,
or raises on every input. Every fix in the group is unsafe, because it changes the result to the
one the code meant, except the fix of positional-tolerance, which only names the argument.
| Rule |
Severity |
Fix |
What it finds |
| compare-to-nan |
error |
unsafe |
A comparison with NaN is False for every value, NaN included. |
| argument-in-wrong-slot |
error |
unsafe |
These calls put an argument where NumPy expects another one, such as a second size where np.zeros expects the dtype, so they raise or give a wrong result. |
| percentile-of-fraction |
warning |
unsafe |
np.percentile takes percents from 0 to 100, so 0.95 asks for a value near the minimum. |
| positional-tolerance |
warning |
safe |
The third argument of np.isclose and np.allclose is rtol, a relative tolerance, and many readers expect an absolute one. |
| arange-float-step |
warning |
none |
With a float step such as 0.1, the length of np.arange depends on rounding, and the stop value can appear in the result. |
| randint-single-value |
error |
unsafe |
The upper bound of randint is excluded, so randint(0, 1) always returns 0. |
| result-is-none |
error |
unsafe |
Methods such as sort() and shuffle() change the array in place and return None, so x = x.sort() sets x to None. |
| result-discarded |
error |
unsafe |
Functions such as np.sort and np.append return a new array and leave the input alone, so a call on its own line does nothing. |
| write-to-copy |
error |
unsafe |
Indexing with a mask or a list of positions gives a copy, so writing into that result never reaches the array. |
| array-truth-value |
error |
unsafe |
and, or, not and chained comparisons ask for the truth of a whole array, which raises ValueError. |
| mask-operator-precedence |
error |
unsafe |
Python applies & and | before comparisons, so x > 2 & x < 8 compares against the wrong value. |
| int-mask-as-index |
error |
unsafe |
mask.astype(int) turns a mask into 0s and 1s, so indexing with it picks element 0 and element 1 over and over. |
| iterator-to-array |
error |
unsafe |
np.array and NumPy reductions do not read values out of a generator, map or a set, so the call gives a wrong value or raises. |
| nan-to-int |
warning |
none |
Casting an array that holds NaN to an int type turns each NaN into -9223372036854775808. |
| read-uninitialized |
error |
unsafe |
np.empty leaves the values unset, so computing with them uses whatever bytes were in memory. |
| array-default-argument |
warning |
none |
A NumPy array as a default argument is one array that every call shares, so a write to it shows up in later calls. |
| append-flattens |
error |
none |
Without axis, np.append and np.insert flatten both arrays, so a row you add becomes loose values in one dimension. |
| compare-reductions |
error |
unsafe |
a.all() == b.all() compares two single booleans, so it is True for many arrays that differ. |
| where-tuple-length |
error |
unsafe |
np.where with one argument returns a tuple with one array per dimension, so its length counts dimensions and not matches. |
Replace each name, method, module or keyword that NumPy removed or deprecated with the form that
the rule's message names, because the old form raises or warns on NumPy 2.5.
| Rule |
Severity |
Fix |
What it finds |
| removed-in-numpy-2 |
error |
unsafe |
These names, modules and keywords were removed between NumPy 1.24 and 2.5, so the code raises. |
| deprecated-in-numpy-2 |
warning |
unsafe |
These names and modules warn on NumPy 2.5 and will stop working in a later release. |
Write one expression on whole arrays in place of code that calls Python once per value or copies
the whole array once per pass, because the whole array form ran up to 530 times faster. The
message of each rule names the whole array form. The table shows the speed that the message
reports, as medians of 7 runs on one core with NumPy 2.5.5, on float64 and int64.
| Rule |
Write instead |
Faster by |
element-loop |
one expression on whole arrays |
70 to 530 times on 1,000,000 values |
growth-in-loop |
a list, and one np.concatenate after the loop |
130 to 170 times for 1,000 passes over 1,000,000 values |
per-value-comprehension |
the expression on np.asarray(...) |
60 to 470 times on an array of 1,000,000 values, and 2 to 24 times on a list |
builtin-on-array |
the method, such as .sum(axis=0) |
37 to 47 times on 1,000,000 values |
np-vectorize |
code on whole arrays, such as np.where or np.select |
not measured |
| Rule |
Severity |
Fix |
What it finds |
| element-loop |
warning |
none |
A loop that computes one element of an array per pass runs Python once for every value, which was 70 to 530 times slower. |
| growth-in-loop |
warning |
none |
Growing an array with np.append or np.concatenate in a loop copies every earlier value on each pass. |
| per-value-comprehension |
warning |
unsafe |
A comprehension that applies arithmetic or a math function to each value runs Python once for every value, which was 60 to 470 times slower. |
| builtin-on-array |
warning |
unsafe |
Python's sum, min, max, sorted and list walk an array one value at a time, which was 37 to 47 times slower. |
| np-vectorize |
warning |
none |
np.vectorize, np.frompyfunc and np.apply_along_axis still call a Python function once per value or slice. |
| recurrence-loop |
warning |
none |
A loop that computes each value from the one before runs Python once for every step. |
| slice-fill-loop |
warning |
none |
A loop that fills an array from one start after another writes the same positions many times. |
| mask-per-label |
warning |
none |
Comparing an array with one label per pass makes one pass for every label. |
| random-draw-loop |
warning |
none |
Drawing from a Generator once per pass of a loop makes one call for every pass. |
Build, copy and store each array with the NumPy function that the rule names, because it ran 1.6
to 880 times faster on 1,000,000 values. The table shows the speed that the message of each rule
reports.
| Rule |
Write instead |
Faster by |
array-construction |
np.arange, np.zeros or np.full |
150 times for np.arange |
needless-copy |
.ravel() |
1.6 to 1.8 times |
file-roundtrip |
np.save and np.load |
27 to 880 times to read, and 420 to 510 times to write |
| Rule |
Severity |
Fix |
What it finds |
| array-construction |
warning |
unsafe |
Building an array from range or a Python list makes a Python int for every value first, and np.ones(n) * 3 builds the values twice. |
| needless-copy |
warning |
unsafe |
flatten() and copy() on an array that is already a new copy copy it again. |
| file-roundtrip |
warning |
none |
Writing an array as text with np.savetxt and parsing it back is slow and loses the dtype and the shape. |
Call the NumPy function that each rule names, because it does the same work without a full sort,
a temporary array or a step that loses digits. Every rule in the group is a warning. The table
shows the speed that the message of each rule reports, measured with NumPy 2.5.5 on one core,
with BLAS and LAPACK, the linear algebra libraries that NumPy calls, on one thread. The other rules in the group give a more exact result and have no
measured speed.
| Rule |
Reported form |
Size |
sort-then-select |
39 times slower than np.min |
1,000,000 values |
count-nonzero |
1.7 times slower than np.count_nonzero |
1,000,000 values |
where-shortcut |
17 times slower than np.asarray |
1,000,000 values |
reduce-last |
8 times slower than np.sum |
1,000,000 values |
power-half |
2 times slower than np.sqrt |
1,000,000 values |
sum-of-products |
2.4 times slower than np.vdot |
1,000,000 values |
norm-by-hand |
3.9 times slower than np.linalg.norm |
1,000,000 values |
vecdot |
2 to 3.5 times slower than np.vecdot |
1,000,000 values |
inv-then-multiply |
3.8 times slower than np.linalg.solve for a vector |
a 1,000 by 1,000 matrix |
eig-on-symmetric |
3.6 times slower than np.linalg.eigh |
a 1,000 by 1,000 matrix |
matmul-diagonal |
34 times slower than np.einsum |
1,000 by 1,000 matrices |
einsum-optimize |
339 times slower than optimize=True |
three 100 by 100 matrices |
| Rule |
Severity |
Fix |
What it finds |
| sort-then-select |
warning |
unsafe |
Sorting every value to read the smallest or largest few was up to 39 times slower. |
| count-nonzero |
warning |
unsafe |
Summing a comparison or taking the length of np.where counts matches the slow way. |
| where-shortcut |
warning |
unsafe |
np.where(x > 0, True, False) copies a mask that is already an array of booleans, which was 17 times slower. |
| expm1-log1p |
warning |
unsafe |
np.exp(x) - 1 and np.log(1 + x) lose digits to rounding when x is near 0. |
| logaddexp |
warning |
unsafe |
np.exp overflows to inf above 709 and underflows to 0 below -745, so a log of a sum of exponentials can be inf where the answer is finite. |
| log-of-product |
warning |
unsafe |
A product can overflow to inf or underflow to 0 before the log is taken. |
| reduce-last |
warning |
unsafe |
np.cumsum(x)[-1] builds every running total to read the last one, which was 8 times slower and adds rounding. |
| power-half |
warning |
unsafe |
np.power(x, 0.5) runs the general power function, which was 2 times slower. |
| sum-of-products |
warning |
unsafe |
np.sum(a * b) builds the array of products to add it up, which was 2.4 times slower. |
| norm-by-hand |
warning |
unsafe |
Building the array of squares to add them up was 3.9 times slower. |
| vecdot |
warning |
unsafe |
Summing squares or products along the last axis builds a temporary array, which was 2.8 times slower. |
| inv-then-multiply |
warning |
unsafe |
Multiplying by an inverse loses more digits than solving, and was 3.8 times slower. |
| log-of-det |
warning |
unsafe |
The determinant can overflow to inf or underflow to 0 before the log is taken. |
| eig-on-symmetric |
warning |
unsafe |
eig treats a symmetric matrix as a general one, which was 3.6 times slower and returns complex values. |
| matmul-diagonal |
warning |
unsafe |
Computing a whole matrix product to read its diagonal or trace was 34 times slower than needed. |
| einsum-optimize |
warning |
unsafe |
Without optimize=, einsum loops over every index at once, which was 339 times slower for three 100 by 100 matrices. |
Write the form that NumPy's docs prefer in place of each old form that the rules report, although
the old form still runs without a warning on NumPy 2.5.5. A rule reports an old form when NumPy's
docs call it legacy, not recommended or kept for backwards compatibility, or when the newer form
ran at least 1.5 times faster. A rule that states only a preference of the docs is info, and the
rest are warnings, as the table shows. xpd skips the rules that name an API added in NumPy 2.0 on
an older NumPy.
| Rule |
Severity |
Fix |
What it finds |
| legacy-random |
warning |
none |
np.random.seed and the functions beside it use a hidden global state that NumPy keeps only for old code. |
| rng-in-loop |
warning |
none |
np.random.default_rng() inside a loop builds a new Generator on every pass. |
| clock-seed |
warning |
unsafe |
A seed taken from the clock repeats whenever two runs start in the same tick. |
| legacy-polynomial |
info |
unsafe |
np.polyfit and its kin are NumPy's legacy polynomial API, and the docs recommend Polynomial.fit as more stable. |
| almost-equal-assert |
info |
unsafe |
assert_almost_equal and its kin are on NumPy's not recommended list, and their default test hides an absolute tolerance of 1.5e-7. |
| assert-on-arrays |
info |
unsafe |
A failed assert np.allclose(a, b) reports only False. |
| legacy-testing-helper |
info |
unsafe |
numpy.testing.assert_ is the legacy form of the assert statement. |
| char-module |
info |
unsafe |
numpy.char is legacy, and NumPy's docs point to numpy.strings. |
| dot-on-matrices |
info |
unsafe |
For two 2-D arrays, NumPy's docs prefer the @ operator to np.dot. |
| mergesort-kind |
info |
unsafe |
kind="mergesort" is kept only for old code and runs the same stable sort as kind="stable". |
| printoptions-restore |
info |
none |
Restoring print options by hand leaves them changed when an error is raised. |
| as-strided |
info |
none |
as_strided reads memory outside the array when a shape or a stride is wrong. |
| numpy-import-alias |
info |
none |
NumPy's docs import it as np, and other code and the fixes of this pack expect that name. |
| unpackbits-popcount |
warning |
none |
Unpacking every byte into eight bits to count the set ones was 37 times slower. |