Skip to content

NumPy

The NumPy pack has 63 rules, and xpd can fix 45 of them.

The NumPy rules find code that gives a wrong result or raises on NumPy 2.5, and code that has a faster or more exact form. They run on files that import numpy, and examples assume import numpy as np. Each claim in a message was measured on NumPy 2.5.5, and the table says what each claim means.

Claim What it means on NumPy 2.5.5
a form is removed it raises
a form is wrong it gives the wrong value
a form is faster the median of 7 runs on one core, at 1,000,000 values unless the rule says otherwise

A fix that writes np. applies only in a file with import numpy as np, and a fix that writes assert_allclose only in a file with from numpy.testing import assert_allclose. A safe fix returns a value of the same dtype that is equal bit for bit, and the Fix: line of an unsafe fix names what can differ.

xpd skips a rule, or one form of a rule, that asks for an API newer than the NumPy in the project's virtual environment (venv). When the venv has no NumPy, or there is no venv, every rule runs.

Rule Runs on
char-module, log-of-det, unpackbits-popcount and vecdot NumPy 2.0 and later
some forms of removed-in-numpy-2 NumPy 1.22, 1.25 or 2.0 and later, by form
some forms of deprecated-in-numpy-2 NumPy 2.0 and later
every other NumPy rule every NumPy version

Code that gives a wrong result or raises on NumPy 2.5

Section titled “Code that gives a wrong result or raises on NumPy 2.5”

Fix every report in the group, because the reported code gives a wrong value without an error, or raises on every input. Every fix in the group is unsafe, because it changes the result to the one the code meant, except the fix of positional-tolerance, which only names the argument.

Rule Severity Fix What it finds
compare-to-nan error unsafe A comparison with NaN is False for every value, NaN included.
argument-in-wrong-slot error unsafe These calls put an argument where NumPy expects another one, such as a second size where np.zeros expects the dtype, so they raise or give a wrong result.
percentile-of-fraction warning unsafe np.percentile takes percents from 0 to 100, so 0.95 asks for a value near the minimum.
positional-tolerance warning safe The third argument of np.isclose and np.allclose is rtol, a relative tolerance, and many readers expect an absolute one.
arange-float-step warning none With a float step such as 0.1, the length of np.arange depends on rounding, and the stop value can appear in the result.
randint-single-value error unsafe The upper bound of randint is excluded, so randint(0, 1) always returns 0.
result-is-none error unsafe Methods such as sort() and shuffle() change the array in place and return None, so x = x.sort() sets x to None.
result-discarded error unsafe Functions such as np.sort and np.append return a new array and leave the input alone, so a call on its own line does nothing.
write-to-copy error unsafe Indexing with a mask or a list of positions gives a copy, so writing into that result never reaches the array.
array-truth-value error unsafe and, or, not and chained comparisons ask for the truth of a whole array, which raises ValueError.
mask-operator-precedence error unsafe Python applies & and | before comparisons, so x > 2 & x < 8 compares against the wrong value.
int-mask-as-index error unsafe mask.astype(int) turns a mask into 0s and 1s, so indexing with it picks element 0 and element 1 over and over.
iterator-to-array error unsafe np.array and NumPy reductions do not read values out of a generator, map or a set, so the call gives a wrong value or raises.
nan-to-int warning none Casting an array that holds NaN to an int type turns each NaN into -9223372036854775808.
read-uninitialized error unsafe np.empty leaves the values unset, so computing with them uses whatever bytes were in memory.
array-default-argument warning none A NumPy array as a default argument is one array that every call shares, so a write to it shows up in later calls.
append-flattens error none Without axis, np.append and np.insert flatten both arrays, so a row you add becomes loose values in one dimension.
compare-reductions error unsafe a.all() == b.all() compares two single booleans, so it is True for many arrays that differ.
where-tuple-length error unsafe np.where with one argument returns a tuple with one array per dimension, so its length counts dimensions and not matches.

Replace each name, method, module or keyword that NumPy removed or deprecated with the form that the rule's message names, because the old form raises or warns on NumPy 2.5.

Rule Severity Fix What it finds
removed-in-numpy-2 error unsafe These names, modules and keywords were removed between NumPy 1.24 and 2.5, so the code raises.
deprecated-in-numpy-2 warning unsafe These names and modules warn on NumPy 2.5 and will stop working in a later release.

Write one expression on whole arrays in place of code that calls Python once per value or copies the whole array once per pass, because the whole array form ran up to 530 times faster. The message of each rule names the whole array form. The table shows the speed that the message reports, as medians of 7 runs on one core with NumPy 2.5.5, on float64 and int64.

Rule Write instead Faster by
element-loop one expression on whole arrays 70 to 530 times on 1,000,000 values
growth-in-loop a list, and one np.concatenate after the loop 130 to 170 times for 1,000 passes over 1,000,000 values
per-value-comprehension the expression on np.asarray(...) 60 to 470 times on an array of 1,000,000 values, and 2 to 24 times on a list
builtin-on-array the method, such as .sum(axis=0) 37 to 47 times on 1,000,000 values
np-vectorize code on whole arrays, such as np.where or np.select not measured
Rule Severity Fix What it finds
element-loop warning none A loop that computes one element of an array per pass runs Python once for every value, which was 70 to 530 times slower.
growth-in-loop warning none Growing an array with np.append or np.concatenate in a loop copies every earlier value on each pass.
per-value-comprehension warning unsafe A comprehension that applies arithmetic or a math function to each value runs Python once for every value, which was 60 to 470 times slower.
builtin-on-array warning unsafe Python's sum, min, max, sorted and list walk an array one value at a time, which was 37 to 47 times slower.
np-vectorize warning none np.vectorize, np.frompyfunc and np.apply_along_axis still call a Python function once per value or slice.
recurrence-loop warning none A loop that computes each value from the one before runs Python once for every step.
slice-fill-loop warning none A loop that fills an array from one start after another writes the same positions many times.
mask-per-label warning none Comparing an array with one label per pass makes one pass for every label.
random-draw-loop warning none Drawing from a Generator once per pass of a loop makes one call for every pass.

Arrays built, copied and stored the slow way

Section titled “Arrays built, copied and stored the slow way”

Build, copy and store each array with the NumPy function that the rule names, because it ran 1.6 to 880 times faster on 1,000,000 values. The table shows the speed that the message of each rule reports.

Rule Write instead Faster by
array-construction np.arange, np.zeros or np.full 150 times for np.arange
needless-copy .ravel() 1.6 to 1.8 times
file-roundtrip np.save and np.load 27 to 880 times to read, and 420 to 510 times to write
Rule Severity Fix What it finds
array-construction warning unsafe Building an array from range or a Python list makes a Python int for every value first, and np.ones(n) * 3 builds the values twice.
needless-copy warning unsafe flatten() and copy() on an array that is already a new copy copy it again.
file-roundtrip warning none Writing an array as text with np.savetxt and parsing it back is slow and loses the dtype and the shape.

Math on whole arrays with a faster or more exact function

Section titled “Math on whole arrays with a faster or more exact function”

Call the NumPy function that each rule names, because it does the same work without a full sort, a temporary array or a step that loses digits. Every rule in the group is a warning. The table shows the speed that the message of each rule reports, measured with NumPy 2.5.5 on one core, with BLAS and LAPACK, the linear algebra libraries that NumPy calls, on one thread. The other rules in the group give a more exact result and have no measured speed.

Rule Reported form Size
sort-then-select 39 times slower than np.min 1,000,000 values
count-nonzero 1.7 times slower than np.count_nonzero 1,000,000 values
where-shortcut 17 times slower than np.asarray 1,000,000 values
reduce-last 8 times slower than np.sum 1,000,000 values
power-half 2 times slower than np.sqrt 1,000,000 values
sum-of-products 2.4 times slower than np.vdot 1,000,000 values
norm-by-hand 3.9 times slower than np.linalg.norm 1,000,000 values
vecdot 2 to 3.5 times slower than np.vecdot 1,000,000 values
inv-then-multiply 3.8 times slower than np.linalg.solve for a vector a 1,000 by 1,000 matrix
eig-on-symmetric 3.6 times slower than np.linalg.eigh a 1,000 by 1,000 matrix
matmul-diagonal 34 times slower than np.einsum 1,000 by 1,000 matrices
einsum-optimize 339 times slower than optimize=True three 100 by 100 matrices
Rule Severity Fix What it finds
sort-then-select warning unsafe Sorting every value to read the smallest or largest few was up to 39 times slower.
count-nonzero warning unsafe Summing a comparison or taking the length of np.where counts matches the slow way.
where-shortcut warning unsafe np.where(x > 0, True, False) copies a mask that is already an array of booleans, which was 17 times slower.
expm1-log1p warning unsafe np.exp(x) - 1 and np.log(1 + x) lose digits to rounding when x is near 0.
logaddexp warning unsafe np.exp overflows to inf above 709 and underflows to 0 below -745, so a log of a sum of exponentials can be inf where the answer is finite.
log-of-product warning unsafe A product can overflow to inf or underflow to 0 before the log is taken.
reduce-last warning unsafe np.cumsum(x)[-1] builds every running total to read the last one, which was 8 times slower and adds rounding.
power-half warning unsafe np.power(x, 0.5) runs the general power function, which was 2 times slower.
sum-of-products warning unsafe np.sum(a * b) builds the array of products to add it up, which was 2.4 times slower.
norm-by-hand warning unsafe Building the array of squares to add them up was 3.9 times slower.
vecdot warning unsafe Summing squares or products along the last axis builds a temporary array, which was 2.8 times slower.
inv-then-multiply warning unsafe Multiplying by an inverse loses more digits than solving, and was 3.8 times slower.
log-of-det warning unsafe The determinant can overflow to inf or underflow to 0 before the log is taken.
eig-on-symmetric warning unsafe eig treats a symmetric matrix as a general one, which was 3.6 times slower and returns complex values.
matmul-diagonal warning unsafe Computing a whole matrix product to read its diagonal or trace was 34 times slower than needed.
einsum-optimize warning unsafe Without optimize=, einsum loops over every index at once, which was 339 times slower for three 100 by 100 matrices.

The API that NumPy's docs prefer on NumPy 2.5

Section titled “The API that NumPy's docs prefer on NumPy 2.5”

Write the form that NumPy's docs prefer in place of each old form that the rules report, although the old form still runs without a warning on NumPy 2.5.5. A rule reports an old form when NumPy's docs call it legacy, not recommended or kept for backwards compatibility, or when the newer form ran at least 1.5 times faster. A rule that states only a preference of the docs is info, and the rest are warnings, as the table shows. xpd skips the rules that name an API added in NumPy 2.0 on an older NumPy.

Severity Rules
warning legacy-random, rng-in-loop, clock-seed and unpackbits-popcount
info the other 10 rules of the group
Rule Severity Fix What it finds
legacy-random warning none np.random.seed and the functions beside it use a hidden global state that NumPy keeps only for old code.
rng-in-loop warning none np.random.default_rng() inside a loop builds a new Generator on every pass.
clock-seed warning unsafe A seed taken from the clock repeats whenever two runs start in the same tick.
legacy-polynomial info unsafe np.polyfit and its kin are NumPy's legacy polynomial API, and the docs recommend Polynomial.fit as more stable.
almost-equal-assert info unsafe assert_almost_equal and its kin are on NumPy's not recommended list, and their default test hides an absolute tolerance of 1.5e-7.
assert-on-arrays info unsafe A failed assert np.allclose(a, b) reports only False.
legacy-testing-helper info unsafe numpy.testing.assert_ is the legacy form of the assert statement.
char-module info unsafe numpy.char is legacy, and NumPy's docs point to numpy.strings.
dot-on-matrices info unsafe For two 2-D arrays, NumPy's docs prefer the @ operator to np.dot.
mergesort-kind info unsafe kind="mergesort" is kept only for old code and runs the same stable sort as kind="stable".
printoptions-restore info none Restoring print options by hand leaves them changed when an error is raised.
as-strided info none as_strided reads memory outside the array when a shape or a stride is wrong.
numpy-import-alias info none NumPy's docs import it as np, and other code and the fixes of this pack expect that name.
unpackbits-popcount warning none Unpacking every byte into eight bits to count the set ones was 37 times slower.