NumPy Support#

The magnitude of a Pint quantity can be of any numerical scalar type, and you are free to choose it according to your needs. For numerical applications requiring arrays, it is quite convenient to use NumPy ndarray (or ndarray-like types supporting NEP-18), and therefore these are the array types supported by Pint.

Pint follows Numpy’s recommendation (NEP29) for minimal Numpy/Python versions support across the Scientific Python ecosystem. This ensures compatibility with other third party libraries (matplotlib, pandas, scipy).

First, we import the relevant packages:

[1]:

# Import NumPy
import numpy as np

# Import Pint
import pint

ureg = pint.UnitRegistry()
Q_ = ureg.Quantity

# Silence NEP 18 warning
import warnings

with warnings.catch_warnings():
    warnings.simplefilter("ignore")
    Q_([])

and then we create a quantity the standard way

[2]:

legs1 = Q_(np.asarray([3.0, 4.0]), "meter")
print(legs1)

[3.0 4.0] meter

[3]:

legs1 = [3.0, 4.0] * ureg.meter
print(legs1)

[3.0 4.0] meter

All usual Pint methods can be used with this quantity. For example:

[4]:

print(legs1.to("kilometer"))

[0.003 0.004] kilometer

[5]:

print(legs1.dimensionality)

[length]

[6]:

try:
    legs1.to("joule")
except pint.DimensionalityError as exc:
    print(exc)

Cannot convert from 'meter' ([length]) to 'joule' ([length] ** 2 * [mass] / [time] ** 2)

NumPy functions are supported by Pint. For example if we define:

[7]:

legs2 = [400.0, 300.0] * ureg.centimeter
print(legs2)

[400.0 300.0] centimeter

we can calculate the hypotenuse of the right triangles with legs1 and legs2.

[8]:

hyps = np.hypot(legs1, legs2)
print(hyps)

[5.0 5.0] meter

Notice that before the np.hypot was used, the numerical value of legs2 was internally converted to the units of legs1 as expected.

Similarly, when you apply a function that expects angles in radians, a conversion is applied before the requested calculation:

[9]:

angles = np.arccos(legs2 / hyps)
print(angles)

[0.6435011087932843 0.9272952180016123] radian

You can convert the result to degrees using usual unit conversion:

[10]:

print(angles.to("degree"))

[36.86989764584401 53.13010235415599] degree

Applying a function that expects angles to a quantity with a different dimensionality results in an error:

[11]:

try:
    np.arccos(legs2)
except pint.DimensionalityError as exc:
    print(exc)

Cannot convert from 'centimeter' ([length]) to 'dimensionless' (dimensionless)

Function/Method Support#

The following ufuncs can be applied to a Quantity object:

  • Math operations: add, subtract, multiply, divide, logaddexp, logaddexp2, true_divide, floor_divide, negative, remainder, mod, fmod, absolute, rint, sign, conj, exp, exp2, log, log2, log10, expm1, log1p, sqrt, square, cbrt, reciprocal

  • Trigonometric functions: sin, cos, tan, arcsin, arccos, arctan, arctan2, hypot, sinh, cosh, tanh, arcsinh, arccosh, arctanh

  • Comparison functions: greater, greater_equal, less, less_equal, not_equal, equal

  • Floating functions: isreal, iscomplex, isfinite, isinf, isnan, signbit, sign, copysign, nextafter, modf, ldexp, frexp, fmod, floor, ceil, trunc

And the following NumPy functions:

[12]:

from pint.facets.numpy.numpy_func import HANDLED_FUNCTIONS

print(sorted(list(HANDLED_FUNCTIONS)))

['all', 'allclose', 'amax', 'amin', 'any', 'append', 'argmax', 'argmin', 'argsort', 'around', 'atleast_1d', 'atleast_2d', 'atleast_3d', 'average', 'block', 'broadcast_arrays', 'broadcast_to', 'clip', 'column_stack', 'compress', 'concatenate', 'copy', 'copyto', 'count_nonzero', 'cross', 'cumprod', 'cumproduct', 'cumsum', 'delete', 'diagonal', 'diff', 'dot', 'dstack', 'ediff1d', 'einsum', 'empty_like', 'expand_dims', 'fix', 'flip', 'full_like', 'gradient', 'hstack', 'insert', 'interp', 'intersect1d', 'isclose', 'iscomplex', 'isin', 'isreal', 'lib.stride_tricks.sliding_window_view', 'linalg.solve', 'linspace', 'max', 'mean', 'median', 'meshgrid', 'min', 'moveaxis', 'nan_to_num', 'nanargmax', 'nanargmin', 'nancumprod', 'nancumsum', 'nanmax', 'nanmean', 'nanmedian', 'nanmin', 'nanpercentile', 'nanprod', 'nanquantile', 'nanstd', 'nansum', 'nanvar', 'ndim', 'nonzero', 'ones_like', 'pad', 'percentile', 'prod', 'ptp', 'quantile', 'ravel', 'reshape', 'resize', 'result_type', 'rollaxis', 'rot90', 'round', 'round_', 'searchsorted', 'shape', 'size', 'sort', 'squeeze', 'stack', 'std', 'sum', 'swapaxes', 'tile', 'transpose', 'trapz', 'trim_zeros', 'unwrap', 'var', 'vstack', 'where', 'zeros_like']

And the following NumPy ndarray methods:

  • argmax, argmin, argsort, astype, clip, compress, conj, conjugate, cumprod, cumsum, diagonal, dot, fill, flatten, flatten, item, max, mean, min, nonzero, prod, ptp, put, ravel, repeat, reshape, round, searchsorted, sort, squeeze, std, sum, take, trace, transpose, var

Pull requests are welcome for any NumPy function, ufunc, or method that is not currently supported.

Array Type Support#

Overview#

When not wrapping a scalar type, a Pint Quantity can be considered a “duck array”, that is, an array-like type that implements (all or most of) NumPy’s API for ndarray. Many other such duck arrays exist in the Python ecosystem, and Pint aims to work with as many of them as reasonably possible. To date, the following are specifically tested and known to work:

  • xarray: DataArray, Dataset, and Variable

  • Sparse: COO

and the following have partial support, with full integration planned:

  • NumPy masked arrays (NOTE: Masked Array compatibility has changed with Pint 0.10 and versions of NumPy up to at least 1.18, see the example below)

  • Dask arrays

  • CuPy arrays

Technical Commentary#

Starting with version 0.10, Pint aims to interoperate with other duck arrays in a well-defined and well-supported fashion. Part of this support lies in implementing `__array_ufunc__ to support NumPy ufuncs <https://numpy.org/neps/nep-0013-ufunc-overrides.html>`__ and `__array_function__ to support NumPy functions <https://numpy.org/neps/nep-0018-array-function-protocol.html>`__. However, the central component to this interoperability is respecting a type casting hierarchy of duck arrays. When all types in the hierarchy properly defer to those above it (in wrapping, arithmetic, and NumPy operations), a well-defined nesting and operator precedence order exists. When they don’t, the graph of relations becomes cyclic, and the expected result of mixed-type operations becomes ambiguous.

For Pint, following this hierarchy means declaring a list of types that are above it in the hierarchy and to which it defers (“upcast types”) and assuming all others are below it and wrappable by it (“downcast types”). To date, Pint’s declared upcast types are:

  • PintArray, as defined by pint-pandas

  • Series, as defined by Pandas

  • DataArray, Dataset, and Variable, as defined by xarray

(Note: if your application requires extension of this collection of types, it is available in Pint’s API at pint.compat.upcast_types.)

While Pint assumes it can wrap any other duck array (meaning, for now, those that implement __array_function__, shape, ndim, and dtype, at least until NEP 30 is implemented), there are a few common types that Pint explicitly tests (or plans to test) for optimal interoperability. These are listed above in the overview section and included in the below chart.

This type casting hierarchy of ndarray-like types can be shown by the below acyclic graph, where solid lines represent declared support, and dashed lines represent planned support:

[13]:

from graphviz import Digraph

g = Digraph(graph_attr={"size": "8,5"}, node_attr={"fontname": "courier"})
g.edge("Dask array", "NumPy ndarray")
g.edge("Dask array", "CuPy ndarray")
g.edge("Dask array", "Sparse COO")
g.edge("Dask array", "NumPy masked array", style="dashed")
g.edge("CuPy ndarray", "NumPy ndarray")
g.edge("Sparse COO", "NumPy ndarray")
g.edge("NumPy masked array", "NumPy ndarray")
g.edge("Jax array", "NumPy ndarray")
g.edge("Pint Quantity", "Dask array", style="dashed")
g.edge("Pint Quantity", "NumPy ndarray")
g.edge("Pint Quantity", "CuPy ndarray", style="dashed")
g.edge("Pint Quantity", "Sparse COO")
g.edge("Pint Quantity", "NumPy masked array", style="dashed")
g.edge("xarray Dataset/DataArray/Variable", "Dask array")
g.edge("xarray Dataset/DataArray/Variable", "CuPy ndarray", style="dashed")
g.edge("xarray Dataset/DataArray/Variable", "Sparse COO")
g.edge("xarray Dataset/DataArray/Variable", "NumPy ndarray")
g.edge("xarray Dataset/DataArray/Variable", "NumPy masked array", style="dashed")
g.edge("xarray Dataset/DataArray/Variable", "Pint Quantity")
g.edge("xarray Dataset/DataArray/Variable", "Jax array", style="dashed")
g

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/backend/execute.py:79, in run_check(cmd, input_lines, encoding, quiet, **kwargs)
     78         kwargs['stdout'] = kwargs['stderr'] = subprocess.PIPE
---> 79     proc = _run_input_lines(cmd, input_lines, kwargs=kwargs)
     80 else:

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/backend/execute.py:99, in _run_input_lines(cmd, input_lines, kwargs)
     98 def _run_input_lines(cmd, input_lines, *, kwargs):
---> 99     popen = subprocess.Popen(cmd, stdin=subprocess.PIPE, **kwargs)
    101     stdin_write = popen.stdin.write

File ~/.asdf/installs/python/3.9.15/lib/python3.9/subprocess.py:951, in Popen.__init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds, user, group, extra_groups, encoding, errors, text, umask)
    948             self.stderr = io.TextIOWrapper(self.stderr,
    949                     encoding=encoding, errors=errors)
--> 951     self._execute_child(args, executable, preexec_fn, close_fds,
    952                         pass_fds, cwd, env,
    953                         startupinfo, creationflags, shell,
    954                         p2cread, p2cwrite,
    955                         c2pread, c2pwrite,
    956                         errread, errwrite,
    957                         restore_signals,
    958                         gid, gids, uid, umask,
    959                         start_new_session)
    960 except:
    961     # Cleanup if the child failed starting.

File ~/.asdf/installs/python/3.9.15/lib/python3.9/subprocess.py:1821, in Popen._execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, gid, gids, uid, umask, start_new_session)
   1820         err_msg = os.strerror(errno_num)
-> 1821     raise child_exception_type(errno_num, err_msg, err_filename)
   1822 raise child_exception_type(err_msg)

FileNotFoundError: [Errno 2] No such file or directory: PosixPath('dot')

The above exception was the direct cause of the following exception:

ExecutableNotFound                        Traceback (most recent call last)
File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/IPython/core/formatters.py:974, in MimeBundleFormatter.__call__(self, obj, include, exclude)
    971     method = get_real_method(obj, self.print_method)
    973     if method is not None:
--> 974         return method(include=include, exclude=exclude)
    975     return None
    976 else:

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/jupyter_integration.py:98, in JupyterIntegration._repr_mimebundle_(self, include, exclude, **_)
     96 include = set(include) if include is not None else {self._jupyter_mimetype}
     97 include -= set(exclude or [])
---> 98 return {mimetype: getattr(self, method_name)()
     99         for mimetype, method_name in MIME_TYPES.items()
    100         if mimetype in include}

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/jupyter_integration.py:98, in <dictcomp>(.0)
     96 include = set(include) if include is not None else {self._jupyter_mimetype}
     97 include -= set(exclude or [])
---> 98 return {mimetype: getattr(self, method_name)()
     99         for mimetype, method_name in MIME_TYPES.items()
    100         if mimetype in include}

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/jupyter_integration.py:112, in JupyterIntegration._repr_image_svg_xml(self)
    110 def _repr_image_svg_xml(self) -> str:
    111     """Return the rendered graph as SVG string."""
--> 112     return self.pipe(format='svg', encoding=SVG_ENCODING)

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/piping.py:104, in Pipe.pipe(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
     55 def pipe(self,
     56          format: typing.Optional[str] = None,
     57          renderer: typing.Optional[str] = None,
   (...)
     61          engine: typing.Optional[str] = None,
     62          encoding: typing.Optional[str] = None) -> typing.Union[bytes, str]:
     63     """Return the source piped through the Graphviz layout command.
     64
     65     Args:
   (...)
    102         '<?xml version='
    103     """
--> 104     return self._pipe_legacy(format,
    105                              renderer=renderer,
    106                              formatter=formatter,
    107                              neato_no_op=neato_no_op,
    108                              quiet=quiet,
    109                              engine=engine,
    110                              encoding=encoding)

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/_tools.py:171, in deprecate_positional_args.<locals>.decorator.<locals>.wrapper(*args, **kwargs)
    162     wanted = ', '.join(f'{name}={value!r}'
    163                        for name, value in deprecated.items())
    164     warnings.warn(f'The signature of {func.__name__} will be reduced'
    165                   f' to {supported_number} positional args'
    166                   f' {list(supported)}: pass {wanted}'
    167                   ' as keyword arg(s)',
    168                   stacklevel=stacklevel,
    169                   category=category)
--> 171 return func(*args, **kwargs)

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/piping.py:121, in Pipe._pipe_legacy(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
    112 @_tools.deprecate_positional_args(supported_number=2)
    113 def _pipe_legacy(self,
    114                  format: typing.Optional[str] = None,
   (...)
    119                  engine: typing.Optional[str] = None,
    120                  encoding: typing.Optional[str] = None) -> typing.Union[bytes, str]:
--> 121     return self._pipe_future(format,
    122                              renderer=renderer,
    123                              formatter=formatter,
    124                              neato_no_op=neato_no_op,
    125                              quiet=quiet,
    126                              engine=engine,
    127                              encoding=encoding)

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/piping.py:149, in Pipe._pipe_future(self, format, renderer, formatter, neato_no_op, quiet, engine, encoding)
    146 if encoding is not None:
    147     if codecs.lookup(encoding) is codecs.lookup(self.encoding):
    148         # common case: both stdin and stdout need the same encoding
--> 149         return self._pipe_lines_string(*args, encoding=encoding, **kwargs)
    150     try:
    151         raw = self._pipe_lines(*args, input_encoding=self.encoding, **kwargs)

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/backend/piping.py:212, in pipe_lines_string(engine, format, input_lines, encoding, renderer, formatter, neato_no_op, quiet)
    206 cmd = dot_command.command(engine, format,
    207                           renderer=renderer,
    208                           formatter=formatter,
    209                           neato_no_op=neato_no_op)
    210 kwargs = {'input_lines': input_lines, 'encoding': encoding}
--> 212 proc = execute.run_check(cmd, capture_output=True, quiet=quiet, **kwargs)
    213 return proc.stdout

File ~/checkouts/readthedocs.org/user_builds/pint/envs/latest/lib/python3.9/site-packages/graphviz/backend/execute.py:84, in run_check(cmd, input_lines, encoding, quiet, **kwargs)
     82 except OSError as e:
     83     if e.errno == errno.ENOENT:
---> 84         raise ExecutableNotFound(cmd) from e
     85     raise
     87 if not quiet and proc.stderr:

ExecutableNotFound: failed to execute PosixPath('dot'), make sure the Graphviz executables are on your systems' PATH

[13]:

<graphviz.graphs.Digraph at 0x7f0bdd75bbb0>

Examples#

xarray wrapping Pint Quantity

[14]:

import xarray as xr

# Load tutorial data
air = xr.tutorial.load_dataset("air_temperature")["air"][0]

# Convert to Quantity
air.data = Q_(air.data, air.attrs.pop("units", ""))

print(air)
print()
print(air.max())

<xarray.DataArray 'air' (lat: 25, lon: 53)>
<Quantity([[241.2     242.5     243.5     ... 232.79999 235.5     238.59999]
 [243.79999 244.5     244.7     ... 232.79999 235.29999 239.29999]
 [250.      249.79999 248.89    ... 233.2     236.39    241.7    ]
 ...
 [296.6     296.19998 296.4     ... 295.4     295.1     294.69998]
 [295.9     296.19998 296.79    ... 295.9     295.9     295.19998]
 [296.29    296.79    297.1     ... 296.9     296.79    296.6    ]], 'kelvin')>
Coordinates:
  * lat      (lat) float32 75.0 72.5 70.0 67.5 65.0 ... 25.0 22.5 20.0 17.5 15.0
  * lon      (lon) float32 200.0 202.5 205.0 207.5 ... 322.5 325.0 327.5 330.0
    time     datetime64[ns] 2013-01-01
Attributes:
    long_name:     4xDaily Air temperature at sigma level 995
    precision:     2
    GRIB_id:       11
    GRIB_name:     TMP
    var_desc:      Air temperature
    dataset:       NMC Reanalysis
    level_desc:    Surface
    statistic:     Individual Obs
    parent_stat:   Other
    actual_range:  [185.16 322.1 ]

<xarray.DataArray 'air' ()>
<Quantity(302.6000061035156, 'kelvin')>
Coordinates:
    time     datetime64[ns] 2013-01-01

Pint Quantity wrapping Sparse COO

[15]:

from sparse import COO

np.random.seed(80243963)

x = np.random.random((100, 100, 100))
x[x < 0.9] = 0  # fill most of the array with zeros
s = COO(x)

q = s * ureg.m

print(q)
print()
print(np.mean(q))

<COO: shape=(100, 100, 100), dtype=float64, nnz=99598, fill_value=0.0> meter

0.09462606529121113 meter

Pint Quantity wrapping NumPy Masked Array

[16]:

m = np.ma.masked_array([2, 3, 5, 7], mask=[False, True, False, True])

# Must create using Quantity class
print(repr(ureg.Quantity(m, "m")))
print()

# DO NOT create using multiplication until
# https://github.com/numpy/numpy/issues/15200 is resolved, as
# unexpected behavior may result
print(repr(m * ureg.m))

<Quantity([2 -- 5 --], 'meter')>

masked_array(data=[<Quantity(2, 'meter')>, --, <Quantity(5, 'meter')>, --],
             mask=[False,  True, False,  True],
       fill_value='?',
            dtype=object)

Pint Quantity wrapping Dask Array

[17]:

import dask.array as da

d = da.arange(500, chunks=50)

# Must create using Quantity class, otherwise Dask will wrap Pint Quantity
q = ureg.Quantity(d, ureg.kelvin)

print(repr(q))
print()

# DO NOT create using multiplication on the right until
# https://github.com/dask/dask/issues/4583 is resolved, as
# unexpected behavior may result
print(repr(d * ureg.kelvin))
print(repr(ureg.kelvin * d))

<Quantity(dask.array<arange, shape=(500,), dtype=int64, chunksize=(50,), chunktype=numpy.ndarray>, 'kelvin')>

<Quantity(dask.array<mul, shape=(500,), dtype=int64, chunksize=(50,), chunktype=numpy.ndarray>, 'kelvin')>
<Quantity(dask.array<mul, shape=(500,), dtype=int64, chunksize=(50,), chunktype=numpy.ndarray>, 'kelvin')>

xarray wrapping Pint Quantity wrapping Dask array wrapping Sparse COO

[18]:

import dask.array as da

x = da.random.random((100, 100, 100), chunks=(100, 1, 1))
x[x < 0.95] = 0

data = xr.DataArray(
    Q_(x.map_blocks(COO), "m"),
    dims=("z", "y", "x"),
    coords={
        "z": np.arange(100),
        "y": np.arange(100) - 50,
        "x": np.arange(100) * 1.5 - 20,
    },
    name="test",
)

print(data)
print()
print(data.sel(x=125.5, y=-46).mean())

<xarray.DataArray 'test' (z: 100, y: 100, x: 100)>
<Quantity(dask.array<COO, shape=(100, 100, 100), dtype=float64, chunksize=(100, 1, 1), chunktype=sparse.COO>, 'meter')>
Coordinates:
  * z        (z) int64 0 1 2 3 4 5 6 7 8 9 10 ... 90 91 92 93 94 95 96 97 98 99
  * y        (y) int64 -50 -49 -48 -47 -46 -45 -44 -43 ... 43 44 45 46 47 48 49
  * x        (x) float64 -20.0 -18.5 -17.0 -15.5 ... 124.0 125.5 127.0 128.5

<xarray.DataArray 'test' ()>
<Quantity(dask.array<mean_agg-aggregate, shape=(), dtype=float64, chunksize=(), chunktype=numpy.ndarray>, 'meter')>
Coordinates:
    y        int64 -46
    x        float64 125.5

Compatibility Packages#

To aid in integration between various array types and Pint (such as by providing convenience methods), the following compatibility packages are available:

(Note: if you have developed a compatibility package for Pint, please submit a pull request to add it to this list!)

Additional Comments#

What follows is a short discussion about how NumPy support is implemented in Pint’s Quantity Object.

For the supported functions, Pint expects certain units and attempts to convert the input (or inputs). For example, the argument of the exponential function (numpy.exp) must be dimensionless. Units will be simplified (converting the magnitude appropriately) and numpy.exp will be applied to the resulting magnitude. If the input is not dimensionless, a DimensionalityError exception will be raised.

In some functions that take 2 or more arguments (e.g. arctan2), the second argument is converted to the units of the first. Again, a DimensionalityError exception will be raised if this is not possible. ndarray or downcast type arguments are generally treated as if they were dimensionless quantities, whereas Pint defers to its declared upcast types by always returning NotImplemented when they are encountered (see above).

To achive these function and ufunc overrides, Pint uses the __array_function__ and __array_ufunc__ protocols respectively, as recommened by NumPy. This means that functions and ufuncs that Pint does not explicitly handle will error, rather than return a value with units stripped (in contrast to Pint’s behavior prior to v0.10). For more information on these protocols, see https://docs.scipy.org/doc/numpy-1.17.0/user/basics.dispatch.html.

This behaviour introduces some performance penalties and increased memory usage. Quantities that must be converted to other units require additional memory and CPU cycles. Therefore, for numerically intensive code, you might want to convert the objects first and then use directly the magnitude, such as by using Pint’s wraps utility (see wrapping).

Attempting to access array interface protocol attributes (such as __array_struct__ and __array_interface__) on Pint Quantities will raise an AttributeError, since a Quantity is meant to behave as a “duck array,” and not a pure ndarray.