sparse icon indicating copy to clipboard operation
sparse copied to clipboard

sparse.random fails for sizes greater than 2**31

Open wecassidy opened this issue 4 years ago • 0 comments

Describe the bug sparse.random fails with a ValueError when creating an array of size 2 ** 31 + 1, but 2 ** 31 works:

>>> sparse.random((2**31,), nnz=1)
<COO: shape=(2147483648,), dtype=float64, nnz=1, fill_value=0.0>
>>> sparse.random((2**31 + 1,), nnz=1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\<path_redacted>\sparse\_utils.py", line 183, in random
    j = random_state.randint(elements)
  File "mtrand.pyx", line 741, in numpy.random.mtrand.RandomState.randint
  File "_bounded_integers.pyx", line 1350, in numpy.random._bounded_integers._rand_int32
ValueError: high is out of bounds for int32

To Reproduce Use sparse.random to create an array of shape (2 ** 31 + 1,) with one non-zero element

Expected behavior Return a COO array of shape (2147483649,) with exactly one non-zero element.

System

  • OS and version: Windows 10
  • sparse version: 0.12.0+44.g765e297 (bug is also present in 0.12.0, installed from pip)
  • NumPy version: 1.18.5
  • Numba version: 0.53.1

Additional context The following all work:

  • sparse.random((2 ** 31 + 1,), nnz=0)
  • sparse.zeros((2 ** 31 + 1,))
  • Creating a DOK array of shape (2 ** 31 + 1,) then converting it to COO:
    >>> d = sparse.DOK((2 ** 31 + 1,))
    >>> d[0] = 1
    >>> c = sparse.COO(d)
    >>> c[0]
    1.0
    

wecassidy avatar Jun 27 '21 05:06 wecassidy