digikar99
concat doesn't work for displaced array
This is a problem for both numericals and dense-numericals. To reproduce:
(numericals:concat
CL-USER> (list (numericals:reshape
(make-array
'(3)
:initial-contents
'(1.0 2.0 3.0)
:element-type 'single-float)
'(3 1))
(make-array
'(3 1)
:initial-contents
'((1.0) (2.0) (3.0))
:element-type 'single-float))
:axis 1)
CL-USER> (dense-numericals:concat
(list (dense-numericals:reshape
(dense-numericals:make-array
'(3)
:initial-contents
'(1.0 2.0 3.0)
:element-type 'single-float)
'(3 1))
(dense-numericals:make-array
'(3 1)
:initial-contents
'(1.0 2.0 3.0)
:element-type 'single-float))
:axis 1)
I can get these to run by changing the (declare (type (simple-array <type>) array-like)) declaration to (declare (type (array <type>) array-like)), e.g. https://github.com/digikar99/numericals/blob/ea50654e6c4372bd0e7c7ae69d7aa07b1c93d326/dense-numericals-src/utils/concat.lisp#L60 for dense-numericals. Not sure if this is the right way, though (no idea about performance).
Thank you for reporting!
The proposed solution seems good for cl:array, since these arrays are always contiguous. It fails with discontiguous arrays permitted by dense-arrays and dense-numericals:
(dense-numericals:concat
(list (print
(dense-numericals:aref*
(dense-numericals:asarray '((1 2) (3 4) (5 6))
:type 'single-float)
nil '(1 :end 2)))
(dense-numericals:make-array
'(3 1)
:initial-contents
'(1.0 2.0 3.0)
:element-type 'single-float))
:axis 1)
It's indeed true that currently, concat assumes each of the elements of array-likes to be a simple array. In the ideal solution, it should be able to handle all the different array-likes, the way asarray handles it. I hope to add this capability this week; until then, the input to concat may be passed through copy or asarray to ensure a simple array.
Turns out sum has some issue with displaced array as well, specifically if :out is a view with stride. Somehow the (out (nu:simple-array <type>)) method still runs (?), but the results are wrong:
(let ((out (zeros '(4))))
(sum
(full '(2 10) :value 1.0)
:axes 1
:out (aref* out (list 2)))
out)
#<STANDARD-DENSE-ARRAY :ROW-MAJOR 4 DOUBLE-FLOAT
0.000e+00
0.000e+00
0.000e+00
0.000e+00
{101103F0D3}>
The particular polymorph responsible for this case is here: https://github.com/digikar99/numericals/blob/e4d8995459888eb0a4aa05bf9abf5c6f53dabcad/src/basic-math/sum.lisp#L175
Thankfully the pointers are operating all correct and the issue is with the higher level code!
dense-arrays-plus-lite:reshape or equivalently dense-numericals:reshape may create a new array if the supplied array is a non-simple array. Fixing the issue with sum would either require (i) compelling out to be simple or (ii) modifying reshape to always work with the original array. (i) can simplify a lot of tasks if applied uniformly across other operations too, although I gotta check how numpy behaves with non-simple arrays if we are aiming for numpy freedom. (ii) is complex but provides more freedom. A third solution can involve (iii) using a different algorithm for the particular sum polymorph.