HDF5.jl icon indicating copy to clipboard operation
HDF5.jl copied to clipboard

Consistent array slicing with Base

Open IlyaOrson opened this issue 8 years ago • 2 comments

Maybe the getindex() behaviour should be the same on the data stored in HDF5 and the one stored in a normal array in julia:

julia> h5open("test.h5","w") do file
           rows, columns = 3, 12
           data = zeros(rows, columns)
           group = g_create(file, "group")
           data_hdf5 = d_create(group, "data", datatype(Float64), dataspace(rows, columns), "chunk", (rows, 1))
           for i = 1:columns
               vec = rand(3)
               data[:,i] = vec
               data_hdf5[:,i] = vec
           end
           typeof(data[:,1]), typeof(data_hdf5[:,1])
       end
(Array{Float64,1}, Array{Float64,2})

IlyaOrson avatar Jul 02 '17 05:07 IlyaOrson

A related example where the normal syntax that works on a julia array fails with HDF5 data set:

julia> h5open("test.h5", "w") do file
       b = d_create(file, "b", Int, ((1000,50),(-1,-1)), "chunk", (1000,1)) 
       b[:,:] = collect(1:length(b)) # This works
       b[:] = collect(1:length(b)) # This fails
       end
ERROR: number of elements in range and length of array must be equal
Stacktrace:
 [1] _setindex!(::HDF5.HDF5Dataset, ::Type{T} where T, ::Array{Int64,1}, ::UnitRange{Int64}, ::Vararg{UnitRange{Int64},N} where N) at /home/ilya/.julia/v0.6/HDF5/src/HDF5.jl:1644
 [2] setindex!(::HDF5.HDF5Dataset, ::Array{Int64,1}, ::UnitRange{Int64}) at /home/ilya/.julia/v0.6/HDF5/src/HDF5.jl:1633
 [3] (::##35#36)(::HDF5.HDF5File) at ./REPL[39]:4
 [4] h5open(::##35#36, ::String, ::String) at /home/ilya/.julia/v0.6/HDF5/src/HDF5.jl:590

My version info:

julia> versioninfo()
Julia Version 0.6.1-pre.0
Commit dcf39a1dda (2017-06-19 13:06 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-4510U CPU @ 2.00GHz
  WORD_SIZE: 64
  BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell)
  LAPACK: libopenblas64_
  LIBM: libopenlibm
  LLVM: libLLVM-3.9.1 (ORCJIT, haswell)

julia> Pkg.status("HDF5")
 - HDF5                          0.8.1+             master

IlyaOrson avatar Jul 03 '17 21:07 IlyaOrson

the first example works as expected since #603. The second example still doesn't work because hyperslab doesn't support 1D indexing of multidimensional arrays. This seems definitely doable to implement if we want to.

kleinhenz avatar Nov 14 '20 22:11 kleinhenz