hdf5 dimension labels and bin values

[PrometheusPi]

HDF5 seems to support dimension scales and labels [1]. Is this already included in libSplash? If no, do you think this might be a good feature to add? I think, this would be extremely useful for documenting physics quantities stored in arrays.

I have not found this option in the HDF5 documentation, but in some tutorials [2] of the hdf5 group. Analyzing *.h5 files with such scales using h5dump looks like scales are actually attributes.

DATASET "data" {
      DATATYPE  H5T_IEEE_F32LE
      DATASPACE  SIMPLE { ( 4, 3, 2 ) / ( 4, 3, 2 ) }
      DATA {
      (0,0,0): 1, 1,
      (0,1,0): 1, 1,
      (0,2,0): 1, 1,
      (1,0,0): 1, 1,
      (1,1,0): 1, 1,
      (1,2,0): 1, 1,
      (2,0,0): 1, 1,
      (2,1,0): 1, 1,
      (2,2,0): 1, 1,
      (3,0,0): 1, 1,
      (3,1,0): 1, 1,
      (3,2,0): 1, 1
      }
      ATTRIBUTE "DIMENSION_LABELS" {
         DATATYPE  H5T_STRING {
            STRSIZE H5T_VARIABLE;
            STRPAD H5T_STR_NULLTERM;
            CSET H5T_CSET_ASCII;
            CTYPE H5T_C_S1;
         }
         DATASPACE  SIMPLE { ( 3 ) / ( 3 ) }
         DATA {
         (0): "z", NULL  , "x"
       }
      ATTRIBUTE "DIMENSION_LIST" {
         DATATYPE  H5T_VLEN { H5T_REFERENCE { H5T_STD_REF_OBJECT }}
         DATASPACE  SIMPLE { ( 3 ) / ( 3 ) }
         DATA {
         (0): (DATASET 8560 /z1 ), (DATASET 8288 /y1 ),
         (2): (DATASET 1536 /x1 , DATASET 1808 /x2 )
         }
      }
   }

[1] http://docs.h5py.org/en/latest/high/dims.html [2] http://www.hdfgroup.org/HDF5/Tutor/h5dimscale.html

[ax3l]

That is pretty cool and could be a front-end related to https://github.com/ComputationalRadiationPhysics/picongpu/issues/408 https://github.com/ComputationalRadiationPhysics/picongpu/issues/115

Given that the information provided here is indeed the generic solution, than we should add this.

[f-schmitt]

We already achieve the same by creating attributes ourselves, however, it might not be available using the same interfaces in h5py. If this is of interest for us, than we should definitely add this in libsplash. I will mark this for 1.3.

[ax3l]

H5DS looks like a general high-level API feature of HDF5 that is "just" supported by h5py, too.

Pretty awesome if we add the scales and labels in 1.3 :)

[PrometheusPi]

what is the status of this issue?

[ax3l]

Labels for dimensions are part of openPMD with the "axisLabels" attribute for mesh records.

It's available since openPMD 1 and also implemented in openPMD-api which is the re-implementation of libSplash with generalized backends (ADIOS1, HDF5, JSON, ...) and various frontends such as C++11 and Python. (We did call it for a short initial devel phase libSplash2, but decided for openPMD-api for consistency with all the other projects and tools around it.)

In openPMD 2 (upcoming), this attribute's internal order has been simplified. An automatic updater exists as well for openPMD 1 files.

openPMD 2.0 is quite far and soon being feature-frozen and released. Tools will then follow in the months after.

We are not planning to implement new features in libSplash.