storing non `ndarray` types as blocks
Is there a way to store any other types besides ndarray as a block in an ASDF file?
I am looking for a simple way to serialize (and tag) bytes, bytearray, BytesIO or similar without going through numpy
Could this be done in an extension library or would this need changes in asdf?
Currently the code requires that anything written to a block be wrapped in ndarray, but I think it's possible to wrap bytes in ndarray without copying the memory:
In [21]: some_bytes = b'a' * 1024
In [22]: array = np.frombuffer(some_bytes, dtype=np.uint8)
In [23]: array.flags
Out[23]:
C_CONTIGUOUS : True
F_CONTIGUOUS : True
OWNDATA : False
WRITEABLE : False
ALIGNED : True
WRITEBACKIFCOPY : False
UPDATEIFCOPY : False
Now that array can be assigned to a block using the BlockManager.find_or_create_block_for_array method.
On the way back I don't know how to retrieve the bytes except to call ndarray.tobytes, which does make a copy.
yes that is a possible workaround we used before
However I was looking for a possibility to explicitly differentiate from ndarray but in this case I will probably just wrap the numpy calls in a custom tag/object