blitzpp
Build efficient serialization to persist / cache / share Blitz Array
Thomas Vincent Wrote:
One thing that we had to develop (only for 3d array) and I do thing should be part of blitz++ itself is efficient file serialization to persist / cache / share blitz array.
I have stuff that does that in NetCDF, and the idea is to include those subroutines in the Blitz++ library. Is this what you are talking about, or are you think of something different?
No not really, I found volume dump (read/write) really useful when working with big (huge) volume. So we implemented a generic binary read/write based on simple std::fstream::write() / read() to share volume across processes / cache to disk them temporally. Format is potentially not cross platform at all but it's really convenient. We did it for 2 / 3 dimensional array of contiguous data but it should scale nicely to higher rank ... not sure how to approach the non-contiguous part though. I think it's something blitz should offer.
After that being able to read/write specific format is necessary too to spread blitz++ usage I think. I have vtk / paraview structured data skills and code (write part only) if it could help.
Non contiguous should be ok, as long as you can generate an element wise iterator. also when reading you may want control over contiguity/alignment...
Not sure how relevant it is, but when you do
#define BZ_HAVE_BOOST_SERIALIZATION
#include <blitz/array.h>
it comes with Boost serialization support out of the box
