NeuralEnsemble
Consistent multi-object handling
Currently users need to know a lot about the specifics of their IO class to read it properly. One of the biggest problems is that, without reading the documentation, the users can't tell whether their IO class should support one or more base objects. All IO classes support the read method. However, in IO classes that have, for example, multiple Block objects, there is no external indication that is possible. There is currently no way that someone can write code that will read all data from any IO class, they have to check whether it can contain multiple objects or not.
So I suggest adding a more consistent interface to deal with this. Specifically in BaseIO, implement a read_all method, which is the multi-object version of read. By default, this will just return [self.read()], but IO classes that support multiple objects can override it. Similarly, there should be read_all_* methods for each read_* method, which again by default are just [self.read_*()] (which would fall through to an AssertionError for classes that don't support that object).
With this approach, a user can just call read_all(), and they will always get all the data from the file, no matter how it is implemented.
Sorry I do not get it.
Today IO.read() do a full read. With this you get a list of all Blocks in your file. It is often just one Block. For some IOs that implement read_all_blocks() it can be several.
Something I do not get is: does read_all_segment() return all segment even if they do not belong to same the same block ? I prefer to force the user to loop over Block and nested loop over Segment: for block in MyIO.read(): for seg in block.segments: do_process(seg)
I think I do not get the idea of the proposal.
Agreed, I think multiple blocks and just getting all data from a file is already covered since 6cdacce.
Concerning read_* / read_all_* methods, I think I would rather remove these methods entirely: For the single read_* methods, it is not clear for most objects what is being loaded. A random spike train from the file, for example, is not very useful. Loading with a read_all_* method loses context but would still require that the whole file is loaded / parsed for many formats.
I think I agree with Robert read_xx method was introduced at first version of neo to load only few object. But it is useless and hard to anderstand for dev. +1 for removing. Just keep a read() that do read all blocks.
For me, the best scenario for big files is read(lazy = True) and then do a load on targeted object.
Update: .read() should always return a list of blocks. To be checked if this is already the case for all IOs
todo: make a checklist of IOs, for each one, check the read() method returns a list.
