NeuralEnsemble
MaxwellIO: Fix issue due to disparity between routed and mapped channels
As mentioned in #1702, there can be a potential disparity between the list of mapped channels and routed channels when using stimulation electrodes. This PR attempts to fix this by ignoring the additional electrodes since they have no data associated to them/ neo can currently only pull data from the routed group. I don't know if there can be additional groups in the data though.
The PR name change was just because it is easier for me to make our release notes if I can quickly glance to see which IOs have been worked on :)
Hi Zach, I tried to answer your comments and improve the style of the code a little. I'm pretty new to creating PRs on public repos so I'm guessing that I didn't use the review feature correctly and just commited a new file version. I hope you can still see the changes.
You're also on the probeinterface team right? Because I noticed that this issue propagates to probeinterface as well so I'll open a similar PR there too.
I'll try and "filet" a piece out of the dataset. I think it should be possible.
Hey again,
I created a data slice that basically represent a single well, single segment recording that contains the mismatch between routed channels and mapped channels. It's only 100 datapoints long, but I can make it longer if you like. If you inspect the file using https://myhdf5.hdfgroup.org/ (or any tool of your choice) and navigate into the data_store/data0000/routed/raw group you can see that the channels array has 998 entries but data_store/data0000/settings/mapping is actually 1000 entries long due to 2 duplicate channels.
Hey again,
I had the time to do some further investigation on my file. It seems to be the case that the issue always takes the form that two channels can be mapped to the same two electrodes. There can also be more than one pair of these duplicates per recording segment.
Initially I thought that this would be physically impossible for the device but after looking into the data, I am not quite sure about this anymore so it is be possible that the mapping file is indeed correct. I'm not sure how neo would navigate a surjective mapping of channels to electrodes since that is usually not a very useful situation.
I'll do some further digging once I get back to the lab tomorrow but for now I'd put this PR on ice until I found the source of this issue in the device and can give more insight into how to navigate the situation. Maybe this results in a more meaningful error message for future users.
Sounds good @LeMuellerGuy thanks for checking on that. It does seem a bit strange unless it really is a recording electrode and a stimulating electrode being associated with the same channel. But even then we would expect the need to unpack the two different information streams in order to be useful. Just ping us when you have some more info!
@philipp-mxw, maybe you would be good to comment on this PR? Whenever you have a moment, just so we can better understand some of this mapping?
Sorry for the late reply, I was away for a few days.
Due to the flexibility of the switch matrix design, it is possible to configure it such that one electrode connects to multiple readout channels, or multiple electrodes share the same readout channel. This can occur unintentionally during stimulation routing, e.g. if two stimulation electrodes are routed to the same stimulation unit. In such cases, the electrodes become electrically connected via the stimulation unit, and the mapping (correctly) reports that both electrodes are routed to the same set of readout channels.
The MxW H5 mapping structure does not support a mapping of one readout channel to a list of electrodes. Instead, one-to-many mappings appear as multiple rows in the mapping table. One for each channel-electrode pair.
If the stimulation is meant to activate both electrodes simultaneously, this setup works fine, as the stimulation unit can drive multiple electrodes. However, if independent stimulation is required, this configuration would not be valid due to the shared connection and the routing must be changed by the user, for example by selecting different electrodes. We are currently working on improving the routing logic to ensure each stimulation unit is only connected to one electrode, which will prevent such routing conflicts in the future.
For practical purposes when loading the mapping from file, it is reasonable to ignore cases where multiple readout channels connect to the same electrode (or electrode set), as this typically has no functional impact, unless they are shared due to a common stimulation unit. In this case the data will only show the voltage that the voltage stimulation unit applies at the electrode. However, cases where multiple electrodes share the same readout channel should be preserved if possible, as they reflect actual electrical connections and may affect data interpretation.
For example, in the H5 file attached to this PR, the mapping is as follows:
Channel 399 -> electrodes [25732 13364] Channel 431 -> electrodes [25732 13364] Electrode 13364 -> channels [399 431] Electrode 25732 -> channels [399 431] In this case, it is sufficient to retain channel 399 and ignore channel 431, since both channels connect to the same electrode set. However, it is important to preserve the information that both electrodes [25732, 13364] are connected, e.g.:
Channel 399 -> electrodes [25732 13364]
I know that was a lot, so please let me know if that was clear enough or you have any more questions!
Thanks for the clarification. I was already suspecting that it may be due to the stimulation buffers.
This might be a little off topic but if I understand it correctly there is not really a way to prevent the two electrodes being connected to the same stimulation buffer is there? To the user it's basically just chance how the connection ends up after routing, since you can only query the stim buffer connections and not set them.
Nevertheless, it seems quite unfortunate since I am not sure if neo, probeinterface, and spikeinterface are currently set up for a one to many mapping of device channels, which essentially makes these kind of files unusable in this context. It's probably also unreasonable to expect to change this since it would require rather hefty changes while the phyisological relevance of a two electrode one channel setup is probably questionable in most common applications.
Yes, unfortunately, there is currently no way to avoid two electrodes from being connected to the same stimulation unit. However, we are aware of this issue and are working on implementing changes to prevent this from happening.
For the time being, it is probably best to just remove any duplicates in order to be compatible with neo/spikeinterface etc. since it will most likely either contain the stimulation voltage or a mixed signal from two electrodes.