sappelhoff
Weird handling of initial bad-by-SNR channels during robust referencing
(came first up in https://github.com/sappelhoff/pyprep/pull/93#issuecomment-868924385 and following comment in that thread.)
This isn't a PyPREP bug so much as a weirdness in the original PREP that PyPREP now matches. Essentially, during robust referencing, PREP performs an initial NoisyChannels pass and flags any bad-by-NaN, bad-by-flat, and bad-by-SNR channels as "unusable", permanently excluding them from being included in the calculated average reference signals:
https://github.com/sappelhoff/pyprep/blob/1abda4d0cbda7ed9e9cef8981c63747609b4ce7b/pyprep/reference.py#L230-L235
https://github.com/sappelhoff/pyprep/blob/1abda4d0cbda7ed9e9cef8981c63747609b4ce7b/pyprep/reference.py#L324-L329
However, PREP doesn't include initial bad-by-SNR channels in the full set of bad channels to interpolate (see #91), so a channel that's initially bad-by-SNR prior to initial average referencing but is no longer bad afterwards will still get used for interpolating bads during the reference loop, despite being fully excluded from the average reference signal itself. This seems wrong, and like something worth fixing.
My gut instinct here would be to default to only marking initial bad-by-flat and bad-by-NaN channels as fully "unusable". I'd imagine that the original intent was to removing channels that contribute more noise than signal from contaminating the average reference, but given how sensitive the 'bad-by-correlation' default settings are (if over 1% of all correlation windows are below threshold, the whole channel is considered bad) and how that can change after average referencing, I don't think initial bad-by-SNR channels should be treated as permanently bad.
As an additional note, because bad-by-SNRs are included in the set of "unusable" channels, that means they also get interpolated during re-referencing no matter what, even if they're totally fine after re-referencing.
Oh, and also: since unusable channels always get interpolated, doesn't that mean flat channels will be made no longer flat when the robust reference signal is applied, but will then immediately be interpolated? On its face it seems like this would be undesirable, but maybe I'm missing something.