spikeinterface icon indicating copy to clipboard operation
spikeinterface copied to clipboard

Compare multiple sorters

Open paul-aparicio opened this issue 11 months ago • 4 comments

Hi, I have been trying to see if I could speed up the compare_multiple_sorters function (it has been taking a very long time to complete). I have tried changing the n_jobs parameter, but it seems to not have any effect? In every case I see only 1 CPU as active. I am running si version 0.102.0, and using the results from 4 sorters. Thanks for any help or suggestions!

paul-aparicio avatar Mar 04 '25 15:03 paul-aparicio

I am testing this, everything seems to be working fine without error, but it hasn't changed the time for me much (it's still running 12+ hours so far). My computer only shows 1 CPU running.

paul-aparicio avatar Mar 05 '25 17:03 paul-aparicio

Mmm maybe the n_jobs parameter isn't working as expected. I'll reopen the issue and check the implementation

alejoe91 avatar Mar 06 '25 07:03 alejoe91

What OS are you using? And are you doing thread or process?

zm711 avatar Mar 07 '25 22:03 zm711

Hello. I have tried multiple times to install the pr 3737. I seem to get similar results when using agreement_method='count' or agreement_method='distance'. The first three steps are very fast (less than few minutes), and the 'count' method does seem to use multiple CPUs. Then 'step 4: extract agreement form graph' typically takes 12+ hours and only seems to use 1 CPU (well at least at the times I looked in on it). Here is a screen shot of the code I used. Is this the correct behavior, or maybe I haven't installed the pr correctly? I am using Ubuntu 20.04 and I am getting the number of CPUs from the system monitor app. Thank you!

Image

paul-aparicio avatar Mar 25 '25 15:03 paul-aparicio

If it's helpful, I tried this again with version 0.103.0 and multicomparison step 1 was clearly using multiple processors and worked quickly. Steps 2 & 3 seemed to complete quickly as well. Steps 4 took > 10 hours though, and seemed to only use 1 CPU (mainly, as far as I could tell).

paul-aparicio avatar Aug 27 '25 16:08 paul-aparicio