craffel
Chroma transcription metrics
Thanks Caleb. Apart from PEP8 stuff, you'll also need to add separate evaluation for chroma metrics to mir_eval.transcription.evaluate, e.g. forcing the value of chroma in the kwargs dict to be True, then False, so that both are evaluated (right @justinsalamon)?
Do we have any implementations to test this against? It's a small enough change that I am ok with not testing it against something existing, though we should be very certain in its correctness then :)
Thanks Caleb. Apart from PEP8 stuff, you'll also need to add separate evaluation for chroma metrics to mir_eval.transcription.evaluate, e.g. forcing the value of chroma in the kwargs dict to be True, then False, so that both are evaluated (right @justinsalamon)?
yes
Do we have any implementations to test this against?
yes, see here.
The music-ir link is dead for me. Do you know of any others?
The MIREX website is temporarily down, my understanding is that they're working on it. I'll check if I have the result files stored locally somewhere, otherwise we'll have to wait till it's back up.
I'm a bit confused about comparing to MIREX results. The existing output*.json files don't match the differences from this comment, so I'm not sure what to look for when comparing the results.
I'm a bit confused about comparing to MIREX results. The existing output*.json files don't match the differences from this comment, so I'm not sure what to look for when comparing the results.
@justinsalamon can you provide some guidance here?
Is it still planned to integrate this functionality and does anybody know which points are still open?
This needs to be picked up again. I don't think anyone is planning on doing so.