scoringutils icon indicating copy to clipboard operation
scoringutils copied to clipboard

Let `get_pairwise_comparison()` run with two models?

Open athowes opened this issue 7 months ago • 2 comments

Currently get_pairwise_comparison() will stop you if you have two models:

https://github.com/epiforecasts/scoringutils/blob/158ba6cf9e7f6460a5d47f8658459812bdb77170/R/pairwise-comparisons.R#L161-L171

While it's fair enough to say that you don't really need relative scores if you just have two models, there is not a conceptual problem with computing a comparison between two models. (Also if you are running more models and setting up the infrastructure with just the first one!)

So I think this should be a warning rather than an abort.

(At the moment, my solution is to create a fake model with one row, append it on, then remove it afterwards.)

athowes avatar Jul 03 '25 12:07 athowes

Aren't you just taking a ratio in this case i.e it isn't a pairwise comparison? One action here if we wan't to keep the action of the function strict is to make the error message more informative.

seabbs avatar Jul 03 '25 14:07 seabbs

I don't mind much, happy to turn it into a warning if that would make your life easier, @athowes. Have you tried doing that locally and made sure it is what you want?

nikosbosse avatar Aug 02 '25 06:08 nikosbosse