rtmdrr
Question about results
Hi. Thank you for sharing your code. I tried it. Technically it works fine. However, I am puzzled when examining its results.
I've generated two score files, almost identical. Only 4 scores out of 10000 are different, and the difference is very small. Nevertheless, when comparing alg2.txt to alg1.txt (alg2 is the first argument), I get very small values of epsilon. This is the case for various values of alpha, big and small ones. This means that the comparison software concludes that in a very high confidence alg2 is better than alg1. However, the reasonable conclusion should be that there is no difference and no edge to any of the score files.
Similarly, when I compare a score file to itself (say alg1 to alg1), it prints "The denominator is 0" but returns a result, and that result is that epsilon is very small for the first algorithm. Again, this makes no sense since it compares the same score-file to itself. Epsilon should be 0.5 in such a case.
The files are attached here (see below).
Thank you very much in advance.
Hi @asher-stern ,
I don't know if this will help you but you may want to try the fix in #2 and check if it solves your issue :)
Hicham
