Possibly documentation error?

[Chenchaoyou]

In the user guide 6.6.2.1. Resilience Performance, the description of each figure seems to be wrong.

1. Worst sample figure: "The curve depicted in the plot is monotonic, indicating that as the worst sample ratio increases, the model’s performance tends to decrease". The figure however seems to be displaying a monotonically increasing AUC as WSR increases.
2. Outer sample figure: "Interestingly, the model’s performance reaches its peak around the worst sample ratio of approximately 0.7, after which it gradually declines". The figure displays a 0.83 AUC as WSR tends to 1.0.

It could be that I understood it all wrong though. Please let me know if it's an error or that I should understand it all differently. Thank you.

[ZebinYang]

Hi @Chenchaoyou,

Thanks for mentioning that, and the contents in user guide 6.6 were not updated to the example figures.

When the worst-sample ratio is 10%, only the worst-10% samples are selected as "worst samples"; When the worst-sample ratio is 100%, all the test samples are selected as "worst samples". Therefore, the performance is expected to improve as worst-sample ratios increase, which means higher ACC / AUC.