[dataset] Collecting rankings from different annotators

[dain5832]

Hello, I wonder how you collected the rankings from different annotators, especially when there's a tie. For instance, say there are six annotators, and three ranked an image as rank2 and another three ranked as rank3. Then, what would be the final ranking of the image, 2 or 3??

[xujz18]

Hello, thanks for your attention! In this case, we don't use a pair of tie for training.