Standard InfoNCE loss vs Debiased Contrastive Loss under a supervised framework

[AthanasiosKwn]

Hello! I just read your paper, it was well writen and very informative.

I have one question:

What is expected to happen if we compare the standard InfoNCE contrastive loss with the proposed Debiased Contrastive Loss under a supervised contrastive learning framework, where false negatives are guaranteed to not exist through the utilization of labeled data? Will Debiased Contrastive Loss give the same results as the classic InfoNCE loss (which in this case is the unbiased loss) or do we expect for some reason a drop in performance? Did you run any relevant experiments ?

I am asking because I am interested about cases where the dataset is labeled but there is a probability - suspicion that some of the labels are incorrect. In such cases, if Debiased Contrastive Loss actually achieves equal results as the classic InfoNCE in supervised contrastive learning setups, then it can be used without any fear of hindering model performance in every case of supervised learning making sure that, IF false negatives exist we get the best possible outcome.

But if, it turns out to produce worse results, then it should only be used when it is verified that false negatives exist otherwise it will be sub optimal compared to the classic InfoNCE.

I look forward to your reply. Thanks in advance!