Transformers4Rec icon indicating copy to clipboard operation
Transformers4Rec copied to clipboard

Fix recall metric calculation

Open sungho-ham opened this issue 1 year ago • 1 comments

Goals :soccer:

  • Fix recall metric error when batch size is 1

Implementation Details :construction:

https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/23d5e3ba73b3d490400e45bf4feba94ed473432f/transformers4rec/torch/ranking_metric.py#L134 The squeeze() function removes all dimensions when batch size is 1. I've added dim=1 argument to keep dimension along batch.

Testing Details :mag:

Test code for error reproduction

import torch
from transformers4rec.torch import ranking_metric

ks = [1, 2]
scores = torch.tensor( [ [1,2,3], ])
labels = torch.tensor( [ [0,1,0], ])
ranking_metric.RecallAt()._metric(ks, scores, labels)

Error message

    135 rel_indices = (num_relevant != 0).nonzero().squeeze()
    136 rel_count = num_relevant[rel_indices]
--> 138 if rel_indices.shape[0] > 0:
    139     for index, k in enumerate(ks):
    140         rel_labels = topk_labels[rel_indices, : int(k)]

IndexError: tuple index out of range

sungho-ham avatar May 14 '24 09:05 sungho-ham

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

copy-pr-bot[bot] avatar May 14 '24 09:05 copy-pr-bot[bot]

rerun tests

rnyak avatar Aug 02 '24 17:08 rnyak

/ok to test

jperez999 avatar Aug 06 '24 17:08 jperez999