How to get confidence for prediction?

[max-grzanna]

Hello people,

is there a way to get a confidence score for the generated predictions? I am looking for a score like a probability or something to see how confident the model is regarding the first 3 predictions.

Thanks and regards!

[YannisPap]

I believe the faktorizedTop3 on the testing dataset will give you the percentage of the times where the actual selection was in the top 3 recommendations, which may suit you.

https://www.tensorflow.org/recommenders/api_docs/python/tfrs/metrics/FactorizedTopK

[maciejkula]

AFAIK prediction uncertainty like this is actually an open research problem, especially given that the scores returned by two-tower models are usually unnormalized and not probabilities.

[rlcauvin]

What about using tf.sigmoid(logits) to convert the unnormalized logits to "probabilities" ranging from 0 to 1?