Improve documentation about rerankers: which ones are supported?

[AlexanderFillbrunn]

System Info

I am currently mostly working with the ghcr.io/huggingface/text-embeddings-inference:cpu-1.2 Docker image on MacOS. Currently, I am only trying to find out which reranker models with a context size larger than 512 tokens are supported by TEI, but apart from the two BAAI ones mentioned in the readme, I haven't been able to run any other reranker models from Huggingface. jinaai/jina-reranker-v1-tiny-en for example does not work:

reranker-1  | Error: The `--pooling` arg is not set and we could not find a pooling configuration (`1_Pooling/config.json`) for this model.
reranker-1  | 
reranker-1  | Caused by:
reranker-1  |     No such file or directory (os error 2)
reranker-1 exited with code 1

Apparently, this is because TEI treats it as an embedding model in this case. According to the readme, BERT models are supported, but the Jina reranker's description says:

JinaBERT itself is a unique variant of the BERT architecture Would that already tell me that it is not supported?

Additionally, the BAAI models are listed with a revision in the readme, but I have found no mention of revisions on the model page on Huggingface. How do I find the correct revision to use?

I would appreciate if documentation could be added how to find out if a model is supported. Additionally, showing a more meaningful error message when a model is not supported would help, if at all possible.

Information

• [X] Docker
• [ ] The CLI directly

Tasks

• [X] An officially supported command
• [ ] My own modifications

Reproduction

Docker-compose for jinaai/jina-reranker-v1-tiny-en that fails.

services:
  reranker:
    image: ghcr.io/huggingface/text-embeddings-inference:cpu-1.2
    restart: always
    #Works: command: ["--model-id", "BAAI/bge-reranker-base", "--revision", "refs/pr/5"]
    command: ["--model-id", "jinaai/jina-reranker-v1-tiny-en"] #Does not work
    platform: linux/amd64
    ports:
      - 8888:80
    volumes:
      - ${PWD}/reranker-data:/data

Expected behavior

Show a clear error message why the model is not supported. Provide a list with supported models containing more than just the two BAAI/bge-reranker-large and BAAI/bge-reranker-base.

[OlivierDehaene]

jinaai/jina-reranker-v1-tiny-en for example does not work:

Models that are not officialy supported by Transformers are always hard to support and parse because they don't need to respect our config files. For example, Jina is notorious for saying in their config that their architecture is Bert, whereas that's absolutely NOT the case (they add alibi and other modifications) making it very hard for us to parse their configs accurately. In the case of this specific model, they also ommit to say that's it's a classifier in the config. I will add a small change on TEI's side to hardcode this one but there is not much we can do since Huggingface doesn't own this model.

I have found no mention of revisions on the model page on Huggingface. How do I find the correct revision to use?

Models in the huggingface hub are git repositories. Revision == git commit. For example, the latest revision of BAAI/bge-base-en-v1.5 is a5beb1e3e68b9ab74eb54cfd186867f64f240e1a.

[OlivierDehaene]

It can be important to pin revisions if you want to be sure that you always serve the correct version of a model. It can also be important if you want to serve your own model and want to A/B test between two different trainings.

[AlexanderFillbrunn]

Hi @OlivierDehaene, thank you for the clarification! If you don't mind this follow-up question: How do I know which models are officially supported? Say I am looking for all supported rerankers, is there a list or some other way to find out?