404 error has occurred when registering Azure AI Cohere rerank model

[potofo]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.7.3

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

1. Deploy Cohere-rerank-v3-multilingual to the Azure AI Service Model Catalog
2. Register the model provider with Dify [Model Provider]->[Azure AI Studio]

Model Type: Rerank Model Name: Cohere-rerank-v3-multilingual-xx Azure AI Studio Endpoint: https://Cohere-rerank-v3-multilingual-xx.eastus.models.ai.azure.com API Key:************************

Error On Web screen: HTTP Error 404:Not Found

If you set the Model Type to LLM, you can register it, but it will not work and will not be recognized as a Rerank model.

✔️ Expected Behavior

Azure AI Studio's Cohere-rerank-v3-multilingua can be registered

❌ Actual Behavior

Error On Web screen: HTTP Error 404:Not Found

If you set the Model Type to LLM, you can register it, but it will not work and will not be recognized as a Rerank model.

[dosubot[bot]]

I found a similar unsolved discussion: Add Azure OpenAI Model Provider Failed ^[1].

To resolve the HTTP Error 404: Not Found when registering the Cohere-rerank-v3-multilingual model in Azure AI Studio with Dify, ensure that you have correctly followed these steps:

1. Register and Apply for API Key: Visit Cohere Rerank, register, and apply for the Rerank model usage qualification to obtain the API key.

2. Configure the Re-rank Model in Dify:

   • Go to "Model Providers -> Cohere" in Dify.
   • Fill in the API key for the Re-rank model.

3. Set the Re-rank Model in Dataset Retrieval Mode:

   • Navigate to "Dataset -> Create Dataset -> Retrieval Settings" to add the Re-rank settings.
   • You can also change the Re-rank configuration in the settings of an existing dataset and in the dataset recall mode settings in application orchestration.

4. Ensure Correct API Endpoint: Verify that the API endpoint you are using is correct and accessible. A 404 error typically indicates that the endpoint is incorrect or the resource does not exist. According to the Dify code, the endpoint and API key must be provided in the credentials:

   endpoint = credentials.get("endpoint")
   api_key = credentials.get("jwt_token")
   
   if not endpoint or not api_key:
       raise ValueError("Azure endpoint and API key must be provided in credentials")
   

5. Verify Model Version: Ensure that you are using a supported model version. According to the Dify documentation, the supported versions for the multilingual model are:

   • rerank-multilingual-v2.0
   • rerank-multilingual-v3.0

If you have followed these steps and are still encountering the error, double-check the API endpoint and ensure that the API key is valid and has the necessary permissions ^[2][3][4].

Additionally, there is a known issue documented in the Dify codebase regarding the registration of rerank models with Azure AI Studio. Specifically, the validate_credentials function raises a CredentialsValidateFailedError when invalid credentials are provided ^[5][4].

To continue talking to Dosu, mention @dosu.

[jumpei-tsutsui]

I am encountering the same issue. Why is a JWT token required instead of an API token for the Rerank model? How can I obtain a persistent JWT token from Azure AI Studio?

[grapswiz]

I alse see in https://github.com/langgenius/dify/pull/7549#issuecomment-2325557714