External Knowledge base will considering as Multi-path retrieval

[gavinlichn]

Self Checks

• [x] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [x] I have searched for existing issues search for existing issues, including closed ones.
• [x] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [x] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [x] Please do not modify this template :) and fill in all the required fields.

Dify version

latest (commit fcd9fd8513c63ae22c24cfcaba4989a71f5406c5 (HEAD -> main, origin/main, origin/HEAD))

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Setup external Knowledge base, as attachment settings. Create an app with the external knowledge base. the knowledge base setting shown as "Multi-path retrieval is used by default. Knowledge is etrieved from multiple knowledge bases and then re-ranked."

✔️ Expected Behavior

External Knowledge base is a single knowledge base, not multi-path retrieval

❌ Actual Behavior

External Knowledge base is treated as multi-path retrieval

[dosubot[bot]]

The Dify platform treats external knowledge bases as multi-path retrieval by default to enhance the quality of results by querying related text fragments from multiple datasets simultaneously and re-ranking them semantically using a Rerank model ^[1].

To configure Dify to treat an external knowledge base as a single knowledge base, you can adjust the retrieval settings. On the "Knowledge" page, when connecting to an external knowledge base, you can modify the "Retrieval Setting" parameters. Use the "Top K" setting to specify the number of relevant content chunks to retrieve, and set the "Score Threshold" to determine the similarity level required for text chunk filtering. Additionally, ensure that you include the knowledge_id in your API request body to uniquely identify your knowledge base, and configure the retrieval_setting object with the appropriate top_k and score_threshold values to fine-tune the retrieval process ^[2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[gavinlichn]

@dosu

Can you help to show me the detail parameter combinations that will treated external knowledge base as a single knowledge base?

I tried to let Dify worked once with setup the external knowledge base with top_k=1 and score_threshold=false

But still not work with other parameters combinations.

[dosubot[bot]]

To configure Dify to treat an external knowledge base as a single knowledge base, follow these steps:

1. Knowledge Base Configuration:

   • Ensure you have provided a unique name and description for your knowledge base.

2. External Knowledge API Setup:

   • Navigate to the "Knowledge" page and select "External Knowledge API."
   • Add a new External Knowledge API with the necessary details:
     • Name: A unique identifier for the API.
     • API Endpoint: The URL of your external knowledge base API.
     • API Key: The secure key for accessing the external knowledge base.

3. Connect External Knowledge Base:

   • On the "Knowledge" page, choose "Connect External Knowledge Base."
   • Configure the following parameters:
     • External Knowledge Base API: Select the API you set up earlier.
     • External Knowledge Base ID: Specify the ID of the external knowledge base.

4. Retrieval Settings:

   • Top K: Set this to 1 to retrieve the most relevant content chunk.
   • Score Threshold: Set this to a value that ensures only highly relevant chunks are retrieved. If you set it to false, ensure it aligns with your retrieval strategy.

5. Testing:

   • Use the "Recall Test" feature to simulate queries and verify that the retrieval is working as expected. Adjust the parameters if necessary.

These settings should help Dify treat your external knowledge base as a single source rather than using multi-path retrieval ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[JohnJyong]

This is just some misunderstanding in the description, we will optimize this description. You can use this function normally，thanks @gavinlichn

[dosubot[bot]]

Hi, @gavinlichn. I'm Dosu, and I'm helping the Dify team manage their backlog. I'm marking this issue as stale.

Issue Summary:

• The issue involved a bug where an external knowledge base was treated as multi-path retrieval.
• I provided instructions to configure the system for single-path retrieval.
• You reported continued issues despite following the instructions.
• @JohnJyong acknowledged a misunderstanding in the function's description.
• The function can be used normally, with plans to optimize the description for clarity.

Next Steps:

• Please confirm if this issue is still relevant to the latest version of the Dify repository by commenting here.
• If no updates are provided, the issue will be automatically closed in 15 days.

Thank you for your understanding and contribution!