Feature: Enhanced Conversational Retrieval Tool Agent (QA Chain + Tools!)

[Nek-11]

Problem:

The current Conversational Retrieval Tool Agent node (introduced in PR #2430 ) lacks two important capabilities:

1. When using follow-up questions, the node doesn't have the ability to rephrase the follow-up into a standalone form using chat history, unlike the Conversational Retrieval QA Chain.
2. When tools with returnDirect: true are used (such as Custom Tools), the UI doesn't properly display the returned content until the user refreshes the chat interface.

Solution:

This PR enhances the Conversational Retrieval Tool Agent (visible with SHOW_COMMUNITY_NODES=true) node with two new features:

1. Rephrase Prompt Support:

   • Added an optional rephrasePrompt parameter that allows automatically converting follow-up questions into standalone queries
   • Implemented the same rephrasing logic that exists in the Conversational Retrieval QA Chain
   • Added support for variable substitution within the prompt template (e.g., {{$vars.variable_name}})

2. Fix for Tool returnDirect Behavior:

   • Updated the streaming logic to properly handle tools with returnDirect: true property
   • Added support for proper streaming of tool outputs to the UI
   • Ensured the UI receives proper stream completion signals (streamEndEvent)

The changes maintain backward compatibility - existing flows using this agent will continue to work without modification. When the optional rephrase prompt is provided, the agent will use it to convert follow-up questions into standalone queries before retrieving relevant documents.

Testing:

• Verified the rephrase prompt works correctly when provided, converting follow-up questions into standalone queries
• Tested with Custom Tools that have returnDirect: true and confirmed that outputs (including UI elements like buttons) appear immediately in the chat interface
• Verified backward compatibility with existing chatflows

How to apply: To update your current Conversational Retrieval Tool Agent to this new one, you must:

1. Start your flowise instance
2. Delete your current Conversational Retrieval Tool Agent (store the prompt somewhere)
3. Rebuild and restart your flowise instance
4. Add the new Conversational Retrieval Tool Agent node (under the same name)
5. That's it ! You now have upgraded your Conversational Retrieval Tool Agent.

@HenryHengZJ and @niztal, I took the liberty of adding my name after yours (@niztal ) in the node, hope you don't mind :) Thanks for building this node btw!

[niztal]

@nikitas-novatix thanks, I don't mind you added your name as author, enjoy 🥂

Can you tell what use cases are you using for this agent?

[Nek-11]

@nikitas-novatix thanks, I don't mind you added your name as author, enjoy 🥂

Can you tell what use cases are you using for this agent?

Thanks :)

Sure ! I'm using it to develop a chatbot on a large public website with lots of webpages, PDFs and other documents.

The reason why I added the rephrase prompt is because the performance is better with a rephrased query, and this way I can choose how the original question gets rephrased, which adds a lot of flexibility.

[Nek-11]

Hey @HenryHengZJ

Any possibility of integrating this? Tested extensively with no failures. Given the number of people I saw on Discord asking for a "node that does what Conversational Retrieval QA Chain does, but with tools", this change could help many users I think!

[Nek-11]

@HenryHengZJ Just a ping for this simple feature, I know you've been busy lately with the new version and login so maybe you haven't had time for this yet, but it should be fairly straightforward :)

[Nek-11]

Pinging this once more just in case

[Blecoeur]

Thanks for the PR, that would be life changing for me :)

[niztal]

@HenryHengZJ I see many people need this PR to be merged, can you please do so?

Thanks 🙏

[Nek-11]

@HenryHengZJ Another ping for this PR 🙏

[niztal]

@HenryHengZJ Another ping for this PR 🙏

Unfortunately @HenryHengZJ doesn't reply I also tried to contact him via discord 😞

[Nek-11]

Please @HenryHengZJ this is an old PR very easy to merge, affects only community nodes, and has no conflict. What's the problem with merging it?

@chungyau97 @vinodkiran I see you are the top contributors. Do you also merge PRs ?

Thanks!!

[Nek-11]

@HenryHengZJ I will continue to ping this until I have a valid answer as to why it's not getting implemented :')