Incorrect Tool Recognition with Multiple Tool Invocation Using Openrouter Models

[XiaoLey]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.13

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

I'm encountering an issue with the Openrouter models when invoking multiple tools via agents. The problem manifests as tool name concatenation errors. This means, when attempting to call multiple tools sequentially or simultaneously, the tool names get concatenated or wrongly identified. However, when calling a single tool, it appears to work correctly.

1. Configure the agent to use Openrouter's model.
2. Attempt to invoke multiple tools, such as current_time and searxng_search.
3. Observe the tool recognition error, where the tool names appear concatenated or incorrectly identified (e.g., current_time;searxng_search).

✔️ Expected Behavior

The agent should correctly identify and invoke each tool individually and sequentially without concatenation or misidentification of tool names.

❌ Actual Behavior

No response

[dosubot[bot]]

To resolve the issue of tool name concatenation errors when invoking multiple tools via agents in Dify version 0.6.13, self-hosted using Docker, you need to ensure that each tool is correctly identified and invoked individually. The ToolEngine class's agent_invoke method is responsible for handling tool invocations.

Here is a snippet of the agent_invoke method that shows how the tool is invoked and how errors are handled:

class ToolEngine:
    """
    Tool runtime engine take care of the tool executions.
    """
    @staticmethod
    def agent_invoke(
        tool: Tool, tool_parameters: Union[str, dict],
        user_id: str, tenant_id: str, message: Message, invoke_from: InvokeFrom,
        agent_tool_callback: DifyAgentCallbackHandler,
        trace_manager: Optional[TraceQueueManager] = None
    ) -> tuple[str, list[tuple[MessageFile, bool]], ToolInvokeMeta]:
        """
        Agent invokes the tool with the given arguments.
        """
        # check if arguments is a string
        if isinstance(tool_parameters, str):
            # check if this tool has only one parameter
            parameters = [
                parameter for parameter in tool.get_runtime_parameters() 
                if parameter.form == ToolParameter.ToolParameterForm.LLM
            ]
            if parameters and len(parameters) == 1:
                tool_parameters = {
                    parameters[0].name: tool_parameters
                }
            else:
                raise ValueError(f"tool_parameters should be a dict, but got a string: {tool_parameters}")

        # invoke the tool
        try:
            # hit the callback handler
            agent_tool_callback.on_tool_start(
                tool_name=tool.identity.name, 
                tool_inputs=tool_parameters
            )

            meta, response = ToolEngine._invoke(tool, tool_parameters, user_id)
            response = ToolFileMessageTransformer.transform_tool_invoke_messages(
                messages=response, 
                user_id=user_id, 
                tenant_id=tenant_id, 
                conversation_id=message.conversation_id
            )

            return plain_text, message_files, meta
        except ToolProviderCredentialValidationError as e:
            error_response = "Please check your tool provider credentials"
            agent_tool_callback.on_tool_error(e)
        except (
            ToolNotFoundError, ToolNotSupportedError, ToolProviderNotFoundError
        ) as e:
            error_response = f"there is not a tool named {tool.identity.name}"
            agent_tool_callback.on_tool_error(e)
        except (
            ToolParameterValidationError
        ) as e:
            error_response = f"tool parameters validation error: {e}, please check your tool parameters"
            agent_tool_callback.on_tool_error(e)
        except ToolInvokeError as e:
            error_response = f"tool invoke error: {e}"
            agent_tool_callback.on_tool_error(e)
        except ToolEngineInvokeError as e:
            meta = e.args[0]
            error_response = f"tool invoke error: {meta.error}"
            agent_tool_callback.on_tool_error(e)
            return error_response, [], meta
        except Exception as e:
            error_response = f"unknown error: {e}"
            agent_tool_callback.on_tool_error(e)

        return error_response, [], ToolInvokeMeta.error_instance(error_response)

Additionally, the DifyAgentCallbackHandler class provides methods to handle tool start, end, and error events, which can help in debugging and ensuring that each tool is correctly identified and invoked individually. Here is a relevant snippet from the DifyAgentCallbackHandler class:

class DifyAgentCallbackHandler(BaseModel):
    """Callback Handler that prints to std out."""
    color: Optional[str] = ''
    current_loop: int = 1

    def __init__(self, color: Optional[str] = None) -> None:
        super().__init__()
        """Initialize callback handler."""
        # use a specific color if not specified
        self.color = color or 'green'
        self.current_loop = 1

    def on_tool_start(
        self,
        tool_name: str,
        tool_inputs: dict[str, Any],
    ) -> None:
        """Handle tool start event."""
        print_text("\n[on_tool_start] ToolCall:" + tool_name + "\n" + str(tool_inputs) + "\n", color=self.color)

    def on_tool_end(
        self,
        tool_name: str,
        tool_inputs: dict[str, Any],
        tool_outputs: str,
        message_id: Optional[str] = None,
        timer: Optional[Any] = None,
        trace_manager: Optional[TraceQueueManager] = None
    ) -> None:
        """Handle tool end event."""
        print_text("\n[on_tool_end]\n", color=self.color)
        print_text("Tool: " + tool_name + "\n", color=self.color)
        print_text("Inputs: " + str(tool_inputs) + "\n", color=self.color)
        print_text("Outputs: " + str(tool_outputs)[:1000] + "\n", color=self.color)
        print_text("\n")

        if trace_manager:
            trace_manager.add_trace_task(
                TraceTask(
                    TraceTaskName.TOOL_TRACE,
                    message_id=message_id,
                    tool_name=tool_name,
                    tool_inputs=tool_inputs,
                    tool_outputs=tool_outputs,
                    timer=timer,
                )
            )

    def on_tool_error(
        self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any
    ) -> None:
        """Handle tool error event."""
        print_text("\n[on_tool_error] Error: " + str(error) + "\n", color='red')

To ensure that each tool is correctly identified and invoked individually, you should:

1. Check Tool Parameters: Ensure that tool_parameters is correctly formatted as a dictionary if the tool requires multiple parameters.
2. Callback Handling: Use the DifyAgentCallbackHandler to handle the start, end, and error events for each tool invocation.
3. Error Handling: Properly handle different types of errors such as ToolProviderCredentialValidationError, ToolNotFoundError, ToolParameterValidationError, etc., to ensure that the correct tool is being invoked and any issues are reported accurately.
4. Agent Logs: Utilize the get_agent_logs method from the AgentService class to debug and verify the tool invocations and their parameters.

By following these steps, you can resolve the issue of tool name concatenation errors and ensure that each tool is correctly identified and invoked individually ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[XiaoLey]

[XiaoLey]

Based on my debugging tracking, I found that the Openrouter's model actually has two tool calls responding, but only one is recorded

[XiaoLey]

I don't have time to continue tracking breakpoints. I hope someone familiar with this area can fix this bug. I really appreciate it!