[FEATURE] Agent tools instructions

[Saicheg]

Feature Area

Agent capabilities

Is your feature request related to a an existing bug? Please link it here.

NA

Describe the solution you'd like

Yesterday, during a live demo by @vladeziegler on AI Agent Week—Portugal, I realized that he had the same problem as I had multiple times.

The idea is that we have an Agent with a role, goal, and backstory. The agent also has tools that are semantically correct to assign to it.

The problem is that you often want to instruct agents on how to use some tools. But semantically, it is not clear where to do that. Usually your best options are task description itself of backstory.

The task option does not sound very good since tasks are detached from agents. The backstory also looks semantically incorrect since it must be some story about an agent and not a specific tool.

So I propose to create an additional level of abstractions to make things look better:

from crew import ToolWithInstruction
from crewai_tools import ScrapeWebsiteTool

tool_with_instruction = ToolWithInstruction(
  ScrapeWebsiteTool(),
  """
  ALWAYS use this tool when making a joke.
  NEVER use this tool when making joke about someones mom.
  """
)

agent = Agent(
           role="Comedian",
           goal="Create hilarious and engaging jokes",
           backstory="""
                 You are a professional stand-up comedian with years of experience in crafting jokes.
                  You have a great sense of humor and can create jokes about any topic while keeping them appropriate and entertaining.
           """,
            tools=[tool_with_instruction],
        )

So now we will somehow include these instructions when prompting LLM with tool and from framework it looks consistent.

Describe alternatives you've considered

They way i am doing it right now is:

from crewai_tools import ScrapeWebsiteTool

agent = Agent(
           role="Comedian",
           goal="Create hilarious and engaging jokes",
           backstory="""
                 You are a professional stand-up comedian with years of experience in crafting jokes.
                 You have a great sense of humor and can create jokes about any topic while keeping them appropriate and entertaining.
                 ALWAYS use ScrapeWebsiteTool tool when making a joke.
                 NEVER use ScrapeWebsiteTool when making joke about someones mom.
           """,
            tools=[ScrapeWebsiteTool()],
        )

It does a job, but it always feels a bit wrong to put such things in the backstory. With multiple tools assigned it also looks messy.

Additional context

No response

Willingness to Contribute

Yes, I'd be happy to submit a pull request

[Vidit-Ostwal]

I believe these instructions would be more appropriately configured at the agent level rather than the tool level

[Saicheg]

I believe these instructions would be more appropriately configured at the agent level rather than the tool level

I like it when instructions and tools for them live under the same roof. Anyway, with current configurations for agents, it is not clear semantically where such instructions should be put.

[Vidit-Ostwal]

I believe these instructions would be more appropriately configured at the agent level rather than the tool level

I like it when instructions and tools for them live under the same roof. Anyway, with current configurations for agents, it is not clear semantically where such instructions should be put.

What if you have two tools that are shared among several agents, and you want each agent to use these tools in different scenarios, as you've described? Defining usage rules at the tool level would limit flexibility, whereas setting them at the agent level would allow for more adaptable and situation-specific usage.

[Saicheg]

@Vidit-Ostwal nothing keeps you from using two different agents with different instructions. You might be missing the concept. Here is an example:

from crew import ToolWithInstruction
from crewai_tools import ScrapeWebsiteTool

tool = ScrapeWebsiteTool()
tool_with_always_scrape = ToolWithInstruction(tool, "ALWAYS use this tool when URL if mentioned")
tool_wtih_conditiona_scrape = ToolWithInstruction(tool, "Use this tool for scraping only if URL contains https://www.crewai.com/")

agent1 = Agent(tools=[tool_with_always_scrape], ...)
agent2 = Agent(tools=[tool_wtih_conditiona_scrape], ...)

[Vidit-Ostwal]

Agreed, you can definitely initialise two different class objects.

[mouramax]

I think if adding specific instructions for tools really makes sense within CrewAI's overall design, it might be a bit more elegant to just have a new, optional instructions attribute directly in the BaseTool class. This attribute would be checked, and if it had something, it could be processed, maybe in the _generate_description method, leading to more customized descriptions. That way, all tools inheriting from BaseTool would automatically benefit from this feature.

Honestly, though, I'm not really sold on this idea myself. I lean towards the belief that when I give my Agent tools, it's my job to guide how they're used through clear instructions in the Task.description. After all, that's where all the details on how the task should be carried out belong, including every little interaction with the environment – and in my view, interacting with the environment is exactly what tools are for.

So, long story short, my opinion is that, on a basic level, just the tool's description should be enough for the task (and in that case, you don't need to do anything extra – just pass the tool, and the framework handles the rest). But, if I need to steer the use of tools more precisely, that's really just a prompt engineering challenge within the task description itself.

[Saicheg]

I think if adding specific instructions for tools really makes sense within CrewAI's overall design, it might be a bit more elegant to just have a new, optional instructions attribute directly in the BaseTool class. This attribute would be checked, and if it had something, it could be processed, maybe in the _generate_description method, leading to more customized descriptions. That way, all tools inheriting from BaseTool would automatically benefit from this feature.

Honestly, though, I'm not really sold on this idea myself. I lean towards the belief that when I give my Agent tools, it's my job to guide how they're used through clear instructions in the Task.description. After all, that's where all the details on how the task should be carried out belong, including every little interaction with the environment – and in my view, interacting with the environment is exactly what tools are for.

So, long story short, my opinion is that, on a basic level, just the tool's description should be enough for the task (and in that case, you don't need to do anything extra – just pass the tool, and the framework handles the rest). But, if I need to steer the use of tools more precisely, that's really just a prompt engineering challenge within the task description itself.

Hey @mouramax, huge thanks for your thoughts. There are several comments from my side:

• You are saying it's our job to describe how tools should be used in task descriptions. The problem is that tools are not linked to tasks, but to agents. One task can be shared among agents. For example for crew with a manager who work on a task collectively. This means the description should go to the agent for me.

• I agree that tool instructions are just prompt engineering, but to be honest, everything here is prompt engineering. CrewAI gives us a nice framework to think of agents to be defined with role, goal, and backstory. Trying to write instructions for tools on one of these buckets is possible, but conceptually, it always feels wrong for me.

• Having instructions on BaseTool can work, but only if you can have them as part of initializer.

from crewai_tools import ScrapeWebsiteTool

tool = ScrapeWebsiteTool(instructions="ALWAYS do this. Never do this")

Totally works for me.

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[Saicheg]

Still waiting from somone from core team to give their opinion!

[mouramax]

Hey, since this Issue is still Open, I figured I'd also mention that you can just tweak your tool's description. Simply tack on the extra info you need, and you'll get the same net result — your Agent gets the additional prompt:

from crewai_tools import ScrapeWebsiteTool

tool = ScrapeWebsiteTool()

print(f"{tool.description=}")

# | Output:                                                                 |
# | -------                                                                 |
# | tool.description="Tool Name: Read website content\nTool Arguments:      |
# | {'website_url': {'description': 'Mandatory website url to read the      |
# | file', 'type': 'str'}}\nTool Description: A tool that can be used to    |
# | read a website content."                                                |

instructions = " / Use this tool for when you're utterly lost."
tool.description += instructions

print(f"{tool.description=}")

# | Output:                                                                 |
# | -------                                                                 |
# | tool.description="Tool Name: Read website content\nTool Arguments:      |
# | {'website_url': {'description': 'Mandatory website url to read the      |
# | file', 'type': 'str'}}\nTool Description: A tool that can be used to    |
# | read a website content. / Use this tool for when you're utterly lost."  |

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.