Add ContextRouter

[cmsparks]

I've expanded the scope of this PR per discussion w/ @threepointone on MCP Routers. This PR adds the ContextRouter interface, which does the following:

• Exposes a listTools and listResources function to filter tools and resources
• Exposes a systemPrompt function, which composes the tool, resource, and MCP state into an LLM consumable system prompt. This is meant to be a snippet which you can include in your existing system prompt, rather than a standalone prompt.
• Exposes a setMessages() method, which updates the ContextRouter

I have purposely excluded prompts and resource templates from the ContextRouter interface because those features are not intended to be model controlled. Tools are intended to be LLM controlled, and resources may be LLM controlled.

Given this interface, I've implemented two ContextRouters:

• BaseContextRouter, which preserves the same functionality as before (no filtering of tools) and provides an additional system prompt
• LLMContextRouter, which lets an LLM filter tools and resources using tool calling.

Here is a sample agent using the LLMContextRouter:

export class MyAgent extends Agent<Env, never> {
  private contextRouter;
  private model;
  constructor(ctx: DurableObjectState, env: Env) {
    super(ctx, env);
    const ai = createWorkersAI({
      binding: env.AI,
    });
    this.model = ai("@cf/meta/llama-4-scout-17b-16e-instruct")
    this.contextRouter = new LLMContextRouter(this.model);
    this.mcp = new MCPClientManager("my-agent", "1.0.0", this.contextRouter);
  }

  async generateResponse(message: CoreMessage[]) {
    await this.contextRouter.setMessages(messages);

    const res = await generateText({
      model,
      tools: this.mcp.unstable_getAITools()
      // ...
    })
  }
}

[changeset-bot[bot]]

🦋 Changeset detected

Latest commit: c8f0e8c8ccc651dd9e80d685f887d7cdccb02254

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
agents	Patch
hono-agents	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[mattzcarey]

I like this approach in that you can expose some very clean functionality for the user. I dont like that you are hardcoding a workflow to create an agent (llm + set some active tools). ie,

1. start the contextmanager.
2. it starts servers
3. it picks tools/resources to activate.

imo the agent should be free to create itself over time and we should accomodate. It should have a tool which can start server, stop server and the tools should be updated for the next step.

[cmsparks]

imo the agent should be free to create itself over time and we should accomodate. It should have a tool which can start server, stop server and the tools should be updated for the next step.

The ContextRouter itself is more of a way for users to define their own ways of filtering tools/resources. This could be an LLM workflow to filter tools (LLMContextRouter included in this PR), well defined state machine, or agentic tool inclusion like you describe.

This design doesn't preclude implementing a ContextRouter that exposes filtering tools to the Agent. For example, to support the usecase you describe, something like should work:

class ToolContextRouter extends BaseContextRouter {
  private activeServers = []
  
  listTools() {
      // filter list tools to only include active servers
      const filteredConnections = {}
      for (const serverId in activeServers) {
        filteredConnections[serverId] = this.connectionManager[serverId]
      }
      
      return getNamespacedData(filteredConnections, "tools");
  }

  // returns tools of activeServers and two utility tools to add/remove servers from context
  getAITools() {
    return {
      ...super.getAITools(),
      addMcpServer: {
          // adds a server to this.activeServers
      },
      removeMcpServer: {
          // removes a server from this.activeServers
      }
    }
  }
  
  systemPrompt() {
    // describes the servers that are accessible and how to use addMcpServer/removeMcpServer
  }
}

This is an abbreviated implementation, but I think I'll play around with adding something like the above as an included ContextRouter implementation in this PR like you describe.

[cmsparks]

Gonna close this for now bc this is probably stale and I probably should rethink the implementation anyways