claude-code icon indicating copy to clipboard operation
claude-code copied to clipboard

[BUG] plugin-dev:agent-development skill documents wrong tools format (JSON array vs comma-separated)

Open kylesnowschwartz opened this issue 2 months ago • 4 comments

Preflight Checklist

  • [x] I have searched existing issues and this hasn't been reported yet
  • [x] This is a single bug report (please file separate reports for different bugs)
  • [x] I am using the latest version of Claude Code

What's Wrong?

The plugin-dev:agent-development skill teaches the wrong format for the tools field in agent frontmatter.

The skill documents:

tools: ["Read", "Write", "Grep", "Bash"]

The official Claude Code documentation says:

tools: Read, Write, Grep, Bash

Source: https://code.claude.com/docs/en/sub-agents.md

The tools field in agent frontmatter is optional and accepts a comma-separated list format

The plugin-dev skill explicitly states the opposite:

Format: Array of tool names

tools: ["Read", "Write", "Grep", "Bash"]

This causes agents created using the plugin-dev skill to have no access to any tools - the JSON array syntax is parsed as a literal string rather than individual tool names.

What Should Happen?

The plugin-dev:agent-development skill should document the correct comma-separated format:

# Correct
tools: Read, Grep, Glob, Bash

# Wrong (parsed as literal string, agent gets NO tools)
tools: ["Read", "Grep", "Glob", "Bash"]

Error Messages/Logs

No error - agents silently get zero tools because the format is wrong. The JSON array is parsed as a single string value rather than a list of tool names.

Steps to Reproduce

  1. Run skill: plugin-dev:agent-development
  2. Follow the documented format for tools
  3. Create an agent with tools: ["Read", "Grep"]
  4. Invoke the agent
  5. Agent has access to none of the specified tools

Claude Model

Opus 4.5

Is this a regression?

I don't know

Claude Code Version

Latest

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

Ghostty

Additional Information

Location of incorrect documentation: plugins/plugin-dev/skills/agent-development/agent-development.md

Sections that need correction:

  • The "tools (optional)" section under "Frontmatter Fields"
  • The "Common tool sets" examples
  • The "Quick Reference" section

All instances show JSON array syntax when it should be comma-separated.

kylesnowschwartz avatar Dec 07 '25 20:12 kylesnowschwartz

cc @dhollman

kylesnowschwartz avatar Dec 07 '25 20:12 kylesnowschwartz

This deserves more attention. Here's my take, anyway.

Just spent time debugging this exact issue (I know, I should have caught it...).

My subagents were outputting tool calls as literal XML text instead of executing them:

<function_calls>...</function_calls>

Root cause: The YAML parser treats tools: ["Read", "Write"] as a single string literal ["Read", "Write"] rather than parsing it as an array. The agent receives zero tools because the string doesn't match any valid tool name.

The fix: Change from JSON array to comma-separated:

Broken (agent gets 0 tools)

tools: ["Read", "Write", "Edit", "Grep"]

Working (agent gets 4 tools)

tools: Read, Write, Edit, Grep

Scope of incorrect examples in plugin-dev:

  • SKILL.md line 357: documents ["Read", "Grep"] as the format
  • complete-agent-examples.md: 4 examples with wrong format
  • agent-creation-prompt.md: 2 examples with wrong format
  • plugin-validator.md, skill-reviewer.md, agent-creator.md: all use wrong format

26 total occurrences in plugin-dev alone. Anyone using these examples to create agents will have silent tool failures.

Impact: No error message, no warning - agents just silently fail to execute tools. Very difficult to diagnose without knowing the exact parser behavior.

This is more than a documentation problem. When the documentation is the instructions for an agent the impact is FAR greater.

lisaross avatar Dec 28 '25 21:12 lisaross

There are many many fallacies in the official plugin development skills repos from Anthropic unfortunately.

I've been using my own fork of https://github.com/sjnims/plugin-dev (cc @sjnims) instead which has heaps of corrections.

I with the good folks at Anthropic would focus more on this area to ensure the world stops burning tokens on hallucinated instructions in the official repos!

kylesnowschwartz avatar Dec 28 '25 21:12 kylesnowschwartz

I’m working on a Claude Code plugin automated testing framework that should help plugin developers ensure that their plugins do actually trigger appropriately. It uses the Claude Agent SDK (typescript version) and the Anthropic SDK to simulate conversations with the Agent SDK. The Claude Agent SDK has the ability to load local plugins, thus simulating Claude Code with your plugin loaded. My hope is that we can use the testing framework to iterate on plugin agent/command/skill/hook trigger keywords. Check it out if you want, it’s still pre v1.0 so might be rough around the edges. I purposely leave linting to external tools, there’s enough out there that can do it better and I’d rather not re-invent the wheel. https://github.com/sjnims/claude-code-plugin-eval

sjnims avatar Dec 29 '25 02:12 sjnims