Enhanced server.json validation (phase 1)

[BobDickinson]

Added schema validation and support for exhaustive and more detailed validation to existing validators. Added new mcp-publisher validate command. No change to any existing tests or behavior other than the new command.

Motivation and Context

Many currently published servers fail schema validation. Of those that pass, many have semantic/logical errors or other errors that make them impossible to configure, or their configuration when applied doesn't generate correct MCP server configuration (per their own documentation).

To address this, we need better tooling to validate servers and to inform server publishers of errors, issues, and best practices in a way that is clear and actionable, both during server development and at time of publishing.

I have started a discussion about the broader topic: https://github.com/modelcontextprotocol/registry/discussions/635

There is also a fairly detailed document describing the design, architecture, implementation, future plans, etc. See Enhanced Validation Design

This PR is the first step in the process of improving validation. It adds schema validation and updates all existing validators to use a new model that allows them to track context and return rich and exhaustive results. This has been done in a way that is backward compatible (ValidateServerJSON is unchanged as are all existing validation unit tests).

There is a new mcp-publisher command called validate that is currenty the only place that schema validation and enhanced (exhaustive/rich) validation results are exposed.

Changes

Internal Validation improvements

• Schema validation has been implemented and is available as an option in validation methods (defaults to off)
• Existing validators track the JSON path (context) of the attribute they're evaluating, passing it to any child validators
• Existing validators are exhaustive (return all issues, not failing fast on the first error)
• Existing validators return rich validation results, including:
  • type (schema, semantic, linter)
  • severity (error, warn, info)
  • path (JSON path to server.json element)
  • reference (to the schema element or rule that triggered the result)
  • message (currently the same as previous error message)

A new validate command has been added to mcp-publisher so publishers can evaluate their server.json before publishing

• Includes new schema validation and previous semantic validation
• Shows full details of all issues encountered during validation

Usage

Given a server.json that looks like this:

{
  "$schema": "https://static.modelcontextprotocol.io/schemas/2025-09-29/server.schema.json",
  "name": "io.github.BobDickinson/registry",
  "description": "An MCP server that provides [describe what your server does]",
  "repository": {
    "url": "",
    "source": ""
  },
  "version": "=1.0.0",
  "packages": [
    {
      "registryType": "oci",
      "registryBaseUrl": "https://docker.io",
      "identifier": "registry",
      "version": "1.0.0",
      "transport": {
        "type": "sse"
      },
      "packageArguments": [
        {
          "type": "named",
          "name": "--mode",
          "description": "Operation mode",
          "format": "foo"
        }
      ],
      "environmentVariables": [
        {
          "description": "Your API key for the service",
          "isRequired": true,
          "format": "string",
          "isSecret": true,
          "name": "YOUR_API_KEY"
        }
      ]
    }
  ]
}

Run the command:

mcp-publisher validate server.json

Which will produce the output:

❌ Validation failed with 5 issue(s):

1. [error] packages.0.packageArguments.0.format (schema)
   value must be one of "string", "number", "boolean", "filepath"
   Reference: #/definitions/Input/properties/format/enum from: [#/definitions/ServerDetail]/properties/packages/items/[#/definitions/Package]/properties/packageArguments/items/[#/definitions/Argument]/else/[#/definitions/NamedArgument]/allOf/0/[#/definitions/InputWithVariables]/allOf/0/[#/definitions/Input]/properties/format/enum

2. [error] packages.0.transport (schema)
   missing required fields: 'url'
   Reference: #/definitions/SseTransport/required from: [#/definitions/ServerDetail]/properties/packages/items/[#/definitions/Package]/properties/transport/else/else/[#/definitions/SseTransport]/required

3. [error] repository.url (schema)
   '' has invalid format 'uri'
   Reference: #/definitions/Repository/properties/url/format from: [#/definitions/ServerDetail]/properties/repository/[#/definitions/Repository]/properties/url/format

4. [error] version (semantic)
   version must be a specific version, not a range: "=1.0.0"
   Reference: version-looks-like-range

5. [error] packages[0].transport.url (semantic)
   url is required for sse transport type
   Reference: streamable-transport-url-required

Error: validation failed

Note in the results above:

• All errors contain the JSON path of the triggering element from server.json.
• The first three errors are schema errors detected during schema validation. The Reference provided contains an absolute path to the schema rule that was violated, followed by the full schema path that enforced that rule (with $ref redirect values substited and enclosed in square brackets).
• The error messages provided by the existing semantic validators are exactly the same as the previous error messages those validators produced.

In the final solution we would remove semantic validators that are redundant to schema validation (as in number 5 above). We have the option to use the structured data produced by schema validation to replace the generic schema validation message with a more descriptive message if that is an issue (in particular, if the only argument for an ad-hoc validator is that is produces a better/cleaner message, we would just move that message creation code to the schema error result formatter and special case it as opposed to having a redundant validator).

What's Next

If this general direction is supported and this PR is accepted, a fast follow would include:

• Add new validators (infliuenced by observed issues) for both semantic errors and best practices
• Require passing schema validation (in addition to improved semantic validation) in order to publish
• Remove existing validation code that is redundant to schema validation (ensuring error messages remain intelligible)
• Update validation client code and tests to handle the new rich return types natively

Issues

Schema validation requires embedding the server.schema.json file into the validators package via go:embed, which is restricted from embedding files outside of the package. For this reason we copy the schema file into a schema subdir (via a prep target in the makefile) and we .gitignore it. I tried cleaning up the copy in the make clean target, but then the linter would complain about the embed target being missing if it wasn't there during development. I also considered just checking in the schema file and assuming a separate workflow for updating the embedded schema file, but it's not clear what that workflow would be or how it would be maintained. I'm open to a different approach, but the schema file does need to be embedded somehow for schema validation to work.

How Has This Been Tested?

All existing validation tests pass, new tests implemented and all pass.

Breaking Changes

No breaking changes, no behavior change except new validate command

Types of changes

• [ ] Bug fix (non-breaking change which fixes an issue)
• [X] New feature (non-breaking change which adds functionality)
• [ ] Breaking change (fix or feature that would cause existing functionality to change)
• [ ] Documentation update

Checklist

• [X] I have read the MCP Documentation
• [X] My code follows the repository's style guidelines
• [X] New and existing tests pass locally
• [X] I have added appropriate error handling
• [X] I have added or updated documentation as needed

[BobDickinson]

There is an issue to consider with this PR, which is that I am embedding schema.server.json and using it to determine the current version string. This is in conflict (potentially) with the hard coded model CurrentSchemaVersion and CurrentSchemaURL. If we go with the embedded schema approach and we have a workflow that assures the correct schema file is used, we should use that as the source of truth and remove the hardcoded values (IMO). Since they're used extremely widely throughout the codebase, I didn't feel comfortable making that change in this PR.

[tadasant]

@claude review this PR

[claude[bot]]

Claude encountered an error —— View job

Failed with exit code 128

---

I'll analyze this and get back to you.

[BobDickinson]

OK, I'm back to work on this. I think the first order of business is to determine what schema versions we want to validate and where we get them (this is also related to the workflow around the embedded schema file copied at build time). Cleary using the checked-in "draft" schema is not correct (and tests added since this PR are failing because they expect 2025-10-17).

In my TypeScript validator what I did was to make a GHA that runs daily and gets any new schema directories from modelcontextprotocol/static, copies them into my repo, and checks them in. The validator then runs with all published schemas and validates server.json against the schema that it specifies (with a warning if that is not the latest schema).

I think it would make sense to do the same thing here: use a GHA to sync official schema versions to the validator (checking them directly into /validators/schema). This allows us to validate against all available schemas (with a warning if not current schema), and eliminates the embedding workflow step and concerns (including cleanup/gitignore). I would suggest a README.md in the schema directory to explain why those files are there and how they got there. Does that sound OK?

Should we make the GHA run daily or on demand (meaning it needs to be run whenever a new schema version is published)?

Is there any use case where we'd want to also be able to validate against the draft schema (as generated from the OpenAPI spec)?

[tadasant]

use a GHA to sync official schema versions to the validator (checking them directly into /validators/schema).

We could potentially do this on the static repo, so instead of "daily" it's triggered by actual changes to the repo. Would that work? That would open a PR here that we could merge into a new release.

Is there any use case where we'd want to also be able to validate against the draft schema (as generated from the OpenAPI spec)?

I think we'd want this ability for development? e.g. we make a big change to a schema and we want to make sure our semantic validators are playing nicely with it before locking everything in.

[tadasant]

@BobDickinson I raised the GHA line of thinking with @modelcontextprotocol/registry-wg maintainers -- feedback was: can we avoid this kind of git-ops based workflow and instead just treat the public URLs as the canonical place where these live without trying to duplicate/sync them across repos.

That would mean we either on-the-fly pull the schemas for every validation we process, or we cache them on e.g. startup of the registry app. Seems reasonable and less complex than GHA flows I think?

[rdimitrov]

@BobDickinson I raised the GHA line of thinking with @modelcontextprotocol/registry-wg maintainers -- feedback was: can we avoid this kind of git-ops based workflow and instead just treat the public URLs as the canonical place where these live without trying to duplicate/sync them across repos.

That would mean we either on-the-fly pull the schemas for every validation we process, or we cache them on e.g. startup of the registry app. Seems reasonable and less complex than GHA flows I think?

@BobDickinson - hey, just to add to what @tadasant shared.

The idea is that we already have a const pointing to the current/latest schema supported within the registry server - https://github.com/modelcontextprotocol/registry/blob/fdcb38537fa08e3392630e38a3d59cac51407b04/pkg/model/constants.go#L41

Instead of ensuring a replicated state between the two repositories (static & registry), can we use that constant that points to the latest schema and initialise the schema validator by leveraging it?

[BobDickinson]

I implemented the git-ops workflow before I saw this feedback and committed it because it was working and I was at a good checkpoint. I think the GHA could be improved (triggered by the commit of a new schema in static, as @tadasant suggested).

One note is that I want the validator to be able to know if a schema version is valid and to be able to validate non-current schemas (I think there are several uses cases for this), so this means we need all of the schemas from the static site, not just the current one.

On the question of git-ops versus dynamic/runtime schema download, I think the real question is, do we want the schemas to be built-in to the applications versus dynamically fetched at runtime.

Dynamically fetching the schemas has the following properties:

• It's significantly more complex (the downloading, caching, error handling, etc, is all extra code to write and maintain, above and beyond the current schema management code, which is very simple, fast, and literally incapable of failing)
• Will be somewhat slower (even if fetching schemas on demand, we still have to download the schema before we verify it). This not a concern for something like the server, as it's long running and we can cache schemas, but it might be a concern for the mcp-publish command (which can't really cache and will have to get the schema being published/validated every time) - if all schema validation, including mcp-publish, goes to using a live endpoint (instead of being done locally in the command code) this would be mitigated
• Will introduce a new potential failure mode (couldn't download schema) in the REST API and mcp-publish
• Will require both the server and the mcp-publish tool to be able to access the open internet at runtime - this is a big deal in an enterprise environment (more so for the server) - having a server that has to phone home means that when deploying in an enterprise you have extra security work to do (get an exception from your security team, punch a hole in the firewall, etc) - my recommendation is if you don't absolutely have to phone home, you should avoid it, especially if you think the product will be used internally in an enterprise (as I think this code will)
• On the plus side, both the server and mcp-publish could potentially support new, dynamic schema versions without updating the software (assuming we also had a dynamic method of determining current schema, such as most recent schema available)

Anyway, I'd personally like to avoid the dynamic downloading approach for now - maybe reconsider when we have a final resolution on the upstream schema sourcing (ServerCard or otherwise). But if there's strong consensus that it needs to be dynamic, I can work on it.

[tadasant]

Will require both the server and the mcp-publish tool to be able to access the open internet at runtime - this is a big deal in an enterprise environment (more so for the server) - having a server that has to phone home means that when deploying in an enterprise you have extra security work to do (get an exception from your security team, punch a hole in the firewall, etc) - my recommendation is if you don't absolutely have to phone home, you should avoid it, especially if you think the product will be used internally in an enterprise (as I think this code will)

This is pretty compelling to me. And because these versions are immutable (i.e. after we publish 2025-10-17, we'll never change it), I'm not that concerned about getting confused about drift / having more than one canonical version. So I'm back in favor of a git-ops flow. Though I still think it'd be cleaner to put the ops on the static repo so that we're not doing a time-based daily thing (and instead can trigger on any push to main on the static repo).