Proposal: decouple the JSON schema representation

[findleyr]

As I go through our API for v1.0.0, and in light of a few recent issues in google/jsonschema-go, I wonder if it would be wise to decouple the API of the SDK from the jsonschema-go library. I am loath to change anything at this point, but after experimenting with this, the impact of this breaking change on server authors would be negligible, and the impact on clients would be non-trivial but also beneficial.

Here are my concerns / arguments:

1. It's always uncomfortable to put another module's types in our API, even if we maintain that module.
2. jsonschema-go has gotten relatively less traction than the SDK, and is therefore less mature. For example, hash of unhashable type main.ID (struct containing slice) google/jsonschema-go#26 revealed an API oversight that would have been painful to live with if we'd cut v1.0.0 of the SDK.
3. The meaning of schema fields is under-specified and evolving. For example, consider SEP-834 Support full JSON Schema 2020-12 modelcontextprotocol#834. Fortunately, that issue proposes standardising on the 2020-12 schema version, which is also the default for the jsonschema-go library. But what if an alternate proposal were to be approved?
4. The jsonschema-go library does not support all drafts, and our tight coupling to a particular representation means that our clients can't talk to certain servers (see support http://json-schema.org/draft-07/schema# google/jsonschema-go#6), or Handle incompatible schemas (e.g. required of type []string) from older jsonschema drafts #455.

All of these suggest that going to v1.0.0 with a tight coupling to the google/jsonschema-go library may be a risk.

As an alternative, I tried two experiments, which you can see in draft PRs:

1. Use a proxy package for the jsonschema API (jsonschema: experiment with a proxy package #515). This technically broke the dependency, but was horrendously ugly and didn't address concerns 2-4 above.
2. Just set schema fields to any, with the contract that they must be JSON-marshallable (mcp: allow any value for schemas #517). Internally, we need to convert these to a jsonschema.Schema in order to do validation, but that becomes an implementation detail.

After investigating, (2) looks rather attractive. It completely breaks the dependency in the API, and requires effectively no change for server code (observe that almost no tests or examples had to be updated), since custom schemas can still be set on the Tool as before. It will break clients that were inspecting the schema, but were any clients doing that? On the other hand, it means that clients will be able to talk to any server, even if we can't unmarshal its schemas to a jsonschema.Schema.

The downside of this change would be that the schema fields are less self-documenting, with less type safety.

CC @jba @samthanawalla @jpmcb @rsc @hyangah @mazas-google

[rsc]

It seems like the answer depends on how much the library is using the schemas versus just passing them along. I would have naively expected that if I write an MCP server that presents a tool with a given schema, that the MCP library would check for me that any tool calls conform to the schema (and send back an error if not), all before invoking my actual tool code. And similarly it would be good for the MCP library to check that the tool's responses conform to the promised schema. Are there no such validation checks? If there are no such checks, then I suppose it doesn't matter if these fields become 'any'. But if there are validation checks, then how do those get implemented without reference to a specific jsonschema library?

On the flip side, as an MCP client, it would be good to check outgoing tool calls and incoming tool responses, but there it is less important, since the MCP server is the one publishing the schema, so it's not a statement of what the client needs or expects.

[findleyr]

@rsc there are such validation checks on the server (not on the client). We still use google/jsonschema-go, but only internally for validation. To get it into the right representation, when you add a tool with AddTool, we would marshal the schema as JSON and unmarshal it into a jsonschema.Schema (and then resolve references in that schema). The point is that this becomes an implementation detail, rather than an attribute of the API, and if we wanted to internally use a different library, or different types depending on the $schema field, we could do that. The compatibility requirement would be at the JSON level, which makes sense to me.

You might note the double marshaling as inefficient, but consider that we can actually keep the json.RawMessage of the schema around for serving tools/list requests, thereby saving a marshal on every request, and the unmarshaling into jsonschema.Schema only occurs when the tool is bound. Additionally, if the provided schema happens to already be a *jsonschema.Schema, we can just use it directly.

So there would still be an API constraint for servers that the schema you provide in Tool.InputSchema is something we can understand, but that constraint is not enforced by the representation as a jsonschema.Schema, which has implications for forward compatibility. It also allows people to use a different library entirely for constructing their schema, if they want. This also allows users to work around issues like #434.

If our users don't provide a schema at all, and instead rely entirely on inference, we will use google/jsonschema-go for inference.

On the client side this change is beneficial because there is an asymmetry: we can easily enforce that servers use the 2020-12 draft, but clients need to be able to talk to arbitrary servers, many of which use draft-07. Decoupling the types allows those clients to work.

[rsc]

Sounds good to me.

[slacaze]

Just one datapoint, I currently depend on the ability do to something like this (pseudo code):

type MyInputType struct{
    SomeField string `json:"some_field" jsonschema:"some description"
}

someDescriptionOverride := "some other description"

inputSchema, _ := jsonschema.For[MyInputType](&jsonschema.ForOptions{})

inputSchema.Properties["some_field"].Description = someDescriptionOverride

&mcp.Tool{
    InputSchema: inputSchema,
    ...
}

Which is to say, while I have schema elements set at compile time, I have a runtime ability to change them (for reasons :p).

I think you're proposing is that InputSchema is not just a *jsonschema.Schema, but an any so that we can pass anything that can serialize to JSON.

In that sense, I don't think I would get affected, and I think the proposal makes sense.

However, I wonder if you could be more explicit, and not use any, but instead json.Marshaller or something like that.

Essentially, detecting at compile time that if you pass (I'm being a bit silly) chan struct{} to InputSchema, it bombs here, not at runtime.

[dpasiukevich]

not use any, but instead json.Marshaller

+1 to explicitly declaring what is expected from the user, if that's possible. Especially as this sdk does operate on the field and not just passes it as opaque object.

[findleyr]

In that sense, I don't think I would get affected, and I think the proposal makes sense.

Yes, that wouldn't be affected. The contract is that you can pass anything that serializes as a JSON schema. In fact, I expect that using json.RawSchemajson.RawMessage will be a use case, when people are forwarding schemas from one system to another.

+1 to explicitly declaring what is expected from the user, if that's possible. Especially as this sdk does operate on the field and not just passes it as opaque object.

We'll document that the field must serialize to a JSON schema, but restricting it to json.Marshaller would be too much. IMO people should be able to pass map[string]any.

In mcp.AddTool, the SDK will panic if it can't understand the schema (by unmarshalling it to the internal jsonschema.Schema representation).

[jba]

json.RawSchema

json.RawMessage

[findleyr]

I think google/jsonschema-go#33 is yet another reason to do this: the user needed to work around schema problems, and was able to do this with mark3labs/mcp-go thanks to the RawInputSchema field, which would be the equivalent of setting input schema to json.RawMessage(...).