Improve 3.1+ schema `$id` UX

[handrews]

Currently, we have the following schema document $ids for OAS 3.1 (shown relative to a base URI of https://https://spec.openapis.org/oas/3.1/, and without the trailing date segment that all schemas need to support updates:

• schema # Validates everything except the Schema Object, which is only validated against type: [boolean, object]
• schema-base # Extends schema to validate Schema Objects against dialect/base and require both $schema and jsonSchemaDialect to be set to schema's $id if they are present
• dialect/base # A dialect equivalent to the default 2020-12 dialect plus the OAS extensions
• meta/base # The vocabulary metaschema for the OAS extensions

schema-base was named to indicate that it incorporates dialect/base. No one is certain why the dialect and vocabulary are referred to as "base", but they are called that in the OAS text as well. I vaguely recall discussing the dialect as a base for further OAS-relevant extensions? Perhaps that is what the intent was.

AFAICT, the advantage of schema is that it's self-contained. But giving it that name means that it is the schema that most people assume is what they want, yet it really does not produce the behavior they'd expect.

schema-base is closer to what people would expect, but is (in my view) overly restrictive in locking $schema to dialect/base, as it is valid to switch metaschemas if $schema appears alongside $id, and conforming JSON Schema implementations know to treat $id-containing subschemas as embedded resources and check for a changed metaschema. However, this does not make schema-base wrong: Users might indeed want to enforce using only dialect/base, and this is the correct way to do that.

On the other hand (and contrary to some of my comments earlier on Slack), schema-base locking jsonSchemaDialect to dialect/base is probably correct. If you want to set jsonSchemaDialect to something else, you would want a schema that validates Scheam Objects accordingly by default.

---

We probably shouldn't change how schema behaves, but we should make it more obvious that it is incomplete. We should also come up with some sort of intuitive name for a schema that is like schema-base but does not restrict $schema. This name needs to convey that most users will want this new schema, and not the plain schema.

Ideally, I'd prefer to make copies of the vocabulary and dialect and give them the URIs dialect/oas and meta/oas (3.1 is already present earlier in the URI), but that is probably more trouble than it is worth given that I imagine very few people pay attention to those other than to just load them correctly when using schema-base.

The important thing is to have an intuitively-named schema that performs the default accepted Schema Object validation but still allows switching metaschemas on embedded resources. That is the default behavior of the OAS, and it's not currently provided.

[fpapon]

Hi,

I'm sure to understand which url to use to validate the 3.1.0 and the 3.1.1 version of the specification. Why using the date in the url instead of the last version number?

https://spec.openapis.org/oas/3.0/schema/2024-10-18 => https://spec.openapis.org/oas/3.0.0/schema

I am asking because in the openapi.json files, we are using the full version number so it's easier to find the right jsonschema to use.

[handrews]

@fpapon There are no schema-visible differences between patch releases. That is part of the definition of a patch release. The dates allow us to make bug fixes and other improvements. But those have nothing to do with patch releases. All 3.0 schemas apply to 3.0.0, 3.0.1, 3.0.2, 3.0.3, 3.0.4, etc., and all 3.1 schemas apply to 3.1.0, 3.1.1, 3.1.2 (forthcoming), etc. but the most recently dated one is the most accurate.

Again, there is no difference among patch releases that can be reflected in a schema.

[handrews]

@fpapon but see also #4877.

[rmannibucau]

@handrews while I fully get what you mean I think François has two points:

• there is no real point in not releasing the schema with the spec so at some point the versioning should be 1-1 and it will help end users to not have to fight to find the schema (it is "common" and "natural")
• the date in the current url is misleading and implies a double versioning unrelated to versions so it is very awckward even if I get from where it comes

So overall I think it should be fixed, even if it is just to get a better "documentation" wording in another patch release

[ralfhandl]

@rmannibucau

there is no real point in not releasing the schema with the spec so at some point the versioning should be 1-1

We actually have eight iterations of the v3.1 schemas on https://spec.openapis.org/#openapi-specification-schemas while the v3.1 spec only has reached patch version 1, on average four schema patches per spec patch.

Once we publish spec v3.1.2 and a subsequent schema iteration, the ratio will drop to 3-1, still way off 1-1.

the date in the current url is misleading and implies a double versioning unrelated to versions

Schema fixes ("iterations") can and do happen more often than spec fixes ("patch versions"), so they are versioned differently.

We could have used sequence numbers for the iterations instead of dates, so the schemas would be 3.1-0, ..., 3.1-7. Not sure whether that would be less misleading.

[rmannibucau]

@ralfhandl

We actually have eight iterations of the v3.1 schemas on https://spec.openapis.org/#openapi-specification-schemas while the v3.1 spec only has reached patch version 1, on average four schema patches per spec patch.

which is an issue

it means specifying a spec version doesn't target a schema and a schema doesn't align on a spec version.

By itself no issue until you can only specify either the spec version or the schema iteration in the spec document which is the case today.

I clearly vote to not enable to specify both but to just do a minor with everything every time it happens, it is not a crazy work and will help everyone.

Side note: if the answer is "you set the version and the $schema" this is just not true cause schema are often injected or handled transversally so we literally have only the spec version to deduce the schema to use and changes like uri/uri-reference are impacting, even in patch releases and totally missed today (jsonSchemaDialect as an example).

So long story short: I think schema should be versionned 1-1 with the spec since they are part of it from an user (and integrator) standpoint.

[fpapon]

Hi, My concern was, in a openapi.json we can have:

{
  "openapi": "3.1.0"
}

which should match on 1-1 version of the jsonSchema, this is used by editors to automatically run the json validation without having the jsonSchema referenced in the document.

{
  "$schema": "https://spec.openapis.org/oas/3.1/schema/2025-02-13"
  "openapi": "3.1.0"
}

So here, I don't know which jsonSchema matched the 3.1.0 version. As I understand on this thread, the latest 3.1 jsonSchema should match all 3.1.x version but why not having just one version 3.1 or follow the incremental patch version?

I saw that the rule of matching 3.1.x in the jsonSchema checking that compatibility:

    "openapi": {
      "type": "string",
      "pattern": "^3\\.1\\.\\d+(-.+)?$"
    },

So maybe publishing only the latest spec should be enough and easier to use:

{
  "$schema": "https://spec.openapis.org/oas/3.1/schema
  "openapi": "3.1.0"
}

[ralfhandl]

@rmannibucau

I think schema should be versionned 1-1 with the spec since they are part of it from an user (and integrator) standpoint.

That would be a possibility going forward.

It would mean that all schema fixes after publication of a spec patch version are "parked" in the corresponding spec minor-version dev branch (for example v3.1-dev) until a new patch version for that minor version is published.

[ralfhandl]

@fpapon Not sure what the JSON examples should convey, $id is not a valid field in the root object of an OpenAPI Document.

publishing only the latest spec should be enough

We do not want to change a published schema „in place“, so a new schema iteration needs a new retrieval URI and a new $id value.

We would like to offer a convenience URI for each spec minor version that redirects (responds with HTTP 307 Temporary Redirect) to the latest published iteration, see

• #4152

[rmannibucau]

think std publication is fine:

• 3.x.y/...
• 3.x/...

[fpapon]

@fpapon Not sure what the JSON examples should convey, $id is not a valid field in the root object of an OpenAPI Document.

Sorry I fixed my comment, it's $schema.

publishing only the latest spec should be enough

We do not want to change a published schema „in place“, so a new schema iteration needs a new retrieval URI and a new $id value.

We would like to offer a convenience URI for each spec minor version that redirects (responds with HTTP 307 Temporary Redirect) to the latest published iteration, see

* [Make the "latest" schema accessible programmatically #4152](https://github.com/OAI/OpenAPI-Specification/issues/4152)

[ralfhandl]

Sorry I fixed my comment, it's $schema.

Well, $schema isn't allowed either next to openapi in the document root.

I know that some editors/IDEs use this to reference a JSON Schema file for validation, but OpenAPI doesn't support this convention.

[rmannibucau]

@ralfhandl i get what you mean but it also means openapi doesn't have any schema since it is a requirement even if almost all tools support other configurations for convenience -> https://json-schema.org/draft/2020-12/draft-bhutton-json-schema-00#rfc.section.8.1.1 , anyway this part is less important since all tools will tolerate this additional keyword

[handrews]

@rmannibucau the JSON Schema section that you linked to has nothing to do with using $schema in non-JSON Schema documents, which has never been part of the JSON Schema specification.

[handrews]

@rmannibucau also, there is no need for such a usage of $schema because the openapi field gives you the major and minor version, which is sufficient to locate the most up-to-date schema.

[karenetheridge]

This confusion with dates and versions has been a concern of mine as well. It's exacerbated by the fact that schemas aren't released at the same time as a specification release - historically they have come weeks or months later, even though the specification isn't generally usable without having the schemas in hand (implementations are often based around the schemas, and users will want to have the new schemas as well to make sure they're correctly using any new features or fixes).

Recognizing that point releases of the specification are meant to be fully backwards-compatible, and therefore the latest version of the schemas in the (for example) 3.1 series should be fully usable with any 3.1.x implementation, this is not actually true in reality because the reverse is not true: OpenAPI documents written for a particular point release or schema version are not backwards compatible with earlier schemas in the series. jsonSchemaDialect is an example of this: it is now permitted to be a uri-reference, not a uri, so such a uri-reference in an OAD written with the latest schema in hand will be fine, but it cannot be validated with earlier schemas in the 3.1 series; consequently implementations using an earlier version of the schema will not accept this OAD.

This is not a problem that the user can mitigate in any way, nor signal to the implementation what minimum schema version it is expecting, because (as just described above, while I was writing this long reply), the only such signals are:

• openapi: indicates the version of the OpenAPI specification that it is targeting (which as said above, can span many different schema iterations);
• jsonSchemaDialect: indicates the identifier of the JSON Schema version to use for embedded JSON Schemas (which is not helpful for versioning of the OAD as a whole).

However, if specification releases and schema releases were always done together, then the user could correctly indicate in their OAD with the openapi field the information: "this document is expecting the implementation to understand the syntax defined by OAS version 3.1.5, for both specification and schemas". Then the implementation can at least provide a helpful error that this version is beyond its understanding, or perhaps attempt to download new versions from the openapis.org website.

(And if this doesn't convince you, ask yourself: If indeed point releases are truly insignificant, then why is the openapi field specified to always include the point release, rather than ONLY EVER containing the major.minor version? It's not even optional to include the point version -- it's MANDATORY!)

I find it quite frustrating that there is resistance to multiple instances of users coming forward indicating that this is actually a real problem in the real world, and being rebuffed because of a belief that "it shouldn't matter, therefore it doesn't matter".

[handrews]

jsonSchemaDialect is an example of this: it is now permitted to be a uri-reference, not a uri, so such a uri-reference in an OAD written with the latest schema in hand will be fine, but it cannot be validated with earlier schemas in the 3.1 series; consequently implementations using an earlier version of the schema will not accept this OAD.

I don't see why implementations shouldn't let you say which schema to use if you want to manage it on that level.

why is the openapi field specified to always include the point release, rather than ONLY EVER containing the major.minor version?

It shouldn't in my view, but that discussion stalemated and we don't have a proper decision on it so we're continuing by inertia, which bothers me. See #4233.

We continuously tell people that it doesn't matter and then ask them to specify it. I think that is the root of so many UX problems with OpenAPI and I wish we'd fix it.

[I'm moslty not trying to argue what the schema policy should be, just explaining what it is, and I wanted to comment on the patch release in the openapi field thing because it really bothers me.]

[handrews]

@ralfhandl

We do not want to change a published schema „in place“, so a new schema iteration needs a new retrieval URI and a new $id value.

Yeah this is where I'm confused. If we had per-patch-release schemas we'd need to have a whole line of dated schemas for each patch release, with the schemas with the same date being identical across all patch releases. What does that get us? How is that not more confusing?

Alternatively, would we really want eight patch releases just for schema fixes, which is the only way I can think of to avoid that problem?

Really, I think the fundamental problem is putting the patch release in the openapi field. My vote is still to fix that instead.

[mehulchauhan]

what is alternative to

yaml-language-server: $schema=https://raw.githubusercontent.com/OAI/OpenAPI-Specification/main/schemas/v3.1/schema.json

[rmannibucau]

@handrews you mix things up, and yes $schema is the standard way to référence a schema - on the phone, maybe I mixed links but it is like that since 10+ years. Yes openapi does have pr does not have to référence it depending the tooling, this is why I said having it or not in the schema is not a topic. But also to référence a schema automatically (tool) or implicitly + deterministically you must either be explicit - where $schema topic comes from - or implicitly, ie with openapi version. When do you think the new "all in one release" approach can be available please?

[handrews]

@rmannibucau I wrote/maintained musch of the JSON Schema specification. I am quite certain that $schema to reference a schema from an instance is not part of the JSON Schema specification.

[rmannibucau]

@handrews yep, doesn't change any of the points or the outcome, does it? Do you need help to set it up to make it Moving forward? Happy to help if needed

[karenetheridge]

I don't see why implementations shouldn't let you say which schema to use if you want to manage it on that level.

Assuming you mean "..but not in the OAD itself":

• that forces this information to be sent out-of-band somehow (a configuration parameter to the application? a change in code somewhere?), which is an extra burden on the user
• the user has to know what schema they need to use, which is hard enough for us insiders that I think it would be nearly impossible to the point of incredible frustration for any user, because (drum roll) there are no unique versions assigned to schema releases and no release announcements made for them. This forces users to download every iteration of the timestamped schemas to figure out which one actually works, and then write down "this date is required for this change, and that date is required for that change".

All of this would go away if the user could just say in the document "this document was written with release 3.1.25 in mind" and then it's implicit that if only schema version 3.1.24 is available, it might not work. And we already have a field for that, and it doesn't even need changing: that's the openapi field at the very top level of the document (https://spec.openapis.org/oas/latest#fixed-fields).

(In my implementation, how I'd handle it is to throw an error explaining clearly that the user asked for a version of the schema that I don't have bundled, which is a signal to the user that either a) they have an old version of the implementation and they need to upgrade, or b) they should file a bug report because the author (moi) hasn't pushed a new release yet that bundles the latest schemas.)

Yeah this is where I'm confused. If we had per-patch-release schemas we'd need to have a whole line of dated schemas for each patch release, with the schemas with the same date being identical across all patch releases. What does that get us? How is that not more confusing?

Because: 1. the version can actually appear in the OAD itself, and 2. there would be a release announcement for it, allowing us to communicate the potentially impactful changes.

Alternatively, would we really want eight patch releases just for schema fixes?

So what? What's wrong with having a series of 3.1.1 through 3.1.8, each with their own small release notes on a 3.1 changelog page? There's nothing wrong with iterating quickly -- and it demonstrates to the outside world that something is still happening and the OpenAPI project is still alive. We like automation, right? If we're already up to pushing point releases like 3.1.1 with minor clarifications in them, why can't we do that for schema changes too? If it's a tiny fix, I don't think it would be a big deal to hold back the release so the cadence is no more than monthly, but since we love automation, the release process is not a huge hassle, right? Right? (padme and anakin)

[karenetheridge]

@mehulchauhan: you should use the schemas as identified here: https://spec.openapis.org/#openapi-specification-schemas -- specifically, the URIs found in the $id property in each of those files. Whatever OpenAPI tool you are using should have these URIs documented and ideally bundle the files themselves right into the application so they don't need to be downloaded from the network.

[rmannibucau]

If I summarize facts:

1. Schema are either deduced from the content (implicitly) or explicit ($schema in most editors, even in yaml but in a comment) -> issue is there is not 1-1 relationship between an openapi version and schema (see 2.)
2. Determinism and compatible between schema révision is not guaranteed today (even if desired but it already broke) -> need to link explicitly either one release with a schema. Note that "latest" link matches only one not enterprise friendly case - same you pinpoint docker images at sha level for ex.

So easiest for everyone is to release schema with openapi release and do a new one when at least one of both change. Alternative - enable to explicit the schema and require it etc... - are not as nice for end users.

Can we work towards that goal?

[ralfhandl]

@rmannibucau

compatible between schema révision is not guaranteed today (even if desired but it already broke)

With which schema iteration did that happen?

And what changed incompatibly?

[rmannibucau]

Side note: if the answer is "you set the version and the $schema" this is just not true cause schema are often injected or handled transversally so we literally have only the spec version to deduce the schema to use and changes like uri/uri-reference are impacting, even in patch releases and totally missed today (jsonSchemaDialect as an example).

between 3.1.0 and 3.1.1 IIRC - didn't dig more to see if there was others. The trick there is that it looks like opening so backward compat but it is not true since a working 3.1.0 impl will totally break with 3.1.1. cause it doesn't support relative refs

[karenetheridge]

The trick there is that it looks like opening so backward compat but it is not true since a working 3.1.0 impl will totally break with 3.1.1. cause it doesn't support relative refs

Yes exactly -- except the schemas that make these changes aren't synchronized with 3.1.1, but rather came after 3.1.1 and before 3.1.2 (which hasn't been released yet).

[handrews]

[EDIT: discussion of the real-world jsonSchemaDialect example ended up elsewhere without me quite realizing it was in the wrong place. TL;DR: I see it as supporting the current policy, see also this later comment in this issue.]

so we literally have only the spec version to deduce the schema to use

There is no deducing. The schema to use is the latest schema for that minor release line. If you have a choice, it is always the latest on the minor release line. The only reason the schemas with earlier dates are left published rather than taken down (and why we put dates in the $id so that they can be left published) is because it's possible that an error in an older schema could theoretically incorrectly allow an OAD with an error, and someone may depend on that error. So we keep them published so folks can use them until they can fix their OADs and use the correct (latest) schema.

except the schemas that make these changes aren't synchronized with 3.1.1, but rather came after 3.1.1 and before 3.1.2 (which hasn't been released yet).

There is no "synchronizing". 3.1.2 replaces 3.1.1, which replaces 3.1.0. They are not distinct versions the way 3.1.x is distinct from 3.0.x. It's like HTTP RFCs. While an HTTP implementation does need to know whether to use HTTP/1.0, HTTP/1.1, HTTP/2, or HTTP/3 (distinct versions), if it is implementing HTTP/1.1 does not care about RFC2068 vs RFC2616 vs RFC7230 (and friends) vs RFC9112 (and friends). Those RFCs replace each other. All HTTP/1.1 implementations are expected to follow RFC9112, even if they were first written at the time of RFC2068. There is no way in HTTP to say "I'm using the RFC2616 version of HTTP/1.1." That's not a thing. OAS point releases are the same.

(I am aware of some RFCs that obsolete older RFCs in problematic ways, and there may even be examples in the HTTP RFCs, but no system is perfect and that is how it is intended to work. If your HTTP client connects to a server that talks HTTP/1.1, there is no way to tell or set the RFC involved.)

[karenetheridge]

There is no deducing. The schema to use is the latest schema for that minor release line.

This is another case of theory bumping up against reality. "The schema to use is the latest schema for the release line" sounds simple, but it isn't. If the implementation doesn't have the latest schemas built into it (or rather, it doesn't have a recent enough schema to properly interpret the OAD that the user has provided), things break. This is not theory. This is reality. The user cannot simply cause the implementation into having the latest schema through wishing or praying. This is something that needs to be set up in advance.

I have tried, multiple times, to describe a real world example where this can happen, and how it can be made better. How can I explain it better?