✨ Add `json_schema_extra` and deprecate `schema_extra`

[KimigaiiWuyi]

According to what this link mentions,

PydanticV1's Field class supported passing arbitrary keyword arguments to the JSON schema. In SQLModel counterpart, it was supported by passing schema_extra (see this).

PydanticV2 still supports passing extra things to the JSON schema but in a slightly different way. One has to pass a dictionary in the json_schema_extra argument (see this).

Field no longer supports arbitrary keyword arguments to be added to the JSON schema. Instead, any extra data you want to add to the JSON schema should be passed as a dictionary to the json_schema_extra keyword argument.

So I modified the Field's parameters to be compatible and consistent with the Pydantic v2 parameters, and for backward compatibility, I didn't remove the schema_extra field, which is logically compatible with both the schema_extra and json_schema_extra passes, and schema_extra should be deprecated at some point in the future. field at some point in the future, but for now schema_extra should be retained.

This Pull Reqeust is easy and simple, and easy to verify.

[Kaplas85]

Any news about this PR?

[ndeybach]

Any possibility of a merge @tiangolo ? I would also be interested in this support being added / long term.

[paulomtts]

Please merge this.

[arnoldasjan]

can you merge this?

[KimigaiiWuyi]

@KimigaiiWuyi, thanks for your interest and efforts!

Currently people use schema_extra as a workaround to pass parameters to Field that are accepted by Pydantic's Field, but not accepted by SQLModel's Field.

As an example:

class A(SQLModel):
    id_: int = Field(schema_extra={"validation_alias": "id"})

With pure Pydantic you would just pass it directly:

class A(BaseModel):
    id_: int = Field(validation_alias="id")

But SQLModel's Field doesn't have such parameter and we need to use that workaround.

Your implementation treats schema_extra as a duplicate of json_schema_extra and overrides it if both are present. But it may lead to issues in this case (stupid example just to demonstrate the idea):

class A(SQLModel):
    id_: int = Field(schema_extra={"validation_alias": "id"}, json_schema_extra={"type": "number"})

In the example above, the validation_alias will be ignored with your implementation.

In this PR we can just modify it to add json_schema_extra to json_extra:

    current_schema_extra = schema_extra or {}
    if json_schema_extra:
        current_schema_extra["json_schema_extra"] = json_schema_extra

In long perspective we need to deprecate schema_extra parameter and introduce something like pd_extra_params (Pydantic extra parameters) parameter so that people would be able to pass parameters not accepted by SQLModel's Field

Also, we need to add test to verify that new json_schema_extra field works (fields are added to json schema)

Indeed, you are correct. Although this situation is rare, it must be taken into consideration.

[github-actions[bot]]

As this PR has been waiting for the original user for a while but seems to be inactive, it's now going to be closed. But if there's anyone interested, feel free to create a new PR.

[YuriiMotov]

PR was closed due to inactivity for 30 days..

@KimigaiiWuyi, would you like to continue working on this? I can re-open it

[TATOAO]

This has already become a kind of bug now,

For the current version: sqlmodel==0.0.24

from sqlmodel import SQLModel, Field
class TestField(SQLModel):
    name: str = Field(max_length=100)
    
if __name__ == "__main__":
    print(TestField.model_fields['name'].json_schema_extra)
    # print None

Since this is legal, printed None. The model_fields actually accepts the attribute "json_schema_extra"

But,

class TestField(SQLModel):
    name: str = Field(max_length=100, json_schema_extra={"is_index": True})

will raise error:

        name: str = Field(max_length=100, json_schema_extra={"is_index": True})
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   TypeError: Field() got an unexpected keyword argument 'json_schema_extra'

And

class TestField(SQLModel):
   name: str = Field(max_length=100,  schema_extra={"is_index": True})

Even though it is legal, I don't see a way to retrieve the information from schema_extra={"is_index": True}.

Therefore, please consider applying @KimigaiiWuyi 's PR.

[KimigaiiWuyi]

PR was closed due to inactivity for 30 days..

@KimigaiiWuyi, would you like to continue working on this? I can re-open it

yes, I can handle it. Help me reopen it. Should I add test cases for it?

[YuriiMotov]

Yes, I think we need to add some tests. And, we need to resolve this: https://github.com/fastapi/sqlmodel/pull/1035#issuecomment-3216354850

Indeed, you are correct. Although this situation is rare, it must be taken into consideration.

Actually, I think we should deprecate schema_extra and introduce separate pydantic_kwargs and json_schema_extra. I started working on this, but struggle to find time to continue this work. You can take a look at: https://github.com/fastapi/sqlmodel/compare/main...YuriiMotov:sqlmodel:refactor-schema-extra

[github-actions[bot]]

📝 Docs preview

Last commit 249ad47368b6bde33b63240742582affc6692b25 at: https://5ef99040.sqlmodel.pages.dev

[KimigaiiWuyi]

Alright, I've now added json_schema_extra and pydantic_kwargs while retaining schema_extra, and also included unit tests.

When handling schema_extra, I maintained maximum compatibility. I retrieved possible parameters from PydanticField. If a parameter in schema_extra exists among them, it gets added to pydantic_kwargs. Otherwise, it's automatically added to json_schema_extra. I'm unsure if this approach is ideal, but it does preserve compatibility. Regardless, this parameter should be completely deprecated in a few versions.

FIELD_ACCEPTED_KWARGS = set(inspect_module.signature(PydanticField).parameters.keys())
if "schema_extra" in FIELD_ACCEPTED_KWARGS:
    FIELD_ACCEPTED_KWARGS.remove("schema_extra")

def Field(...):
    if current_schema_extra:
        for key, value in current_schema_extra.items():
            if key in FIELD_ACCEPTED_KWARGS:
                current_pydantic_kwargs[key] = value
            else:
                current_json_schema_extra[key] = value

If this approach is deemed inappropriate, please also suggest making schema_extra fully equivalent to pydantic_kwargs.

@YuriiMotov

[github-actions[bot]]

This pull request has a merge conflict that needs to be resolved.

[YuriiMotov]

I think we should change:

    if current_schema_extra:
        for key, value in current_schema_extra.items():
            if key in FIELD_ACCEPTED_KWARGS:
                current_pydantic_kwargs[key] = value
            else:
                current_json_schema_extra[key] = value

to

    if IS_PYDANTIC_V2:
        for key, value in current_schema_extra.items():
            if key in FIELD_ACCEPTED_KWARGS:
                current_pydantic_kwargs[key] = value
            else:
                current_json_schema_extra[key] = value
        current_pydantic_kwargs["json_schema_extra"] = current_json_schema_extra
    else:
        current_pydantic_kwargs.update(current_json_schema_extra)
        current_pydantic_kwargs.update(current_schema_extra)

And remove passing current_json_schema_extra to FieldInfo

[YuriiMotov]

We just discussed the idea of adding pydantic_kwargs parameter with Sebastian and he thinks we don't need to add it (we will later support passing all Pydantic-related parameters directly). So, we should remove pydantic_kwargs from the signature. Everything is the same - we add json_schema_extra, mark schema_extra as deprecated, but keep supporting it for some time

@KimigaiiWuyi, will you be able to work on this? Or, I can suggest changes

[KimigaiiWuyi]

我们刚刚和 Sebastian 讨论了添加参数的想法pydantic_kwargs，他认为我们不需要添加它（我们稍后会支持直接传递所有与 Pydantic 相关的参数）。所以，我们应该从签名中 删除它。 一切都一样——我们添加了它，标记为已弃用，但会继续支持一段时间。pydantic_kwargs``json_schema_extra``schema_extra

@KimigaiiWuyi你能帮我修改一下吗？或者我可以建议修改

Of course, but I don't know how you plan to support passing all Pydantic-related parameters directly. So I just need to brutally delete everything related to pydantic_kwargs? (Since you'll be adding it directly to the arguments later)

But I might need to come back tomorrow to finish it.

[YuriiMotov]

@KimigaiiWuyi, I suggested my way of fixing this, please review:

• Removed pydantic_kwargs parameter, updated texts that mentioned it
• Renamed pydantic_kwargs local var into field_info_kwargs as it's used in parallel PR we are working on
• Only show deprecation warning when deprecated schema_extra parameter is used - we can't raise RuntimeError as we don't support all Pydantic-related parameters yet
• Simplified logic of handling schema_extra with Pydantic V2 installed
• Conditionally use model_json_schema or schema in tests depending on Pydantic version
• Only test workaround with json_schema_extra being passed via schema_extra for Pydantic V2 - it doesn't make sense with Pydantic V1