pydantic-xml icon indicating copy to clipboard operation
pydantic-xml copied to clipboard

Inconsistent XmlEntityInfo Handling with Multiple Annotated Metadata

Open lclbm opened this issue 7 months ago • 7 comments

from typing import Annotated

from pydantic import Field
from pydantic_xml import BaseXmlModel, attr

from pydantic.fields import FieldInfo
from pydantic_xml.fields import XmlEntityInfo


class Node(BaseXmlModel):
    name1: Annotated[str, attr(serialization_alias="A"), Field(default="1")]
    name2: Annotated[str, attr(serialization_alias="A"), attr(default="5")]
    name3: Annotated[str, attr(serialization_alias="A")] = attr(default="3")
    name4: Annotated[str, attr(serialization_alias="A")]


assert all(isinstance(Node.model_fields[f"name{i}"], FieldInfo) for i in range(1, 4))
assert isinstance(Node.model_fields["name4"], XmlEntityInfo)
...

pydantic==2.11.5 pydantic-xml==2.17.1

Hi @dapper91 , I've encountered an issue with Annotated metadata handling in pydantic-xml. Regardless of whether multiple metadata entries are FieldInfo, XmlEntityInfo, or a mix of both, the presence of more than one metadata annotation consistently causes XmlEntityInfo to be incorrectly overridden by FieldInfo in attribute field definitions.

When a field uses Annotated with a single attr metadata, Pydantic's merge_field_infos correctly returns a copy of XmlEntityInfo. However, when a field contains two or more metadata entries, merge_field_infos erroneously returns Pydantic's standard FieldInfo type instead. This causes XML attribute fields to be incorrectly typed when multiple metadata annotations are present.

https://github.com/pydantic/pydantic/blob/v2.11.5/pydantic/fields.py#L471

lclbm avatar Jun 23 '25 01:06 lclbm

pydantic 2.11.7 pydantic-core 2.33.2 pydantic-xml 2.17.2

I've verified this behavior persists after upgrading both pydantic and pydantic-xml to their latest versions.

lclbm avatar Jun 23 '25 01:06 lclbm

@lclbm Hi,

Due to pydantic internal implementation of annotations handling (pydantic substitutes XmlEntityInfo by its own implementation - FieldInfo), the only way to work with multiple annotations right now is to use default values from pydantic-xml (attr, element), like this:

class Node(BaseXmlModel):
    name1: Annotated[str, attr(serialization_alias="A"), Field(default="1")] = attr()
    name2: Annotated[str, attr(serialization_alias="A"), attr(default="5")] = attr()
    name3: Annotated[str, attr(serialization_alias="A")] = attr(default="3")
    name4: Annotated[str, attr(serialization_alias="A")]  # this works because if a single annotation is found, field info kept as is: https://github.com/pydantic/pydantic/blob/v2.11.5/pydantic/fields.py#L479

dapper91 avatar Jun 23 '25 15:06 dapper91

Thank you very much for your explanation. I had the same understanding before - it seems the only solution is to report this issue to the pydantic team. I wonder if there are any good improvement approaches?

Speaking of which, I'd like to ask: In what scenario is the XmlEntityInfo.merge_field_infos method actually used? During my breakpoint debugging, I never saw execution enter this function. I'd greatly appreciate it if you could clarify this confusion for me. Thank you!

https://github.com/dapper91/pydantic-xml/blob/e5f21f5a7b3a88369a0ff70be799854dbb6650fe/pydantic_xml/fields.py#L40-L73

lclbm avatar Jun 24 '25 00:06 lclbm

That's weird... According to my experience XmlEntityInfo.merge_field_infos called four times, once for each field.

dapper91 avatar Jun 24 '25 17:06 dapper91

Hi, dapper91 When I downgraded pydantic from version 2.11.7 to 2.10.0, the XmlEntityInfo.merge_field_infos method executed successfully, indicating that pydantic modified the invocation requirements for merge_field_infos—now apparently allowing it to be called only from pd.FieldInfo—which suggests pydantic reduced support for custom extensions.

pydantic 2.10.0 pydantic-core 2.27.0 pydantic-xml 2.17.2

from typing import Annotated

from pydantic import Field
from pydantic_xml import BaseXmlModel, attr
from pydantic.fields import FieldInfo
from pydantic_xml.fields import XmlEntityInfo

f = XmlEntityInfo.merge_field_infos

IS_EXECUTED = False


def _merge_field_infos(*field_infos, **overrides):
    global IS_EXECUTED
    IS_EXECUTED = True

    print("merging field infos:", field_infos, overrides)
    return f(*field_infos, **overrides)


XmlEntityInfo.merge_field_infos = _merge_field_infos


class Node(BaseXmlModel):
    name1: Annotated[str, attr(serialization_alias="A"), Field(default="1")]
    name2: Annotated[str, attr(serialization_alias="A"), attr(default="5")]
    name3: Annotated[str, attr(serialization_alias="A")] = attr(default="3")
    name4: Annotated[str, attr(serialization_alias="A")]


assert all(isinstance(Node.model_fields[f"name{i}"], FieldInfo) for i in range(1, 4))
assert isinstance(Node.model_fields["name4"], XmlEntityInfo)
assert IS_EXECUTED, "merge_field_infos should be executed"

v2.11.7 from_annotated_attribute https://github.com/pydantic/pydantic/blob/910bc54b6d5c05e4eabb4a9076def9e7ed23d38a/pydantic/fields.py#L376-L383

v2.10.0 from_annotated_attribute https://github.com/pydantic/pydantic/blob/5f033e46c54fea1b59b6894d6527daf49475e690/pydantic/fields.py#L419-L434

lclbm avatar Jun 27 '25 06:06 lclbm

I've found the pydantic maintainers' response acknowledging significant architectural changes. I'm curious whether better approaches might exist to support both custom FieldInfo implementations with multiple annotations and annotation-based field assignments.

https://github.com/pydantic/pydantic/issues/11461#issuecomment-2668062467

lclbm avatar Jun 27 '25 08:06 lclbm

Thanks. I will try to research it, but right now I can't see any other way to achieve that except explicitly defining the field default value.

dapper91 avatar Jun 29 '25 13:06 dapper91