danielgtaylor
Add support for pydantic dataclasses
Pydantic dataclasses are a drop-in replacement for builtin dataclasses to offer a stronger type validation. This PR adds support for generating pydantic dataclasses instead of the builtin variant.
This enable usage of protobuf classes in API endpoint (like FastAPI) which supports parsing pydantic objects but not builtin dataclasses.
I am pitching this as an additional feature, feel free to let me know what you think. I will maintain my own fork otherwise; I do feel like this is a trivial improvement to implement.
edit It was rather foolish to think it was trivial, it is not;
I'll put as draft for now. Some issues to note:
- Does not play well with oneOf
- ForwardRefs need to be updated
Right now the imports are done through the template, but it might be cleaner to do the pydantic imports through the builtin import system.
ForwardRefs are an easy fix, oneOf can be partially achieved through validators and overriding __post_init__ to set to None the fields that aren't set (otherwise a validation error from pydantic occurs)
Plan for the oneOf (with pydantic dataclasses):
- Set each field part of a group to
Optional - Set the
message_fieldoptionalattribute toTrue - Add a pydantic validator (for the entire dataclass) that checks that at least one of the field is defined
@root_validator()
def check_oneof(cls, values):
meta = cls._betterproto_meta.oneof_field_by_group
for group, field_set in meta.items():
if not any([values[field.name] is not None for field in field_set]):
raise ValueError(f"Group {group} has no value; all fields are None")
return values
... that checks that at least one of the field is defined
Changed the validator to ensure that exactly one field is defined.
Further tasks
- [x] Only add the call to update the forward refs if the message has other messages as fields
- [x] Add tests
Hey @danielgtaylor. I've completed these changes a while back. This code has been in production at Garda World for well over two months now without any specific incident. I just completed the unit test portion for windows (generation part). Let me know if this is something you are interested in having in the mainline version of python-betterproto. I did not add any specific test to protobuf but instead relied on use-cases; all previous tests were also ported so they are running on the protobuf version (insuring no regression)
@Kludex Ill finish the requested changes today :) We are actively using this at Garda so I have allocated time to finish this feature.
Thinking a bit ahead... Would there be a way to create validators and not lose them when regenerating again?
Thinking a bit ahead... Would there be a way to create validators and not lose them when regenerating again?
Ah! Great question. I personally think this is out of scope for a project like this, since the goal is to translate protobuf to python. The validation is enforced by the underlying protobuf model since this is a 1:1 translation (mainly to enforce types and groups). If I had other business logic to validate, I would add methods dynamically in a __init__.py file to perform the pydantic validation.
@Gobot1234 Fixed all suggestions and added text in the README. Double checking that everything still works since I merged w/master.
@Gobot1234 All is good on my side; the merge w/master did not affect anything. If this ever need maintenance or issues pop-up, feel free to @ me and I'll be right there.