powertools-lambda-python icon indicating copy to clipboard operation
powertools-lambda-python copied to clipboard
verified publisher icon aws-powertools

Bug: Possible regression in handling of Annotated[] fields in 3.22.1

Open chriselion opened this issue 2 months ago • 5 comments

Expected Behaviour

Previously (3.22.0 and earlier), annotating a field like

body: Annotated[MyRequest, Body],

would correctly validate the request body.

This might have been contrary to the recommended approach of using Body() (I'm not sure how it got so widespread in our codebase), but it did appear to work as intended.

Current Behaviour

Since 3.22.1, routes that use fields like this will return a 422, even if the body is valid for the specified class. The response message is of the form:

{"statusCode":422,"detail":[{"loc":["query","body"],"type":"missing"}]}

Code snippet

# repro.py
from typing import Annotated

import aws_lambda_powertools
from aws_lambda_powertools.event_handler import APIGatewayHttpResolver
from aws_lambda_powertools.event_handler.openapi.params import Body
from aws_lambda_powertools.utilities.typing import LambdaContext
import pydantic


class MyRequest(pydantic.BaseModel):
    foo: str
    bar: str = "bar2"


class MyResponse(pydantic.BaseModel):
    concatenated: str


app = APIGatewayHttpResolver(enable_validation=True)


# Endpoint using the Body class in the annotation
@app.patch("/test_class/")
def test_endpoint_class(
    body: Annotated[MyRequest, Body],
) -> MyResponse:
    return MyResponse(concatenated=(body.foo + body.bar))


# Endpoint using a Body instance in the annotation
@app.patch("/test_instance/")
def test_endpoint_instance(
    body: Annotated[MyRequest, Body()],
) -> MyResponse:
    return MyResponse(concatenated=(body.foo + body.bar))


def run_test_event(path):
    event = {
        "rawPath": path,
        "requestContext": {
            "http": {
                "method": "PATCH",
            },
            "stage": "$default",
        },
    }
    event["body"] = '{"foo": "foo1"}'

    try:
        response = app.resolve(event, LambdaContext())
        print(f"request to {path} returned status {response['statusCode']}")
        if response["statusCode"] == 422:
            print(f"  error: {response['body']}")
    except Exception as e:
        print(f"Raise an exception resolving event: {e}")


def main():
    print(f"Running on aws-lambda-powertools=={aws_lambda_powertools.__version__} pydantic=={pydantic.__version__}")
    run_test_event("/test_class/")
    run_test_event("/test_instance/")


if __name__ == "__main__":
    main()

Possible Solution

Currently the only workaround I have is to change occurrences of Annotated[..., Body] to Annotated[..., Body()]. However, this doesn't work with pydyantic>=2.12 and aws-lambda-powertools<=3.19.0 (approximately), I believe due to already logged issues about pydyantic 2.12

Steps to Reproduce

I used the following bash script to run the example code, in a local uv environment with python 3.12.0. I don't believe the python version is a factor, and it should reproduce in a normal virtual environment with pip instead of uv too.

# Run the above script for several versions of pydantic x aws-lambda-powertools
for pydanticVersion in 2.11.10 2.12.4; do
    uv pip install -q pydantic==${pydanticVersion}
    for powertoolsVersion in 3.19.0 3.21.0 3.22.0 3.22.1; do
        uv pip install -q aws-lambda-powertools==${powertoolsVersion}
        uv run python repro.py
        echo
    done
done

In particular, note in the output

Running on aws-lambda-powertools==3.22.0 pydantic==2.11.10
request to /test_class/ returned status 200
request to /test_instance/ returned status 200

Running on aws-lambda-powertools==3.22.1 pydantic==2.11.10
request to /test_class/ returned status 422
  error: {"statusCode":422,"detail":[{"loc":["query","body"],"type":"missing"}]}
request to /test_instance/ returned status 200

...

Running on aws-lambda-powertools==3.22.0 crequest to /test_class/ returned status 200
request to /test_instance/ returned status 200

Running on aws-lambda-powertools==3.22.1 pydantic==2.12.4
request to /test_class/ returned status 422
  error: {"statusCode":422,"detail":[{"loc":["query","body"],"type":"missing"}]}
request to /test_instance/ returned status 200

Which indicates that the Annotated[..., Body] type hint worked with multiple versions of pydantic in 3.22.0 but not 3.22.1

Notes the lambda function runtime and packaging format fields below aren't accurate - this was just run locally.

Powertools for AWS Lambda (Python) version

3.22.1

AWS Lambda function runtime

3.12

Packaging format used

PyPi

Debugging logs

Not sure these give anything useful, but here is the output for pydantic==2.12.4 and aws-lambda-powertools=={3.22.0, 3.22.1}

/Users/celion/code_clean/aws_powertools_repro/.venv/lib/python3.12/site-packages/aws_lambda_powertools/package_logger.py:20: UserWarning: POWERTOOLS_DEBUG environment variable is enabled. Setting logging level to DEBUG.
  if powertools_debug_is_set():
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /test_class/ and methods: PATCH
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /test_instance/ and methods: PATCH
Running on aws-lambda-powertools==3.22.0 pydantic==2.12.4
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIRequestValidationMiddleware object at 0x105d8c920>, <aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIResponseValidationMiddleware object at 0x105d8cbf0>]
2025-11-12 17:38:57,349 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIRequestValidationMiddleware] next call chain is OpenAPIRequestValidationMiddleware -> OpenAPIResponseValidationMiddleware
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIRequestValidationMiddleware handler
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIResponseValidationMiddleware] next call chain is OpenAPIResponseValidationMiddleware -> _registered_api_adapter
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIResponseValidationMiddleware handler
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> test_endpoint_class
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Calling API Route Handler: {'body': MyRequest(foo='foo1', bar='bar2')}
2025-11-12 17:38:57,350 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
request to /test_class/ returned status 200
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIRequestValidationMiddleware object at 0x105d8c920>, <aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIResponseValidationMiddleware object at 0x105d8cbf0>]
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIRequestValidationMiddleware] next call chain is OpenAPIRequestValidationMiddleware -> OpenAPIResponseValidationMiddleware
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIRequestValidationMiddleware handler
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIResponseValidationMiddleware] next call chain is OpenAPIResponseValidationMiddleware -> _registered_api_adapter
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIResponseValidationMiddleware handler
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> test_endpoint_instance
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Calling API Route Handler: {'body': MyRequest(foo='foo1', bar='bar2')}
2025-11-12 17:38:57,351 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
request to /test_instance/ returned status 200

/Users/celion/code_clean/aws_powertools_repro/.venv/lib/python3.12/site-packages/aws_lambda_powertools/package_logger.py:20: UserWarning: POWERTOOLS_DEBUG environment variable is enabled. Setting logging level to DEBUG.
  if powertools_debug_is_set():
2025-11-12 17:38:58,045 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /test_class/ and methods: PATCH
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Adding route using rule /test_instance/ and methods: PATCH
Running on aws-lambda-powertools==3.22.1 pydantic==2.12.4
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIRequestValidationMiddleware object at 0x103e3eff0>, <aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIResponseValidationMiddleware object at 0x103a45730>]
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIRequestValidationMiddleware] next call chain is OpenAPIRequestValidationMiddleware -> OpenAPIResponseValidationMiddleware
2025-11-12 17:38:58,046 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIRequestValidationMiddleware handler
request to /test_class/ returned status 422
  error: {"statusCode":422,"detail":[{"loc":["query","body"],"type":"missing"}]}
2025-11-12 17:38:58,047 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Converting event to API Gateway HTTP API contract
2025-11-12 17:38:58,047 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Found a registered route. Calling function
2025-11-12 17:38:58,047 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Building middleware stack: [<aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIRequestValidationMiddleware object at 0x103e3eff0>, <aws_lambda_powertools.event_handler.middlewares.openapi_validation.OpenAPIResponseValidationMiddleware object at 0x103a45730>]
2025-11-12 17:38:58,047 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIRequestValidationMiddleware] next call chain is OpenAPIRequestValidationMiddleware -> OpenAPIResponseValidationMiddleware
2025-11-12 17:38:58,047 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIRequestValidationMiddleware handler
2025-11-12 17:38:58,048 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [OpenAPIResponseValidationMiddleware] next call chain is OpenAPIResponseValidationMiddleware -> _registered_api_adapter
2025-11-12 17:38:58,048 aws_lambda_powertools.event_handler.middlewares.openapi_validation [DEBUG] OpenAPIResponseValidationMiddleware handler
2025-11-12 17:38:58,048 aws_lambda_powertools.event_handler.api_gateway [DEBUG] MiddlewareFrame: [_registered_api_adapter] next call chain is _registered_api_adapter -> test_endpoint_instance
2025-11-12 17:38:58,048 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Calling API Route Handler: {'body': MyRequest(foo='foo1', bar='bar2')}
2025-11-12 17:38:58,048 aws_lambda_powertools.event_handler.api_gateway [DEBUG] Simple response detected, serializing return before constructing final response
request to /test_instance/ returned status 200

chriselion avatar Nov 12 '25 23:11 chriselion

I'm fully prepared to accept that we were using Annotated incorrectly 😄

chriselion avatar Nov 13 '25 02:11 chriselion

Hello @chriselion ! Thanks for opening this issue. @leandrodamascena can you take a look, please?

anafalcao avatar Nov 13 '25 13:11 anafalcao

I set a breakpoint in get_field_info_annotated_type(), since that's where most of the changes for https://github.com/aws-powertools/powertools-lambda-python/pull/7609 were.

with 3.22.0

# start of get_field_info_annotated_type
annotation: typing.Annotated[__main__.MyRequest, <class 'aws_lambda_powertools.event_handler.openapi.params.Body'>]

# end of get_field_info_annotated_type
annotation: typing.Annotated[__main__.MyRequest, <class 'aws_lambda_powertools.event_handler.openapi.params.Body'>]
field_info: None
type_annotation: <class '__main__.MyRequest'>

with 3.22.1

# start of get_field_info_annotated_type
annotation: typing.Annotated[__main__.MyRequest, <class 'aws_lambda_powertools.event_handler.openapi.params.Body'>]

# end of get_field_info_annotated_type
annotation: typing.Annotated[__main__.MyRequest, <class 'aws_lambda_powertools.event_handler.openapi.params.Body'>]
field_info: None
type_annotation: typing.Annotated[__main__.MyRequest, <class 'aws_lambda_powertools.event_handler.openapi.params.Body'>]

(in both cases, using pydantic==2.12.4)

So it looks like in 3.22.0 and before, get_field_info_annotated_type() helpfully discarded the malformed Annotation. In 3.22.1, it stopped doing this and kept the original annotation.

This is probably a case of Hyrum's law and/or xkcd 1172; feel free to close if you don't consider it a regression.

chriselion avatar Nov 13 '25 17:11 chriselion

Hey @chriselion I'm working to release a new version with support for Python 3.14 and will take a look on this tomorrow, ok?

leandrodamascena avatar Nov 13 '25 17:11 leandrodamascena

Sounds good, that's probably more important!

chriselion avatar Nov 13 '25 17:11 chriselion