Zappa icon indicating copy to clipboard operation
Zappa copied to clipboard

[Migrated] [async] Check if args/kwargs are JSON Serializable while running locally

Open jneves opened this issue 4 years ago • 2 comments

Originally from: https://github.com/Miserlou/Zappa/issues/1791 by mariamrf

Context

  • When running an async @task locally, the args and kwargs are passed as they are (even if they're not JSON Serializable).
  • It would be nice to have this check when running locally so we can test for and predict failures of that kind.

Expected Behavior

  • When a @task is run locally with invalid args/kwargs (non-serializable), it should break/raise an exception as this is the actual behavior when it's deployed.

Actual Behavior

  • When a @task is run locally with invalid args/kwargs (non-serializable), it just runs as a normal function and therefore, we can't catch errors of this kind in the tests.

Possible Fix

In async.py:

            lambda_function_name = lambda_function_name_arg or os.environ.get('AWS_LAMBDA_FUNCTION_NAME')
            aws_region = aws_region_arg or os.environ.get('AWS_REGION')

            if (service in ASYNC_CLASSES) and (lambda_function_name):
                send_result = ASYNC_CLASSES[service](lambda_function_name=lambda_function_name,
                                                     aws_region=aws_region,
                                                     capture_response=capture_response).send(task_path, args, kwargs)
                return send_result
            else:
                validate_json_serializable({'args': args, 'kwargs': kwargs})
                return func(*args, **kwargs)

and in utilities.py:

def validate_json_serializable(thing):
    try:
        json.dumps(thing)
    except (TypeError, OverflowError):
        # raise more human-readable error here

Steps to Reproduce

  1. Write a function that accepts a non-serializable type (like an instance of a class)
  2. Run this function locally
  3. Function runs!
  4. Deploy the code to Lambda
  5. Function breaks

Your Environment

  • Zappa version used: 0.45.1
  • ~~Operating System and~~ Python version: 3.6
  • ~~The output of pip freeze:~~
  • ~~Link to your project (optional):~~
  • ~~Your zappa_settings.py:~~

jneves avatar Feb 20 '21 12:02 jneves

I can add these changes if you need contributors, this has bitten us a few times where a dev will pass Django objects, see that it is working locally, and push to release.

jottenlips avatar Feb 03 '22 19:02 jottenlips

I had a branch up, but I like your changes more.

jottenlips avatar Feb 03 '22 19:02 jottenlips

released in 0.56.0, closing.

monkut avatar Dec 01 '22 10:12 monkut