Key error on deployment - set comprehension issue?

[achard]

I'm getting a key error when attempting to deploy my code.

I have isolated the issue to one of the functions, it seems to me that Chalice may not be able to understand set comprehensions? My code works fine when running locally. It deploys fine when I comment out the function/route in question.

The line that seems to be causing my issues it this one: dynamodb_address_set |= {address_object['address'] for address_object in response.get('Items', [])}

I was able to work around this issue by replacing that line with these two:

address_page_list = [address_object['address'] for address_object in response.get('Items', [])]
dynamodb_address_set |= set(address_page_list)

The full function causing the error is:

@app.route('/update_ips', methods=['POST', 'GET'])
def update_ips():
    start_key = None
    scan_kwargs = {}
    dynamodb_address_set = set()
    request = app.current_request
    if request is not None and request.method == 'POST':
        fields = request.json_body
        prisma_ip_set = set(fields.get('ip_list'))

    dynamodb = boto3.resource('dynamodb')
    table = dynamodb.Table("prisma_addresses")
    done = False
    while not done:
        if start_key:
            scan_kwargs['ExclusiveStartKey'] = start_key
        response = table.scan(**scan_kwargs)
        dynamodb_address_set |= {address_object['address'] for address_object in response.get('Items', [])}
        start_key = response.get('LastEvaluatedKey', None)
        done = start_key is None
    added_addresses = prisma_ip_set - dynamodb_address_set
    removed_addresses = dynamodb_address_set - prisma_ip_set
    for address in removed_addresses:
        table.delete_item(Key={'address': address})
    for address in added_addresses:
        table.put_item(Item={'address': address})

    return {
        'current_addresses': prisma_ip_set,
        'added_addresses': added_addresses,
        'removed_addresses': removed_addresses,
        'address_count': len(prisma_ip_set)
    }

And the traceback I get when attempting to deploy is this:

PS prisma_bypass_reports> chalice deploy --stage prod
Traceback (most recent call last):
  File "%projectpath%\venv\lib\site-packages\chalice\cli\__init__.py", line 636, in main
    return cli(obj={})
  File "%projectpath%\venv\lib\site-packages\click\core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "%projectpath%\venv\lib\site-packages\click\core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "%projectpath%\venv\lib\site-packages\click\core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "%projectpath%\venv\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "%projectpath%\venv\lib\site-packages\click\core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "%projectpath%\venv\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "%projectpath%\venv\lib\site-packages\chalice\cli\__init__.py", line 189, in deploy
    deployed_values = d.deploy(config, chalice_stage_name=stage)
  File "%projectpath%\venv\lib\site-packages\chalice\deploy\deployer.py", line 376, in deploy
    return self._deploy(config, chalice_stage_name)
  File "%projectpath%\venv\lib\site-packages\chalice\deploy\deployer.py", line 386, in _deploy
    self._build_stage.execute(config, resources)
  File "%projectpath%\venv\lib\site-packages\chalice\deploy\deployer.py", line 652, in execute
    step.handle(config, resource)
  File "%projectpath%\venv\lib\site-packages\chalice\deploy\deployer.py", line 419, in handle
    handler(config, resource)
  File "%projectpath%\venv\lib\site-packages\chalice\deploy\deployer.py", line 637, in handle_autogeniampolicy
    policy = self._policy_gen.generate_policy(config)
  File "%projectpath%\venv\lib\site-packages\chalice\policy.py", line 93, in generate_policy
    app_policy = policy_from_source_code(app_source)
  File "%projectpath%\venv\lib\site-packages\chalice\policy.py", line 28, in policy_from_source_code
    client_calls = get_client_calls_for_app(source_code)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 74, in get_client_calls_for_app
    binder = t.bind_types()
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 684, in bind_types
    self.visit(self._parsed_code.parsed_ast)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "C:\Program Files\Python38\lib\ast.py", line 368, in generic_visit
    self.visit(item)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 694, in visit_FunctionDef
    child_infer.bind_types()
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 352, in bind_types
    self.visit(self._current_ast_namespace)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 540, in visit_FunctionDef
    self.visit(child)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "C:\Program Files\Python38\lib\ast.py", line 368, in generic_visit
    self.visit(item)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "C:\Program Files\Python38\lib\ast.py", line 370, in generic_visit
    self.visit(value)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "C:\Program Files\Python38\lib\ast.py", line 370, in generic_visit
    self.visit(value)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "C:\Program Files\Python38\lib\ast.py", line 370, in generic_visit
    self.visit(value)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 662, in visit
    return ast.NodeVisitor.visit(self, node)
  File "C:\Program Files\Python38\lib\ast.py", line 360, in visit
    return visitor(node)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 401, in visit_Name
    self._symbol_table.get_inferred_type(node.id)
  File "%projectpath%\venv\lib\site-packages\chalice\analyzer.py", line 249, in get_inferred_type
    symbol = self._local_table.lookup(name)
  File "C:\Program Files\Python38\lib\symtable.py", line 95, in lookup
    flags = self._table.symbols[name]
KeyError: 'address_object'

[hugos94]

The KeyError when deploying also happened to me (locally it works fine). I saw your issue (@achard) and went to change my set comprehensions and it worked again. In my case it was enough to change the braces to set().

Before:

access_plan_ids = {access.plan_id for access in accesses}

After:

access_plan_ids = set(access.plan_id for access in accesses)

[gbottari]

I recently faced the same problem with Chalice version 1.27.3. Changing from set comprehension to the set() function worked out, i.e.

this does not work:

# app.py
{item['value'] for item in collection}

while this works:

# app.py
set(item['value'] for item in collection)

I believe this is due to the policy generation procedure. You can reproduce the problem if you just call chalice gen-policy without necessarily calling chalice deploy.