Contract tests test_input_equals_output does not work with arrays of primitive values

[brooks-mikkelsen]

Hey, I believe there is a bug in the test_input_equals_output portion of several contract tests.

Issue

When comparing a list of strings, the tests will call compare(), which tries to iterate through the input. In the case of primitives including strings, it should probably just assert their equality rather than trying to iterate through the primitive type.

Our inputs_1_create.json:

{
  "Id": "cfn-contract-test-endpoint",
  ...
  "HlsPackage": {
    ...
    "AdTriggers": [
      "SPLICE_INSERT",
      "PROVIDER_ADVERTISEMENT",
      "DISTRIBUTOR_ADVERTISEMENT",
      "PROVIDER_PLACEMENT_OPPORTUNITY",
      "DISTRIBUTOR_PLACEMENT_OPPORTUNITY"
    ],
    ...
  }
}

Stack trace:

________________________________________________________________________________________________________________________ contract_create_delete _________________________________________________________________________________________________________________________
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/suite/handler_create.py:47: in contract_create_delete
    test_input_equals_output(resource_client, input_model, delete_model)
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/suite/handler_commons.py:168: in test_input_equals_output
    pruned_output_model,
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
    raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:354: in compare
    self.compare(inputs[key], outputs[key])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
    raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:357: in compare
    self.compare_list(inputs[key], outputs[key])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:368: in compare_list
    self.compare(inputs[index], outputs[index])
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:364: in compare
    raise e
/Users/bmmikkel/.pyenv/versions/3.7.10/lib/python3.7/site-packages/rpdk/core/contract/resource_client.py:353: in compare
    if isinstance(inputs[key], dict):
E   TypeError: string indices must be integers

I am using cfn cli version 0.2.8. 0.2.7 worked with the same inputs payload.

Workaround

In our inputs/input_1_create.json (and _update and _invalid) files, we can temporarily just specify an empty array, which will prevent the tests from trying to iterate through characters of a string or iterate through a number/bool. Changing the AdTriggers in mentioned above to [] in each input file made the contract tests pass.

[youhaoqs]

+1, seeing this issue with recent contract tests.