localstack icon indicating copy to clipboard operation
localstack copied to clipboard

CI: correct test selection propagation for service-specific tests and run them conditionally

Open k-a-il opened this issue 7 months ago β€’ 8 comments

Motivation

This PR addresses two current issues with test selections, specifically:

  • missing propagation of test selection files to service-specific tests
  • issue with unnecessary job startup if no service-specific code was updated

Changes

  • Add TESTSELECTION_PYTEST_ARGS to PYTEST_ARGS for provider specific tests
  • Add a step to identify changed services and use its output to conditionally trigger service-specific jobs

Testing

  • Test that only Integration tests are triggered if S3 test has been changed https://github.com/localstack/localstack/pull/12750
  • Test that Integration tests and DynamoDB-v2 tests are triggered https://github.com/localstack/localstack/pull/12751
  • Test that all tests are triggered when workflow is triggered on master

k-a-il avatar Jun 12 '25 14:06 k-a-il

Test Results - Preflight, Unit

21β€ˆ630 tests  +17   19β€ˆ973 βœ… +15   7m 2s ⏱️ +2s β€‡β€‡β€ˆβ€‡β€‡1 suites ± 0    1β€ˆ657 πŸ’€ + 2  β€‡β€‡β€ˆβ€‡β€‡1 files   ± 0β€‚β€ƒβ€ƒβ€‡β€‡β€ˆβ€‡β€‡0 ❌ ± 0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results (MA/MR) - Preflight, Unit

21β€ˆ630 tests  +17   19β€ˆ973 βœ… +15   6m 8s ⏱️ -13s β€‡β€‡β€ˆβ€‡β€‡1 suites ± 0    1β€ˆ657 πŸ’€ + 2  β€‡β€‡β€ˆβ€‡β€‡1 files   ± 0β€‚β€ƒβ€ƒβ€‡β€‡β€ˆβ€‡β€‡0 ❌ ± 0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results (amd64) - Acceptance

7 tests  Β±0   5 βœ… Β±0   3m 22s ⏱️ +15s 1 suites Β±0   2 πŸ’€ Β±0  1 files   Β±0   0 ❌ Β±0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results (amd64, MA/MR) - Acceptance

7 tests  Β±0   5 βœ… Β±0   3m 11s ⏱️ -8s 1 suites Β±0   2 πŸ’€ Β±0  1 files   Β±0   0 ❌ Β±0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results - Alternative Providers

987 tests   589 βœ…β€ƒβ€ƒ30m 19s ⏱️   4 suites  398 πŸ’€   4 files      0 ❌

Results for commit 82634230.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results (MA/MR) - Alternative Providers

987 tests   589 βœ…β€ƒβ€ƒ30m 34s ⏱️   4 suites  398 πŸ’€   4 files      0 ❌

Results for commit 82634230.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 14:06 github-actions[bot]

Test Results (amd64) - Integration, Bootstrap

β€‡β€ˆβ€‡β€‡5 files  ± 0β€‚β€ƒβ€‡β€ˆβ€‡β€‡5 suites  Β±0   2h 22m 34s ⏱️ + 2m 13s 5β€ˆ242 tests +12  4β€ˆ313 βœ… +12  929 πŸ’€ Β±0  0 ❌ Β±0  5β€ˆ248 runsβ€Š +12  4β€ˆ313 βœ… +12  935 πŸ’€ Β±0  0 ❌ Β±0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 15:06 github-actions[bot]

Test Results (amd64, MA/MR) - Integration, Bootstrap

β€‡β€ˆβ€‡β€‡5 files  ± 0β€‚β€ƒβ€‡β€ˆβ€‡β€‡5 suites  Β±0   2h 22m 50s ⏱️ +24s 5β€ˆ242 tests +12  4β€ˆ313 βœ… +12  929 πŸ’€ Β±0  0 ❌ Β±0  5β€ˆ248 runsβ€Š +12  4β€ˆ313 βœ… +12  935 πŸ’€ Β±0  0 ❌ Β±0 

Results for commit 73f835e4. ± Comparison against base commit 3bbf9442.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jun 12 '25 15:06 github-actions[bot]