[Bug] [TC-SWTCH-2.2, 2.3, 2.5, 2.6] Python tests are failing due to Timeout value.

[Ashwinigrl]

Describe the bug

While executing TC-SWTCH-2.2, 2.3, 2.5, 2.6 Python scripts in UI, these test cases are failing with below error. In matter_Testing_support file line number 1904 get_accepted_endpoints_for_test function has been called using asyncio.run with timeout value of 30 this cause the asyncio.exceptions.CancelledError while the wildcard read. hence the timeout value needs to be increased.

[MatterTest] 08-06 10:02:37.962 ERROR Exception occurred in test_TC_SWTCH_2_2. Traceback (most recent call last): File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for return await fut ^^^^^^^^^ File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 1794, in get_accepted_endpoints_for_test wildcard = await self.default_controller.Read(self.dut_node_id, [()]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/chip/ChipDeviceCtrl.py", line 1534, in Read return await future ^^^^^^^^^^^^ asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/mobly/base_test.py", line 818, in exec_one_test test_method() File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 1833, in per_endpoint_runner endpoints = asyncio.run(runner_with_timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/tasks.py", line 519, in wait_for async with timeouts.timeout(timeout): File "/usr/lib/python3.12/asyncio/timeouts.py", line 115, in aexit raise TimeoutError from exc_val TimeoutError

Python Script References:

• https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/TC_SWTCH.py

Matter_testing_support file:

• https://github.com/project-chip/connectedhomeip/blob/master/src/python_testing/matter_testing_support.py

Steps to reproduce the behavior

1. Open the UI
2. Upload the PICS
3. Ensure the TC-SWTCH-2.2 test case is selected
4. Executed the below mentioned command to put DUT into a commissionable state, ./chip-all-clusters-app
5. Click the Start button on Test-Harness user interface.

Expected behavior

Wild card read on endpoint is terminated due to the 30s timeout constraints in asyncio.run. So Time out value needs to be increased.

Log files

TC-switch-2-2-dut.log TC-Swtch-2-2.log

PICS file

Switch Cluster Test Plan .txt

Environment

TH Version: v2.11-beta2+fall2024 Sha: bbeda31

Reference Issue: https://github.com/project-chip/connectedhomeip/issues/34623

[Ashwinigrl]

Facing same issue in TC-CC-2.2 Test case.

[MatterTest] 08-06 13:15:23.117 ERROR Exception occurred in test_TC_CC_2_2. Traceback (most recent call last): File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for return await fut ^^^^^^^^^ File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 1794, in get_accepted_endpoints_for_test wildcard = await self.default_controller.Read(self.dut_node_id, [()]) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/dist-packages/chip/ChipDeviceCtrl.py", line 1534, in Read return await future ^^^^^^^^^^^^ asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/mobly/base_test.py", line 818, in exec_one_test test_method() File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 1833, in per_endpoint_runner endpoints = asyncio.run(runner_with_timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/tasks.py", line 519, in wait_for async with timeouts.timeout(timeout): File "/usr/lib/python3.12/asyncio/timeouts.py", line 115, in aexit

Reference log: TC-CC-2.2.log

[fabiowmm]

@cecille can you comment here?

[Ashwinigrl]

Still facing same issue in TC-SWTCH-2.3, 2.4, 2.5, 2.6.

[MatterTest] 09-09 10:22:20.271 ERROR Exception occurred in test_TC_SWTCH_2_3. Traceback (most recent call last): File "/usr/lib/python3.12/asyncio/tasks.py", line 520, in wait_for return await fut ^^^^^^^^^ File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 2115, in should_run_test_on_endpoint matching = await (_get_all_matching_endpoints(self, accept_function)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 2100, in _get_all_matching_endpoints wildcard = await self.default_controller.Read(self.dut_node_id, [(Clusters.Descriptor), Attribute.AttributePath(None, None, GlobalAttributeIds.ATTRIBUTE_LIST_ID), Attribute.AttributePath(None, None, GlobalAttributeIds.FEATURE_MAP_ID), Attribute.AttributePath(None, None, GlobalAttributeIds.ACCEPTED_COMMAND_LIST_ID)])

File "/usr/local/lib/python3.12/dist-packages/chip/ChipDeviceCtrl.py", line 1544, in Read return await future ^^^^^^^^^^^^ asyncio.exceptions.CancelledError

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/usr/local/lib/python3.12/dist-packages/mobly/base_test.py", line 818, in exec_one_test test_method() File "/root/python_testing/scripts/sdk/matter_testing_support.py", line 2177, in per_endpoint_runner should_run_test = asyncio.run(runner_with_timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/asyncio/tasks.py", line 519, in wait_for async with timeouts.timeout(timeout): File "/usr/lib/python3.12/asyncio/timeouts.py", line 115, in aexit raise TimeoutError from exc_val TimeoutError [MatterTest] 09-09 10:22:20.344 ERROR

Reference log: TC-SWTCH-2.3.log

Version: v2.11-beta3+fall2024 Sha: f52d40a

[cecille]

use the timeout arg / CLI flag to adjust the timeout to whatever is required. Default timeout is set for the CI.

[cecille]

Can you confirm the SHA you're using? Is this on the new TH? per_endpoint_runner doesn't exist any more.

[Ashwinigrl]

@cecille I'm using the latest TH image Version: v2.11-beta3+fall2024
sdk sha : 17b1a38e909e7874593bcb87c31be03a5866f1d4

per_endpoint_runner is still exists in the matter_testing_support file

[fabiowmm]

This is indeed the approved SVE SHA and per_endpoint_runner still exist there.

[cecille]

.... no it doesn't..... https://github.com/project-chip/connectedhomeip/blob/17b1a38e909e7874593bcb87c31be03a5866f1d4/src/python_testing/TC_SWTCH.py

[fabiowmm]

.... no it doesn't..... https://github.com/project-chip/connectedhomeip/blob/17b1a38e909e7874593bcb87c31be03a5866f1d4/src/python_testing/TC_SWTCH.py

I was referring to the method still existing in the matter_testing_support.py script.

[rquidute]

Hi @Ashwinigrl Could you check if this issue is still happening with the new TH version v2.11-beta3.1+fall2024 ?

[Ashwinigrl]

Issue fixed in Latest TH Version: v2.11-beta3.1+fall2024 Sha: 9c1ef58. Hence closing Issue.