certification-tool [Feature] TH image is not prasing the PIXIT values from the PICS XML files

Feature description

Currently, the TH image is not prasing the PIXIT values from the PICS XML files.

Test Cases

For example, In TC-SMOKECO-xx, there is a PIXIT PIXIT.SMOKECO.TEST_EVENT_TRIGGER_KEY and it's a enable key used to trigger the test event.

The below mentioned PIXIT is available in the PICS XML file:

But once after uploaded the PICS XML file in TH image, the above mentioned PIXIT is not available in project config or test parameter. As of now QA adding the enable key argument manually in test parameters filed.

Added the project config below:

Project Config:

"pics": { "clusters": { "Smoke CO Alarm Cluster Test Plan": { "name": "Smoke CO Alarm Cluster Test Plan", "items": { "SMOKECO.S": { "number": "SMOKECO.S", "enabled": true }, "SMOKECO.C": { "number": "SMOKECO.C", "enabled": false }, "SMOKECO.S.A0000": { "number": "SMOKECO.S.A0000", "enabled": true }, "SMOKECO.S.A0001": { "number": "SMOKECO.S.A0001", "enabled": true }, "SMOKECO.S.A0002": { "number": "SMOKECO.S.A0002", "enabled": true }, "SMOKECO.S.A0003": { "number": "SMOKECO.S.A0003", "enabled": true }, "SMOKECO.S.A0004": { "number": "SMOKECO.S.A0004", "enabled": true }, "SMOKECO.S.A0005": { "number": "SMOKECO.S.A0005", "enabled": true }, "SMOKECO.S.A0006": { "number": "SMOKECO.S.A0006", "enabled": true }, "SMOKECO.S.A0007": { "number": "SMOKECO.S.A0007", "enabled": true }, "SMOKECO.S.A0008": { "number": "SMOKECO.S.A0008", "enabled": true }, "SMOKECO.S.A0009": { "number": "SMOKECO.S.A0009", "enabled": true }, "SMOKECO.S.A000a": { "number": "SMOKECO.S.A000a", "enabled": true }, "SMOKECO.S.A000b": { "number": "SMOKECO.S.A000b", "enabled": true }, "SMOKECO.S.A000c": { "number": "SMOKECO.S.A000c", "enabled": true }, "SMOKECO.S.E00": { "number": "SMOKECO.S.E00", "enabled": true }, "SMOKECO.S.E01": { "number": "SMOKECO.S.E01", "enabled": true }, "SMOKECO.S.E02": { "number": "SMOKECO.S.E02", "enabled": true }, "SMOKECO.S.E03": { "number": "SMOKECO.S.E03", "enabled": true }, "SMOKECO.S.E04": { "number": "SMOKECO.S.E04", "enabled": true }, "SMOKECO.S.E05": { "number": "SMOKECO.S.E05", "enabled": true }, "SMOKECO.S.E06": { "number": "SMOKECO.S.E06", "enabled": true }, "SMOKECO.S.E07": { "number": "SMOKECO.S.E07", "enabled": true }, "SMOKECO.S.E08": { "number": "SMOKECO.S.E08", "enabled": true }, "SMOKECO.S.E09": { "number": "SMOKECO.S.E09", "enabled": true }, "SMOKECO.S.E0a": { "number": "SMOKECO.S.E0a", "enabled": true }, "SMOKECO.S.C00.Rsp": { "number": "SMOKECO.S.C00.Rsp", "enabled": true }, "SMOKECO.S.F00": { "number": "SMOKECO.S.F00", "enabled": true }, "SMOKECO.S.F01": { "number": "SMOKECO.S.F01", "enabled": true }, "SMOKECO.M.ManuallyControlledTest": { "number": "SMOKECO.M.ManuallyControlledTest", "enabled": false }, "SMOKECO.M.ManuallyControlledMute": { "number": "SMOKECO.M.ManuallyControlledMute", "enabled": false } } } } } }

Additional Information

Version: v2.11-beta2+fall2024 Sha: bbeda31

PICS tool used: PICS Tool v2.1.12 - Connectivity Standards Alliance.html.zip PICS XML used: SMOKECO.zip

Aug 12 '24 11:08 KishokG

New Feature Request

Aug 14 '24 16:08 raju-apple

Hi @KishokG I'm looking now at this issue and I have a few questions:

Given the attached file, how do you fill the test_parameters manually accordingly?
AFAIU when adding a PIXIT in test_parameters it should follow this pattern: --[type]-arg [PIXIT INFO:value].

Example:

"test_parameters": { "int-arg": "PIXIT.ACE.APPENDPOINT:1 PIXIT.ACE.APPDEVTYPEID:256", "string-arg": "PIXIT.ACE.APPCLUSTER:OnOff PIXIT.ACE.APPATTRIBUTE:OnOff" }

So looking at Smoke CO Alarm Cluster Test Plan.xml file I can't file the PIXIT type (example int-arg) and the respective value to be informed.

Apr 16 '25 16:04 rquidute

Hi @rquidute, Just to clarify—the Test Harness (TH) should parse all the elements present in the uploaded PICS XML file. However, it's currently not parsing the PIXIT items as expected (as shown in the example above).

Also, this issue is not related to "test_parameters" as mentioned you—it's specifically about the PIXIT elements not being picked up by the TH.

I can see PIXIT items in XML file:

But while seeing in the Project configurations the PIXIT items are missing:

{ "config": { "test_parameters": null, "network": { "wifi": { "ssid": "testharness", "password": "wifi-password" }, "thread": { "rcp_serial_path": "/dev/ttyACM0", "rcp_baudrate": 115200, "on_mesh_prefix": "fd11:22::/64", "network_interface": "eth0", "dataset": { "channel": "15", "panid": "0x1234", "extpanid": "1111111122222222", "networkkey": "00112233445566778899aabbccddeeff", "networkname": "DEMO" }, "otbr_docker_image": null } }, "dut_config": { "discriminator": "3840", "setup_code": "20202021", "pairing_mode": "onnetwork", "chip_timeout": null, "chip_use_paa_certs": false, "trace_log": true, "enhanced_setup_flow": null } }, "pics": { "clusters": { "Smoke CO Alarm Cluster Test Plan": { "name": "Smoke CO Alarm Cluster Test Plan", "items": { "SMOKECO.S": { "number": "SMOKECO.S", "enabled": false }, "SMOKECO.C": { "number": "SMOKECO.C", "enabled": false }, "SMOKECO.S.A0000": { "number": "SMOKECO.S.A0000", "enabled": false }, "SMOKECO.S.A0001": { "number": "SMOKECO.S.A0001", "enabled": false }, "SMOKECO.S.A0002": { "number": "SMOKECO.S.A0002", "enabled": false }, "SMOKECO.S.A0003": { "number": "SMOKECO.S.A0003", "enabled": false }, "SMOKECO.S.A0004": { "number": "SMOKECO.S.A0004", "enabled": false }, "SMOKECO.S.A0005": { "number": "SMOKECO.S.A0005", "enabled": false }, "SMOKECO.S.A0006": { "number": "SMOKECO.S.A0006", "enabled": false }, "SMOKECO.S.A0007": { "number": "SMOKECO.S.A0007", "enabled": false }, "SMOKECO.S.A0008": { "number": "SMOKECO.S.A0008", "enabled": false }, "SMOKECO.S.A0009": { "number": "SMOKECO.S.A0009", "enabled": false }, "SMOKECO.S.A000a": { "number": "SMOKECO.S.A000a", "enabled": false }, "SMOKECO.S.A000b": { "number": "SMOKECO.S.A000b", "enabled": false }, "SMOKECO.S.A000c": { "number": "SMOKECO.S.A000c", "enabled": false }, "SMOKECO.S.E00": { "number": "SMOKECO.S.E00", "enabled": false }, "SMOKECO.S.E01": { "number": "SMOKECO.S.E01", "enabled": false }, "SMOKECO.S.E02": { "number": "SMOKECO.S.E02", "enabled": false }, "SMOKECO.S.E03": { "number": "SMOKECO.S.E03", "enabled": false }, "SMOKECO.S.E04": { "number": "SMOKECO.S.E04", "enabled": false }, "SMOKECO.S.E05": { "number": "SMOKECO.S.E05", "enabled": false }, "SMOKECO.S.E06": { "number": "SMOKECO.S.E06", "enabled": false }, "SMOKECO.S.E07": { "number": "SMOKECO.S.E07", "enabled": false }, "SMOKECO.S.E08": { "number": "SMOKECO.S.E08", "enabled": false }, "SMOKECO.S.E09": { "number": "SMOKECO.S.E09", "enabled": false }, "SMOKECO.S.E0a": { "number": "SMOKECO.S.E0a", "enabled": false }, "SMOKECO.S.C00.Rsp": { "number": "SMOKECO.S.C00.Rsp", "enabled": false }, "SMOKECO.S.F00": { "number": "SMOKECO.S.F00", "enabled": false }, "SMOKECO.S.F01": { "number": "SMOKECO.S.F01", "enabled": false }, "SMOKECO.M.ManuallyControlledTest": { "number": "SMOKECO.M.ManuallyControlledTest", "enabled": false }, "SMOKECO.M.ManuallyControlledMute": { "number": "SMOKECO.M.ManuallyControlledMute", "enabled": false } } } } } }

Apr 23 '25 04:04 KishokG

Thanks for your feedback @KishokG . So, the main idea would be just to display the PIXIT correctly? However, just displaying it doesn't mean this information would actually be used in the tests, right? I'm asking because the way PIXIT data is passed to the tests is through the test_parameters. So, would simply displaying the PIXIT be enough, or should we take into account the PIXIT contained in the file and use that information during test execution?

Apr 23 '25 12:04 rquidute