certification-tool icon indicating copy to clipboard operation
certification-tool copied to clipboard

Feature suggestion - Client test cases launch All-Clusters on the Test Harness

Open raju-apple opened this issue 2 years ago • 6 comments

From chip-certification-tool created by EF-JANC: CHIP-Specifications/chip-certification-tool#472

For DUT as Client test cases, when running the Manual_tests, would it be possible to start the All-Clusters-app as TH in the Test Harness so this can be logged directly with the test case, instead of us having to run 3 separate logging windows (TH + SSH To All-Clusters + SSH to DUT)

raju-apple avatar Oct 19 '23 18:10 raju-apple

My initial concerns are -

  • We need to delete the temp files and run the cluster app between test runs.
  • Looks like there are a bunch of custom apps, should we always pick the all-clusters-app?

raju-apple avatar Oct 19 '23 18:10 raju-apple

Response and design proposal by @mikaelhm: We might need a custom Test Suite for these so the app is just commisioned once. So we can avoid it for all test cases. We could make a special "assisted manual test"

  • Starts app (agree it should be configurable)
  • streams logs without filtering to test engine logger
  • Runs normal manual test case prompt flow.

raju-apple avatar Oct 19 '23 18:10 raju-apple

Visualizing @mikaelhm 's design proposal Screen Shot 2022-08-05 at 2 44 55 PM

raju-apple avatar Oct 19 '23 18:10 raju-apple

@sparampallirajag @mikaelhm Thanks for the feedback!

Please allow me to provide some feedback on the above :)

-Looks like there are a bunch of custom apps, should we always pick the all-clusters-app? <JC> The testplan 'always' reffers to the All-Clusters-App as the TH so I would say yes as this would contain all clusters and should be the most advanced tool foor client tests.

The above approach of having it as 'manual' tests is fine - I just wanted to propose a more user friendly way of working.

And with regards to the deleting the tmp files this is correct, I have made a small script for this to ease the server tests on a SDK ;

sudo rm -rf /tmp/chip_* #remove the CHIP Temp files to allow for new commisioning
 
sleep .5 # Wait 0.5 second.
​
./apps/chip-all-clusters-app #Launch ACA

raju-apple avatar Oct 19 '23 18:10 raju-apple

Work in progress. This is not a blocker and will not be ready in a few days. So, removing the v1.x label.

raju-apple avatar Oct 19 '23 18:10 raju-apple

@fabiowmm any resolution on this ?

raju-apple avatar Jun 17 '24 23:06 raju-apple

@rquidute @ccruzagralopes isso aqui foi realizado pelos commits referenciados acima?

fabiowmm avatar Oct 10 '24 18:10 fabiowmm

I believe this mention is a GitHub bug. The mentioned commit is actually from a PR in the backend repo that has the same issue number as this issue:

  • https://github.com/project-chip/certification-tool-backend/pull/17

And just FIY: I believe you didn't notice that you wrote the message in Portuguese 😆

ccruzagralopes avatar Oct 10 '24 18:10 ccruzagralopes

Deprioritized.

fabiowmm avatar Oct 10 '24 19:10 fabiowmm