no way to tell from spread log output which tasks _succeeded_
Spread makes it very obvious what things failed:
2020-10-26T19:30:59.1053887Z error: unsuccessful run
2020-10-26T19:30:59.1055189Z 2020-10-26 19:30:59 Successful tasks: 5
2020-10-26T19:30:59.1056126Z 2020-10-26 19:30:59 Aborted tasks: 12
2020-10-26T19:30:59.1057609Z 2020-10-26 19:30:59 Failed task prepare: 6
2020-10-26T19:30:59.1058270Z - google-nested:ubuntu-20.04-64:tests/nested/core20/gadget-reseal
2020-10-26T19:30:59.1059005Z - google-nested:ubuntu-20.04-64:tests/nested/core20/kernel-reseal
2020-10-26T19:30:59.1059709Z - google-nested:ubuntu-20.04-64:tests/nested/core20/tpm
2020-10-26T19:30:59.1060604Z - google-nested:ubuntu-20.04-64:tests/nested/manual/grade-signed-above-testkeys-boot:secured
2020-10-26T19:30:59.1061792Z - google-nested:ubuntu-20.04-64:tests/nested/manual/grade-signed-above-testkeys-boot:signed
2020-10-26T19:30:59.1062919Z - google-nested:ubuntu-20.04-64:tests/nested/manual/grade-signed-cloud-init-testkeys
But it does not make it clear from the log or from the end output what passed. I don't think it needs to list out all of the passing tests at the end, as in most cases this will just be noise, but when run with -v I think spread should indicate something in the log that can be grepped for to tell that a test passed, for example from the above run we had:
2020-10-26T18:11:51.7524676Z 2020-10-26 18:11:51 Preparing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775)...
2020-10-26T18:13:57.9025354Z 2020-10-26 18:13:57 Executing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775) (2/17)...
2020-10-26T18:17:59.6419456Z 2020-10-26 18:17:59 Restoring google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775)...
2020-10-26T18:18:01.2846669Z 2020-10-26 18:18:01 Restoring google-nested:ubuntu-20.04-64:tests/nested/classic/ (oct261801-050775)...
(with lots of other tests interspersed in the actual output, the above was filtered for just lines related to the test google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug)
it would be nice if after a test passed it had something like
2020-10-26T18:11:51.7524676Z 2020-10-26 18:11:51 Preparing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775)...
2020-10-26T18:13:57.9025354Z 2020-10-26 18:13:57 Executing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775) (2/17)...
2020-10-26T18:17:59.6419456Z 2020-10-26 18:17:59 Success executing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775) (2/17)...
2020-10-26T18:17:59.6419456Z 2020-10-26 18:17:59 Restoring google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775)...
2020-10-26T18:18:01.2846669Z 2020-10-26 18:18:01 Restoring google-nested:ubuntu-20.04-64:tests/nested/classic/ (oct261801-050775)...
or the Success executing line could even be combined with Restoring to be something like:
2020-10-26T18:11:51.7524676Z 2020-10-26 18:11:51 Preparing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775)...
2020-10-26T18:13:57.9025354Z 2020-10-26 18:13:57 Executing google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775) (2/17)...
2020-10-26T18:17:59.6419456Z 2020-10-26 18:17:59 Success executing; restoring google-nested:ubuntu-20.04-64:tests/nested/classic/hotplug (oct261801-050775) (2/17)...
2020-10-26T18:18:01.2846669Z 2020-10-26 18:18:01 Restoring google-nested:ubuntu-20.04-64:tests/nested/classic/ (oct261801-050775)...
Basically something that makes it easy to grep a verbose log to tell:
- [ ] If a specific test I care about passed
- [ ] What the set of tests that passed was
Again, this doesn't need to be in the default output, the verbose output is enough for our use cases I think.