Jetpack Tests: Only include tests related to the selected host
Raised by @alisterscott:
The Jetpack Connect tests use Jurassic Ninja but we include them in Jetpack test runs on other hosts.
Is there any reason for that? If not, we should exclude them when running Jetpack tests on other hosts, e.g. Pressable.
Internal ref: p1521676288000097-slack-flow
Notes from a sync conversation with @brbrr:
We run tests using Jetpack stable against Pressable, and Jetpack master against JN. Generally speaking, the version of Jetpack (stable/master) is more important than the hosting integration. We should continue to include the Jetpack Connect tests in all runs for that reason.
Suggested improvements:
- We can update the
describeblocks to refer to the correct host name in tests that only run on one host. - We should highlight which version of Jetpack we're testing against in each run.
- If we need to decrease the load on JN we could take a different approach, e.g. using one screen size per test run.
@brbrr is this something that still needs to be done? If so, I'll add it to Trello
Hm, I've revisited our convo, and it seems that I misunderstand Rachel's concern.
So yeah, we running Connect specs using stable Jetpack version for both current targets. I think we should update them to use Jetpack master branch for one of the targets. (will create a trello card)