wp-e2e-tests icon indicating copy to clipboard operation
wp-e2e-tests copied to clipboard

Jetpack Tests: Only include tests related to the selected host

Open rachelmcr opened this issue 7 years ago • 3 comments

Raised by @alisterscott:

The Jetpack Connect tests use Jurassic Ninja but we include them in Jetpack test runs on other hosts.

Is there any reason for that? If not, we should exclude them when running Jetpack tests on other hosts, e.g. Pressable.

Internal ref: p1521676288000097-slack-flow

rachelmcr avatar Mar 22 '18 13:03 rachelmcr

Notes from a sync conversation with @brbrr:

We run tests using Jetpack stable against Pressable, and Jetpack master against JN. Generally speaking, the version of Jetpack (stable/master) is more important than the hosting integration. We should continue to include the Jetpack Connect tests in all runs for that reason.

Suggested improvements:

  • We can update the describe blocks to refer to the correct host name in tests that only run on one host.
  • We should highlight which version of Jetpack we're testing against in each run.
  • If we need to decrease the load on JN we could take a different approach, e.g. using one screen size per test run.

rachelmcr avatar Mar 22 '18 18:03 rachelmcr

@brbrr is this something that still needs to be done? If so, I'll add it to Trello

alisterscott avatar May 21 '18 02:05 alisterscott

Hm, I've revisited our convo, and it seems that I misunderstand Rachel's concern.

So yeah, we running Connect specs using stable Jetpack version for both current targets. I think we should update them to use Jetpack master branch for one of the targets. (will create a trello card)

brbrr avatar May 23 '18 13:05 brbrr