wp-e2e-tests icon indicating copy to clipboard operation
wp-e2e-tests copied to clipboard

Look at setting AB test groups on start page

Open alisterscott opened this issue 7 years ago • 2 comments

@Stojdza says on #1488

@alisterscott What do you think about a solution to put back refresh in setABTestControlGroupsInLocalStorage(), but set it to false by default? And where it's needed to call refresh with true. Or otherwise - to be true by default, and where it's not needed or causing problems, set it to false.

I'm not sure is it applicable to all tests, but I think that way we would skip calling /themes page for every test.

alisterscott avatar Sep 20 '18 03:09 alisterscott

@Stojdza: @bsessions85: I think we'll always want to set the AB test groups when we're visiting the start page and always refresh the page when we do which mimics the behaviour when using the AB test icon on for example wpcalypso.wordpress.com:

wpcalypso

I'm still not sure when we wouldn't not want to refresh since the AB tests can entirely change the start page, such as using a different flow which won't take effect until the test does something that moves pages

Reading the original reason for removing the refresh here: https://github.com/Automattic/wp-e2e-tests/pull/1459 it seems that the refresh lost the query strings which is kind of a bug in itself since the user would also lose their place if they refreshed their browser on that page 🤔

But the solution we have works for the issue addressed in #1459 since we set the AB test before visiting start

Maybe we can just choose a better page to visit before start, like maybe /log-in is more lightweight?

alisterscott avatar Sep 20 '18 03:09 alisterscott

I've made an attempt to clean things up a bit in https://github.com/Automattic/wp-e2e-tests/pull/1491

alisterscott avatar Sep 20 '18 04:09 alisterscott