angular-cli icon indicating copy to clipboard operation
angular-cli copied to clipboard

run e2e legacy tests under bazel

Open jbedard opened this issue 3 years ago • 2 comments

This runs the legacy-cli/e2e tests via bazel using the bazel built npm packages from packages/**.

After all the pre-factors it's mainly just build config and a few code changes to use the bazel environment vars if present. This can be simplified if the non-bazel versions are dropped in the future. Currently all the CI jobs are prefixed with bazel-.

Build targets / API:

There are 9 bazel targets invoking the e2e tests with different configurations:

  • //tests/legacy-cli:e2e.{npm,yarn,esbuild,saucelabs}: configured like CI today
  • //tests/legacy-cli:e2e.snapshot.{npm,yarn,esbuild,saucelabs}: ^ using snapshots
  • //tests/legacy-cli:e2e: nothing pre-configured, does not run by default or on CI, is designed to be run and configured manually with bazel cli args.

With any of these you can:

  • bazel test to run as a bazel test
  • bazel run to do the equivalent of --debug and remain running on failure
  • --test_filters="..." which essentially does the e2e_runner --glob="..."
  • --test_arg="..." to specify additional e2e_runner args, --glob would work here too

Sharding Today e2e tests have --shard and --nb-shards cli params which circleci sets based on the parallelism config. Now those cli args are defaulted based on the bazel environment variables set by nodejs_test(shard_count) and/or the E2E_SHARD_TOTAL + E2E_SHARD environment variables I added. This way both bazel and circleci sharding can be combined into one (bazel shards via processes on one circleci environment, circleci sharding in isolated environments).

Prefactors:

  • https://github.com/angular/angular-cli/pull/23130
  • https://github.com/angular/angular-cli/pull/23114
  • https://github.com/angular/angular-cli/pull/23115
  • https://github.com/angular/angular-cli/pull/23245
  • https://github.com/angular/angular-cli/pull/23313
  • https://github.com/angular/angular-cli/pull/23326
  • https://github.com/angular/angular-cli/pull/23372
  • https://github.com/angular/angular-cli/pull/23640
  • https://github.com/angular/angular-cli/pull/23541
  • https://github.com/angular/angular-cli/pull/23648
  • https://github.com/angular/angular-cli/pull/23649
  • https://github.com/angular/angular-cli/pull/23687
  • https://github.com/angular/angular-cli/pull/23694
  • https://github.com/angular/angular-cli/pull/23711
  • https://github.com/angular/angular-cli/pull/23712
  • https://github.com/angular/angular-cli/pull/23686
  • https://github.com/angular/angular-cli/pull/23767
  • https://github.com/angular/angular-cli/pull/23714
  • https://github.com/angular/angular-cli/pull/23753
  • https://github.com/angular/angular-cli/pull/23902
  • https://github.com/angular/angular-cli/pull/23903
  • https://github.com/angular/angular-cli/pull/23926

jbedard avatar May 03 '22 23:05 jbedard

Couple of questions

  • How is it possible to run a single test or using --glob option?
  • How to debug a single test IE the --debug option?

Both of these are paramount for day to day tasks.

alan-agius4 avatar Sep 14 '22 10:09 alan-agius4

I think those were all addressed offline and not in the comments here, just no one had pressed resolve.

jbedard avatar Nov 14 '22 19:11 jbedard

This issue has been automatically locked due to inactivity. Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.