App icon indicating copy to clipboard operation
App copied to clipboard

[HOLD for payment 2024-02-07] [Performance] Update the E2E tests to use real API responses

Open mountiny opened this issue 2 years ago • 8 comments

Problem

The E2E tests currently use hardcoded mocked data, but that does not reflect heavy user usage of the app as the amount of the data is not coming close to it.

Solution

Lets update the tests so they can authenticate with testing account and get actual data for special account, cache that and reuse it throughout the tests. This will ensure the tests mimic more closely actual user usage

@hannojg

mountiny avatar Jan 19 '24 11:01 mountiny

Triggered auto assignment to @kadiealexander (NewFeature), see https://stackoverflowteams.com/c/expensify/questions/14418#:~:text=BugZero%20process%20steps%20for%20feature%20requests for more details.

melvin-bot[bot] avatar Jan 19 '24 11:01 melvin-bot[bot]

@hannojg can you comment so I can assign you

mountiny avatar Jan 19 '24 13:01 mountiny

Servus @mountiny 👋

hannojg avatar Jan 19 '24 14:01 hannojg

The PR is good for review! @AndrewGable as I think you are internal most familiar with the setup of the e2e tests:

  • https://github.com/Expensify/App/pull/34792

hannojg avatar Jan 22 '24 20:01 hannojg

Reviewing label has been removed, please complete the "BugZero Checklist".

melvin-bot[bot] avatar Jan 31 '24 02:01 melvin-bot[bot]

The solution for this issue has been :rocket: deployed to production :rocket: in version 1.4.33-5 and is now subject to a 7-day regression period :calendar:. Here is the list of pull requests that resolve this issue:

  • https://github.com/Expensify/App/pull/34792

If no regressions arise, payment will be issued on 2024-02-07. :confetti_ball:

For reference, here are some details about the assignees on this issue:

  • @hannojg does not require payment (Contractor)

melvin-bot[bot] avatar Jan 31 '24 02:01 melvin-bot[bot]

BugZero Checklist: The PR adding this new feature has been merged! The following checklist (instructions) will need to be completed before the issue can be closed:

  • [ ] [@hannojg] Please propose regression test steps to ensure the new feature will work correctly on production in further releases.
  • [ ] [@kadiealexander] Link the GH issue for creating/updating the regression test once above steps have been agreed upon.

melvin-bot[bot] avatar Jan 31 '24 02:01 melvin-bot[bot]

@hannojg please don't forget about the regression test steps.

kadiealexander avatar Feb 06 '24 22:02 kadiealexander

Skipping the payment summary for this issue since all the assignees are employees or vendors. If this is incorrect, please manually add the payment summary SO.

melvin-bot[bot] avatar Feb 07 '24 15:02 melvin-bot[bot]

@AndrewGable, @NikkiWines, @hannojg, @kadiealexander Huh... This is 4 days overdue. Who can take care of this?

melvin-bot[bot] avatar Feb 12 '24 15:02 melvin-bot[bot]

I think this issue should be all good, right? I don't think we need regression steps since this wasn't a regression but rather an adjustment to our existing E2E tests

NikkiWines avatar Feb 13 '24 23:02 NikkiWines

Ah sweet then!

kadiealexander avatar Feb 14 '24 05:02 kadiealexander