atomic-server icon indicating copy to clipboard operation
atomic-server copied to clipboard

Test passing locally, some are flaky in CI

Open joepio opened this issue 6 months ago • 1 comments

When running on my macbook, the e2e tests are running fine. However, on the (far slower) CI machine some of them fail pretty often.

Runs

  • https://686d0f5d62e047882bb195b8--atomic-tests.netlify.app

Problems

  • [ ] ChatRoom second window doesn't show the new message. Maybe because the subscribe is processed after the commit has been fired. I think a robust solution is to send all the relevant, potentially missed commits (from the past x time) to the client. Or maybe just send the current RESOURCE to the client on subscribe? Could also work. A dirty hack is to add a timeout or something, or maybe add some retries if its flaky.
  • [ ] Bookmark opens the wrong drive, atomic instead of localtest-something. Doesn't seem to happen when running locally. Maybe some SERVER_URL / FRONTEND_URL related issue?
  • [ ] Sveltekit pnpm link failing
  • [ ] New Drive can't be found, because there are too many in the dropdown. We should clean these up. Image
  • [ ] Filepicker can't find txt file. I think this is because our testFilePath function is unreliable / not properly configured. I'd like a more robust solution.
  • [ ] editProfileAndCommit fails when list of drives is too long, the click doesnt work. Try scrollIntoViewIfNeeded. Maybe a cleanup would be better though.

joepio avatar Jul 08 '25 13:07 joepio

Bookmark opens the wrong drive, atomic instead of localtest-something. Doesn't seem to happen when running locally. Maybe some SERVER_URL / FRONTEND_URL related issue?

That's weird, it uses the same signing function as the rest of the tests. There is nothing different that could cause this issue.

New Drive can't be found, because there are too many in the dropdown. We should clean these up.

These are cleaned up. That's why we have the prune test. Did you disable it?

Filepicker can't find txt file. I think this is because our testFilePath function is unreliable / not properly configured. I'd like a more robust solution.

That means the cwd is different in the CI from where you are running it locally. I already modified it before to handle more than one path but apparently the CI runs the test from yet another path?

Polleps avatar Jul 10 '25 11:07 Polleps