ckeditor4-react
ckeditor4-react copied to clipboard
Investigate test flakiness
Are you reporting a feature request or a bug?
Task
Provide detailed reproduction steps (if any)
It happens occasionally that unit tests start failing without any apparent cause even if there were no code changes: https://github.com/ckeditor/ckeditor4-react/pull/292.
While they are called "unit tests", in fact these tests don't really fulfill the requirements of unit testing:
- There is a dependency on CDN
- Singleton
CKEDITORnamespace is used across all tests and there is an unreliable teardown for editor instances - Test cases are run sequentially
- Test cases are run in predefined order
Current stack:
- Karma is used because it allows to easily run tests in real browsers. Real browser is currently necessary due to fact that editor is loaded from CDN.
-
@testing-library/reactand related tools are used because they are pretty much staples when it comes to testing React code. In a typical scenario it's usually used with Jest and jsdom but we need a real browser here. - Jasmine is used because it's a popular choice for Karma.
Perhaps our current approach can be improved?
Expected result
Unit tests are fast and reliable.
Actual result
Unit tests occasionally fail for unclear reasons.