microsoft
[Bug]: Text fill is NOT working as expected in playwright version 1.53.1
Version
1.53.1
Steps to reproduce
In playwright version 1.52.0, things were working as expected and our text field fills were working properly. We were :
- checking to see if text field is editable
- scrollintoviewIfNeeded
- clearing the text field
- fill the text field with user value and assert to see if the value is reflected (expect().toHaveText()).
- fire the blur event.
we recently upgraded playwright to 1.53.1 and see that our fill and clear events are not working properly. We are seeing a case where the fill event either happens too fast and the field does not hydrate properly or the clear event happens (in debug mode, fill seems to have completed) and the field seems to be empty.
We also see this coming up between page loads as well when the test transitions between pages.
I apologize in advance for not sharing a minimal reproduceable repo.
Expected behavior
I expect the value that the user provides to be filled into a text field.
Actual behavior
Text field is empty, waiting for and the text fill is NOT happening as expected
Additional context
No response
Environment
System :
OS: macOS 14.7.6
Binaries:
Node - 20.15.0
npm - 10.7.0
npmPackages:
@playwright/test: 1.53.1
Hi! Could you provide me with a reproduction case? It's hard for me to act on this without a repro.
Hi! Could you provide me with a reproduction case? It's hard for me to act on this without a repro.
Same here, tried 1.53.0, 1.53.1 Fill and click just flaky, and fail most of the time 1.52.0 all test pass
https://github.com/user-attachments/assets/93c9de1f-1011-495f-b0c8-2a35478893c3
flaky fill, there is no repro I can provide, as I said, everything works good in 1.52.0
I will try my best to supply with a sample although i do not have any git repo yet:
<!-- minimal-repro.html -->
<!DOCTYPE html>
<html>
<head>
<script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
</head>
<body>
<input type="text" id="test-input" />
<script>
$('#test-input').on('focus', function() {
console.log('Focus event fired');
setTimeout(() => {
console.log('Focus processing complete');
}, 10);
});
$('#test-input').on('input change', function() {
console.log('Input/change event fired:', this.value);
});
</script>
</body>
</html>
my test:
import { test, expect } from '@playwright/test';
test('test', async ({ page }) => {
await page.goto('file:///path/to/minimal-repro.html');
const input = page.locator('#test-input');
await expect(input).toBeEditable();
await input.focus();
await input.clear();
await input.fill('test value');
await expect(input).toHaveValue('test value');
await input.blur();
});
I am so sorry again. i am not able to provide a full fledged repo.
Hope this helps.
@romankhomitskyi - thank you for providing a video for the issue.
@Skn0tt - thank you for taking a look at the issue.
What we notice is that the event happens too quickly and as stated earlier, the value does NOT show up in the UI after the event is fired.
@harir5 that format is perfect for a reproduction case, I don't need a repository. The important bit is that it's minimal, and you're close to minimal :D
When I execute your test on my machine on Chromium, it passes. I don't think it should, no? Does it fail on your machine?
@Skn0tt - it is very flaky. We have a JS load heavy front end with lots of plugins. We are waiting for a load event to complete. (the page is interactable, elements are editable/available but the page is supposedly not stable yet - visually at least); At the point of the fill(), we see that page.locaotr().fill() is very very quick and the value is NOT getting registered with the field properly. We see that in 1.52.0 it was working fine - as in the fields were properly getting filled.
I see! So it sounds like Playwright caught a bug in your app. If the user is too quick with typing, your app doesn't register it. This probably didn't surface in 1.52 because Playwright was slightly slower on this particular action. Like a race condition that always existed, but never surfaced on your app.
As a fix, i'd recommend disabling the input field by default, and programatically enabling it once your JS loaded far enough that it's safe to type into the field.
We are writing E2E tests with no access to the codebase. but this flakiness is something that we cannot inherently fix directly - we are adding hard waits for now but it is NOT the solution that we intend to use in the long run. We are trying to have it addressed by our development group. In the interim i thought i could ask the playwright team about it as well - this was working in 1.52.0.
I see! So it sounds like Playwright caught a bug in your app. If the user is too quick with typing, your app doesn't register it. This probably didn't surface in 1.52 because Playwright was slightly slower on this particular action. Like a race condition that always existed, but never surfaced on your app.
As a fix, i'd recommend disabling the input field by default, and programatically enabling it once your JS loaded far enough that it's safe to type into the field.
There were definitely some recent changes to fill and click. Are you suggesting that these updates allow Playwright to catch more bugs, rather than making it more flaky?
I agree that some bugs can occur when actions are performed too quickly. However, Playwright executes interactions much faster than a real user ever would, so these kinds of issues are generally low priority in practice.
Whether these kinds of bugs matter very much depends on the product requirements, so we err on the side of safety here.
If you can't change the application, then I recommend using expect.poll:
async function fillPolling(locator, value) {
await expect.poll(async () => {
await locator.fill(value)
return await locator.inputValue()
}).toEqual(value)
}
This retries filling the input until it works, and it should be faster than blanket timeouts.
@Skn0tt - Thank you very much for the work around. We will try using it and hopefully our development group is also able to resolve the issue. I appreciate the help provided.
In the meantime, i would like to keep this thread open for one more day (silly hope that others in the community might be facing an issue similar to mine). I hope that is ok.
Thank you again for looking into this issue for us.
Could you try running with the following in the config? Perhaps this could be related to a new Chromium feature that was shipped recently.
use: {
launchOptions: {
args: ['--disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,ExtensionManifestV2Disabled,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,LensOverlay,MediaRouter,PaintHolding,ThirdPartyStoragePartitioning,Translate,DeferRendererTasksAfterInput'],
},
},
Could you try running with the following in the config? Perhaps this could be related to a new Chromium feature that was shipped recently.
use: { launchOptions: { args: ['--disable-features=AcceptCHFrame,AutoExpandDetailsElement,AvoidUnnecessaryBeforeUnloadCheckSync,CertificateTransparencyComponentUpdater,DestroyProfileOnBrowserClose,DialMediaRouteProvider,ExtensionManifestV2Disabled,GlobalMediaControls,HttpsUpgrades,ImprovedCookieControls,LazyFrameLoading,LensOverlay,MediaRouter,PaintHolding,ThirdPartyStoragePartitioning,Translate,DeferRendererTasksAfterInput'], }, },
I’ve tried all these options and they helped—the tests now run consistently(multiple runs without retry and no one contains failure with that input). But could they be hiding any bugs, and is the browser I’m running still representative of what real users see?
The core thing you did by setting this arg is to enable the DeferRendererTasksAfterInput chromium feature. This was feature shipped to Chromium in the past months, and it slightly changes how element handlers and timers like setTimeout(, 0) interact.
As discussed above, these subtle changes in execution order now break your app when a user behaves as fast as Playwright does. Whether you want to fix that or not is a product decision you need to make. The "proper" decision (whatever that means) in my eyes would be to fix your application. If you can't, then you should follow the evaluate workaround I gave above. Do not use the --disable-feature args I gave above, that was solely for debugging purposes :)