openjdk
8326027: [TEST_BUG]Comparing buffered images of white background frame fails in Mac
On analysis of captured image, it is observed that there are few pixels which vary slightly w.r.t to expected color. Hence tolerance is included in color comparison. Tested in mach5 multiple time and its green.
Progress
- [ ] Change must be properly reviewed (1 review required, with at least 1 Reviewer)
- [x] Change must not contain extraneous whitespace
- [x] Commit message must refer to an issue
Integration blocker
⚠️ Title mismatch between PR and JBS for issue JDK-8326027
Issue
- JDK-8326027: [TEST_BUG] Comparing screenshots of frame with white background fails in Mac (Bug - P4) ⚠️ Title mismatch between PR and JBS.
Reviewing
Using git
Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/20760/head:pull/20760
$ git checkout pull/20760
Update a local copy of the PR:
$ git checkout pull/20760
$ git pull https://git.openjdk.org/jdk.git pull/20760/head
Using Skara CLI tools
Checkout this PR locally:
$ git pr checkout 20760
View PR using the GUI difftool:
$ git pr show -t 20760
Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/20760.diff
Webrev
:wave: Welcome back tr! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.
![bridgekeeper[bot] avatar](https://avatars.githubusercontent.com/u/41768318?v=4 "Published by a pub.dev verified publisher"){kind=small}
❗ This change is not yet ready to be integrated. See the Progress checklist in the description for automated requirements.
@TejeshR13 The following label will be automatically applied to this pull request:
-
client
When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.
Have you checked what is the root cause? Are the pixels drawn incorrectly in the frame, or is the robot distorting the colors when it grabs the pixels?
I wonder whether the tolerance wouldn't be required if
Robot.createMultiResolutionScreenCapturewas used to create the screenshot and then analysing the original image which didn't go through down-scaling in High DPI environment.
Yes, I have observed that some of the HiDPI image capture have this shades of pixels from expected which would require tolerance to be considered while comparing.
Have you checked what is the root cause? Are the pixels drawn incorrectly in the frame, or is the robot distorting the colors when it grabs the pixels?
The issue occurs intermittently/might be even host specific. It occurs when Robot.createMultiResolutionScreenCapture is used to capture HiDPI image and comparing them. I believe it's related to Robot capture, rather than pixel drawn incorrectly since we haven't observed such case in headless test which actually compares colors using Graphics paint to buffered image.
Have you checked what is the root cause? Are the pixels drawn incorrectly in the frame, or is the robot distorting the colors when it grabs the pixels?
The issue occurs intermittently/might be even host specific. It occurs when Robot.createMultiResolutionScreenCapture is used to capture HiDPI image and comparing them. I believe it's related to Robot capture, rather than pixel drawn incorrectly since we haven't observed such case in headless test which actually compares colors using Graphics paint to buffered image.
I assume it's the opposite: if Robot.createMultiResolutionScreenCapture was used, the colours would be accurate (as displayed); whereas Robot.createScreenCapture scales down the screenshot which may introduce colour inaccuracy.
I wonder whether the tolerance wouldn't be required if
Robot.createMultiResolutionScreenCapturewas used to create the screenshot and then analysing the original image which didn't go through down-scaling in High DPI environment.Yes, I have observed that some of the HiDPI image capture have this shades of pixels from expected which would require tolerance to be considered while comparing.
This is why I asked the question: does the comparison remain accurate without the need for tolerance if you use Robot.createMultiResolutionScreenCapture and use the highest resolution from the returned multi-resolution image?
Have you checked what is the root cause? Are the pixels drawn incorrectly in the frame, or is the robot distorting the colors when it grabs the pixels?
The issue occurs intermittently/might be even host specific. It occurs when Robot.createMultiResolutionScreenCapture is used to capture HiDPI image and comparing them. I believe it's related to Robot capture, rather than pixel drawn incorrectly since we haven't observed such case in headless test which actually compares colors using Graphics paint to buffered image.
I assume it's the opposite: if
Robot.createMultiResolutionScreenCapturewas used, the colours would be accurate (as displayed); whereasRobot.createScreenCapturescales down the screenshot which may introduce colour inaccuracy.I wonder whether the tolerance wouldn't be required if
Robot.createMultiResolutionScreenCapturewas used to create the screenshot and then analysing the original image which didn't go through down-scaling in High DPI environment.Yes, I have observed that some of the HiDPI image capture have this shades of pixels from expected which would require tolerance to be considered while comparing.
This is why I asked the question: does the comparison remain accurate without the need for tolerance if you use
Robot.createMultiResolutionScreenCaptureand use the highest resolution from the returned multi-resolution image?
My bad, I misplaced robot.createScreenCapture() with Robot.createMultiResolutionScreenCapture. W.r.t to accuracy of Robot.createMultiResolutionScreenCapture() I don't have any comments now, I'll have to confirm it after testing.
Why are we extending this test to be run on Mac now? Looks like it was Linux specific test.
Also Title "Comparing buffered images of white background frame fails in Mac" makes it looks like some product issue. But changes are related to a single test, if we are making single test related changes its better to have [Testbug] or test in the title instead of issue description.
This color comparison failure is happening when we are checking boundary pixels of a window or is it happening everywhere?
Why are we extending this test to be run on Mac now? Looks like it was Linux specific test.
Also Title "Comparing buffered images of white background frame fails in Mac" makes it looks like some product issue. But changes are related to a single test, if we are making single test related changes its better to have [Testbug] or test in the title instead of issue description.
This color comparison failure is happening when we are checking boundary pixels of a window or is it happening everywhere?
- The product fix was linux specific, yet I guess there were review suggestions on extending the test to other OS. Hence while testing on other platforms few issues w.r.t OS for this test was observed and JBS was raised (For Windows - JDK-8326026 - Mouse drag for a transparent frame fails in Windows)
- Will update the title.
- Its not happening at borders, its at random places, u can refer to the image attached in JBS.
Why are we extending this test to be run on Mac now? Looks like it was Linux specific test. Also Title "Comparing buffered images of white background frame fails in Mac" makes it looks like some product issue. But changes are related to a single test, if we are making single test related changes its better to have [Testbug] or test in the title instead of issue description. This color comparison failure is happening when we are checking boundary pixels of a window or is it happening everywhere?
1. The product fix was linux specific, yet I guess there were review suggestions on extending the test to other OS. Hence while testing on other platforms few issues w.r.t OS for this test was observed and JBS was raised (For Windows - [JDK-8326026](https://bugs.openjdk.org/browse/JDK-8326026) - Mouse drag for a transparent frame fails in Windows) 2. Will update the title. 3. Its not happening at borders, its at random places, u can refer to the image attached in JBS.
Please share product fix for which the original test update was done.
Why are we extending this test to be run on Mac now? Looks like it was Linux specific test. Also Title "Comparing buffered images of white background frame fails in Mac" makes it looks like some product issue. But changes are related to a single test, if we are making single test related changes its better to have [Testbug] or test in the title instead of issue description. This color comparison failure is happening when we are checking boundary pixels of a window or is it happening everywhere?
1. The product fix was linux specific, yet I guess there were review suggestions on extending the test to other OS. Hence while testing on other platforms few issues w.r.t OS for this test was observed and JBS was raised (For Windows - [JDK-8326026](https://bugs.openjdk.org/browse/JDK-8326026) - Mouse drag for a transparent frame fails in Windows) 2. Will update the title. 3. Its not happening at borders, its at random places, u can refer to the image attached in JBS.Please share product fix for which the original test update was done.
JBS - JDK-8187759: Background not refreshed when painting over a transparent JFrame PR - https://github.com/openjdk/jdk/pull/17081
@jayathirthrao It looks as if image comparison is a common issue on macOS, tolerance is usually added for macOS.
Sometimes colour difference is because of a color profile used; that's why the guidelines recommend setting the same common profile on all the test systems.
I suspect the other reason is because of High DPI (Retina) displays on Macs. Yet it has never been confirmed or disconfirmed.
JBS - JDK-8187759: Background not refreshed when painting over a transparent JFrame PR - #17081
I linked the bugs in JBS and added a comment to JDK-8187759.
You could've and should've added these additional details to the bug description. I mean that it was discovered while you were working on another issue.
Have you checked what is the root cause? Are the pixels drawn incorrectly in the frame, or is the robot distorting the colors when it grabs the pixels?
The issue occurs intermittently/might be even host specific. It occurs when Robot.createMultiResolutionScreenCapture is used to capture HiDPI image and comparing them. I believe it's related to Robot capture, rather than pixel drawn incorrectly since we haven't observed such case in headless test which actually compares colors using Graphics paint to buffered image.
I assume it's the opposite: if
Robot.createMultiResolutionScreenCapturewas used, the colours would be accurate (as displayed); whereasRobot.createScreenCapturescales down the screenshot which may introduce colour inaccuracy.I wonder whether the tolerance wouldn't be required if
Robot.createMultiResolutionScreenCapturewas used to create the screenshot and then analysing the original image which didn't go through down-scaling in High DPI environment.Yes, I have observed that some of the HiDPI image capture have this shades of pixels from expected which would require tolerance to be considered while comparing.
This is why I asked the question: does the comparison remain accurate without the need for tolerance if you use
Robot.createMultiResolutionScreenCaptureand use the highest resolution from the returned multi-resolution image?
I couldn't reproduce the issue neither with Robot.createMultiResolutionScreenCapture nor with Robot.createScreenCapture since the failure occurs intermittently, not able to say anything about using Robot.createMultiResolutionScreenCapture instead of Robot.createScreenCapture .
This is why I asked the question: does the comparison remain accurate without the need for tolerance if you use
Robot.createMultiResolutionScreenCaptureand use the highest resolution from the returned multi-resolution image?I couldn't reproduce the issue neither with
Robot.createMultiResolutionScreenCapturenor withRobot.createScreenCapturesince the failure occurs intermittently, not able to say anything about usingRobot.createMultiResolutionScreenCaptureinstead ofRobot.createScreenCapture.
According to your report in JBS, the white color is not exactly white: 0xFFEE'EEEE. White color has value of 0xFFFF'FFFF.
However, it's also common on Macs (because of color profile applied?), I mean that the captured color value is the same as the value used for filling.
At the same time, I expect the color is the same across the entire frame surface. What we see is that there are variations. I wonder how many pixels on the image have different color, other than the first pixel.
This is why I asked the question: does the comparison remain accurate without the need for tolerance if you use
Robot.createMultiResolutionScreenCaptureand use the highest resolution from the returned multi-resolution image?I couldn't reproduce the issue neither with
Robot.createMultiResolutionScreenCapturenor withRobot.createScreenCapturesince the failure occurs intermittently, not able to say anything about usingRobot.createMultiResolutionScreenCaptureinstead ofRobot.createScreenCapture.According to your report in JBS, the white color is not exactly white:
0xFFEE'EEEE. White color has value of0xFFFF'FFFF.However, it's also common on Macs (because of color profile applied?), I mean that the captured color value is the same as the value used for filling.
At the same time, I expect the color is the same across the entire frame surface. What we see is that there are variations. I wonder how many pixels on the image have different color, other than the first pixel.
I guess the number pixels that doesn't match might vary, can't be constant. We can get that from the failure image though, but since the issue is intermittent, can't conclude exactly on this point.
What makes it intermittent ? Does it fail on a non-retina ? or a retina ? or both ? The test systems should all be set up with sRGB color profile, so that ought not to be the issue. Have you seen it fail locally on a retina laptop screen (with sRGB profile of course) ? Mac these days doesn't always do the simple 2x scale they used to .. my macbook is natively 3072x1920 but 1792 x 1120 is the default virtual size. That's a scale of 1.71 .. I would have to change to 1536x960 to get the 2x scale.
Still interpolating white and white ought to result in white ... but the hardware could be inaccurate ...
If you can reproduce it locally check your scale vs the native.
You could use -Dsun.java2d.uiScale=1 but I suspect that in this case it isn't going to help
What makes it intermittent ? Does it fail on a non-retina ? or a retina ? or both ? The test systems should all be set up with sRGB color profile, so that ought not to be the issue. Have you seen it fail locally on a retina laptop screen (with sRGB profile of course) ? Mac these days doesn't always do the simple 2x scale they used to .. my macbook is natively 3072x1920 but 1792 x 1120 is the default virtual size. That's a scale of 1.71 .. I would have to change to 1536x960 to get the 2x scale.
Still interpolating white and white ought to result in white ... but the hardware could be inaccurate ...
If you can reproduce it locally check your scale vs the native.
You could use -Dsun.java2d.uiScale=1 but I suspect that in this case it isn't going to help
I have not seen it fail on local machine, as I have mentioned in JBS the failure occurred on particular host (Which I failed to mention in JBS comment). And also I observed similar issue while working on this bug - JDK-8338153, hence proposing this fix. Since I'm not able to reproduce the issue either on my local machine or in CI system currently, should I close this bug ?
@TejeshR13 This pull request has been inactive for more than 4 weeks and will be automatically closed if another 4 weeks passes without any activity. To avoid this, simply add a new comment to the pull request. Feel free to ask for assistance if you need help with progressing this pull request towards integration!
@TejeshR13 This pull request has been inactive for more than 8 weeks and will now be automatically closed. If you would like to continue working on this pull request in the future, feel free to reopen it! This can be done using the /open pull request command.