sitespeedio
No VisualMetrics when setting a ViewPort of 1920x1080 for specific website
When running a test with the specific viewport (1920x1080), all of the visual metrics come back as 0ms. (see attached report). When I remove the viewport and run with the defaults I get visual metrics back with no issues. I get back lots of other metrics, so I assume it's just the visual metrics from the video that are being impacted. I have tested with other websites and this doesn't seem to be a problem, so I am not sure if there is something specific to our site that could be the cause of this.
- Exact how you run (the exact parameters): docker run --rm -v "$(pwd)":/sitespeed.io sitespeedio/sitespeed.io:14.0.0 https://www.dae.mn --browsertime.connectivity.engine tsproxy --browsertime.connectivity.profile cable --browsertime.viewPort 1920x1080
- Which version you are using: 14.0.0
- OS (if you use Docker, report the base OS): macOS Catalina 10.15.5
- Browser version (if you don't run Docker): N/A - use docker
Hi @shaqb thanks for the great bug report! I get the same problem. Testing with the viewport for https://www.sitespeed.io then it works. I've had a quick look and it looks like Visual Metrics can't handle the video but there's no errors in the Visual Metric log. If I look at the original video (without cutting the video) everything looks ok. Let me look some more.
I could see that the --orange flag for Visual Metrics seems to remove the frames so there's no left. Testing on the original file:
./visualmetrics.py -i 1-original.mp4 --orange --force --json gives only zeroes, removing orange and running ./visualmetrics.py -i 1-original.mp4 --force --json at least gives metrics except that first visual change is wrong since it picks up the orange frame as first change. I need to look into the Visual Metrics log to better understand what happens.
Thanks for the quick turnaround in checking this @soulgalore! Let me know if you need me to run anymore tests.
Thanks, no I need to look into it, I have a original video with the full web site and can reproduce with Visual Metrics and need to find the time to look into the problem. I'm away this week but can hopefully fix it next week.
Hi @soulgalore - did you have a nice vacation? I was wondering if you'd have a chance to look further into this issue? Thanks!
So I know what's causing this. A long time ago I increased the limit on how many pixels that needs to be the same to be the same color. The web site is partly orange and we use an orange color to know when the browser starts to navigate (we add a orange color on the screen and then remove it when the browser starts to navigate). With the current settings, Visual Metrics thinks the orange never goes away. I reconfigured it and will try ity for a while in a test release, to make sure it doesn't affect anything else (the original change tries to fix another problem). It will take some time though until I can release it (wanna make sure it doesn't break anything else).
Hi @soulgalore - I hope you're well! Did you manage to complete your tests to see if this fix can be merged?
No sorry, I haven't focused on that. The fix I did is maybe ok, but I wanna understand the problem better. I think the root cause is that somewhere the code is broken. Let me have a look the coming days and get back after the weekend.