Page MenuHomePhabricator

Wrong last visual change on WebPageReplay mobile
Closed, ResolvedPublic

Description

I've been doing changes to use the upstream version of Visual Metrics and forgot that we where using a specific version that added more "fuzz" when we tried to find the last frames. Without that it looks like this:

And there's no visual change I can see at least. The last time I saw this, Pat was thinking it's Chrome that throws on anti alias but I'm not sure. I will look into it.

This is an example: http://webpagereplay-wikimedia.s3-website-us-east-1.amazonaws.com/?prefix=enwiki/mobile/chrome/100/Barack_Obama/2019-02-04-13-03/

Event Timeline

Peter created this task.Feb 4 2019, 1:32 PM
Restricted Application added subscribers: Gilles, Aklapper. · View Herald TranscriptFeb 4 2019, 1:32 PM
Peter added a comment.Feb 11 2019, 2:04 PM

Hmm this was really annoying. I've been trying to backtrack/understand how this works. I've been trying with two images that is generated with a diff with VisualMetrics:


If I test them in a tool without any fuzz I can see small diff (the red):

However when I run with fuzz factor 10% (as Visual Metrics) there's no diff.

I've tried it manually:

docker run -v “$(pwd)“:/video sitespeedio/visualmetrics-deps convert “/video/ms_006367.png” “/video/ms_006450.png” -crop 241x372+0+8 miff:- | compare -metric AE - -fuzz 10% null:

And I get a 0 back (0 pixels diff). So Visual Metrics shouldn't say it diff ... let me see if I can recreate the full video locally.

Peter added a comment.Feb 15 2019, 5:37 AM

Adding an example file. Running default visual change gives last visual change something like 11 seconds but the "real" one should be more like 6 seconds.

Peter added a comment.Feb 15 2019, 5:58 AM

There's two ways to fix this: when visual metrics removes frames from the end, it uses a fuzz factor of 10% (that's default that is used all over) and then compares pixel by pixel and the there must be zero pixels diff, for making sure that the screenshots are the same. However when looking for Visual Elements, the element also uses a 10 % fuzz but a 2% of pixels that can differs.

I've tried changing both and with that test file a 13% fuzz fixes the problem or a pixels diff of 0.01%. I think adding that pixel diff is better so we keep the same fuzz factor on all places. I'll make the change later today and push it on Monday.

Peter added a comment.Feb 17 2019, 6:11 PM

I've pushed the fix now (with an annotation), I'll check that it really works before I close it.

Peter added a comment.Feb 17 2019, 6:17 PM

No that didn't help, there is still something like 13 s last visual change happening for Obama on mobile. I've changed so that we look a 0.01% of the pixels can differ (that worked locally) but it seems that the diff is too small. Let me increase it a little and try again tomorrow.

Peter added a comment.Feb 17 2019, 6:44 PM

I did some more changes and now uses 0.05% diff that's ok, that made it work so it runs in the servers now. Let me have a look tomorrow that everything is ok and then close.

Peter added a comment.Feb 17 2019, 7:10 PM

Looks much better!

Peter closed this task as Resolved.Feb 18 2019, 6:48 AM

This looks good now. Mobile changed like this for all URLs I checked:

And for desktop no change: