What is tested:

• Ensure the DesktopWebUIClickTracking schema is logging screen resolution as expected

Bugs or potential issues
In the desktopwebuiactionstracking schema , the viewportsizebucket field has mixed formats for the screen size. For example, for the same screen size, it was recored as 1200-2000 or 1200px-2000px. It shows on all wikis and all days since the instrument was enabled.

Query

SELECT DISTINCT event.viewportSizeBucket
FROM event.desktopwebuiactionstracking 
WHERE year = 2022 AND month = 4 AND day=17
AND event.name = 'ui.toc'

What is tested:

• New events logged in the mediawiki_web_ui_scroll schema

Bugs or potential issues
In event.mediawiki_web_ui_scroll schema, the fr.wikipedia.org has most scroll-to-toc events and sessions, 4.25 times of the 2nd place pt.wikipedia.org. It's not proportional to the pageviews ratio of frwiki and ptwiki, which is 2.5 based on the 2021 pageviews in wiki comparison.

Query

SELECT meta.domain,  COUNT(1) AS events,
COUNT(DISTINCT web_session_id) AS sessions,
COUNT(DISTINCT page_id) AS pages ,
min(meta.dt) as first_event_ts
FROM  event.mediawiki_web_ui_scroll
WHERE year=2022 and month=4
AND action='scroll-to-toc'
GROUP BY meta.domain
ORDER BY events DESC
LIMIT 10000

There was a recent drop of events to this schema on https://grafana.wikimedia.org/d/000000566/overview?orgId=1&from=now-30d&to=now which needs explaining and could be contributing to the issue with events.

I've looked at events per session on frwiki and compared to jawiki, ptwiki and dewiki, (to see if maybe a single session responsible for a large number of events) and there doesn't seem to be any obvious outliers that would suggest that.

events by session

frwiki	jawiki	ptwiki	dewiki

Looking at events per day on frwiki, they seem to be evenly distributed as well (other than the drop on April 6th), so no spike that would account for extra events either :(

frwiki events per day

How many editors are there on French compared to Portuguese? Note, we only track scroll to top (currently) for logged-in users (although https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/773628/3/extension.json we introduced the possibility to log anon traffic)

The drop in events on the 7th is still a bit confusing to me and makes me worry there was an unexpected bug/change in behaviour in https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/773628)

Change 785919 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[operations/mediawiki-config@master] [Web scroll] Restore original sampling rate

https://gerrit.wikimedia.org/r/785919

Change 785919 merged by jenkins-bot:

[operations/mediawiki-config@master] [Web scroll] Restore original sampling rate

https://gerrit.wikimedia.org/r/785919

Okay, so I can explain this.

Previously, we disabled the web ui scroll schema if the user was anonymous and in the sample (1% for French Wikipedia, 10% for all other projects)

In Ic7a50b30b0275dc37f09f80f491ab713cbb16294 merged on March 29th, we changed this so that:

• The sampling rate applied to logged in users (1% for French Wikipedia, 10% for all other projects)
• A new sampling rate applied to anonymous users set to 0%

This new change would have gone live on 7th April so this explains the significant drop we see here on the graph:

The patch https://gerrit.wikimedia.org/r/785919 will rectify the issue by restoring the levels. After it's backported I predict we should see volume larger than the previous spike on 5th May as we should be seeing all the original events plus additional events for the scroll to table of contents event (given toc events seem to be on a par with scrolling to top we'll most likely see around 20 events a second (double the existing 10).

In event.mediawiki_web_ui_scroll schema, the fr.wikipedia.org has most scroll-to-toc events and sessions, 4.25 times of the 2nd place pt.wikipedia.org. It's not proportional to the pageviews ratio of frwiki and ptwiki, which is 2.5 based on the 2021 pageviews in wiki comparison.

If I look at the scroll-to-top event [1] before the introduction of the new toc event, the number of scroll to top events on Portuguese is also around 25% of the volume of events on French so it feels like this is a behavioural characteristic of PTwiki compared to FRWIKI. I'd love us to understand that a bit more but i don't think this is a bug in the instrumentation.

[1]

SELECT meta.domain,  COUNT(1) AS events,
COUNT(DISTINCT web_session_id) AS sessions,
COUNT(DISTINCT page_id) AS pages ,
min(meta.dt) as first_event_ts
FROM  event.mediawiki_web_ui_scroll
WHERE year=2022 and month=4 and day < 7
AND action='scroll-to-top'
GROUP BY meta.domain
ORDER BY events DESC
LIMIT 10000

How many editors are there on French compared to Portuguese? Note, we only track scroll to top (currently) for logged-in users (although https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikimediaEvents/+/773628/3/extension.json we introduced the possibility to log anon traffic)

As we only track scrolls by logged-in users, edits by logged-in users could be a reference other than pageviews.
Edits Metrics

Scroll events

The number of scroll events on fr.wikipedai.org is proportional to the number of edits by logged-in users on pt.wikipedia and he.wikipedia, though it's not proportional to other wikis. We believe it's not an issue as long as the sample rate is consistent during the experiment on each wiki.

Mentioned in SAL (#wikimedia-operations) [2022-04-25T21:25:13Z] <catrope@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:785919|[Web scroll] Restore original sampling rate (T305442)]] (duration: 01m 01s)

Please assign the ticket to me after engineer has resolved the issue on their side. I will continue to QA.

@jwang sorry, I should have been more explicit about that issue. There was a typo in the instrumentation as it was initially deployed (my bad).

The values without "px" were a typo. The corrected values, with "px" took effect on about April 12th. The wrong values might still persist for a while because of caching, but at a much lower rate.

Looking at the number of events received with the wrong values, it looks like they are slowly fading away, and the values with "px" are coming in steady since about April 12th.

"720-999" per day (typo)	"720px-999px" per day (correct)

"320-719" per day (typo)	"320px-719px" per day (correct)

The correct values for the schema are

<320px
320px-719px
720px-999px
1000px-1199px
1200px-2000px
>2000px

Both the 'px' and non-px values represent the same thing, but it may be easiest to just disregard the non-px events and use the ones with px.

@Jdrewniak Thank you very much for the info! I have documented it in the sub ticket T306558.