Page MenuHomePhabricator

Lower sampling rate for MobileWebUIClickTracking on English Wikipedia before wmf8 is on English Wikipedia
Closed, ResolvedPublic

Description

Hey analytics team! I have a time sensitive question for which I need a decision on Wednesday (turns out this is on the train this week).
tldr: I will lower sampling rate to 1% on English Wikipedia Wednesday PM unless I hear any objections.

Questions to product analytics:

  • Will this impact any existing analysis you are doing?

Questions to Analytics:

  • What event rate should I be targeting for this schema ?

Background

We recently added an init event to the MobileWebUIClickTracking schema to mirror the init event in the DesktopWebUIClickTracking. This allows us to measure "bounce" rate.

We currently enable the DesktopWebUIClickTracking for 10% of projects in the set of desktop improvements wiki.

We currently enable MobileWebUIClickTracking for 10% on all projects

We can't compare mobile and desktop exactly, but my napkin math [1] suggests that we can expect this to result in at least 1000 events a second (compared to the current 4 events a second) which puts it on par with the VirtualPageViews schema.

I'd like to lower English Wikipedia sampling rate for mobile click tracking from the existing 10% to 1% to be cautious because of this.

[1] https://phabricator.wikimedia.org/T294738#7492873

Event Timeline

Jdlrobson created this task.

Change 737814 had a related patch set uploaded (by Jdlrobson; author: Jdlrobson):

[operations/mediawiki-config@master] Lower mobile web click tracking rate

https://gerrit.wikimedia.org/r/737814

Jdlrobson raised the priority of this task from High to Unbreak Now!.Nov 10 2021, 1:09 AM
Jdlrobson lowered the priority of this task from Unbreak Now! to High.
Jdlrobson updated the task description. (Show Details)

(This will become UBN on Thursday unless https://gerrit.wikimedia.org/r/737814 is merged)

Jdlrobson renamed this task from Lower sampling rate for MobileWebUIClickTracking on English Wikipedia to Lower sampling rate for MobileWebUIClickTracking on English Wikipedia before wmf8 is on English Wikipedia.Nov 10 2021, 1:11 AM

Mentioned in SAL (#wikimedia-operations) [2021-11-10T19:35:15Z] <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737814|Lower mobile web click tracking rate (T295432)]] (duration: 00m 57s)

Change 737814 merged by jenkins-bot:

[operations/mediawiki-config@master] Lower mobile web click tracking rate

https://gerrit.wikimedia.org/r/737814

Mentioned in SAL (#wikimedia-operations) [2021-11-10T19:41:20Z] <cjming@deploy1002> Synchronized wmf-config/InitialiseSettings.php: Config: [[gerrit:737814|Lower mobile web click tracking rate (T295432)]] (duration: 00m 55s)

Jdlrobson added a subscriber: nray.

Currently we only see 4 events a second on this schema so it's hard to confirm
https://grafana.wikimedia.org/d/000000566/overview?viewPanel=22&orgId=1

Events are lower than expected so I think the config change did have the desired impact
https://grafana.wikimedia.org/d/000000566/overview?viewPanel=22&orgId=1&from=now-12h&to=now

@nray while I'm gone could you please monitor https://grafana.wikimedia.org/d/000000566/overview?viewPanel=22&orgId=1&from=now-12h&to=now as the train rolls out to make sure we stay < 1000 events per second. Ideally we're hoping to see < 200.

Hello, I am train conductor this week. Is this resolved now?

Hi @jeena,

Yes the changes have been deployed and this is now resolved.