Page MenuHomePhabricator

Add page view token to UIAction schemas
Closed, ResolvedPublic

Description

Per the approach we converged on in T304036, this ticket involves the work of adding the pageview token to the DesktopWebUIActions and MobileWebUIActions schemas.

Doing the above is necessary to ensure the four schemas listed below are sampling events at the same rates and using the same methods:

  • VisualEditorFeatureUse
  • EditAttemptStep
  • DesktopWebUIActions
  • Talk_Page_Edit

Requirements

  1. A unique one-time token is assigned for each desktop page load in DesktopWebUIActions
  2. A unique one-time token is assigned for each mobile web page load in MobileWebUIActions
  3. The new pageToken fields in the UIAction schemas contain the same values from mw.user.getPageviewToken() that's currently logged in EditAttemptStep as page_token.
  4. It's possible to correlate a page view logged in DesktopWebUIActions or MobileWebUIactions with any corresponding edit events (both page and discussiontool related events) that occur on the same page load logged in EditAttemptStep.
    • This will be used to calculate metrics regarding users that view a talk page and then engage (i.e. click an edit button, start a new topic, reply, etc.).
    • For example, we want to be able to know the timestamp of a pageview (action=init) event from the UI actions schemas and find the timestamp for any subsequent edit attempts (action = init ) event in the editing schemas.

Done

  • 1. @MNeisler drafts ===Requirements
  • 2. @DLynch to proposes a patch that implements the === Requirements
  • 3. The Editing Team consults with relevant members of the Web Team to ensure they support the approach we'll have proposed in "2."
  • 4. Once all patches to resolve this task are merged, @MNeisler to verify instrumentation is working as expected

Related Objects

Event Timeline

Change 789324 had a related patch set uploaded (by DLynch; author: DLynch):

[schemas/event/secondary@master] DesktopUIActions/MobileUIActions: add pageToken field

https://gerrit.wikimedia.org/r/789324

Change 789325 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/WikimediaEvents@master] DesktopUIActions/MobileUIActions: add pageToken field

https://gerrit.wikimedia.org/r/789325

Those patches add a new field to both UIActions schemas called pageToken. It contains the same value from mw.user.getPageviewToken() that's logged in EditAttemptStep as page_token.

I populated the Requirements section in the task description based on the page token approach discussion in T304036. I believe the next step is to discuss and review with the Web team.

@jwang Adding you to this task for awareness. We're looking at adding a pageToken field to the DesktopUIActions and MobileWebUIActions schemas to correlate events logged in these schemas with events logged in EditAttemptStep based on page load. See requirements in the task description and proposed patch. Let us know if you have any questions or concerns about the change or want to discuss (FYI - I'm out next week but will be back on 9 May 2022).

MNeisler triaged this task as Medium priority.May 6 2022, 3:52 PM
MNeisler moved this task from Next 2 weeks to Doing on the Product-Analytics (Kanban) board.

hi @jwang! Per the comment Megan posted in T307640#7910093, I'm assigning this task over to you. Although, please let me know if there's another way we ought to go about requesting you/the Web Team review this proposal.

It's possible to correlate a page view logged in DesktopWebUIActions or MobileWebUIactions with any corresponding edit events (both page and discussiontool related events) that occur on the same page load logged in EditAttemptStep.

Just to make sure this is spelled out here:

  • This is only going to be present on the client-side events (because the page token is only available client-side), so WikiEditor's init and save EditAttemptStep events will lack a page_token.
  • Per T303654's findings, someone making an edit using WikiEditor (i.e. integration=page + editor_interface=wikitext) won't be sampled from the same ID as DesktopWebUIActions on that pageload (and, you know, any UI click to launch said editor will have been on a different pageload anyway).
  • (Not a limitation, just a consideration.) There will potentially be multiple EditAttemptStep sessions per page_token. Someone could, in theory, leave 500 comments via DiscussionTools, interspersed with editing the full page wikitext in the 2017 wikitext editor, all while still being technically on the same pageload.

@MNeisler , I asked web team on slack. we don't have concern on recording page token in desktopwebuiactionstracking schema.

FYI, the current sample rate of desktopwebuiactionstracking is 20% on pilot wikis.

Rebasing required: T306648 just made its own changes to the UIActions schemas, so I'm going to have to put everything back on top of that.

Change 789324 merged by jenkins-bot:

[schemas/event/secondary@master] DesktopUIActions/MobileUIActions: add pageToken field

https://gerrit.wikimedia.org/r/789324

Change 789325 merged by jenkins-bot:

[mediawiki/extensions/WikimediaEvents@master] DesktopUIActions/MobileUIActions: add pageToken field

https://gerrit.wikimedia.org/r/789325

jwang updated the task description. (Show Details)

I've verified that the new pageToken field added to both the dekstopwebui and mobilewebui schemas is logging as expected, starting on 18 May 2022. See summary of confirmed checks below:

A unique one-time token is assigned for each desktop page load in DesktopWebUIActions
A unique one-time token is assigned for each mobile web page load in MobileWebUIActions

✅ Confirmed.

  • There is a unique page token assigned to every pageview (action = 'init') logged within DesktopWebUIActions and MobileWebUIActions.
  • The same page token appears to be applied to any subsequent click events (action = 'click') that occurs on that page load.
  • A browsing session (event.token) can have multiple different page tokens (indicating there were multiple page loads within a session) but there is only one session associated with each page token as expected.

The new pageToken fields in the UIAction schemas contain the same values from mw.user.getPageviewToken() that's currently logged in EditAttemptStep as page_token.
It's possible to correlate a page view logged in DesktopWebUIActions or MobileWebUIactions with any corresponding edit events (both page and discussiontool related events) that occur on the same page load logged in EditAttemptStep.

This will need to be QA'd after the sampling rate configs are synchronized as part of the work being done in T303654

@ppelberg - Reassigning to you for sign off

I've verified that the new pageToken field added to both the dekstopwebui and mobilewebui schemas is logging as expected, starting on 18 May 2022. See summary of confirmed checks below:

Wonderful.

The new pageToken fields in the UIAction schemas contain the same values from mw.user.getPageviewToken() that's currently logged in EditAttemptStep as page_token.
It's possible to correlate a page view logged in DesktopWebUIActions or MobileWebUIactions with any corresponding edit events (both page and discussiontool related events) that occur on the same page load logged in EditAttemptStep.

This will need to be QA'd after the sampling rate configs are synchronized as part of the work being done in T303654

@MNeisler considering the sampling rate configs will be synchronized in T309260, would it be accurate for me to think that the QA you're describing above should happen in T309260 as well?

If so, can you please populate T309260's QA section with what will be required to verify, "...the UIAction schemas contain the same values from mw.user.getPageviewToken() that's currently logged in EditAttemptStep as page_token..."

@ppelberg - Reassigning to you for sign off

Once we resolve the question above, I'm going to close this out.

@MNeisler considering the sampling rate configs will be synchronized in T309260, would it be accurate for me to think that the QA you're describing above should happen in T309260 as well?

Yes, that's correct

If so, can you please populate T309260's QA section with what will be required to verify, "...the UIAction schemas contain the same values from mw.user.getPageviewToken() that's currently logged in EditAttemptStep as page_token..."

I've updated T309260 to include this QA requirement.