Add instrumentation required to analyze engagement with talk page meta content (mobile)
Open, MediumPublic
Actions

Description

In T312309, we are going to implement a link within with the DT-enhanced mobile talk page view.

When people tap this link, they will be shown the meta content (read: templates) that can sometimes appear within talk pages' lead sections.

This task involves the work with adding the instrumentation required to analyze how people are engaging with said link so that we can: A) decide whether it should be made more/less discoverable and B) understand what – if any – impact people tapping this link has on the change(s) they go on to subsequently publish to the talk page on which the link appears.

Research questions

These are the questions the instrumentation we add will need to help us answer.

On mobile talk pages that contain meta content (read: the "About this page" link is shown), how does the quality of talk page edits vary between people who view this meta content and people do not view it?
Of all the visits people make to mobile talk pages that contain the "About this page" link, what percentage of people tap this link? How does engagement (read: taps) with the "About this page" link compare to engagement with other affordances on the page?

Instrumentation requirements

Instrumentation is required to identify the following events:

Clicks to the "About this page" link
Visits to mobile talk pages that contain the "About this page" link
Clicks to other affordances on mobile talk pages that contain the "About this page" link
Edits performed by users that clicks the "About this page" link

Done

===Instrumentation requirements are documented that will enable us to answer the ===Research questions
===Instrumentation requirements are implemented

Details

	Subject	Repo	Branch	Lines +/-
	Trigger a MobileWebUIActions show event for lede-button	mediawiki/extensions/DiscussionTools	master	+1 -0
	Log an event when the lede button is clicked	mediawiki/extensions/DiscussionTools	master	+1 -0

Customize query in gerrit

Related Objects

Mentioned Here: T312309: Unify experience for making lead section content available on mobile talk pages

Event Timeline

ppelberg created this task.Aug 4 2022, 4:18 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 4 2022, 4:18 PM

ppelberg updated the task description. (Show Details)Aug 4 2022, 4:29 PM

ppelberg moved this task from Untriaged to Upcoming on the Editing-team board.

ppelberg moved this task from Backlog to Triaged on the DiscussionTools board.

ppelberg added subscribers: MNeisler, DLynch.

On mobile talk pages that contain meta content (read: the "About this page" link is shown), how does the quality of talk page edits vary between people who view this meta content and people do not view it?

You should be aware that some tmboxes (=the content shown in MobileFrontend’s “About this page” overlay) should affect talk page edits’ quality more or less directly (e.g. talk page headers, maybe archive boxes as well—if some topic has already been discussed, restarting the discussion may get reverted), but others like WikiProject boxes or VRT permission templates shouldn’t (they are metadata of the article, not instructions for users of the talk page). Maybe the instrumentation should record not only the presence or absence of the overlay, but also types of templates included in it.

ppelberg edited projects, added Editing-team (Kanban Board); removed Editing-team.Oct 19 2022, 4:16 AM

Change 868447 had a related patch set uploaded (by DLynch; author: DLynch):

[mediawiki/extensions/DiscussionTools@master] Log an event when the lede button is clicked

https://gerrit.wikimedia.org/r/868447

gerritbot added a project: Patch-For-Review.Dec 15 2022, 5:27 PM

That would make it go into MobileWebUIActions as a click with the value lede-button.

Change 868447 merged by jenkins-bot:

[mediawiki/extensions/DiscussionTools@master] Log an event when the lede button is clicked

https://gerrit.wikimedia.org/r/868447

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.17; 2023-01-02).Dec 15 2022, 8:00 PM

Maintenance_bot removed a project: Patch-For-Review.Dec 15 2022, 8:30 PM

@MNeisler / @ppelberg that proposal-patch was done-and-launched last year, so I think this ticket just needs approval that it actually meets the not-written-yet requirements. :D

ppelberg edited projects, added Editing-team (Tracking); removed Editing-team (Kanban Board).Mar 13 2023, 5:23 PM

ppelberg moved this task from Backlog to Analytics on the Editing-team (Tracking) board.

ppelberg added a project: Product-Analytics.

MNeisler triaged this task as Medium priority.Mar 21 2023, 3:44 PM

MNeisler moved this task from Triage to Current Quarter on the Product-Analytics board.

MNeisler updated the task description. (Show Details)Apr 27 2023, 7:51 PM

Sorry for the delay! I'm finally getting around to looking at this.

I've updated the description with the instrumentation requirements for us to be able to accurately measure the identified research questions. Below is my review of it the current instrumentation meets these requirements.

✅ Clicks to the "About this page" link (Instrumented)
The new event added to mobilewebuiactiontracking (event.name = 'lede-button') will allow us to track clicks to this new link.
I confirmed that this event is logging as expected in mobilewebuiactiontracking so no further action is needed here.

✅ Visits to mobile talk pages that contain the "About this page" link (Instrumented)
This can be determined using existing instrumentation in mobilewebuiactionstracking (views logged as action = init)

✅ Edits performed by users that clicks the "About this page" link (Instrumented)
The page token can be used to join mobile page view data (in Mobilewebuiactionstracking) to editing data (in Editattemptstep) (Note EditAttemptStep is sampled 100% but events in MobileWebUIActionsTracking have different sampling rates depending on the wiki)

❓ Mobile talk pages that contain the "About this page" link (Still needs to be instrumented)
@DLynch Will these new links appear on all mobile talk pages? If not, I'll need some way I identify the talk pages where this is shown during data analysis. Would it be possible to add an event when this new link is shown (using the existing show action in the schema)

MNeisler updated the task description. (Show Details)Apr 28 2023, 3:42 PM

MNeisler reassigned this task from MNeisler to DLynch.Apr 28 2023, 3:45 PM

MNeisler moved this task from Current Quarter to Tracking on the Product-Analytics board.

Will these new links appear on all mobile talk pages?

Short answer: no.

Long answer: It'll appear on all mobile talk pages on which DT is enabled AND EITHER talkpageheader is shown OR there's non-comment content in the lede section. You'd need to do some fairly in-depth inspection of content to work this out during analysis.