Page MenuHomePhabricator

Differentiate between edits to existing sections and the creation of new sections
Closed, ResolvedPublic

Description

This task is about implementing a new event that will enable the software to distinguish edits to existing sections from edits associated with the creation of new sections.

Background

Current behavior

Init events with type=section and mechanism=url are going to be a mixture of new and existing sections. Intuitively I'd expect mechanism=url to be comparatively uncommon, on the assumption that most ways we'd create a new section involve just clicking the link on the page.

Implications of the "current behavior"

...we are unable to identify new section events created using the existing workflow because the init_mechanism behavior is inconsistent. This is needed if we want to make any comparisons between the two workflows and to accurately calculate metrics such as Adoption metric #4 in T263053 (how many people used the New Discussion Tool to add X proportions of new topics ).

Requirements

  • Add instrumentation that enables us to distinguish edits to existing sections from edits associated with the creation of new sections so that we can know things like what percentage of new talk page sections did people use the New Discussion Tool to create. See: #4 in T263053.

Open questions

  • 1. What should the new event(s) needed to satisfy the ===Requirements above be called?

Done

  • The instrumentation needed to satisfy the ===Requirements is implemented
  • Verify the instrumentation needed to satisfy the ===Requirements is working as expected
  • Update the necessary schema with the instrumentation changes needed satisfy the ===Requirements

Related Objects

StatusSubtypeAssignedTask
OpenNone
OpenNone
OpenNone
OpenNone
Resolvedppelberg
ResolvedRyasmeen
Resolvedmatmarex
ResolvedEsanders
Resolvedmatmarex
ResolvedWhatamidoing-WMF
ResolvedWhatamidoing-WMF
Resolvedppelberg
Resolvedmatmarex
ResolvedWhatamidoing-WMF
ResolvedWhatamidoing-WMF
Resolvedmatmarex
ResolvedWhatamidoing-WMF
Resolvedmatmarex
ResolvedWhatamidoing-WMF
OpenNone
OpenNone
OpenNone
OpenMNeisler
Resolvedppelberg
Resolvedppelberg

Event Timeline

Just to note, this is explicitly a change to the existing logging in VisualEditor / WikiEditor to get the semantics of init_mechanism tweaked so that (a) they're consistent, and (b) you can always tell whether there's a new section being created.

I don't think I'll need to make any changes to DiscussionTools for this, unless and until we implement a takeover for the section=new URL.

Change 660069 had a related patch set uploaded (by DLynch; owner: David Lynch):
[mediawiki/extensions/WikimediaEvents@master] Bump EditAttemptStep schema version to 21024286

https://gerrit.wikimedia.org/r/660069

Change 660075 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/WikiEditor@master] Logging init mechanism for new section direct navigation

https://gerrit.wikimedia.org/r/660075

Change 660077 had a related patch set uploaded (by DLynch; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Logging init mechanism consistency

https://gerrit.wikimedia.org/r/660077

These patches add a new possible value for init_mechanism: url-new. It will be logged when direct navigation occurs to a URL that triggers an editor for either a new page or a new section.

The VE patch also makes its use of init_mechanism=new consistent with WikiEditor/the-schema-docs. It was previously not using new when you clicked the "Create" tab after navigating to a non-existent page. (You could still distinguish this case by looking for page_id=0.)

Change 660069 merged by jenkins-bot:
[mediawiki/extensions/WikimediaEvents@master] Bump EditAttemptStep schema version to 21024286

https://gerrit.wikimedia.org/r/660069

Change 660077 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Logging init mechanism consistency

https://gerrit.wikimedia.org/r/660077

Change 660075 merged by jenkins-bot:
[mediawiki/extensions/WikiEditor@master] Logging init mechanism for new section direct navigation

https://gerrit.wikimedia.org/r/660075

META
Assigning this over to @MNeisler considering the necessary patches have been merged.

Although, QA is currently blocked considering these new events being logged depend on 1.36.0-wmf.29 being fully rolled out which has not yet happened. See: T271343.

Although, QA is currently blocked considering these new events being logged depend on 1.36.0-wmf.29 being fully rolled out which has not yet happened. See: T271343.

Update, this ticket is now ready for QA. considering 1.36.0-wmf.30 has arrived at all Wikipedias.

@ppelberg
I completed QA of the new instrumentation changes and confirmed that the events are logging as expected and we can now differentiate between edits to existing sections vs the creation of new section using current VE/Wikieditor workflows.

See summary of checks and findings below:
(Based on data from 15 Feb 2021 through 23 Feb 2021 logged in EditAtttemptStep)

  • We are logging both init_mechanism = 'url-new' and init_mechanism = 'url' events for page and section init_types. This will allow us to now distinguish between new and existing section or page VisualEditor/WikiEditor edits that occur from direct navigation to a URL.
  • The number of logged events for each init_mechanism type seem reasonable given the likelihood of occurrence. The majority of edits to create a new section (78%) (as indicated by init_mechanism = 'new' or 'url-new and init_type = section) are created by clicking on a link to a page vs direct navigation to a URL that triggers an editor for a new section. The majority of edits from direct navigation to a url (98.9%) are page (init_type = page) vs section edits.
  • Discussion tool events (as indicated by event.integration = 'discussiontools) are only recorded for init_mechnaism = click events. This is expected as these instrumentation changes were not implemented for DiscussionTools. We can still distinguish new vs existing events for discussion tools based on current instrumentation. See T265099 for details

Edit Attempts by Init Mechanism and Type
Note: Data reflects all init events across all namespaces and wikis.

init_mechanisminit_typeintegrationn_events
clickpagediscussiontools9418
clickpagepage235317
clicksectiondiscussiontools459
clicksectionpage181493
newpagepage156614
newsectionpage7989
urlpagepage112989
urlsectionpage165317
url-newpagepage223159
url-newsectionpage2285

New and existing sections edit attempts to talk pages
Note: Data reflects all init events across all talk namespaces and wikis

section_statusinit_mechanismn_events
existingclick7319
existingurl3226
newnew6097
newurl-new1543
  • The number of logged events for each section edit type seem reasonable given the liklihood of occurrence.
    • Since deployment of the instrumentation changes, a slight majority of section edits on talk pages have been to existing sections (58%).
    • Most edits (79.8%) to create new sections on talk pages are completed by clicking the link on the page vs direct navigation to a url as expected.

Further details documented in the QA notebook.

This looks great – thank you, @MNeisler.