Page MenuHomePhabricator

Rework how we track newcomer task edits
Closed, ResolvedPublic

Description

Following up on @Tgr's comment in T296818#7544212:

The way we tag edits (based on the Tracker class, ie. essentially setting an "in progress" flag on the page when the user clicks on the task card, clearing it when the article is saved or after 7 days, and looking at that flag for adding change tags) is inherently fragile. For non-structured edits that's unavoidable - they are just normal edits, so we don't really have a better way of detecting them. For structured edits, we might want to fix that and rely on onVisualEditorApiVisualEditorEditPreSave (a fairly reliable indicator of whether something was a structured edit) instead.

This has been brittle for a long time, I'd like to propose we scrap the current implementation and do something like the following instead:

  • Remove the Tracker (includes/NewcomerTasks/Tracker/Tracker.php) class entirely.
  • Create a generic ext.growthExperiments.NewcomerTask module with NewcomerTaskDesktopArticleTarget and NewcomerTaskMobileArticleTarget classes.
    • Those classes can inherit from the corresponding VisualEditor article target classes and override the save method, so that when a user has an active SuggestedEditSession, we can pass along data via the options array.
    • Load the NewcomerTask article target when the user starts an unstructured task, or if the user switches from structured editing to VE in AddLink/AddImage, load NewcomerTaskDesktopArticleTarget / NewcomerTaskMobileArticleTarget.
    • Then in includes/VisualEditorHooks.php in onVisualEditorApiVisualEditorEditPostSave, we can check if the save was associated with a NewcomerTask, and add the relevant tags.
    • This might also be a good time to make change tags that are specific for each task type (e.g. "Newcomer task: copyedit") in additon to the generic "Newcomer task" tag.
  • For newcomers who are not using VisualEditor, we could either decide to not care about tracking those edits (from an analytics perspective, is this important to us?), or we could hook into the 'postEdit' and 'postEditMobile' hooks and POST some data from the SuggestedEdit to an API endpoint which would then handle applying the relevant change tags

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
mewoph added a subscriber: mewoph.

Moving to needs discussion column (will add this to the agenda for next engineer sync)

Change 744911 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Quick fix for task type handling in change tagging

https://gerrit.wikimedia.org/r/744911

Change 744911 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Quick fix for task type handling in change tagging

https://gerrit.wikimedia.org/r/744911

Tgr added a subscriber: Etonkovidova.

Moving back to ready for development for the more complex part of doing something about non-structured tasks. @Etonkovidova heads up that the part about structured tasks has been merged.

kostajh triaged this task as Medium priority.Dec 13 2021, 6:52 PM

Change 754571 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/VisualEditor@master] [WIP] Add method for setting save options

https://gerrit.wikimedia.org/r/754571

Change 754572 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Track newcomer task edits through VisualEditor

https://gerrit.wikimedia.org/r/754572

Change 757153 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/extensions/GrowthExperiments@master] Suggested edits: Fix tracking

https://gerrit.wikimedia.org/r/757153

Change 757456 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Tag newcomer task edits through post-edit dialog

https://gerrit.wikimedia.org/r/757456

Change 757153 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Suggested edits: Fix tracking

https://gerrit.wikimedia.org/r/757153

Test wiki created on Patch demo by KHarlan (WMF) using patch(es) linked to this task:

https://patchdemo.wmflabs.org/wikis/6fcf545cfa/w/

@RHo @Trizek-WMF @MMiller_WMF there's a change in this patch to the edit tags that are applied. All tasks done via Suggested Edits should get the newcomer task tag (no change as before). What's new is that copyedit/references/update/expand/links tasks will each get their own change tag in addition to newcomer task, e.g. newcomer task: copyedit, newcomer task: references, and so on. That's not strictly necessary for this patch and task but it seemed like a good time to do it, to apply the same consistency in tagging edits with the entrypoint a user takes for starting a task. Does that sound OK to you? The updated tags would be:

"tag-newcomer_task": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task]]",
	"tag-newcomer_task-description": "Edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_copyedit": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: copyedit]]",
	"tag-newcomer_task_copyedit-description": "\"Copyedit\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_references": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: references]]",
	"tag-newcomer_task_references-description": "\"References\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_update": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: update]]",
	"tag-newcomer_task_update-description": "\"Update\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_expand": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: expand]]",
	"tag-newcomer_task_expand-description": "\"Expand\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_links": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: links]]",
	"tag-newcomer_task_links-description": "\"Links\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_add_link": "[[mw:Special:MyLanguage/Growth/Personalized first day/Structured tasks/Add a link|Suggested: add links]]",
	"tag-newcomer_task_add_link-description": "\"Add link\" edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_image_suggestion": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Structured_tasks/Add_an_image|Suggested: add images]]",
	"tag-newcomer_task_image_suggestion-description": "\"Add images\" edit suggested by the suggested edits module of the newcomer homepage",

@RHo @Trizek-WMF @MMiller_WMF there's a change in this patch to the edit tags that are applied. All tasks done via Suggested Edits should get the newcomer task tag (no change as before). What's new is that copyedit/references/update/expand/links tasks will each get their own change tag in addition to newcomer task, e.g. newcomer task: copyedit, newcomer task: references, and so on. That's not strictly necessary for this patch and task but it seemed like a good time to do it, to apply the same consistency in tagging edits with the entrypoint a user takes for starting a task. Does that sound OK to you? The updated tags would be:

"tag-newcomer_task": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task]]",
	"tag-newcomer_task-description": "Edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_copyedit": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: copyedit]]",
	"tag-newcomer_task_copyedit-description": "\"Copyedit\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_references": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: references]]",
	"tag-newcomer_task_references-description": "\"References\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_update": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: update]]",
	"tag-newcomer_task_update-description": "\"Update\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_expand": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: expand]]",
	"tag-newcomer_task_expand-description": "\"Expand\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_links": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Newcomer_tasks|Newcomer task: links]]",
	"tag-newcomer_task_links-description": "\"Links\" task edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_add_link": "[[mw:Special:MyLanguage/Growth/Personalized first day/Structured tasks/Add a link|Suggested: add links]]",
	"tag-newcomer_task_add_link-description": "\"Add link\" edit suggested by the suggested edits module of the newcomer homepage",
	"tag-newcomer_task_image_suggestion": "[[mw:Special:MyLanguage/Growth/Personalized_first_day/Structured_tasks/Add_an_image|Suggested: add images]]",
	"tag-newcomer_task_image_suggestion-description": "\"Add images\" edit suggested by the suggested edits module of the newcomer homepage",

Adding the extra tag SGTM but a minor thought is whether it matters to have shorter tag names for easier scanning? For example instead of Newcomer task: update, making it new task: update (or something pithier). Will defer to @Trizek-WMF and/or @MMiller_WMF who are more familiar with edit tag nomenclature.

We thought of the tag system to scale so adding new tags is easily doable. It is nice to have more precise tagging!

@RHo, it may be interesting for English, but not for other languages (I checked for French, and I don't see a gain: "Tâche du novice" -> "Nouvelle tâche").

IMO, "Newcomer task" really defines the type of tasks users scanning RCs or watchlist will find. "New task" sounds more generic than "Newcomer task". :)

Urbanecm_WMF changed the task status from Open to In Progress.Jan 31 2022, 11:33 AM

Change 754572 abandoned by Kosta Harlan:

[mediawiki/extensions/GrowthExperiments@master] [WIP] Track newcomer task edits through VisualEditor

Reason:

Going with I289c3a5ea759c97775eca1b31929d1c7f42361c2, which incorporates some of this work

https://gerrit.wikimedia.org/r/754572

Change 754571 abandoned by Kosta Harlan:

[mediawiki/extensions/VisualEditor@master] Add method for setting plugin data for save

Reason:

Actually, saveFields seems to work fine as is for what I need. Thank you!

https://gerrit.wikimedia.org/r/754571

Change 757456 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Tag newcomer task edits via VisualEditor or post-edit dialog

https://gerrit.wikimedia.org/r/757456

Rewording some text from the commit message:

What:

  • Replace the best-guess system for tracking newcomer task edits with an implementation using VisualEditor and a REST endpoint for non-VE

newcomer edits.

Why:

When a user makes an edit to an article via Special:Homepage Suggested Edits, we tag it with "newcomer task" and other change tags that are more specific ("add links", "add images"). The way we track this is a best-case approximation: when a user clicks on an article, we add an entry to a per-user cache . Then, after an edit, we check the user's cache, and if so we assume the user must have made the edit via Suggested Edits; the rationale being that the user would be very unlikely to have otherwise edited that page through some other workflow.

This has mostly worked, but there are weird edge cases where occasionally edits that don't appear to be e.g. "add image" edits are nonetheless tagged with "add image". We can be more precise about how we are tagging suggested edit edits, and that's what this patch is implementing.

How:

  • Create edit tags for each template-based task type (e.g. "newcomer task copyedit"). Those are applied in addition to the generic "newcomer task" tag.
  • For VisualEditor edits, use VE's PostSave hook to tag the edits with "newcomer task" and task type specific change tags
  • For non-VisualEditor edits, use the post-edit dialog loading process to send an an HTTP request to an endpoint and handle edit tags in the handler for that endpoint.

Change 776934 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] TemplateBasedTaskTypeHandler: Fix name of links tag

https://gerrit.wikimedia.org/r/776934

Change 776935 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] KPIs: Track reverts by task type

https://gerrit.wikimedia.org/r/776935

Change 776934 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] TemplateBasedTaskTypeHandler: Fix name of links tag

https://gerrit.wikimedia.org/r/776934

Checked in testwiki wmf.6 - all looks good.

not Switching editors

A user gets to SE article via Homepage and makes an editThe edits will have Visual edit, Newcomer task tags and the more specific Newcomer task tag, e.g. Tags: Visual edit Newcomer task Suggested: add images
A user gets to SE article but not via Homepage and makes an editThe edits won't have Newcomer task
A user gets to a non-structured SE article via Homepage, makes an edit and makes a subsequent edit(s)All edits will have e.g. for copyedit task - Tags: Visual edit Newcomer task Newcomer task: copyedit
A user gets to a structured SE article via Homepage, makes an edit and makes a subsequent edit(s)The subsequent edit will only have Tags: Visual edit

Switching editors

a non-structured task: a user makes an edit in VE, switches to Source Editor, publishesTags: Visual edit: Switched Newcomer task Newcomer task: copyedit
subsequent edits after the scenario above in Source editorTags: Newcomer task Newcomer task: copyedit

@kostajh - the description below is an edge case. Should the console error be fixed?
For structured tasks - when a user does a subsequent edit(s) in the Source editor, there will be an error in the Console. The tags are applied correctly.

  1. A user gets to a structured task via Homepage
  2. A user makes an edit in Suggestions mode, publishes an edit.
  3. In the post-edit dialog, a user clicks on "Close and edit this article again".
  4. A user then makes an edit in Source editor - the Console displays: tagNonVisualEditorEditWithGrowthChangeTags error:
    Screen Shot 2022-04-06 at 6.21.48 PM.png (660×2 px, 354 KB)

Moving back to code review, as there's one more patch to be reviewed.

Change 776935 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] KPIs: Track reverts by task type

https://gerrit.wikimedia.org/r/776935

Test wiki on Patch demo by KHarlan (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmflabs.org/wikis/6fcf545cfa/w/