Page MenuHomePhabricator

Suggested Edits: Decide how to avoid scenarios where one article receives excessive #Newcomer Task attention
Closed, ResolvedPublic

Description

User story & summary:

As a newcomer, I want to make productive edits, so that my contribution is valued and not reverted.
As an experienced editor, I don't want newcomers to edit the same article again and again, because sometimes the article is in a good state but the maintenance template hasn't yet been removed. (Or perhaps the maintenance template is still applicable, but the newcomers simply don't yet have the knowledge to meaningfully improve the article).

Background & research:

This task is important because:

  • Experienced editors from multiple communities have suggested we consider an improvement to avoid situations like this.
  • User feedback: https://www.mediawiki.org/wiki/Topic:Ydiefxefrduyrhop "At a minimum, set up throttling, and limit newcomer tasks per article. There are 6.9 million of them, please don't send all the new editors to one article I am trying to maintain. Secondly, please do not send any newcomers to fix or add links that are not needed per the guideline, improperly formed, or piped when they should not be."
Potential Solutions:
  • Set up throttling, and limit newcomer tasks per article.
  • If there are less than X suggestions of one task type, then don't suggest that task type to newcomers.
  • If an article's last edit was a #newcomer-task, then remove it from the Suggested Edit task queue.
  • If there are less than X of either "easy" task type, then start to surface "medium" task types.
  • Intelligent throttling. By that I mean, create some heuristic about what constitutes reasonable newcomer edits at an article, and not just by a dumb, "per-article" figure (although that is a great start), but more based on human resources available to evaluate newcomer edits, and deal with the edits, and the editors. For example, if an article has a lot of watchers and much recent activity from a dozen active experienced editors (who are not reverted), then that is a strong assistance pool that could handle three or four or half a dozen newcomers editing the article. Otoh, if the article was released from Draft the day before yesterday, with nearly all of the edits by a single editor, or a couple of AWB typos or a bot edit, then sending twelve newcomers there is going to swamp the single author of the Draft in dealing solely with that article.
  • Some way to reduce lead edits. Short of forbidding them, try to think of a way to guide newcomers to the body of the article, where all of their edits should rightly be as a starting point. Which is not to say article leads don't need links or any of the other things, but it's overwhelmingly the destination of newcomers (just like it is for article readers) with a far higher chance of a questionable edit that doesn't improve the article, or one that violates a guideline.
  • Some way to provide basic policy & guideline support to the newcomer task. Maybe an edit notice or something with do's and dont's summarizing the main points of guidance we offer on links (or whatever the task may be).
  • Handle scoped maintenance templates ('article' vs 'section' scope). As a quick fix to avoid slamming experienced editors, you could just turn off newcomer tasks for maintenance templates with non-default scope (usually., 'section'; rarely also 'paragraph', 'table' or other terms). Or o direct the newcomer editor into doing a section edit of the section with a scoped maintenance template.
  • Other ideas...?
NOTE: Communities can enable the "Add a link (Structured task)" via Special:CommunityConfiguration if the issue is mainly in relation to not having enough link suggestions available.
Acceptance Criteria:
  • Document potential solutions to this problem
  • Discuss the pros and cons of various approaches (with Ambassadors and community)
  • Add a task/epic to address the problem

Event Timeline

KStoller-WMF moved this task from Inbox to Needs Discussion on the Growth-Team board.

Notes from discussion:

  • We might want to consider addressing the underlying issue.
  • Should we offer suggested edits at all if there are only x (a small number of) articles in the pool? (Will this be a problem on smaller wikis).
  • Ideally maintenance templates are removed once they are no longer relevant.
  • Should there be a new step to suggest removing the template if there isn't anything else to do? (Then prompt to make the change, or surface the article to the mentor, or suggest the change on the talk page?)
  • Should this option be community configurable so wikis can decide what works best for them.
KStoller-WMF renamed this task from Suggested Edits: Don't suggest an article in Suggeted Edits, if the last edit was a Suggested Edit (Newcomer Task) to Suggested Edits: Decide how to avoid scenarios where one article receives excessive #Newcomer Task attention.Aug 1 2024, 9:16 PM
KStoller-WMF claimed this task.
KStoller-WMF updated the task description. (Show Details)
KStoller-WMF moved this task from Needs Discussion to Backlog on the Growth-Team board.

This issue has been raised again: https://www.mediawiki.org/wiki/Topic:Ydiefxefrduyrhop

Although I agree that ideally we should fix the underlying issue (and make sure there are enough tasks to ensure newcomers aren't being directed to a limited set of articles), I also think at this point it makes sense to consider improvements where there is a very limited set of "easy" Suggested Edits available.

Further ideas:

  1. Intelligent throttling.

By that I mean, create some heuristic about what constitutes reasonable newcomer edits at an article, and not just by a dumb, "per-article" figure (although that is a great start), but more based on human resources available to evaluate newcomer edits, and deal with the edits, and the editors. For example, if an article has a lot of watchers and much recent activity from a dozen active experienced editors (who are not reverted), then that is a strong assistance pool that could handle three or four or half a dozen newcomers editing the article.

Otoh, if the article was released from Draft the day before yesterday, with nearly all of the edits by a single editor, or a couple of AWB typos or a bot edit, then sending twelve newcomers there is going to swamp the single author of the Draft in dealing solely with that article.

  1. Some way to reduce lead edits.

Short of forbidding them, try to think of a way to guide newcomers to the body of the article, where all of their edits should rightly be as a starting point. Which is not to say article leads don't need links or any of the other things, but it's overwhelmingly the destination of newcomers (just like it is for article readers) with a far higher chance of a questionable edit that doesn't improve the article, or one that violates a guideline.

  1. Some way to provide basic policy & guideline support to the newcomer task.

Maybe an edit notice or something with do's and dont's summarizing the main points of guidance we offer on links (or whatever the task may be).

  1. Handle scoped maintenance templates ('article' vs 'section' scope)

I had wondered why I got slammed with newcomers adding links (mostly wrong/not needed, and sometimes the same link added repeatedly by other newcomers after I unlinked it) to the lead of Brazilian criminal justice when there was no template at the top of the page, and the lead did not need links.

The reason (I presume; can you confirm?) turned out to be the presence of a More links needed template with section scope, at the top of section § Recent reforms and initiatives. Of the 20 newcomer edits with added links, all were applied to the lead where they were not needed, and none to the section where they were needed. (Note: that article has several other maint. templates with section scope, like expansion needed, and update needed, but I presume those are not connected with newcomer tasks, and so did not cause any activity.)

As a quick fix to avoid slamming experienced editors, you could just turn off newcomer tasks for maintenance templates with non-default scope (usually., 'section'; rarely also 'paragraph', 'table' or other terms). If that would involve a long release window due to testing or for whatever reason, please lmk, and I can modify some of the templates temporarily to conditionally include a {{no newcomer tasks}} template within the maintenance template, when the template is for non-article scope. But imho that should be strictly temporary, and any such modification should be removed when handled in the Newcomer tasks code.

A more robust solution would be to direct the newcomer editor into doing a section edit of the section with a scoped maintenance template. (Does VE even support that? if not, maybe some instructions to the newcomer editor or a section link to get them directly to the section in question.)

Thanks for all of the ideas!

The reason (I presume; can you confirm?) turned out to be the presence of a More links needed template with section scope, at the top of section § Recent reforms and initiatives.

Yes, I believe that is likely what happened. The unstructured links task doesn't limit editors to just a specific section.
The Add a link (Structured Task) would make this a non-issue.

A more robust solution would be to direct the newcomer editor into doing a section edit of the section with a scoped maintenance template. (Does VE even support that? if not, maybe some instructions to the newcomer editor or a section link to get them directly to the section in question.)

I think that this might be a general UX improvement to consider, but I don't think it will address the main frustration here where a small subset of articles end up getting a lot of newcomer attention because we have few tasks identified as "easy" for newcomers to start doing. That being said, the Add a link (Structured Task) does something similar to what you are suggesting... it essentially presents a limited version of VE so it limits the mistakes newcomers can make.

Thanks again for the feedback! I've started a discussion with engineers. And I added your ideas to the tasks "Potential Solutions" list.

I’ve spoken with several Growth team engineers to explore both the solutions outlined in this task and additional potential ideas.

While "intelligent throttling" might offer an ideal user experience, it would likely be inefficient to compute and complex to implement, as it would require us to process all recent edits of an article to gather the necessary data.

A simpler and more feasible solution would be to avoid surfacing an article as a Suggested Edit if the article has been edited within the last x days. Essentially, we’d filter out any articles edited recently, regardless of the edit type, whether it’s a Newcomer Task or a regular edit. This approach could address multiple issues, such as preventing newcomers from being directed to articles involved in ongoing edit wars. It also creates a lag between Newcomer edits, allowing more experienced editors time to update maintenance tags if needed.

The main drawback that comes to mind is this could reduce the number of available Suggested Edits for newcomers, so we’ll need to closely monitor the task queue to ensure there are still enough "easy" tasks available.

@Mathglot and @Trizek-WMF What other downsides do you foresee with this approach? Additionally, what do you think would be a reasonable initial value for x?

Resolving this task with the decision documented here:
T364350#10208821