Page MenuHomePhabricator

GrowthExperiment 0 edits message is unclear (does not mention that the count is article edits)
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

  • Create a new account
  • Ask a talk page query
  • Get a echo notification telling you you've made your first edit
  • Go to your home page

What happens?:

Screenshot from 2023-11-27 20-09-45.png (704×810 px, 81 KB)

We end up at a situation where the software tells us that we have 0 edits and congratulates us for making our first edit at the same time

What should have happened instead?:
The '0 edits so far' message should make it clear that it only considers article edits maybe '0 edits to articles so far' ?

Reported by @Stuartyeates in the NPP Discord :)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

Change 977777 had a related patch set uploaded (by Sohom Datta; author: Sohom Datta):

[mediawiki/extensions/GrowthExperiments@master] Reword confusing message when 0 articles have been edited

https://gerrit.wikimedia.org/r/977777

Urbanecm_WMF subscribed.

Hello @Soda,

Thank you very much for taking the time to report the task and to submit a patch. At the very moment, the empty state of the Impact module is displayed when the user has some mainspace edits. However, once the Impact module gets enabled (=you have at least one mainspace edit), it looks like this:

image.png (1×596 px, 157 KB)

Here, for the "Total edits" number, it makes use of the total edit count (the same one can find in their Special:Preferences page), not the mainspace count. In other words, the Impact module appears to be self-inconsistent: when deciding whether to display, it makes use of the mainspace edit count, but when generating the numbers to display, it makes use of the total edit count.

Personally, I find this self-inconsistency potentially confusing and semantically incorrect. I think that ideally, we should be using only one type of edits across the whole module. In the past, we used to be displaying article editcount instead of total editcount, so that's probably where this inconsistency originates (see T325336 for the change to total edit count). Personally, I think we should continue using the total edit count, which would mean that we'd display the module when the user has any edit (instead of changing the copy here).

I've informed Growth's product manager, @KStoller-WMF, of this task. Until she has a chance to take a look and comment, I'm moving this task to the Needs Discussion column on the team's workboard.

Thanks again for creating the topic and bringing this issue to our attention.

Hello @Soda,

Thank you very much for taking the time to report the task and to submit a patch. At the very moment, the empty state of the Impact module is displayed when the user has some mainspace edits. However, once the Impact module gets enabled (=you have at least one mainspace edit), it looks like this:

image.png (1×596 px, 157 KB)

Here, for the "Total edits" number, it makes use of the total edit count (the same one can find in their Special:Preferences page), not the mainspace count. In other words, the Impact module appears to be self-inconsistent: when deciding whether to display, it makes use of the mainspace edit count, but when generating the numbers to display, it makes use of the total edit count.

Personally, I find this self-inconsistency potentially confusing and semantically incorrect. I think that ideally, we should be using only one type of edits across the whole module. In the past, we used to be displaying article editcount instead of total editcount, so that's probably where this inconsistency originates (see T325336 for the change to total edit count). Personally, I think we should continue using the total edit count, which would mean that we'd display the module when the user has any edit (instead of changing the copy here).

I did notice the inconsistency when reading the code but assumed it was intentional :)

I've informed Growth's product manager, @KStoller-WMF, of this task. Until she has a chance to take a look and comment, I'm moving this task to the Needs Discussion column on the team's workboard.

Thanks again for creating the topic and bringing this issue to our attention.

👍

First of all, thank you @Soda for creating this task and associated patch! I agree that we need to improve something here, and your patch might be the simplest fix.

This might be too much information, but for some context about how we ended up here:

The impact module was originally designed to only account for article edits, but that evolved based on editor feedback. It seemed editors wanted the Impact module edit count to show the same total as Special:Contributions. Showing just an article edit count was confusing for some and also it meant that we needed to calculate the metric, which added additional confusion and frustration because we capped the metric at 1,000 edits so it wasn't too costly to calculate.

My understanding is that currently the scorecard data (Total edits, Thanks received, Last edited, and Longest streak) relate to all edits, not just article edits. But then the "Views on articles you've edited" data only relates to article edits. I personally think that is OK, and seems to fit what the majority of editors want to see.

In T352080#9361034, @Urbanecm_WMF wrote:
Personally, I think we should continue using the total edit count, which would mean that we'd display the module when the user has any edit (instead of changing the copy here).

That was my original inclination as well, but from our A/B test, it appears that may have had (a very slight) negative impact on activation. More details are here: T338640: Impact Module: New editors who have edited outside of the main namespace should see the Impact module empty state & here: https://www.mediawiki.org/wiki/Growth/Positive_reinforcement#Impact_module_experiment_results

In other words, I'm hesitant to revert that change and show the impact module to accounts that have any edit, unless we also consider some design improvements for editors who have only edited outside of mainspace (and then A/B test again). Until we can refocus on further Impact Module improvements, I think the change @Soda is suggesting is at least a step in the right direction.

@JFernandez-WMF Do you have any thoughts on this? Or preference between copy: '0 edits to articles so far' vs '0 article edits so far'

thank you @Soda for filing this and Kirsten for the ping! I agree that making improvements to copy is a great first step - both messages suggested look great to me, so I am in favor of both options!

Thanks everyone. Those points make a lot of sense. In that case, let's go ahead and merge @Soda's patch that implements the change. +2'ed in Gerrit and moving this to QA.

Change 977777 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Reword confusing message when 0 articles have been edited

https://gerrit.wikimedia.org/r/977777

Etonkovidova claimed this task.
Etonkovidova subscribed.

Checked in wmf.9 - the message is updated: "0 edits to articles so far"

Screen Shot 2023-12-14 at 3.41.01 PM.png (1×792 px, 198 KB)
mobile
Screen Shot 2023-12-14 at 3.41.41 PM.png (1×842 px, 130 KB)
Screen Shot 2023-12-14 at 3.41.53 PM.png (1×876 px, 162 KB)

Thanks for testing, Elena!

And thanks again for working on this, @Soda !