Page MenuHomePhabricator

Profile stats on the SE home page
Closed, ResolvedPublic2 Estimate Story Points

Description

Profile stats are shown, if users have edited in Suggested edits before. It consists of:

  • Total amount of contributions (Label: Contributions)
  • The record of consecutive edit days (Label: Edit streak)
    • If streak is below 2 days, the interface outputs a date stamp (local format) of the last edit and the label changes to “Last edited": Check out this visual on Zeplin that conveys the idea.
  • Pageviews of articles edited by the user in the last 30 days [T229940] (Label: Pageviews)
  • Edit quality / reverts (Label: Edit quality)
    • Check out the section “About edit quality and blocking” below
    • To make editing more newcomer friendly, we’re going to tolerate 5 reverts and are applying an absolute scale for the first 100 edits:
      • 0 reverts = Perfect
      • 1 revert = Excellent
      • 2 reverts = Very good
      • 3 reverts = Good
      • 4 reverts = Okay
      • 5 reverts = Sufficient
        • ⛔️ Feature is paused if more than 5 reverts
      • 6 reverts = Poor
        • ⛔️ Feature is disabled if more than 7 reverts
      • > 7 reverts = Bad
    • After 100 edits, a relative scale will be applied:
      • 0.0% revert rate = Perfect
      • ≤ 1.0% revert rate = Excellent
      • ≤ 2.0% revert rate = Very good
      • ≤ 3.0% revert rate = Good
      • ≤ 4.0% revert rate = Okay
      • ≤ 5.0% revert rate = Sufficient
        • ⛔️ Feature is paused if revert rate’s higher than 5%
      • ≤ 7.0% revert rate = Poor
        • ⛔️ Feature is disabled if revert rate’s higher than 7%
      • > 7.0% revert rate = Bad
    • The icon/illustration will adapt its colors (green, orange, red) and appearance based on quality value (see "About edit quality and blocking" section below)
    • Animation: Edit quality indicator should animate to the current value when accessing the Suggested edits home for the the first time on a particular day. Check out this example animation to get an idea.
    • Copy for paused/disabled states:
      • Pause: Suggested edits is paused until October 8, 2019. Sorry %username, some of your recent contributions have been reverted. Learn how to improve your edit skills on Wikipedia with the links below.
      • Disabled (too many reverts): Suggested edits is disabled. Sorry %username, too many of your recent contributions have been reverted. Learn how to improve your edit skills on Wikipedia with the links below.
      • Disabled (blocked):
        • IP: The IP you are using is blocked from editing Wikipedia. Learn more about blocks by visiting the link below.
        • Username: Your username is blocked from editing Wikipedia. Learn more about blocks by visiting the link below.
  • Message to thank user & introduce tasks:
    • Copy: %username, thanks for your edits. Below you’ll find more opportunities to contribute to Wikipedia.
  • Tooltips
    • Uses Material Design’s standard tooltip component
    • Behavior
      • Are revealed on a single tap to provide more info about a profile stats item
      • Stays visible until users tap outside the tooltip again
    • Copy:
      • For Contributions: Your total number of contributions with Suggested edits.
      • For Edit streak: How many days without break you've contributed using Suggested edits.
      • For Pageviews: Total pageviews of items you contributed to in the last 30 days with Suggested edits.
      • For Edit quality: Based on how many times one of your contributions was reverted. Reverted edits: %numberOfRevertedEdits.

States

DefaultPausedDisabled (too many reverts)Disabled (blocked)

Tooltips

ContributionsEdit streakPageviewsEdit quality

About edit quality and blocking (Zeplin)

Reverts
First 100 edits (absolute scale)012345> 5 (paused)6≥ 7 (disabled)
After 100 edits (relative scale)0%≤ 1.0%≤ 2.0%≤ 3.0%≤ 4.0%≤ 5.0%> 5% (paused)≤ 7.0%≥ 7% (disabled)
LabelPerfectExcellentVery goodGoodOkaySufficientPoor
Screen

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 28 2019, 1:20 PM

@Jhernandez - please speak with @Dbrant and @JoeWalsh and @schoenbaechler about back-end requirements for this ticket and the blocking. @LGoto will schedule something for you.

schoenbaechler updated the task description. (Show Details)
schoenbaechler updated the task description. (Show Details)
schoenbaechler added a subscriber: Johan.
schoenbaechler renamed this task from Profile stats on the SE home page to WIP: Profile stats on the SE home page.Sep 4 2019, 3:20 PM
schoenbaechler updated the task description. (Show Details)

Action items & notes from the discussions with @Johan & @Charlotte this afternoon below.


Action items

Johan
  • Check if we should link to Wikidata diffs or not
Charlotte
  • Will check Wikidata revert rates with Mikhail (normal revert rates for new editors)
    • Robin will take into account Charlotte’s findings and come with an improved scale by the end of this week (Sep 13)
Robin
  • Define concept that it‘s about pausing/blocking the Suggested edits feature and not a general edit block
  • Detach edit quality from revert rate and go into a separate screen direction for pausing/blocking F30222649
  • Checks if Wikidata & Commons send out echo notifications
  • Will take a detailed look at Mobile talk pages
  • Will tweak pausing/blocking screen
    • Charlotte will prepare info for tech news with the updated screen
    • Johan will send tech news that Charlotte has prepared

Agenda with discussion notes

Different states of blocking vs one general block?

Johan:

  • No strong opinion
  • Possible to reach out to Wikidata community about that
  • A link to diff on Commons is fine, not sure if there’s one for Wikidata

Charlotte:

  • What’s the state of Mobile talk pages? (concerned about their state/design)
    • Johan: Unlikely to see a new Mobile talk page in the near future, as I understand it
  • Link to FAQ and improve it
Where do we lead users? Talk page? Diff? Other links?

Johan: Maybe to the Wikidata item talk page that was reverted? Not sure, will check...

How far do we want to get involved with the community’s business? (Re: Dmitry’s concerns)

Johan:

  • Pausing function not an issue with the community
  • Assumes that community would be happy about it
  • We’re creating a completely new workflow here
  • If we really want to make sure, we could ask, but Johan’s fairly certain that it’s acceptable
  • As long as we don’t interfere with account outside this function, it’s fine
Re: Level of education in the app or FAQ page

Johan: More education is better. Community is not that good in creating information flows from his experience. A lot of education would be appreciated

Thresholds for pausing/blocking?

≤ 2.0% revert rate = 99% edit quality
≤ 2.5% revert rate = 84% edit quality
≤ 3.0% revert rate = 69% edit quality
≤ 3.5% revert rate = 54% edit quality
≤ 4.0% revert rate = 49% edit quality
≤ 4.5% revert rate = 34% edit quality

5.0% revert rate = Feature is blocked* (< 20% edit quality)

Johan: Initial reaction: “It makes sense!” but would like to see some basic research about Wikidata and Commons revert rates. What are normal revert rates for new editors? Thinks that Wikidata probably has lower revert rates. Would likes this to be confirmed.

@Charlotte, updated the screens for tech news as discussed earlier.

States

DefaultPausedDisabled
Johan added a comment.Sep 10 2019, 9:57 PM

I think Wikidata diffs are useful enough that we could link to them, but they're not something we can depend on will have the information that a newcomer will need to get what's wrong. It's an additional tool rather than the explanation.

Right, action items from yesterday largely done. This has enabled us to come up with a scale which @schoenbaechler will memorialise in the ticket, combining an absolute scale for the first 100 edits, and a % scale thereafter. We will also not link to the diff of the last-reverted edit, because this will already have been sent to the user in a notification. @cooltey has a PR (https://github.com/wikimedia/apps-android-wikipedia/pull/635 ) that will enable us to show the read notifications from Commons in the app - this was not happening before. I will speak with @Jhernandez this afternoon about the scales.

Sounds good @Charlotte & @Johan. I updated the task’s description with the newly defined edit quality concept.

schoenbaechler added a subscriber: cchen.EditedSep 18 2019, 1:38 PM

By looking at timeframe: 4/23/2019 - 9/10/2019

  • Total number of unique suggested edit users with a reverted suggested edit:
Total # of users with a reverted suggested edit% of unique suggested edit users% of total unique users
1003.24% (out of 3,081)1.3% (out of 7,748)
  • Break down by edit type:
Edit Type# of users with a reverted suggested edit% of unique suggested edit users% of total unique users
image-captions893.35% (out of 2,656)3.34% (out of 2,657)
title-descriptions111.61% (out of 681)0.19% (out of 5,824)

Queries refer to suggested-edits-v2 .

Based on the low relatively low revert rates, I suggest to go with the following states:

DefaultPausedDisabled (too many reverts)Disabled (blocked)

Copy

  • Pause: Suggested edits is paused until October 8, 2019. Sorry %username, some of your recent contributions have been reverted. Learn how to improve your edit skills on Wikipedia with the links below.
  • Disabled (too many reverts): Suggested edits is disabled. Sorry %username, too many of your recent contributions have been reverted. Learn how to improve your edit skills on Wikipedia with the links below.
  • Disabled (blocked): Suggested edits is disabled. Sorry %username, the IP you are using is blocked from editing Wikipedia. Learn more about IP address blocking by visiting the help page below.

@Johan & @Charlotte:

  • Could you please review the copy above?
  • Also, in regards of helpful links, I’m mainly thinking about enhancing the Suggested edits help, respectively in case of an IP block, the Help:I_have_been_blocked page.
  • How does that sound? Or are there other (essential) links, that we should list here?

Thanks!

Johan added a comment.Sep 18 2019, 1:43 PM

I would avoid linking to the https://en.wikipedia.org/wiki/Help:I_have_been_blocked page – it's specific for English Wikipedia in some ways, with links to appeal to the decision that will be confusing since they're not blocked in that sense, they just can't use this specific workflow in the app.

Thanks @Johan, I think wasn’t quite clear enough about the disabled states. There are 3 states of ”disabled”:

  1. Feature’s disabled for a week, when 5 edits (absolute scale) or 5% (relative scale) are reverted
  2. Feature’s disabled permanently, when 7 edits (absolute scale) or 7% (relative scale) are reverted
  3. Feature’s disabled permanently, when the user’s IP or username is blocked from editing in the first place.
    • This has nothing to do with Suggested edits, we just need to communicate it when a logged in user that meets this criteria taps on Suggested edits.

Not quite sure if I understood blocking completely, my knowledge mostly bases on Help:Blocking users.

Johan added a comment.Sep 18 2019, 2:28 PM

Sorry, my misunderstanding. I'd still not link to English Wikipedia, unless we're specifically showing it in an instance where the user is trying to edit English Wikipedia while blocked on English Wikipedia – i.e. not in our suggested edits, where they actually edit Wikidata.

Charlotte added a comment.EditedSep 18 2019, 2:56 PM

Sorry, my misunderstanding. I'd still not link to English Wikipedia, unless we're specifically showing it in an instance where the user is trying to edit English Wikipedia while blocked on English Wikipedia – i.e. not in our suggested edits, where they actually edit Wikidata.

I agree. We definitely do not want to give the false impression that a user's account is blocked when we are only pausing or disabling a feature within the app, not doing anything to their account. In the case of an IP block, we still don't want to give the impression that something they've done wrong within the app is what caused that.

My suggestion is that we (meaning, I) create a new FAQ page about pausing/disabling of SE where we can address all three of these cases. And for the editing tips/tricks, we can go to the normal FAQ.

schoenbaechler added a comment.EditedSep 18 2019, 3:13 PM

@Charlotte

My suggestion is that we (meaning, I) create a new FAQ page about pausing/disabling of SE where we can address all three of these cases. And for the editing tips/tricks, we can go to the normal FAQ.

Sounds good!

We definitely do not want to give the false impression that a user's account is blocked when we are only pausing or disabling a feature within the app, not doing anything to their account.

We’re on the same page here. Still, we need to reflect the IP/username (edit) block in the interface and communicate it when users tap on the SE nav item.

My suggestion was to go with this: F30389184, but how about removing the illustration and leave out the first part of the copy (Suggested edits is disabled) to detach it from SE even more? Something unemotional, like this:

We’re on the same page here. Still, we need to reflect the IP/username (edit) block in the interface and communicate it when users tap on the SE nav item.
My suggestion was to go with this: F30389184, but how about removing the illustration and leave out the first part of the copy (Suggested edits is disabled) to detach it from SE even more? Something unemotional, like this:

I think that's a great solution. Unemotional, conveys that this doesn't have anything to do with the SE feature. @Johan, thoughts?

Johan added a comment.Sep 18 2019, 3:33 PM

Looks good to me.

schoenbaechler updated the task description. (Show Details)

@schoenbaechler - Like the updates to the task description re: streak. That makes sense. Don't forget to update the image of the Disabled (blocked) state though. 😄

schoenbaechler renamed this task from WIP: Profile stats on the SE home page to Profile stats on the SE home page.Sep 20 2019, 8:08 PM
Charlotte triaged this task as Medium priority.Sep 26 2019, 3:51 PM

Updated the task’s description with the latest infos in T231445#5555636.

Thanks @schoenbaechler - just a note to tweak this portion of the task description, which is not up to date:

✅ Green: ≥ 60%
⚠️ Orange: ≥ 30%
⛔️ Red: < 30%

Charlotte set the point value for this task to 2.Oct 22 2019, 4:11 PM

A couple of questions for folks:

  • Do we have a URL for the "Editing tips and tricks" link?
  • Do we actually want to call it "Editing tips and tricks"? (i.e. should there be tricks to making edits?)
  • Can we just have a single link instead of two, namely the one for "Suggested edits help page"?
  • The help page for blocked IP addresses seems to exist only in English. Is there a list of similar pages for other language wikis? At least the major ones?

A couple of questions for folks:

  • Do we have a URL for the "Editing tips and tricks" link?

@Johan - Is there something specific we want to send people to? Most of the on-wiki guides I've found are either language-specific, project-specific, or essays that may or may not reflect a general consensus.

  • Do we actually want to call it "Editing tips and tricks"? (i.e. should there be tricks to making edits?)

I think we can go with just "Editing tips"

  • Can we just have a single link instead of two, namely the one for "Suggested edits help page"?

Yes, if there's nothing specific that Johan can suggest as a good general guide, then I think the link to our own SE help page is probably going to be the best - and only - thing to link to.

  • The help page for blocked IP addresses seems to exist only in English. Is there a list of similar pages for other language wikis? At least the major ones?

@Johan - do you know if there are ones in other languages? The ones I had sent @Dbrant were https://utrs.wmflabs.org/ and https://en.wikipedia.org/wiki/Wikipedia:Guide_to_appealing_blocks

Johan added a comment.Oct 28 2019, 3:43 PM

I've no good solution for this at the top of my head, but I'll ask around.

schoenbaechler added a comment.EditedNov 4 2019, 12:23 PM

Listed most of the remaining issues in T231445#563096.

The only thing that’s left here is:

Can we just have a single link instead of two, namely the one for "Suggested edits help page"?

Yes, if there's nothing specific that Johan can suggest as a good general guide, then I think the link to our own SE help page is probably going to be the best - and only - thing to link to.

One link sounds good to me as well (Suggested edits help page). The screens that need to be updated are these two:

I think we can still add an “Editing tips" section on the FAQ page itself if @Johan comes up with a helpful link or a page with good infos.

Dbrant added a comment.Nov 4 2019, 7:15 PM

@Charlotte @schoenbaechler
And just one other question about "disabling" the feature because of too many reverts:
What does the user have to do to re-enable the feature? (What is the path to redemption?)

@Dbrant @Charlotte

And just one other question about "disabling" the feature because of too many reverts: What does the user have to do to re-enable the feature? (What is the path to redemption?)

Instinctively I’d say we should not make it easy to re-enable the feature and therefore not offer a way to perform this action via app. We’re increasing the feature’s visibility more and more and should be thoughtful about possible vandalism. ¨

What we definitely should include IMO is a section about the different states on the Suggested edits FAQ page. But yeah, this is one of the areas where @Johan’s expertise is highly appreciated (👋).

Johan added a comment.Nov 5 2019, 12:17 AM

Regarding the helpful links: No, as far as I can tell, we've got a new use case here – the wikis have been solving this in different ways, and maybe we could manually compile a list but that would be basically going around everywhere asking people what their best page would be, and then getting wildly varying results. We're tying to get people to set something up, but for the time being I'd assume that if we want something we'd have to write it ourselves.

Johan added a comment.Nov 5 2019, 12:34 AM

When it comes to re-enabling the feature ... we should be aware there's a fair number of reverts of good-faith edits because people don't grasp what the wikis are about, but our workflows should mainly avoid running into them as we're streamlining what people can do here. If we have already paused the functionality in a first step, then unpaused, and then it gets disabled, I've got no good idea for where to go from there.

Charlotte added a comment.EditedNov 5 2019, 12:06 PM

Regarding the helpful links: No, as far as I can tell, we've got a new use case here – the wikis have been solving this in different ways, and maybe we could manually compile a list but that would be basically going around everywhere asking people what their best page would be, and then getting wildly varying results. We're tying to get people to set something up, but for the time being I'd assume that if we want something we'd have to write it ourselves.

I concur. So before the release I'll send you an updated version of the FAQ for review @Johan. It's something that may attract community attention in the future - and of course their input on how best to explain policies, etc would be very welcome - but since this is a new feature I think we've got to take the lead in initially explaining it.

When it comes to re-enabling the feature ... we should be aware there's a fair number of reverts of good-faith edits because people don't grasp what the wikis are about, but our workflows should mainly avoid running into them as we're streamlining what people can do here. If we have already paused the functionality in a first step, then unpaused, and then it gets disabled, I've got no good idea for where to go from there.

It is precisely for this reason that we didn't envision a way for "redemption", so to speak. The user will have seen their edit quality going down over time, and will have been given a chance to pause, reflect and learn during the initial pausing of the feature - so there is relatively little chance that we will accidentally permanently lock the feature for someone who has been acting in genuine good faith. We should note that the feature locking will affect only a tiny minority of users. Even with our one-and-done locking system now, it's still only 1-3% of users who get the feature permanently locked for them. I expect actually that this percentage will hold steady, rather than going down, because as @schoenbaechler says the point of SE v3 is also to open out the usage of the feature to more people who are not as experienced as those who currently find the feature.

Of course we'll keep an eye on numbers: as with the new pausing/locking feature, which came out of a rethinking of how to balance quality with volume, so may future modifications. Hope that answers the question @Dbrant.

Congrats on the smooth work --> ready for QA signoff 🚀

Dbrant closed this task as Resolved.Dec 9 2019, 5:22 PM