Page MenuHomePhabricator

Impact module: scorecards
Closed, ResolvedPublic

Assigned To
Authored By
KStoller-WMF
Jun 14 2022, 10:29 PM
Referenced Files
F35747657: image.png
Nov 10 2022, 9:15 PM
F35747652: image.png
Nov 10 2022, 9:14 PM
F35747654: image.png
Nov 10 2022, 9:14 PM
F35747605: image.png
Nov 10 2022, 9:14 PM
F35747607: image.png
Nov 10 2022, 9:14 PM
F35747573: image.png
Nov 10 2022, 9:05 PM
F35747570: image.png
Nov 10 2022, 9:05 PM
F35715372: Screenshot 2022-11-07 at 17.04.40.png
Nov 7 2022, 5:08 PM

Description

As a new editor, I want to quickly see how many edits I've completed, and if anyone has Thanked me for my edits, because then I will better understand my impact and involvement and be more likely to return and edit again.

This task is about creating the Scorecard component that will be utilized to create the Summary Panel: T310665.

Description

  • A new component for displaying stats will be added to different parts of the impact module
  • A simple “inline” version of the scorecard made up of a label and value is used in the main impact module (in a two-column split)
  • A “block” variant will be used in the summary panel section of the impact module, with an additional icon and optional tooltip component.

image.png (1×1 px, 151 KB)

Figma design.

Acceptance Criteria

Given I'm building the Impact Module Summary Panel T310665,
Then I have the Score Card components needed to display the Summary Panel

QQQ translator instructions

Completion checklist

Functionality

  • The patches have been code reviewed and merged
  • The task passes its acceptance criteria

Engineering

  • There are existing and passing unit/integration tests
  • Tests for every involved patch should pass
  • Coverage for every involved project should have improved or stayed the same

Design & QA

  • If the task is UX/Design related: it must be reviewed and approved by the UX/Design team
  • Must be reviewed and approved by Quality Assurance.

Documentation

  • Related and updated documentation done where necessary

Event Timeline

kostajh changed the task status from Open to Stalled.Jul 1 2022, 2:32 PM
kostajh subscribed.

@KStoller-WMF do you have notes for how the "Quality" score should be calculated? If so, could you please link them here or add to the task description? Thank you.

@kostajh I'm already missing @RHo . In designs, she mentioned that we would try to match Android app's scoring. The logic is here: https://github.com/wikimedia/apps-android-wikipedia/blob/main/app/src/main/java/org/wikipedia/userprofile/UserContributionsStats.kt#L113-L115

Quality ratings are:

  • Perfect
  • Excellent
  • Very Good
  • Good
  • Okay
  • Sufficient
  • Poor

According to the Android team: quality rating relates to the number OR percentage of a user's edits that have been reverted, depending on the number of total edits the user has:

  • If the user has fewer than 100 edits, then the number is literally the total reverts, regardless of total edits. In other words, 0 reverts would be a "perfect" quality standing, while 6 or more would be considered poor, and the feature is disabled.
  • If the user has more than 100 edits, it's the percentage of reverts over total edits.

However, in user testing it was expressed that it would be demotivating to have a Perfect quality score and then lose it and never be able to return to Perfect. I'm also somewhat concerned that there will not be community consensus on what is considered Perfect / Excellent / Very Good / Good / Okay / Sufficient / Poor.

Is there a way to move forward with this task without finalizing the quality score? Or perhaps we can keep it very simple at first, like simply using the percentage of "Live edits" as the Quality score? (I assume "Live edits" percentage is easy to access as it's included in xtools but maybe I'm mistaken?) Then I can chat more with @KieranMcCann when he returns and we can iterate from there? Or I'm totally fine if we need to keep this task stalled if that sounds like significant work to iterate on the quality score based on final design input.

@kostajh I'm already missing @RHo . In designs, she mentioned that we would try to match Android app's scoring. The logic is here: https://github.com/wikimedia/apps-android-wikipedia/blob/main/app/src/main/java/org/wikipedia/userprofile/UserContributionsStats.kt#L113-L115

Me too :)

Quality ratings are:

  • Perfect
  • Excellent
  • Very Good
  • Good
  • Okay
  • Sufficient
  • Poor

According to the Android team: quality rating relates to the number OR percentage of a user's edits that have been reverted, depending on the number of total edits the user has:

  • If the user has fewer than 100 edits, then the number is literally the total reverts, regardless of total edits. In other words, 0 reverts would be a "perfect" quality standing, while 6 or more would be considered poor, and the feature is disabled.
  • If the user has more than 100 edits, it's the percentage of reverts over total edits.

However, in user testing it was expressed that it would be demotivating to have a Perfect quality score and then lose it and never be able to return to Perfect. I'm also somewhat concerned that there will not be community consensus on what is considered Perfect / Excellent / Very Good / Good / Okay / Sufficient / Poor.

Is there a way to move forward with this task without finalizing the quality score?

Two options I can think of:

  • replace it with another data point, like the user's revert count (which the quality score is indirectly expressing anyway)
  • remove it entirely, and have two data boxes instead of three

Or perhaps we can keep it very simple at first, like simply using the percentage of "Live edits" as the Quality score? (I assume "Live edits" percentage is easy to access as it's included in xtools but maybe I'm mistaken?) Then I can chat more with @KieranMcCann when he returns and we can iterate from there? Or I'm totally fine if we need to keep this task stalled if that sounds like significant work to iterate on the quality score based on final design input.

What does "Live edits" mean, exactly? Is it edits you've made that are not reverted?

What does "Live edits" mean, exactly? Is it edits you've made that are not reverted?

Exactly. So the Live edit percentage is the percentage of edits you've made that have not been reverted. So it's a somewhat more positive way to express the revert count. So if I made 10 edits and 1 was reverted, my "quality" metric would be 90%.

That being said, the fact that you had to ask is likely a good sign that it's not a very intuitive. :)

I'll chat with Ambassadors next week and work with Kieran to finalize the plan for the quality metric.

@kostajh This task was broken out to cover just creating those basic "scorecards", and then the three scorecards make up the summary panel: T310665.

So I suppose we could either make this a subtask of T310665, or just decline this task if you think it's excessive to break up the summary panel work into two tasks. I was breaking up tasks based on Figma designs, but I can see how these might not need separate tasks:

Screen Shot 2022-07-19 at 12.07.27 PM.png (824×2 px, 265 KB)

@KieranMcCann @KStoller-WMF could you please link to the latest Figma that doesn't include "Edit quality"? The existing link https://www.figma.com/file/fOa1x7hw6EM9VnaMJv7vib/Positive-reinforcement?node-id=1468%3A121646 goes to the older version.

Hi @kostajh. I’ve updated the designs on that page so you can use the original Figma link.

I notice lot of the other pages in this part of the Figma file need to be updated to reflect the recent changes to the impact module so I’ll get this done as soon as we have the designs finalised.

Hi @kostajh. I’ve updated the designs on that page so you can use the original Figma link.

I notice lot of the other pages in this part of the Figma file need to be updated to reflect the recent changes to the impact module so I’ll get this done as soon as we have the designs finalised.

Thank you @KieranMcCann, much appreciated.

Change 819113 had a related patch set uploaded (by Sergio Gimeno; author: Sergio Gimeno):

[mediawiki/extensions/GrowthExperiments@master] User impact: add edit count and thanks score cards

https://gerrit.wikimedia.org/r/819113

Sgs changed the task status from Open to In Progress.Sep 22 2022, 10:48 PM
Sgs moved this task from In Progress to Code Review on the Growth-Team (Sprint 0 (Growth Team)) board.

Change 819113 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] User impact: add edit count and thanks score cards

https://gerrit.wikimedia.org/r/819113

This can be tested in beta using the url https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Special:Impact/Sergi0&new-impact=true. Current version does not provide the informative contextual popover. That interface is still in discussion and will be implemented in T310665.

Sgs changed the task status from In Progress to Open.Sep 30 2022, 1:47 PM
Sgs moved this task from Code Review to QA on the Growth-Team (Sprint 0 (Growth Team)) board.

Just noting here that after discussion on T310666 it has been proposed that the ‘last edited’ and ‘best streak’ data points should be styled as scorecards and moved into the summary panel. Figma designs have been updated to reflect this.

These look good to me @Etonkovidova. The only thing I noticed is that we decided to change the icon used for the Thanks data point. Would it be possible to change this from the heart icon to the speech bubble with the smile? (Figma)

Screenshot 2022-11-07 at 17.04.40.png (174×394 px, 10 KB)

Other than that it looks good to me! @RHo may have additional comments though.

Change 854086 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] NewImpact: Switch Thanks scorecard icon to UserTalk

https://gerrit.wikimedia.org/r/854086

These look good to me @Etonkovidova. The only thing I noticed is that we decided to change the icon used for the Thanks data point. Would it be possible to change this from the heart icon to the speech bubble with the smile? (Figma)

Screenshot 2022-11-07 at 17.04.40.png (174×394 px, 10 KB)

Other than that it looks good to me! @RHo may have additional comments though.

Done in this patch: https://gerrit.wikimedia.org/r/854086

Change 854086 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] NewImpact: Switch Thanks scorecard icon to UserTalk

https://gerrit.wikimedia.org/r/854086

Hi @Sgs or @Etonkovidova - I am getting only a blank Impact module (on either Homepage or the standalone Impact page:

Homepage
image.png (508×1 px, 54 KB)
Impact page
image.png (816×726 px, 55 KB)

Am I doing missing something to see it? It looks mostly fine from screenshots, but I wanted to check the alignment of the "i" icon with the label (expect it to bebe vertically centered but the screenshot seems to show top-aligned), and similarly the vertically centre alignment of scorecard icons with the value:

image.png (162×848 px, 19 KB)
image.png (334×510 px, 38 KB)

Hi @Sgs or @Etonkovidova - I am getting only a blank Impact module (on either Homepage or the standalone Impact page:

Homepage
image.png (508×1 px, 54 KB)
Impact page
image.png (816×726 px, 55 KB)

Am I doing missing something to see it? It looks mostly fine from screenshots, but I wanted to check the alignment of the "i" icon with the label (expect it to bebe vertically centered but the screenshot seems to show top-aligned), and similarly the vertically centre alignment of scorecard icons with the value:

image.png (162×848 px, 19 KB)
image.png (334×510 px, 38 KB)

@KieranMcCann Do you mind giving this a quick check? You can view the new impact module by adding this URL parameter &new-impact=1 to the newcomer Homepage.

https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Special:Homepage&new-impact=1

https://test.wikipedia.org/w/index.php?title=Special:Homepage&new-impact=1

Does this look OK, or do you think we need to adjust alignment?

@KStoller-WMF Ideally we would adjust the alignment of the ‘i’ icons. I think the issue is more obvious because they are in close vicinity to the other text + icon pairing where the alignment looks correct. If it is proving to be a major blocker then I guess it could be filed as a big to fix later?

@KStoller-WMF Ideally we would adjust the alignment of the ‘i’ icons. I think the issue is more obvious because they are in close vicinity to the other text + icon pairing where the alignment looks correct. If it is proving to be a major blocker then I guess it could be filed as a big to fix later?

Thanks. Since this task is done except for this minor change, and we can't release any fixes for the next few weeks, I've created a separate task for this: T325325