Page MenuHomePhabricator

Personalized praise Echo notification
Closed, ResolvedPublic

Description

User Story:

As a mentor,
I want a reminder when I have mentees that are "praise-worthy"
Because I'm busy and I won't visit the Mentor dashboard unless I get an occasional reminder

Background:

Research shows that praise and encouragement from other users increases newcomer retention.
Growth documentation:

Mediawiki overview of Positive Reinforcement project.

Designs

https://www.figma.com/file/fOa1x7hw6EM9VnaMJv7vib/Positive-reinforcement?node-id=3378%3A113899&t=ROWF6HCiMqrcKtCU-0:

Frame 5033.png (146×550 px, 9 KB)

Note from KS: Copy is still not finalized. I think we should consider using "encouragement" not "appreciation". Alternate copy ideas:

Send encouragement to new editors via your Mentor dashboard.
Encouragement from mentors improves newcomer retention.
View Mentor dashboard

or:

Send encouragement to mentees via your Mentor dashboard.
Research shows that praise and encouragement from other users increases newcomer retention.
View Mentor dashboard

Questions:

How to trigger notifications?
What is the final copy of the notification(s)?
Where does "Learn more" link to or display?
Many notifications have a "..." option to unsubscribe. Should we follow that standard or link to settings under that menu?

Acceptance Criteria:

Given I'm a Mentor,
When I have praiseworthy mentees,
Then I receive weekly notifications (or at the frequency which I have set).

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
KStoller-WMF updated the task description. (Show Details)
KStoller-WMF added a subscriber: BKots-WMF.

Change 900322 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] [WIP] Personalized praise: Add notifications

https://gerrit.wikimedia.org/r/900322

Hi @KStoller-WMF,

here is a WIP version of the notifications. Since the wording isn't finalized it yet, I mixed the various versions from the description as it seems best to me.

image.png (148×617 px, 16 KB)

It looks we'd need to significantly shorten the notification body though -- the available space seems to be very short.

Thanks for the update! It looks like the designs didn't take into account the notification character limits. I'll consider this as I finalize copy.

And in regards to finalizing copy, there are a few other places we need to standardize the language we are using (current copy includes "encouragement," "praise-worthy," and "appreciation").
@JFernandez-WMF let's work through this together when we meet on Tuesday: T326961: Positive Reinforcement: Personalised praise - finalize copy and QQQ descriptions for TranslateWiki.net

(I have to admin that writing copy isn't my favorite... so if anyone has a suggestion, please feel free to chime in.)

We should probably update the notification header/body similar to how it was done in T332744: Leveling up notifications shouldn't be truncated from Notices panel.

Change 900322 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Personalized praise: Add notifications

https://gerrit.wikimedia.org/r/900322

We should probably update the notification header/body similar to how it was done in T332744: Leveling up notifications shouldn't be truncated from Notices panel.

It currently fits the available space:

image.png (199×634 px, 18 KB)

Not sure how it changes with translations though. Let's wait for that, maybe? @KStoller-WMF, do you have any opinion on this?

In T322452#8789155, @Urbanecm_WMF wrote:
It currently fits the available space:

image.png (199×634 px, 18 KB)

This looks good to me.
Let's leave as is for now, translators can always slightly adjust the language of the second line if it doesn't fit once translated.

It does go against the Echo design guidelines by which the body should be a snippet of user-generated text, though.

Change 915460 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] PersonalizedPraise: Make notifications in line with Echo design guide

https://gerrit.wikimedia.org/r/915460

It does go against the Echo design guidelines by which the body should be a snippet of user-generated text, though.

I see, thansk for the clarification. As discussed yesterday, I'm inclined to follow the guidelines if there is no strong reason to ignore them. Uploaded a patch to fix this, which makes the notification looks like this:

image.png (326×642 px, 32 KB)

Change 915460 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] PersonalizedPraise: Make notifications in line with Echo design guide

https://gerrit.wikimedia.org/r/915460

Urbanecm_WMF changed the task status from In Progress to Open.May 4 2023, 9:22 PM
Etonkovidova subscribed.

Checked in enwiki betalabs - the notifications are in place and work as expected:

Screen Shot 2023-05-15 at 4.14.08 PM.png (620×1 px, 100 KB)

@Urbanecm_WMF - it seems that the Settings on testwiki wmf.8 do not have frequency options?

enwiki betalabstestwiki wmf.8
Screen Shot 2023-05-15 at 4.24.21 PM.png (1×1 px, 244 KB)
Screen Shot 2023-05-15 at 4.23.53 PM.png (1×1 px, 206 KB)

@Etonkovidova
I think that's likely a side-effect of our deployment strategy:
T334630: Personalized praise: Deployment of the new mentor dashboard module to Growth Pilot Wikis

Basically we have notifications disabled temporarily during this "Beta" soft launch. We plan to re-enable notifications soon / before the MVP release. This allowed us to launch the feature quickly without creating a lot of noise for mentors while the feature was still being tested. Sorry for the confusion!

As @KStoller-WMF said. I deployed testwiki in the same setup as the pilot wikis have, which is why I excluded notifications. I can certainly enable notifications on testwiki earlier (or even now) if we want to test it there earlier.

As @KStoller-WMF said. I deployed testwiki in the same setup as the pilot wikis have, which is why I excluded notifications. I can certainly enable notifications on testwiki earlier (or even now) if we want to test it there earlier.

Thx, @Urbanecm_WMF! I think that it's ok to keep testwiki setup as the pilot wikis, so for now it's fine.

For some reasons, notifications are either not being sent or not being properly instrumented:

hive (event)> select count(*) from mediawiki_mentor_dashboard_personalized_praise where year=2023 and action = 'notified' limit 10;
_c0
0
Time taken: 30.768 seconds, Fetched: 1 row(s)
hive (event)>

Moving back to In Progress for investigation.

Change 924911 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924911

Change 924911 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924911

Change 924939 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.41.0-wmf.11] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924939

Change 924940 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@wmf/1.41.0-wmf.10] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924940

Change 924939 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@wmf/1.41.0-wmf.11] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924939

Mentioned in SAL (#wikimedia-operations) [2023-05-31T13:51:31Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:924939|Personalized praise: Fix first-ever notifications (T322452)]], [[gerrit:924576|DeleteAction: Replace remaining OOUI fields (T337809)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-31T13:53:03Z] <urbanecm@deploy1002> daimona and urbanecm: Backport for [[gerrit:924939|Personalized praise: Fix first-ever notifications (T322452)]], [[gerrit:924576|DeleteAction: Replace remaining OOUI fields (T337809)]] synced to the testservers: mwdebug2002.codfw.wmnet, mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-05-31T14:02:43Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:924939|Personalized praise: Fix first-ever notifications (T322452)]], [[gerrit:924576|DeleteAction: Replace remaining OOUI fields (T337809)]] (duration: 11m 11s)

Change 924940 merged by Urbanecm:

[mediawiki/extensions/GrowthExperiments@wmf/1.41.0-wmf.10] Personalized praise: Fix first-ever notifications

https://gerrit.wikimedia.org/r/924940

Mentioned in SAL (#wikimedia-operations) [2023-05-31T14:07:12Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:924941|NewImpact: Cache empty user impact on account creation (T337320)]], [[gerrit:924940|Personalized praise: Fix first-ever notifications (T322452)]]

Mentioned in SAL (#wikimedia-operations) [2023-05-31T14:08:49Z] <urbanecm@deploy1002> urbanecm: Backport for [[gerrit:924941|NewImpact: Cache empty user impact on account creation (T337320)]], [[gerrit:924940|Personalized praise: Fix first-ever notifications (T322452)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-05-31T14:14:38Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:924941|NewImpact: Cache empty user impact on account creation (T337320)]], [[gerrit:924940|Personalized praise: Fix first-ever notifications (T322452)]] (duration: 07m 26s)

Change 925716 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] [tests] Add unit test for PersonalizedPraiseNotificationsDispatcher

https://gerrit.wikimedia.org/r/925716

Change 925716 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] [tests] Add unit test for PersonalizedPraiseNotificationsDispatcher

https://gerrit.wikimedia.org/r/925716

Notifications (wmf.12)email
Screen Shot 2023-06-08 at 3.58.27 PM.png (332×1 px, 50 KB)
Screen Shot 2023-06-08 at 4.00.54 PM.png (976×2 px, 157 KB)