Page MenuHomePhabricator

Homepage: mentorship module
Closed, ResolvedPublic

Description

This is a task for building one of the modules in the newcomer homepage: the mentorship module.

Main audience: Users who are looking for personalized help.

Primary targeted persona(s): Helena (Joiner-Inner); Yankov (Knowledge Sharer)

Secondary persona(s): Mina (Box Checker); Jae-Hee (Social Changer)

Invision mockups: https://wikimedia.invisionapp.com/share/KUQV2QDJ8A7#/350928124_NH-Mentorship_Module (in places where the mockups and the specifications on this task disagree, the specifications take precedent.)

Summary: We know from research that "human-to-human" help can be one of the best methods to engage newcomers. By being helped by another experienced editor, newcomers can ask their exact question, learn that a vibrant community exists, and develop a potential longer-term connection to the community all at once. In this module, we want to follow a similar model to the French Wikipedia community, which assigns an experienced editor to each newcomer randomly, and then automatically sends them a message "from" their mentor greeting them and saying they are available for questions. This module will appear for all newcomers, give the name of their mentor chosen randomly from a list, some basic information about the mentor, and a short message. When the newcomer clicks on a call to action to ask a question, they will go to the mentor's talk page to ask a question.

Specifications for initial version

  • General
    • This module should be present for all newcomers, regardless of whether they checked the box for getting contacted by a mentor on the welcome survey. That box will regulate which newcomers receive mentorship emails.
  • Assigning mentors
    • Mentors should sign up on a wiki page by adding a link to their userpage to a list, similar to French Wikipedia. We may want to recommend that the community protect this page. (technical note: a link will be easier to work with than an unlinked username).
    • Communities are asked to produce their own signup pages via T218237.
    • When a newcomer views their homepage, they should have a mentor assigned to them that will continue to be their mentor each time they view the page. It is worth noting that this will need to be the same mentor as the one listed in an email, once we are sending engagement emails to users (even if the user has not visited their homepage before viewing the email). Also worth noting is the possibility that future features may also need to draw upon these mentorship relationships, whether or not the user has visited their homepage or received an email.
    • Each mentor should be assigned to the newcomers such that no mentor is assigned to too many.
    • We will need to be able to run analysis on which mentors are assigned to which newcomers. (technical note: this could be stored as a hidden user preference).
    • If, at the time of account creation, the page listing mentors contains no valid mentors, or is inaccessible, the mentorship module should not be present on the homepage.
    • If a mentor's name is removed from the page, we should stop assigning them to newcomers. Newcomers that already have those mentors assigned should keep them.
  • Copy and links
    • Title: "Your mentor"
    • Section 1: mentor identity
      • Shows the mentor's username along with a gray person avatar. The username and avatar should link to the mentor's user page in a new tab.
      • Under the username, it should say the number of edits they have on that wiki and the date of the most recent edit in terms of "active days ago", like: "37,243 edits. Active 2 days ago." If their most recent edit was yesterday, it should say "Active yesterday". If their most recent was today, it should say, "Active today."
      • Next, display the text, "This experienced user knows you're new and can help you with editing."
      • Under that text should be the button that reads, "Ask your mentor a question"
      • Under the button should be a line that reads, "View your mentor's conversations." That whole sentence should be a link to the mentor's talk page that opens in a new tab.
  • Asking a question
    • When a user clicks the button to ask their mentor a question, they should be presented with a modal, which follows the same essential workflow as the modal for asking a question via the help module. The modal has two screens.
    • Screen 1: asking the question and email configuration
      • Title: "Ask your mentor"
      • Section 1: "When you ask a question, it gets posted publicly under your username, "[USERNAME]", to your mentor's talk page, which is where they can find and respond to your question." The text that reads "your mentor's talk page" should be a link to the mentor's talk page. That link should open in a new tab.
      • Section 2: this should contain the exact same workflow and business rules for viewing/adding/confirming an email address that exists in the help panel. It should contain the exact same language, which is applicable in both contexts.
      • Section 3: this should be the exact same question-asking box with same header and business rules as in the help panel.
      • There should not be an option to "Include page title with my question", since that is not applicable on the homepage.
      • This screen should have a "Post my question" button that is inactive unless there is text in the question box. This button posts to the mentor's talk page and advances the modal to "Screen 2".
      • This screen should have a "Cancel" button.
    • Screen 2: success
      • Title: "Ask your mentor"
      • Section 1: "Your question has been posted!" should be displayed in the same style, and preceded by the same checkmark as in the help panel.
      • Section 2: "Your question is now on your mentor's talk page, which is where they will answer it as soon as they can."
      • Section 3: "You will receive an email notification when a response is posted, likely within a day or so." This language should be modified according to the user's email status, in the same way as the help panel.
      • Section 4: if this is the user's first edit, they should receive the congratulations message in the same way as the help panel.
      • Section 5: "View your question on your mentor's talk page." This should be a link anchored to the header of the new question.
      • This screen should have a "Done" button.
  • On the mentor's talk page
    • New questions should be posted as new sections on the mentor's user talk page.
    • They should have automatically-generated headers that read like, "Question from [USERNAME] (22:34, 27 February 2019)". The [USERNAME] should link to the user page of the mentee.
    • They should be automatically signed by the newcomer, like with the help panel.
    • These questions should be saved with the edit tag, "Mentorship module question".
  • Note: the "Your recent questions" section is included in the mockups, but is not part of the specifications for the initial version. See T218841 for development of this capability.

Potential future specifications, listed for planning purposes

  • Add the "Your recent questions" capability to easily link users to their questions and responses. See T218841 for this capability.
  • We may want to give different configurations of the module to newcomers who do and do not check the mentorship box on the welcome survey. The idea is that those who did not check the box may still need an opportunity to control their mentorship experience. This distinction may mean that the module includes the opt-in button, or maybe it gives them a choice of mentors (see next bullet).
  • Instead of assigning a single mentor to newcomers, we may want to allow them to choose from three or four choices of mentors. They could choose by viewing User pages, or perhaps we could list topics of interest or a message from each mentor. This gives the user some control over the beginning of their mentorship experience.
  • We may want to build more structure around a mentorship program, perhaps with features that allow mentors to see which newcomers they are assigned to, and to monitor those newcomers' work. (technical note: this would require a new table for performance).
  • Allow mentors to configure parts of what the newcomer sees on their homepage, such as a personal message or avatar, e.g. "My name is User123 and my favorite topics are sports and music. I love to help newcomers get started!" (technical note: this might be easier to administer through a user preference than through a wiki page).
  • If a mentor's name is removed from the page listing mentors, we could assign the user a new mentor when they click the call-to-action to ask a question. Or we could direct the user to the help desk.
  • We could base this module around (or add) the "Email this user" feature, instead of relying on talk page communication. This feature is only available to users who have a confirmed email address.
  • Instead of having users ask the question in a modal, maybe there is a way to take them to the mentor's talk page and use a GuidedTour to take them through posting their question.
  • In the listing describing a mentor, it might be more compelling to newcomers to say how many newcomers their mentor has helped, instead of how many edits they have or how long their Wikipedia tenure has been. This might be calculated by the number of distinct newcomers that the mentor has interacted with, or perhaps just the number who have asked questions to the mentor, which may be easier to calculate. Other things we could list might include how long the mentor has been a user, or a more accurate measure of "recent activity" that could combine edits with login date or the date they last read an echo notification.
  • In the modal for asking a question, we may want to illustrate to newcomers how to check their notifications to receive their response, perhaps by indicating it with a GuidedTour addition. See T220146 for this capability.
  • After a newcomer has a certain amount of experience (e.g. 1,000 edits), the mentorship module could ask them if they are interested in becoming a mentor, and then could convert over into a module design useful for when you are a mentor.

Related Objects

StatusSubtypeAssignedTask
Resolved Rileych
ResolvedSBisson
ResolvedTrizek-WMF
Resolvedkostajh
ResolvedUrbanecm_WMF
Resolvedkostajh
ResolvedNone
Resolved Tim_WMDE
ResolvedTrizek-WMF
ResolvedCatrope
ResolvedDyolf77_WMF
ResolvedJTannerWMF
Resolvedkostajh
OpenNone
OpenNone
OpenNone
ResolvedUrbanecm_WMF
ResolvedSBisson
Declined Cntlsn
DeclinedNone
OpenNone
OpenNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
  • After asking a question
    • When the newcomer has asked a question, a new header should appear under the "Ask your mentor" button that reads, "Your recent questions".
    • Listed under that header should be the most recent three questions that the user has asked via the mentorship module.
    • They should be listed with a snippet from the beginning of the text with an ellipses. So if the user's question was, "hello Cloud atlas, how do i add a photo to an article?", the snippet might read, "hello Cloud atlas, how do i a..." We could also experiment with listing the date of the question alongside.

@MMiller_WMF I was under the impression this part had been cut for now. Can you confirm if it should be in the first version or not? I don't see it in the latest mockups.

I think the mentorship module has plenty of links to the mentor's user page and talk page already. The issue is if there's a lot of activity there and the users can't find their own questions after a while, but showing stale links wouldn't help with that.

Also their notifications would contain links to questions that have been replied to.

@SBisson -- I currently think we will want to include the "Your recent questions" capability. @Cntlsn is about to work on a design for it, and I've asked @Trizek-WMF a question for his advice on the help module task (T215986). Let's continue conversation there and see if this becomes clearer over the next couple days.

Also -- I am about to add the link to the Czech mentorship page to the task description.

Change 496452 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Homepage Mentorship module

https://gerrit.wikimedia.org/r/496452

Note that event logging for mentorship ask a question will be handled in a follow up (currently it's logged as homepage, the same as the help module).

Change 496452 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Homepage Mentorship module

https://gerrit.wikimedia.org/r/496452

Change 498416 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[operations/mediawiki-config@master] Homepage: register mentors page on labs enwiki

https://gerrit.wikimedia.org/r/498416

Change 498416 merged by jenkins-bot:
[operations/mediawiki-config@master] Homepage: register mentors page on labs enwiki

https://gerrit.wikimedia.org/r/498416

@MMiller_WMF, @Etonkovidova: The mentorship module should be on betalabs shortly.

I've made this page for mentors to sign up: https://en.wikipedia.beta.wmflabs.org/wiki/Wikipedia:Mentors Feel free to change it. The format is not important but mentors need to add a link to their user page, a signature is not sufficient.

I've deployed a config change so that the module knows where to find the list of mentors.

Once everything is deployed, you should get assigned a mentor upon visiting your homepage.

Change 498449 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Homepage: use get_class() in error handling

https://gerrit.wikimedia.org/r/498449

Change 498449 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Homepage: use get_class() in error handling

https://gerrit.wikimedia.org/r/498449

Moving to @MMiller_WMF for review
What I marked with
(1)

If a mentor's name is removed from the page, we should stop assigning them to newcomers. Newcomers that already have those mentors assigned should keep them.

The deleted mentor's name seems to be persistent. Logging out/log in, disable the Homepage feature - the mentor that was assigned is still assigned. e.g. Etonkovidova is still assigned to ET188
(2)

Under the username, it should say the number of edits they have on that wiki and the date of the most recent edit in terms of "active days ago", like: "37,243 edits. Active 2 days ago." If their most recent edit was yesterday, it should say "Active yesterday". If their most recent was today, it should say, "Active today."

Not sure where the last activity date comes from. e.g. Etonkovidova according to Special:Contributions, revision, and logging tables has the last activity around Nov 2018 (there is no entries in recentchanges), yet Mentorship module says that the user was active 26 days ago. I logged as Etonkovidova, but the last activity status did not change to more recent, so logging in does not count (or it's not tracked)?

What I marked with
(3)

[...] It is worth noting that this will need to be the same mentor as the one listed in an email, once we are sending engagement emails to users [...]

Is it still in the scope of Mentorship module to create engaging email?

(2)

Section 3: "You will receive an email notification when a response is posted, likely within a day or so." This language should be modified according to the user's email status, in the same way as the help panel.

If I understand it correctly - a mentor when answering a mentee, will mention them and a mention notification will be sent? I still do not see that Notifications in Preferences include email notifications for mentions by default in kowiki, viwki, and cswiki.

A general note:

Screen Shot 2019-03-22 at 5.16.58 PM.png (309×405 px, 36 KB)

Change 498662 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Impact: fix last active days

https://gerrit.wikimedia.org/r/498662

If a mentor's name is removed from the page, we should stop assigning them to newcomers. Newcomers that already have those mentors assigned should keep them.

The deleted mentor's name seems to be persistent. Logging out/log in, disable the Homepage feature - the mentor that was assigned is still assigned. e.g. Etonkovidova is still assigned to ET188

Yes, it is supposed to be persistent.

(2)

Under the username, it should say the number of edits they have on that wiki and the date of the most recent edit in terms of "active days ago", like: "37,243 edits. Active 2 days ago." If their most recent edit was yesterday, it should say "Active yesterday". If their most recent was today, it should say, "Active today."

Not sure where the last activity date comes from. e.g. Etonkovidova according to Special:Contributions, revision, and logging tables has the last activity around Nov 2018 (there is no entries in recentchanges), yet Mentorship module says that the user was active 26 days ago. I logged as Etonkovidova, but the last activity status did not change to more recent, so logging in does not count (or it's not tracked)?

This is a bug in the date arithmetic. I've pushed a new patch to fix it.

Change 498662 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Impact: fix last active days

https://gerrit.wikimedia.org/r/498662

IMO, it feels odd that the only way (as of now) to get back to the question you posted to your mentor is to click a link that says you'll view conversations with other users and your mentor( View your mentor's conversations with other users). Could we change this text to "View your mentor's talk page" or something else that conveys that this is how you can view your previous questions?

I tested the module out today, and I have these notes:

Bugs or issues for @SBisson, @kostajh, @Catrope

  • Mentor assignment: I opened the homepage with 20 accounts that had never opened it before. Below are the proportions of who got which mentor (which are the mentors listed on this page before I started opening accounts. I did this all over the course of about five minutes, and KHarlan (WMF) got assigned to a disproportionately high number of accounts. Can you see any reason for this to happen? Or just bad luck?
    • KHarlan (WMF): 11
    • Sbisson (WMF): 4
    • MMiller (WMF): 3
    • Roan Kattouw (WMF): 2
  • The mentor description stats don't seem to be working right. See the below for Sbisson (WMF). The homepage says "3 edits" and "Active 922 days ago", but then the screenshot of that account's contributions show 2 edits and active about 300 days ago.

image.png (164×234 px, 14 KB)

image.png (389×792 px, 76 KB)

  • Whenever I try to ask a question to the KHarlan (WMF) account, I get the below error message. Is that because that account's talk page is Flow?

image.png (428×537 px, 77 KB)

  • I like @kostajh's suggestion to change this text: "View your mentor's conversations with other users". I think we should change it to "View your mentor's conversations". I will update this in the description.

Questions for @Etonkovidova

  • I think this specification is fine to have a green: "When a newcomer views their homepage, they should have a mentor assigned..."
  • I tested this specification, and it is fine to have a green: "We will need to be able to run analysis on which mentors are assigned..."
  • I tested this specification by removing all mentors from the page, and it is fine to have a green: "If, at the time of account creation, the page listing mentors contains..."
  • I tested this specification, and it is fine to have a green: "If a mentor's name is removed from the page..."
  • Regarding the following specification, I think we have been over this a few times. I created new accounts in our target wikis to verify that they do have mention emails on by default. Have you see it a different way? "You will receive an email notification when a response is posted, likely within a day..."

I am putting this back In Progress so engineers can work on the issues listed above.

  • I like @kostajh's suggestion to change this text: "View your mentor's conversations with other users". I think we should change it to "View your mentor's conversations". I will update this in the description.

@MMiller_WMF I also think this is a good suggestion and I have updated the mockups accordingly, both InVision and Zeplin.
For the future, maybe we might try to make it more clear that the user will see their conversation(s) with their mentor, plus other conversations their mentor had with other users.

Whenever I try to ask a question to the KHarlan (WMF) account, I get the below error message. Is that because that account's talk page is Flow?

Yes, that's right.

I tested the module out today, and I have these notes:

  • Mentor assignment

This is random. We've seen early imbalance with group assignments with the survey but it fixes itself when the numbers become larger. We could implement a round robin selection (without too much trouble, I think) if we believe this is an issue.

  • The mentor description stats don't seem to be working right. See the below for Sbisson (WMF). The homepage says "3 edits" and "Active 922 days ago", but then the screenshot of that account's contributions show 2 edits and active about 300 days ago.

This is because of Flow.

The number of edits is right in this case. Creating a new topic is 2 edits: one for the topic title, and one for the first post right below. However, Special:Contributions (and the other "feed" pages) will show it as one "new topic" entry for clarity. I think the number we show is right and if something should be done, it would be on the Flow side to either better represent that the topic creation is 2 edits or really make it one edit.

Similarly, when we ask Core for the timestamp of the last contribution, it looks at the revision table and give us the last edit from there. It doesn't know anything about Flow. I think we could fix it relatively easily by using the contributions API, which Flow hooks into.

Thanks for the responses. And for the explanation about Flow. I do not think it is important for us to change it to use the Contributions API. So hearing all this, I think the only engineering item left is the following:

  • I like @kostajh's suggestion to change this text: "View your mentor's conversations with other users". I think we should change it to "View your mentor's conversations". I will update this in the description.

And I would like if @Etonkovidova could look over my comments for her above.

Change 506199 had a related patch set uploaded (by Sbisson; owner: Sbisson):
[mediawiki/extensions/GrowthExperiments@master] Update mentor's talk page link text per T216631#5135461

https://gerrit.wikimedia.org/r/506199

Change 506199 merged by jenkins-bot:
[mediawiki/extensions/GrowthExperiments@master] Update mentor's talk page link text per T216631#5135461

https://gerrit.wikimedia.org/r/506199

@MMiller_WMF I removed exclamation points.

  • The mentor description stats don't seem to be working right. See the below for Sbisson (WMF). The homepage says "3 edits" and "Active 922 days ago", but then the screenshot of that account's contributions show 2 edits and active about 300 days ago.

This is because of Flow.

The number of edits is right in this case. Creating a new topic is 2 edits: one for the topic title, and one for the first post right below. However, Special:Contributions (and the other "feed" pages) will show it as one "new topic" entry for clarity. I think the number we show is right and if something should be done, it would be on the Flow side to either better represent that the topic creation is 2 edits or really make it one edit.

It was filed as https://phabricator.wikimedia.org/T220464.

This is now waiting to be checked in our target wikis.

The mentorship module has been functioning well in production for weeks.