Page MenuHomePhabricator

Automatically ping newcomers when their question has been answered
Open, HighPublic

Description

Mentors don't always think about pinging newcomers, or they do it the wrong way (after sending the message, which may not always trigger the notification).

Since we rely on this notification (we say that a notification will be received), a solution that makes pings happening for sure should be implemented. Newcomers are expecting to have a ping, since the popup tells them they will receive one:

image.png (416×507 px, 22 KB)

Source for the discussion: https://www.mediawiki.org/w/index.php?title=Topic:W1goooycq8ddppg1

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Iniquity renamed this task from Automatically ping mentors when their question has been answered to Automatically ping newcomers when their question has been answered.Jan 15 2021, 1:06 PM

I added the parent task, but it seems to me that they are the same :)

As an idea, in order not to create new forms, we can create a bot. It will check if there is a ping in the previous message, and if not, write in small font: @Username: Hello! Your question has been answered. ~~~~.

The second idea is to work with Convenient-Discussions and DiscussionTools projects. They can help remind users to ping participants in certain cases. But don't forget about those who don't use these tools.

As an idea, in order not to create new forms, we can create a bot. It will check if there is a ping in the previous message, and if not, write in small font: @Username: Hello! Your question has been answered. ~~~~.

@Q-bit-array started developing a bot for ruwiki to notify newcomers when they were answered on Help desk or Mentor's page. I hope we get some kind of interaction improvement :)

As an idea, in order not to create new forms, we can create a bot. It will check if there is a ping in the previous message, and if not, write in small font: @Username: Hello! Your question has been answered. ~~~~.

@Q-bit-array started developing a bot for ruwiki to notify newcomers when they were answered on Help desk or Mentor's page. I hope we get some kind of interaction improvement :)

Cool! Could you please provide a link to the code?

@Iniquity, we are still curious about this bot. Also, do you use the new magic word {{#mentor}} there?

Hey! Sorry for such a late reply, I was busy offline :)

@Iniquity, we are still curious about this bot. Also, do you use the new magic word {{#mentor}} there?

We do not use the magic word, as it is not needed in this case.

As an idea, in order not to create new forms, we can create a bot. It will check if there is a ping in the previous message, and if not, write in small font: @Username: Hello! Your question has been answered. ~~~~.

@Q-bit-array started developing a bot for ruwiki to notify newcomers when they were answered on Help desk or Mentor's page. I hope we get some kind of interaction improvement :)

Cool! Could you please provide a link to the code?

@Q-bit-array said that he will post the code when the bot starts to work stably :)

kostajh added a subscriber: RHo.

Moving to needs discussion / analysis. As far as I can tell, what we would want to do is switch the help panel / mentorship question ask feature to use ApiDiscussionToolsEdit, and then we'd get the subscribe functionality for free (and also remove our custom ApiHelpPanelPostQuestion.php code). Sounds like a win-win to me. We'd need to add on the change tag for the help / mentorship panel, but that shouldn't be hard.

@MMiller_WMF @RHo do you have any product concerns with using DiscussionTools API for creating the new section on a mentor's talk page when a question is asked via the help panel?

Moving to needs discussion / analysis. As far as I can tell, what we would want to do is switch the help panel / mentorship question ask feature to use ApiDiscussionToolsEdit, and then we'd get the subscribe functionality for free (and also remove our custom ApiHelpPanelPostQuestion.php code). Sounds like a win-win to me. We'd need to add on the change tag for the help / mentorship panel, but that shouldn't be hard.

@MMiller_WMF @RHo do you have any product concerns with using DiscussionTools API for creating the new section on a mentor's talk page when a question is asked via the help panel?

Hi @kostajh - this sounds good and I can't think of any concerns relating to the help panel and mentor module. However, I'd like to check that this won't interfere with the mentor dashboard, like how it is counting mentee questions. @Urbanecm_WMF - how, if at all, might this change impact the mentor dashboard?

I quickly checked the lastest mentors' replies on 3 wikis and about a third of mentor forget to ping. We need to address this issue.

Moving to needs discussion / analysis. As far as I can tell, what we would want to do is switch the help panel / mentorship question ask feature to use ApiDiscussionToolsEdit, and then we'd get the subscribe functionality for free (and also remove our custom ApiHelpPanelPostQuestion.php code). Sounds like a win-win to me. We'd need to add on the change tag for the help / mentorship panel, but that shouldn't be hard.

@MMiller_WMF @RHo do you have any product concerns with using DiscussionTools API for creating the new section on a mentor's talk page when a question is asked via the help panel?

Hi @kostajh - this sounds good and I can't think of any concerns relating to the help panel and mentor module. However, I'd like to check that this won't interfere with the mentor dashboard, like how it is counting mentee questions. @Urbanecm_WMF - how, if at all, might this change impact the mentor dashboard?

As long as the edits will continue to be tagged with the current tags, mentor dashboard will be fine. If we need to change the tag names for some reason thanks to this change, it should be easy to update mentor dashboard to respect new tag names.

ppelberg added subscribers: matmarex, ppelberg.

Drop-by comment with two questions...

  • Question: @RHo can you please give T290282 a quick read and let us know whether implementing it would provide newcomers the experience you are talking about creating in this ticket?
  • Question: @matmarex, can you please share some context about how ApiDiscussionToolsEdit works in response to the comment @kostajh posted above (T272146#7422839)?
  • Question: @matmarex, can you please share some context about how ApiDiscussionToolsEdit works in response to the comment @kostajh posted above (T272146#7422839)?

Moving to needs discussion / analysis. As far as I can tell, what we would want to do is switch the help panel / mentorship question ask feature to use ApiDiscussionToolsEdit, and then we'd get the subscribe functionality for free (and also remove our custom ApiHelpPanelPostQuestion.php code). Sounds like a win-win to me. We'd need to add on the change tag for the help / mentorship panel, but that shouldn't be hard.

action=discussiontoolsedit API doesn't do anything related to subscriptions right now, so I was a little confused, but then I realized you wrote that with the assumption that we'll have automatic topic subscriptions.

You'd indeed get the subscribe functionality for free in that case, but note that a) this might be some weeks/months away still b) it'll be opt-in first, and opt-out afterwards c) depending on how we do it, you might get the subscribe functionality for free even without making these changes, since we're considering enabling it for all comments regardless of how they're added.

You might instead want to use action=discussiontoolssubscribe API, to manually subscribe the user to the section they just posted. You could actually do that today (even when manual topic subscriptions are a beta feature or entirely unavailable to users, the backend part still works).

Drop-by comment with two questions...

  • Question: @RHo can you please give T290282 a quick read and let us know whether implementing it would provide newcomers the experience you are talking about creating in this ticket?

Thanks for the ping @ppelberg - @Trizek-WMF actually created this ticket so I will do my best to answer, which is yes, it looks to me like T290282 is basically the same request as this this task, to send newcomers an echo notification when their mentor replies to their question even without the mentor pinging the newcomer in the reply. However, per comments from @matmarex above on T272146#7449275 in response to @kostajh on T272146#7422839, it seems like there are ways to do this in the shorter without waiting for the automatic topic subscriptions functionality to be implemented. @kostajh - would using the action=discussiontoolssubscribe API instead be something that could be more easily/quickly implemented to resolve this task?

Drop-by comment with two questions...

  • Question: @RHo can you please give T290282 a quick read and let us know whether implementing it would provide newcomers the experience you are talking about creating in this ticket?

Thanks for the ping @ppelberg - @Trizek-WMF actually created this ticket so I will do my best to answer, which is yes, it looks to me like T290282 is basically the same request as this this task, to send newcomers an echo notification when their mentor replies to their question even without the mentor pinging the newcomer in the reply. However, per comments from @matmarex above on T272146#7449275 in response to @kostajh on T272146#7422839, it seems like there are ways to do this in the shorter without waiting for the automatic topic subscriptions functionality to be implemented. @kostajh - would using the action=discussiontoolssubscribe API instead be something that could be more easily/quickly implemented to resolve this task?

I took a look at this today, it looks like we could do this by updating QuestionPoster.php (or introduce another class specifically for this functionality) to find the comment ID from a newly posted question and then calling the discussion tools subscribe API. There are a few steps (save the comment; get the RESTBase HTML; find the comment ID; call the API) to do this.

Even though the automatic topic subscription aren't available yet, it might be easier for us to switch QuestionPoster to use DiscussionTools API for adding the new question ("section") on the help desk mentors page and getting the comment ID as part of that request, then calling the subscribe API. That way there are only two steps, and we don't have to access the internals or duplicate logic from DiscussionTools needed to get RESTBase HTML and find a comment ID.

In any case, there are some options for moving this forward, so I think the next move is for @MMiller_WMF / @RHo / @DMburugu to decide when we should do this.

Drop-by comment with two questions...

  • Question: @RHo can you please give T290282 a quick read and let us know whether implementing it would provide newcomers the experience you are talking about creating in this ticket?

Thanks for the ping @ppelberg - @Trizek-WMF actually created this ticket so I will do my best to answer, which is yes, it looks to me like T290282 is basically the same request as this this task, to send newcomers an echo notification when their mentor replies to their question even without the mentor pinging the newcomer in the reply. However, per comments from @matmarex above on T272146#7449275 in response to @kostajh on T272146#7422839, it seems like there are ways to do this in the shorter without waiting for the automatic topic subscriptions functionality to be implemented. @kostajh - would using the action=discussiontoolssubscribe API instead be something that could be more easily/quickly implemented to resolve this task?

I took a look at this today, it looks like we could do this by updating QuestionPoster.php (or introduce another class specifically for this functionality) to find the comment ID from a newly posted question and then calling the discussion tools subscribe API. There are a few steps (save the comment; get the RESTBase HTML; find the comment ID; call the API) to do this.

Even though the automatic topic subscription aren't available yet, it might be easier for us to switch QuestionPoster to use DiscussionTools API for adding the new question ("section") on the help desk mentors page and getting the comment ID as part of that request, then calling the subscribe API. That way there are only two steps, and we don't have to access the internals or duplicate logic from DiscussionTools needed to get RESTBase HTML and find a comment ID.

In any case, there are some options for moving this forward, so I think the next move is for @MMiller_WMF / @RHo / @DMburugu to decide when we should do this.

IMO, this is one of the higher priority mentorship tasks, since we want mentees who take the time to ask questions to see the response, which obviously they are far more likely to do if they get the notification. It sounds like the options are not dependent on Editing team and auto topic subscriptions, is that right @kostajh? If so, ideally @Urbanecm_WMF could pick this up as part of upcoming mentorship work.

Note that the features offered by DiscussionTools that'd be of interest for us (new topic tool and topic subscriptions) are at beta at most Wikipedias. For new topic tool, it's in beta mode at all Wikipedias except arwiki and cswiki, topic subscriptions are in beta mode everywhere.

It may be possible to make our code re-use those parts anyway, even when they're effectively turned off for most users. However, I feel that they're in beta mode for a reason, and I don't think it's right to use them while they're in beta.

Drop-by comment with two questions...

  • Question: @RHo can you please give T290282 a quick read and let us know whether implementing it would provide newcomers the experience you are talking about creating in this ticket?

Thanks for the ping @ppelberg - @Trizek-WMF actually created this ticket so I will do my best to answer, which is yes, it looks to me like T290282 is basically the same request as this this task, to send newcomers an echo notification when their mentor replies to their question even without the mentor pinging the newcomer in the reply. However, per comments from @matmarex above on T272146#7449275 in response to @kostajh on T272146#7422839, it seems like there are ways to do this in the shorter without waiting for the automatic topic subscriptions functionality to be implemented. @kostajh - would using the action=discussiontoolssubscribe API instead be something that could be more easily/quickly implemented to resolve this task?

I took a look at this today, it looks like we could do this by updating QuestionPoster.php (or introduce another class specifically for this functionality) to find the comment ID from a newly posted question and then calling the discussion tools subscribe API. There are a few steps (save the comment; get the RESTBase HTML; find the comment ID; call the API) to do this.

Even though the automatic topic subscription aren't available yet, it might be easier for us to switch QuestionPoster to use DiscussionTools API for adding the new question ("section") on the help desk mentors page and getting the comment ID as part of that request, then calling the subscribe API. That way there are only two steps, and we don't have to access the internals or duplicate logic from DiscussionTools needed to get RESTBase HTML and find a comment ID.

In any case, there are some options for moving this forward, so I think the next move is for @MMiller_WMF / @RHo / @DMburugu to decide when we should do this.

IMO, this is one of the higher priority mentorship tasks, since we want mentees who take the time to ask questions to see the response, which obviously they are far more likely to do if they get the notification. It sounds like the options are not dependent on Editing team and auto topic subscriptions, is that right @kostajh? If so, ideally @Urbanecm_WMF could pick this up as part of upcoming mentorship work.

Right, we're not blocked on Editing team. I think it's great if @Urbanecm_WMF wants to do this! :)

Note that the features offered by DiscussionTools that'd be of interest for us (new topic tool and topic subscriptions) are at beta at most Wikipedias. For new topic tool, it's in beta mode at all Wikipedias except arwiki and cswiki, topic subscriptions are in beta mode everywhere.

It may be possible to make our code re-use those parts anyway, even when they're effectively turned off for most users. However, I feel that they're in beta mode for a reason, and I don't think it's right to use them while they're in beta.

There are probably two risks:

  • the mentee doesn't get subscribed to the topic, which isn't any worse than the status quo
  • the mentee gets subscribed to the wrong topic (from glancing at the code seems like pretty low risk, but higher impact to the end user)

I would feel comfortable going ahead using the Discussion Tools API for posting the question (making a new section), unless Editing-team tells us not to.

I'm still not sure about what would should we do to accomplish this task here. We can use action=discussiontoolsedit to add new questions, that's not an issue. However, I'm not sure about subscribing users.

One way would be to use action=discussiontoolssubscribe, as @matmarex suggested. However, that requires us to know the comment ID, which isn't available easily. However, it doesn't look hard to change action=discussiontoolsedit to return the comment ID of the added comment (it already determins it when autosubscriptions are enabled). With that change done, it would be trivial to pass the ID (whatever it is) back to the API, and do the subscription.

An alternative would be to change action=discussiontoolsedit to let callers to say "subscribe the user to the topic they just created" (which would avoid the need for a round-robin with API requests; not an issue for clients on high-speed connections, but significant improvement on slower connections).

I'd also appreciate if action=discussiontoolsedit was directly able to add tags – fortunately, that also doesn't appear to be hard to change.

@matmarex Can you confirm my understanding described above is the correct one? If it is, do you (editing) have any preference?

I'm still not sure about what would should we do to accomplish this task here. We can use action=discussiontoolsedit to add new questions, that's not an issue. However, I'm not sure about subscribing users.

One way would be to use action=discussiontoolssubscribe, as @matmarex suggested. However, that requires us to know the comment ID, which isn't available easily. However, it doesn't look hard to change action=discussiontoolsedit to return the comment ID of the added comment (it already determins it when autosubscriptions are enabled). With that change done, it would be trivial to pass the ID (whatever it is) back to the API, and do the subscription.

An alternative would be to change action=discussiontoolsedit to let callers to say "subscribe the user to the topic they just created" (which would avoid the need for a round-robin with API requests; not an issue for clients on high-speed connections, but significant improvement on slower connections).

I would patch action=discussiontoolsedit to both:

  • provide the comment ID as part of its response
  • accept a parameter to automatically subscribe, if the client wishes to (defaults to whatever the global autosubscription value is if the client doesn't set this parameter)

I'd also appreciate if action=discussiontoolsedit was directly able to add tags – fortunately, that also doesn't appear to be hard to change.

+1 to adding a parameter for this as well.

@matmarex Can you confirm my understanding described above is the correct one? If it is, do you (editing) have any preference?

I'm not @matmarex, just adding my 2c.

I see two scenarios there:

  1. a mentee writes a message on their mentor's talk page. The mentor replies without a ping but the mentee is aware of the reply, because of this feature.
  2. a mentee writes a message on any other page on the wiki. Someone replies without a ping, and the mentee is not aware of the reply.

Maybe some mentees will be confused on scenario 2. It may lead them to be discouraged, or they might contact their mentor in an unhappy manner.

Also it could be interesting to work on a simular feature: T303848: Automatically ping the mentor when their mentees replied to one of their messages at the mentee talk page.

Considering that subscriptions to topics from DiscussionTools will soon come to all projects, I think this can be used in addition to the bot.
T284489: Deploy topic subscriptions (desktop) as opt-out feature at all projects

Considering that subscriptions to topics from DiscussionTools will soon come to all projects, I think this can be used in addition to the bot.
T284489: Deploy topic subscriptions (desktop) as opt-out feature at all projects

Indeed. I think we can consider this task resolved by auto subscriptions. Thanks Editing-team!

KStoller-WMF claimed this task.

I'm not sure if you can call this "resolved" yet. We wrote the code that would make it possible, but it's not enabled on Wikimedia wikis yet – topic subscriptions are only available on 20 projects (T314693), and even if they were, automatic topic subscriptions are only triggered when posting a comment using reply or new topic tool (we wrote the code to trigger them everywhere, but it doesn't seem like it's going to be deployed – T290041).

Thanks for the clarification @matmarex! Reopening the task. So, if I understand this right, to make use of topic subscriptions, we'd have to use the subscribe API (action=discussiontoolssubscribe) or add the topic via action=discussiontoolsedit&paction=addtopic, is that right?

Yeah, or convince my team that we should enable automatic topic subscriptions in all editing interfaces.

Why only newcomers? I would love this.

Oh wait, this is only for mentored users. I would like if this could be added for all questions asked by all users (assuming they enabled it).

Why only newcomers? I would love this.

Oh wait, this is only for mentored users. I would like if this could be added for all questions asked by all users (assuming they enabled it).

I think the wider Topic Subscriptions project by the Editing-team is what you're looking for. Topic Subscriptions are actually available on the Wikimedia wikis now, but you probably need to enable it first (via your preferences). If you want to try this out, check the "Beta features" tab of your preferences and enable "Discussion Tools" there. After you do that, you should be able to enable Topic Subscriptions in the "Discussion pages" section in the "Editing" tab. Hope this helps!

This task is here for the Growth-Team to integrate Topic Subscriptions with the mentorship features.

Why only newcomers? I would love this.

Oh wait, this is only for mentored users. I would like if this could be added for all questions asked by all users (assuming they enabled it).

I think the wider Topic Subscriptions project by the Editing-team is what you're looking for. Topic Subscriptions are actually available on the Wikimedia wikis now, but you probably need to enable it first (via your preferences). If you want to try this out, check the "Beta features" tab of your preferences and enable "Discussion Tools" there. After you do that, you should be able to enable Topic Subscriptions in the "Discussion pages" section in the "Editing" tab. Hope this helps!

This task is here for the Growth-Team to integrate Topic Subscriptions with the mentorship features.

I hate StructuredDiscussions though.

Luckily, StructuredDiscussions is entirely unrelated to everything we're discussing here. Feel free to try out the beta feature and leave comments about it on its discussion page (not here). Thanks!

Luckily, StructuredDiscussions is entirely unrelated to everything we're discussing here. Feel free to try out the beta feature and leave comments about it on its discussion page (not here). Thanks!

Thank God Jimbo Wales.

@KStoller-WMF could you please weigh in on whether you'd like us to pursue this?

Unfortunately, I don't think we have data on:

  • how often mentors use the notification template correctly to ping a mentee in a response
  • whether the mentee sees the notification
  • whether the mentee sees the mentor's reply

So it would be hard to point to the outcome of this task and say that it measurably improved things, but intuitively, doing this task seems like a good idea to me.

Yes, I think this is worth pursuing even if we can't precisely measure the improvement. Newcomers expect to get notified when their question is answered.

That being said, my understanding is that this task is still blocked...

We wrote the code that would make it possible, but it's not enabled on Wikimedia wikis yet – topic subscriptions are only available on 20 projects (T314693), and even if they were, automatic topic subscriptions are only triggered when posting a comment using reply or new topic tool (we wrote the code to trigger them everywhere, but it doesn't seem like it's going to be deployed – T290041).

I'll move to blocked, but @Urbanecm_WMF should feel free to move this back to Needs Discussion or even to our current sprint if I'm mistaken and this work is actionable.

Unfortunately, I don't think we have data on:

  • how often mentors use the notification template correctly to ping a mentee in a response
  • whether the mentee sees the notification
  • whether the mentee sees the mentor's reply

And is it technically impossible (or too much work) to have data on these? At first sight, they seem to be easily measurable (although I don’t know how exactly metrics on Wikimedia work, so I may be wrong).

Yes, I think this is worth pursuing even if we can't precisely measure the improvement. Newcomers expect to get notified when their question is answered.

That being said, my understanding is that this task is still blocked...

We wrote the code that would make it possible, but it's not enabled on Wikimedia wikis yet – topic subscriptions are only available on 20 projects (T314693), and even if they were, automatic topic subscriptions are only triggered when posting a comment using reply or new topic tool (we wrote the code to trigger them everywhere, but it doesn't seem like it's going to be deployed – T290041).

I'll move to blocked, but @Urbanecm_WMF should feel free to move this back to Needs Discussion or even to our current sprint if I'm mistaken and this work is actionable.

AFAICS, that comment is no longer accurate, automatic topic subscriptions are deployed everywhere (@matmarex, please correct me if I'm wrong about this). That said, they're only deployed for DiscussionTools-originating comments only.

I believe the task is now actionable for us, and that we need to do two things:

  1. Create a shared understanding between us and Editing on how should Growth-Team trigger auto subscription for its own post (calling their API sounds reasonably easy, but they're all marked as Internal)
  2. Do the change in our code

Boldly moving this to Upcoming work (and Prioritized on the mentorship board).

Unfortunately, I don't think we have data on:

  • how often mentors use the notification template correctly to ping a mentee in a response
  • whether the mentee sees the notification
  • whether the mentee sees the mentor's reply

And is it technically impossible (or too much work) to have data on these?

When it comes to metrics, the underlying challenge often is specifically and precisely defining what is going to be measured, while not making the implementation terribly complex.

At first sight, they seem to be easily measurable (although I don’t know how exactly metrics on Wikimedia work, so I may be wrong).

For example, in this case, when building the measurements, we'd need to decide what actual action should be instrumented in both web/email notifications (and possibly, also the push notifications in mobile apps; not sure how often those are by newcomers). In another words, for each of those notification venues, it needs to be decided what technical action (clicking which button) is "reading notifications".

We'd also have to decide how to handle excerpts shown in the notification text. Perhaps sometimes people don't open the talk page, because the excerpt in the notification email is sufficient.

Those are two examples of questions that pop to my head as I read this thread. They (and other questions) can definitely be answered, but it's not a trivial thing to do, and it takes a while.

In another words, for each of those notification venues, it needs to be decided what technical action (clicking which button) is "reading notifications".

Perhaps sometimes people don't open the talk page, because the excerpt in the notification email is sufficient.

Good point, I didn’t think about this. Thanks for the explanation!

Change 944955 had a related patch set uploaded (by Urbanecm; author: Urbanecm):

[mediawiki/extensions/GrowthExperiments@master] [WIP] QuestionPoster: Switch to DiscussionTools

https://gerrit.wikimedia.org/r/944955

I think this now needs us to wire into the topic subscriptions, which are already available in production. From our end, seems to be accomplishable by two ways:

  1. Switching to action=discussiontoolsedit API (probably would need an internal API call in QuestionPoster::submitWikitext
    • This seems like the easiest option in the end. The only trickier part is adding the tag. DiscsusionTools can be easily changed to have a tags parameter (which is through a set of indirections passed down to action=edit in MW Core), but that cannot be used to add software-defined tags (so they can serve as a source of truth).
    • The tags issue can be solved by ChangeTagsAllowedAddHook, which would work, but it would (theoretically) allow other users to add the "help panel question" tag manually. Personally, I don't think that's a terrible issue to have.
    • If we want to prohibit users from adding the tag, we can get the rev ID from the API response and add it manually.
  2. Only invoking DiscussionTools subscription (perhaps by directly calling DT's classes).
    • Avoids the need to call the API internally, but doesn't have much other advantages.

Opinions welcomed!

Change 944955 abandoned by Urbanecm:

[mediawiki/extensions/GrowthExperiments@master] [WIP] QuestionPoster: Switch to DiscussionTools

Reason:

https://gerrit.wikimedia.org/r/944955