Page MenuHomePhabricator

[Technical Investigation] What constraints limit what notifications can be sent and how they can be delivered?
Closed, ResolvedPublic

Description

This task is a technical investigation intended to identify the constraints that will limit what on-wiki communication-related notifications can be sent and how they can be delivered. This task also represents the work involved with identifying what changes would need to be made to support the "Use cases" listed below.

We anticipate this investigation will surface new questions that may require additional research to answer.

Background

T233447 encapsulates the work of increasing the likelihood people receive relevant responses to the comments they post and conversations they start on talk pages, across Wikipedia's namespaces.

A key part to, "...increasing the likelihood people receive relevant responses to the comments they post and conversations they start..." is the people capable of and interested in providing said "relevant responses" being aware their input is needed.

As such, this task is about identifying what technical constraints will limit what on-wiki communication notifications can be sent and how they can be delivered.

Use cases

The below is a living list of uses for on-wiki communication notifications.

Use caseKey questionRelated eventsPotential implementation
1."Did someone respond to the question I asked? What did they say?Event 1: Someone adds a comment in response to a comment you posted. Event 2: Someone adds a comment in a conversation (read: section) you started.Enable people to opt-in/out of receiving notifications, delivered via Echo, for comments posted in direct response to things they've said regardless of how those "comments posted in direct response" were posted (e.g. via DiscussionTools or full page editing).
2."Did someone say something new in this conversation I am interested in?"Event 1: Someone adds a comment in a conversation (read: section) you are "following."Enable people to opt-in/out of receiving notifications, delivered via Echo, for comments posted in sections/conversations they are interested in "following" regardless of how those "comments" were posted (e.g. via DiscussionTools or full page editing).
3.Did someone start a new conversation about a topic I am interested in?Event 1: Someone starts a new section on a talk page that is not your own user talk page.Enable people to opt-in/out of receiving notifications, delivered via Echo, when someone starts a new conversation (read: section) on a page they are interested in.

Investigation findings

Below is what we know so far about the feasibility of the delivering the functionality the there "Use cases" above describe.

Use case 1

  • We've developed an initial approach for detecting when another person adds a new comment in response to a comment you posted and when another person adds a comment in a conversation you started, //regardless/ of the interface/tool used to post said comment.
  • We've developed an initial approach for NOT sending notifications when:
    • The entire talk page someone is watching is archived
    • Specific sections on a talk page someone is watching are moved
  • We've developed an initial approach for differentiating between comments when multiple are posted within a single edit, using full-page editing

Use case 2

  • We've developed an initial approach for detecting when the following happens, regardless of the interface/tool used:
    • A new comment is added to the page and
    • The new comment was added to a section you are subscribed to

Use case 3

  • We've developed an initial approach that we think will enable the software to detect when a new section is created on a talk page you are watching, regardless of what tool (e.g. full-page wikitext editor, New Discussion Tool) the person used to create said "new section."

Open questions

Use case 2

  • Where and how should peoples' "subscriptions" to specific sections be stored?
    • These "subscriptions" will be stored in a database. The architecture for said database will be determined in this task: T263817.
  • What happens if the a specific section you are watching gets moved?
    • We will investigate this in T262990.
  • What happens if a specific section you are watching gets renamed?
    • We will investigate this in T262991.

Echo integration

  • Now that we are fairly confident in an approach for identifying new comments, new sections, etc., how might we "tell" Echo when to send notifications for these new comments and new sections? How might we "tel" Echo what content said "notifications" should contain?
    • We'll figure this out later. Engineering does not have any concerns about this at this time.

Overall feasibility

  • What about the "Use cases" described above are not feasible to deliver?
    • All uses cases are technically feasible. Open questions remain about how reliably and robustly they can be supported.
      • For example, for "Use case 2" [i], we are not yet sure how the software will support cases when sections are moved (read: archived or renamed). The thinking about these questions will happen in T262990 and T262991.

Performance

  • How might we detect new comments and new sections at scale?
    • We will investigate this in T262107.

Done

  • All "Open questions" are answered

Event Timeline

Next steps

  • @ppelberg needs to continue filling in "Use cases" and add evidence to the task description that substantiates the use cases included within.

Notes from conversation with Ed

  • A key question for us to answer in the context of delivering people notifications when comments they've written/conversations they've started are responded to: Where should the code will live to detect these event-triggering responses?
    • For responses posted using DiscussionTools, the answer is straightforward: at save, the software explicitly knows:
      • Which comment is being responded to
      • Who the author of the comment being responded to is
      • Who the author of the response is
    • For responses NOT posted using DiscussionTools, the answer is NOT straightforward: at save, the software does NOT explicitly know:
      • Which comment is being responded to
      • Who the author of the comment being responded to is
      • Who the author of the response is

Enable the content's author to subscribe to and receive notifications, delivered via Echo, for comments posted in specific sections/conversations they are interested in "following."

Wikitext talk pages are so flexible that sections (conversations) can easily be renamed, merged or split. How to handle these cases?

  • For responses NOT posted using DiscussionTools, the answer is NOT straightforward: at save, the software does NOT explicitly know:
    • Who the author of the response is

I think it does know—the author is the user who hit the save button. This should be available even without DT. The other two pieces of information aren’t, though, and probably it won’t be possible ever to reliably guess them, like when someone replies to several people in one edit.

Next steps

  • @ppelberg needs to continue filling in "Use cases" and add evidence to the task description that substantiates the use cases included within.

"Use cases" are populated.

Next steps

  • @ppelberg to discuss "Investigation" with @Esanders
  • @ppelberg to link to the evidence that inspired the "Use cases" in the description.
ppelberg moved this task from Incoming to Doing on the Editing-team (Kanban Board) board.

(@Esanders I added you as the assignee of this task considering the work you are doing on it. I need to update the task description with the notes from our last two conversations).

(@Esanders I added you as the assignee of this task considering the work you are doing on it. I need to update the task description with the notes from our last two conversations).

I've populated, what I understand to be, the outcomes of the investigation @Esanders has been doing in the task description's "Investigation findings" and "Open questions" section.

Next step:

  • Ed, can you review the task description to ensure it accurately and exhaustively represents what we currently know?
    • On 28-July, @Esanders reviewed the task description and Ed subsequently confirmed the task description accurately reflects the state of this investigation.

EDIT: marking the "Next step" above as completed and adding the outcome from the conversation Ed and I had on 28-July.

Enable the content's author to subscribe to and receive notifications, delivered via Echo, for comments posted in specific sections/conversations they are interested in "following."

Wikitext talk pages are so flexible that sections (conversations) can easily be renamed, merged or split. How to handle these cases?

This is a good question, @Tacsipacsi and one we're looking for an answer to. In the meantime, I've made sure it's included in the task description's "Open questions" section.

  • For responses NOT posted using DiscussionTools, the answer is NOT straightforward: at save, the software does NOT explicitly know:
    • Who the author of the response is

I think it does know—the author is the user who hit the save button. This should be available even without DT. The other two pieces of information aren’t, though, and probably it won’t be possible ever to reliably guess them, like when someone replies to several people in one edit.

We might have a way to detect all three pieces of information, regardless of the tool/interface being used to post the comment. Although, it's still too early to say for certain.

I think it does know—the author is the user who hit the save button. This should be available even without DT. The other two pieces of information aren’t, though, and probably it won’t be possible ever to reliably guess them, like when someone replies to several people in one edit.

We might have a way to detect all three pieces of information, regardless of the tool/interface being used to post the comment. Although, it's still too early to say for certain.

Indeed, by running the comment parser on the server it would theoretically be possible to detect multiple comments being added in one edit, and who they are replying to. TBC

I think it does know—the author is the user who hit the save button. This should be available even without DT. The other two pieces of information aren’t, though, and probably it won’t be possible ever to reliably guess them, like when someone replies to several people in one edit.

We might have a way to detect all three pieces of information, regardless of the tool/interface being used to post the comment. Although, it's still too early to say for certain.

Indeed, by running the comment parser on the server it would theoretically be possible to detect multiple comments being added in one edit, and who they are replying to. TBC

“Someone replies to several people” doesn’t necessarily mean that there are multiple separate, one-message-per-person comments. Imagine the following conversation:

I like apples. [[User:A]] 12:34, 24 July 2020 (UTC)
:I like bananas. [[User:B]] 23:45, 24 July 2020 (UTC)

I also like apples, but I like bananas, too. ~~~~

This might not be the best example, but I hope you get it: the current user (whose signature is marked with pre-PST ~~~~ here) replies to both A (notice the also) and B, but in a totally not machine-readable manner. False negatives like the one this would most probably produce are the worst possible outcome, as people miss conversations.

@Esanders and @ppelberg will discuss this next week

Notes from the conversations @Esanders and I had today are in line below.

Open questions

Use case 2

  • Where and how should peoples' "subscriptions" to specific sections be stored?

These "subscriptions" will be stored in a database. The architecture for said database will be determined at a future point.

  • What happens if a specific section you are watching gets moved?

We will investigate this in T262990.

  • What happens if a specific section you are watching gets renamed?

We will investigate this in T262991.

Echo integration

  • Now that we are fairly confident in an approach for identifying new comments, new sections, etc., how might we "tell" Echo when to send notifications for these new comments and new sections? How might we "tell" Echo what content said "notifications" should contain?~~

We'll figure this out later. Engineering does not have any concerns about this at this time.

Overall feasibility

  • What about the "Use cases" described above are not feasible to deliver?

All uses cases are technically feasible. Open questions remain about how reliably and robustly they can be supported.

For example, for "Use case 2" [i], we are not yet sure how the software will support cases when sections are moved (read: archived or renamed). The thinking about these questions will happen in T262990 and T262991.

Performance

  • How might we detect new comments and new sections at scale?

We will investigate this in T262107.


i. Use case 2: "Someone adds a comment in a conversation (read: section) you are "following."

I think it does know—the author is the user who hit the save button. This should be available even without DT. The other two pieces of information aren’t, though, and probably it won’t be possible ever to reliably guess them, like when someone replies to several people in one edit.

We might have a way to detect all three pieces of information, regardless of the tool/interface being used to post the comment. Although, it's still too early to say for certain.

Indeed, by running the comment parser on the server it would theoretically be possible to detect multiple comments being added in one edit, and who they are replying to. TBC

“Someone replies to several people” doesn’t necessarily mean that there are multiple separate, one-message-per-person comments. Imagine the following conversation:

I like apples. [[User:A]] 12:34, 24 July 2020 (UTC)
:I like bananas. [[User:B]] 23:45, 24 July 2020 (UTC)

I also like apples, but I like bananas, too. ~~~~

This might not be the best example, but I hope you get it: the current user (whose signature is marked with pre-PST ~~~~ here) replies to both A (notice the also) and B, but in a totally not machine-readable manner. False negatives like the one this would most probably produce are the worst possible outcome, as people miss conversations.

Good spot, @Tacsipacsi. I've added, what I think is, the question this example prompts to the "Open questions" section of T263819's description...can you please review to make sure it captures the case you are describing above? And if not, edit the question so it does?