Meta

• ADDED the === Process @LZaman, @MNeisler, and I independently discussed to the task description.

Will this be using the Pre-mortem toolkit that is part of the DEI Playbook? It seems to include lots of useful forms that could be relevant.

In T290508#7345377, @iamjessklein wrote:

Will this be using the Pre-mortem toolkit that is part of the DEI Playbook? It seems to include lots of useful forms that could be relevant.

Good question; we'll be following the process described in the PRE-MORTEM TOOLKIT and we'll experiment with the forms/tools linked within it for the pre-mortem we'll conduct for Usability work (T249579) in Q2.

Good question; we'll be following the process described in the PRE-MORTEM TOOLKIT and we'll experiment with the forms/tools linked within it for the pre-mortem we'll conduct for Usability work (T249579) in Q2.

great, thanks for looking into it.

4. Prioritize failure scenarios to plan for

This is done [i]. Now on to Steps "5." and "6." [ii] which, I think, can happen independently of one another.

i. You'll notice the Priority within the T290508: Automatic Topic Subscriptions sheet is now populated.
ii. 5. Propose leading indicators (e.g. qualitative and quantitative) for prioritized scenario and 6. Propose a plan of action/mitigation strategy for each prioritized scenario

According to the process above, it seems that @ppelberg and @MNeisler are populating the spreadsheet with leading indicators, is that the correct assumption?

@ppelberg - I've proposed some quantitative leading indicators for the prioritized (Med and High) scenarios and they are ready for your review.

In T290508#7431989, @MNeisler wrote:

@ppelberg - I've proposed some quantitative leading indicators for the prioritized (Med and High) scenarios and they are ready for your review.

The indicators you've proposed look good to me – thank you, @MNeisler.

Next steps

• @ppelberg + @MNeisler to relate each scenario to a planned analysis that will be defined in the experiment plan
• @MNeisler to confirm whether we do or do NOT have instrumentation in place to know whether someone successfully receives a notification for new comments posted to topics they are subscribed to (manually or automatically). More context in this comment.

In T290508#7476325, @ppelberg wrote:

Next steps

• @ppelberg + @MNeisler to relate each scenario to a planned analysis that will be defined in the experiment plan

This is done. I've updated the two analyses we have planned (T280896 + T280898) with the Next steps we defined in Column H of the Talk Pages Project/Pre-Mortems/T290508: Automatic Topic Subscriptions sheet.

• @MNeisler to confirm whether we do or do NOT have instrumentation in place to know whether someone successfully receives a notification for new comments posted to topics they are subscribed to (manually or automatically). More context in this comment.

This is done. Per what @MNeisler shared [i], we do not have the instrumentation in place to directly link notifications to the topics that "emitted" them. As such, we are going to depend on people reporting cases where notifications are not being sent for topics they've been automatically subscribed. This has happened at least once so far (T297264).

i.

With current instrumentation, we can track the following as potential proxies:

• Users that automatically subscribe to a topic and post a comment or response to that topic (using a combo of discussiontools_subscription and the new talk_page_edit schemas)
• Users that have automatically subscribed to (any) topic and received a notification ( does not look like we would not be able to directly link the notification received to the topic subscribed to)