Initial results:

• What outstanding issues need to be resolved before this notification could be deployed?
  • Is T254988 resolved?
    • yes
  • What is the cause of T256297 and does it need work?
    • I was not able to reproduce the issue locally, so I need to look at the differences between the beta deployment and the dev environment. The extension uses a global pref to flag whether the user has already received a notification. The check for that flag is written in such a way that if it isn't retrievable, the user is treated as if they have not been notified. This would result in a user getting notified with every edit, just as reported in the issue. That guard clause needs to be flipped so that we fail without notification. My guess is that there is something about the global preferences setup on beta that isn't accounted for in the code.
• Additionally, I discovered that this extension has no tests of its own. We should really remedy that since this will be run on every page load of every user. It will also make debugging much easier.
• Per T132084#6145961, is it possible to make this a staggered rollout, to avoid notifying up to 25,000 editors in a single day?
  • We could just change the existing setting for the editcount threshold, starting high and dropping it over time by deploying updates to the mediawiki configs. If that would have to be done per-project, it might be better to do it in code. This would be really straightforward to implement. The function that decides whether to send the notification could also check the editcount and the date to sort editors into notification stages. We could do something like:
    • week 1 - editors with > 10k edits
    • week 2 - editors with > 4k edits
    • week 3 - editors with > 1k edits
    • week 4 - editors with > 500 edits
  • We'll need to do some analysis to figure out about how many editors is in each of those buckets to determine the actual thresholds, and of course decide what kind of timeline we want for the rollout.
• What support do we need from the previous developers to get up to speed in working on this?
  • If there are examples of other extensions that exist solely to send notifications, I'd love to have that pointed out to me so I can take a look at them. Otherwise, I don't think we need much. This is a pretty straightforward extension.
• How long do we estimate this work would take to finalise?
  • I don't have that answer yet, because I haven't answered all my own questions yet.

Another take on the staggered rollout: We can roll the extension out to smaller wikis and then to big wikis like English, Swedish, and German. We can do this in addition to the editor notification stages to observe how the Wikipedia Library behaves with the new volume of users.

This is a great first pass, thanks Jason.

If there are examples of other extensions that exist solely to send notifications, I'd love to have that pointed out to me so I can take a look at them.

Some examples that might be helpful, mostly more complex than this:

• Extension:Thanks
• Extension:LoginNotify

We could just change the existing setting for the editcount threshold, starting high and dropping it over time by deploying updates to the mediawiki configs. If that would have to be done per-project, it might be better to do it in code. This would be really straightforward to implement. The function that decides whether to send the notification could also check the editcount and the date to sort editors into notification stages.

We can roll the extension out to smaller wikis and then to big wikis like English, Swedish, and German. We can do this in addition to the editor notification stages to observe how the Wikipedia Library behaves with the new volume of users.

My gut feeling is that the edit count threshold might be more achievable since this is a global notification counting edits across all Wikimedia projects, but I could be wrong.

We'll need to do some analysis to figure out about how many editors is in each of those buckets to determine the actual thresholds, and of course decide what kind of timeline we want for the rollout.

Per modifications on https://quarry.wmflabs.org/query/42517, here's the breakdown for English Wikipedia (only taking into account edits to that project), which I assume is at least approximately representative of all users:

Since we don't currently have a clear figure for how many users we're going to notify (only the % is useful in that table) I think this kind of staggered rollout, starting with the most active, makes a lot of sense.

This is back in Ready for Dev with the following outstanding items:

• Decide on rollout strategy - does the edit count breakdown make sense? Is it achievable?
  • A secondary note here that occurred to me. "dropping it over time by deploying updates to the mediawiki configs"; mediawiki deployments happen weekly so we'd be tied to the weekly deployment train, is that right? Or is there a way to deploy those simple config changes on a more customised timeline?
• Diagnose the likely cause of T256297 a little further.
• Was T256403 fixed in T254988?

Was T256403 fixed in T254988?

Yes; sorry my initial response wasn't clear on that.

• Diagnose the likely cause of T256297 a little further.

I researched other extensions that use GlobalPreferences, and found that they all do varying amounts of setup to check the status of GlobalPreferences and to manage preference caching. Caching seems the most likely difference between beta and local dev. Given the way our guard clause is written, a cached preference value that doesn't show someone already received the notification would result in exactly the reported bug.
I found a nice helper class that we can largely crib for our extension to take care of this for us:
https://github.com/wikimedia/mediawiki-extensions-ContentTranslation/blob/ebe1f3576b5edd3439bc91adb07fe5d7d9c8eaea/includes/PreferenceHelper.php

I think we could update our global pref handling and re-enable the extension on beta to see if things work correctly. If it's still broken, it will not send any notifications.

Decide on rollout strategy - does the edit count breakdown make sense? Is it achievable?

What I like about this breakdown is that the stages start small and gets bigger over time. For me the biggest question is a week enough time between stages? We might want to make the delay between stages longer so that if we're overwhelmed we have time to get a config change deployed to pause further stages.

A secondary note here that occurred to me. "dropping it over time by deploying updates to the mediawiki configs"; mediawiki deployments happen weekly so we'd be tied to the weekly deployment train, is that right? Or is there a way to deploy those simple config changes on a more customised timeline?

The only deploys that don't follow the weekly schedule are backports (not us) and incident response (which we hope to avoid). The simplest way that I can see to customize our timeline is to just do it in code by having the required editcount change based on date and checking the date on pageload. We could either remove all config for this and just hardcode the values, or we could make the dates and related counts config as well. That's probably the right way to do it.

Based on this spike I'm seeing the following work as outstanding:

• T271917: Update the text and link destinations per my update to T132084
• Fix T256297 by improving the guard clause in the case that global preferences can't be read.
• T271919: Write some tests
• T271921: Implement a gradual rollout process based on global edit counts, probably through config changes made manually by us as we're comfortable
• T271922: Request a performance review prior to deployment

Another consideration that came up this week is since a large number of users are going to receive this notification at deployment, we want to make sure it's translated into as many languages as possible. Since the code was already merged for Beta, it's been live on TranslateWiki for some time, with a substantial number of translations completed. Since we're rewording the text, however, these will mostly get invalidated. We'll want to wait until we're happy the updated text has been sufficiently translated before beginning our rollout. It would make sense, therefore, for updating the text/links to be the first task.

We also discussed that config changes can be deployed separately to the deployment train, and are therefore more flexible. This means we could more manually control our rollout strategy over time.

EDIT: Tasks filed, linked above next to the bullet points.

Something that came up in a conversation with @Prtksxna today - can we get data on how many users receive/see/click the notification and break that down by things like project?