A different way to go about this could be - resetting the list of translators so that only the ones currently interested (and/or confirming they are generically interested) would be included in future pings.

Another thing I dream of would be being able to have /multiple/ lists of translators, so I can choose which ones to target. (This is already possible of course, but it requires more work.)
For instance, we could have people who are only interested in Tech news, and that's fine.
Or we could have people interested in a specific product/feature, and we could target only those when necessary.
Or we could have people only interested in helping with non-technical translations. Super, let me target only those if my request is relevant to them.

We're planning to create a maintenance script that will unsubscribe translators from the current wiki who have been inactive. It will send a notification to them via Echo, informing them that they are being unsubscribed and can re-subscribe if they wish to receive notifications about new translations. This script can be run once a week or with an even larger gap.

We plan to consider an account inactive if there are zero edits and log entries over a period of X months, where X will be a configurable parameter. The logic for this will be similar to what we have in the isInactiveAccount function in the removeUnusedAccounts.php maintenance script that checks for no edits. See code here.

Users who have been blocked for a long period of time when the maintenance script runs will also be notified and unsubscribed.

Users blocked for a short duration will continue to receive the translation notifications.

The maintenance script will send notification using Echo extension. The maintenance script can fail with an error message if it detects that the Echo extension is not installed.

I don’t think unsubscribe all blocked users is a good idea. For example, even if one is blocked for getting in revert wars (maybe on more than one wiki, e.g. on their home wiki and on Wikidata), they can still contribute translations. Even indefinite blocks may in fact be temporary (this is often the case when bots malfunction, but it may affect human users as well every now and then). I think only users with indefinite/infinite blocks that started X time before the script run should be unsubscribed (I’d say X=30 days – maybe less would also be fine, but if the script will be run regularly anyway, a 30-day limit means just that 30 days worth of blocked accounts will continue get notifications – since TranslatorSignup is quite a hidden thing, it’s probably not more than one or two users at any given time).

In T323192#8595312, @Tacsipacsi wrote:

I don’t think unsubscribe all blocked users is a good idea. For example, even if one is blocked for getting in revert wars (maybe on more than one wiki, e.g. on their home wiki and on Wikidata), they can still contribute translations. Even indefinite blocks may in fact be temporary (this is often the case when bots malfunction, but it may affect human users as well every now and then). I think only users with indefinite/infinite blocks that started X time before the script run should be unsubscribed (I’d say X=30 days – maybe less would also be fine, but if the script will be run regularly anyway, a 30-day limit means just that 30 days worth of blocked accounts will continue get notifications – since TranslatorSignup is quite a hidden thing, it’s probably not more than one or two users at any given time).

You make good points. I think I can summarize as following:

• Unsubscribe users who have been blocked X time before the unsubscribe script runs.
• For other blocked users, avoid sending them a notification by checking status before delivery, but do not unsubscribe them.

I think blocked users should also get notifications (before they get unsubscribed), so the second bullet point should simply be removed – they may be able to translate after (and in case) the block is removed/expires, or if they’re not blocked on the wiki where the translation happens.

In T323192#8593792, @abi_ wrote:

It will send a notification to them via Echo

I just realized that this sentence implicitly introduces a dependency on Echo, which currently doesn’t exist (cf. T48310). I’m fine with it, but this decision should be explicit (and either the maintenance script should check for Echo’s presence, or it should be a mandatory dependency).

Thanks @Tacsipacsi. I've incorporated your suggestions in the original comment here: https://phabricator.wikimedia.org/T323192#8593792

I don’t see this suggestion incorporated:

In T323192#8607162, @Tacsipacsi wrote:

I think blocked users should also get notifications (before they get unsubscribed), so the second bullet point should simply be removed – they may be able to translate after (and in case) the block is removed/expires, or if they’re not blocked on the wiki where the translation happens.

In T323192#8630228, @Tacsipacsi wrote:

I don’t see this suggestion incorporated:

In T323192#8607162, @Tacsipacsi wrote:

I think blocked users should also get notifications (before they get unsubscribed), so the second bullet point should simply be removed – they may be able to translate after (and in case) the block is removed/expires, or if they’re not blocked on the wiki where the translation happens.

Doesn't this cover that?

Users who have been blocked for a long period of time when the maintenance script runs will also be notified and unsubscribed.

That paragraph is about permanently blocked users, while those who can translate after the block expires are those who are only temporarily blocked, so it’s not even relevant. The relevant sentence is

Users blocked for a short duration will not be unsubscribed but will not receive the translation notifications either.

which says that temporarily blocked users won’t get notifications – this is what I disagree with.

In T323192#8680109, @Tacsipacsi wrote:

That paragraph is about permanently blocked users, while those who can translate after the block expires are those who are only temporarily blocked, so it’s not even relevant. The relevant sentence is

Users blocked for a short duration will not be unsubscribed but will not receive the translation notifications either.

which says that temporarily blocked users won’t get notifications – this is what I disagree with.

Thanks for the clarification. I think we can take this route since this is a separate chunk of work anyway that can be done if we see the need for it.

Thanks!

Change 898446 had a related patch set uploaded (by Wangombe; author: Wangombe):

[mediawiki/extensions/TranslationNotifications@master] (WIP)Introduce UnsubscribeInactiveUsers script

https://gerrit.wikimedia.org/r/898446

The maintenance script will perform the following actions:

• Fetch all translators signed up to receive notifications on that site.
• Check if those translators are inactive across all wikis. If yes, remove them and send them a notification
• Check if those translators are blocked globally for a long time. If yes, remove them and send them a notification

Notifications will be sent using the Echo extension. A dependency will not be added on the extension, but the maintenance script will check for the Echo extension, and refuse to run if its not installed.

One corner case to consider would be that after a user is unsubscribed, and they subscribe again, the maintenance script should not unsubscribe them again when it runs the next time. The re-signup activity should be factored in when determining user activity. I'll be creating a subtask to save the translator signup time in the user preferences in order to use it when determining inactivity.

Some useful code snippets:

• Following code in the Phonos extension fetches all the sites: https://github.com/wikimedia/mediawiki-extensions-Phonos/blob/master/maintenance/countOrphanedFiles.php#L94-L124
• Following code in the Babel extension queries database from different sites: https://github.com/wikimedia/mediawiki-extensions-Babel/blob/master/includes/Database.php#L75-L95

Will the “inactive” duration be a command-line parameter, similarly to “blocked for a long time”? What value do you plan to use? (I’d say one or two years; a few months of inactivity without being blocked may simply mean that the user is busy IRL, but haven’t given up returning to the wiki yet.) Does “blocked globally” mean that there’s at least one wiki the user is indefinitely blocked on, or does it take only global locks (and not blocks!) into account?

In T323192#8691973, @Tacsipacsi wrote:

Will the “inactive” duration be a command-line parameter, similarly to “blocked for a long time”? What value do you plan to use? (I’d say one or two years; a few months of inactivity without being blocked may simply mean that the user is busy IRL, but haven’t given up returning to the wiki yet.) Does “blocked globally” mean that there’s at least one wiki the user is indefinitely blocked on, or does it take only global locks (and not blocks!) into account?

Yes, inactive duration will be a parameter to the script. We've not yet decided the value of this parameter. What you said about it being upto 2 years makes sense but I think we can finalize this at a later point. It might be a good idea to identify how many users the script would affect before we actually unsubscribe them, so we will add a dry-run mode and test with that.

Regarding blocks, I think we can check if the subscriber is blocked on the site where the script is running since users are notified about content to translate from that specific site. So I think I should update the term "blocked globally".

Yes, inactive duration will be a parameter to the script.

If we want to automate the running the script with a timer, that goes into the puppet repository and needs SRE review. If it's a configuration option, we can follow normal backport deployments if we want to change it. For testing, it's probably more convenient to provide it via command line.

Change 898446 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] Add script to unsubscribe inactive or blocked users

https://gerrit.wikimedia.org/r/898446

Change 911248 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Fix indexing of attached accounts

https://gerrit.wikimedia.org/r/911248

Change 911248 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Fix indexing of attached accounts

https://gerrit.wikimedia.org/r/911248

Check if those translators are blocked globally for a long time. If yes, remove them and send them a notification.

@abi_ Since globally locked accounts can't log-in anymore, sending them a notification would not be required as they won't be able to see it. Globally locked accounts could probably be removed without any waiting period, since it's mostly a permanent action (we do not temporarily globally lock accounts).

Would it be possible to incorporate T335053: Don't attempt to send email to users without any configured in Preferences to this script, and remove those users without any email address set in their user preferences at the time this script runs?

Change 914789 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Use LoadBalancer from target wiki

https://gerrit.wikimedia.org/r/914789

Change 914789 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Use LoadBalancer from target wiki

https://gerrit.wikimedia.org/r/914789

In T323192#8823310, @MarcoAurelio wrote:

Check if those translators are blocked globally for a long time. If yes, remove them and send them a notification.

@abi_ Since globally locked accounts can't log-in anymore, sending them a notification would not be required as they won't be able to see it. Globally locked accounts could probably be removed without any waiting period, since it's mostly a permanent action (we do not temporarily globally lock accounts).

Hmm, OK. I'll incorporate this change.

In T323192#8823310, @MarcoAurelio wrote:

Would it be possible to incorporate T335053: Don't attempt to send email to users without any configured in Preferences to this script, and remove those users without any email address set in their user preferences at the time this script runs?

That would make sense to do, but I'd like to do it in a separate task. Additionally I've also submitted a couple of patches to avoid sending emails to users who don't have it configured.

Change 920288 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Fix call to ILoadBalancer::getConnection

https://gerrit.wikimedia.org/r/920288

Change 920289 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Avoid using ActorStore

https://gerrit.wikimedia.org/r/920289

Change 920288 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Fix call to ILoadBalancer::getConnection

https://gerrit.wikimedia.org/r/920288

Change 920289 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers::ActorStore: Use UserIdentity of target wiki

https://gerrit.wikimedia.org/r/920289

Change 924057 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Cast string to int for UserIdentityValue

https://gerrit.wikimedia.org/r/924057

Change 924057 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Cast string to int for UserIdentityValue

https://gerrit.wikimedia.org/r/924057

Ran this script on MediaWiki site today and here are the stats:

Total subscribers: 325

For 60 days (since 2023-04-08):

• 226 inactive subscribers
• 8 blocked subscribers

For 90 days (since 2023-03-09):

• 218 inactive subscribers
• 8 blocked subscribers

For 180 days (since 2022-12-09):

• 205 inactive subscribers
• 8 blocked subscribers

For 365 days (since 2022-06-07):

• 188 inactive subscribers
• 8 blocked subscribers

Will post statistics from Meta-Wiki tomorrow once the latest changes have been deployed.

Change 927977 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Add more verbose logs

https://gerrit.wikimedia.org/r/927977

Change 928159 had a related patch set uploaded (by Abijeet Patro; author: Abijeet Patro):

[operations/puppet@production] TranslationNotifications: Run UnsubscribeInactiveUsers periodically

https://gerrit.wikimedia.org/r/928159

Stats from Meta-Wiki

Total subscribers: 3456

For 60 days (since 2023-04-09):

• 2553 inactive subscribers
• 18 blocked subscribers

For 90 days (since 2023-03-10):

• 2478 inactive subscribers
• 18 blocked subscribers

For 180 days (since 2022-12-10):

• 2322 inactive subscribers
• 18 blocked subscribers

For 365 days (since 2022-06-08):

• 2150 inactive subscribers
• 18 blocked subscribers

Here are the stats from some major wikis:

Based on the data I think its safe to set the inactivity period to 1 year. Concerns?

Wow, 2150 out of 3456 is a whole lot. I have no concerns about the inactivity period of one year, but what about other wikis?

'wmgUseTranslationNotifications' => [
	'default' => false,
	'testwiki' => true,
	'betawikiversity' => true, // T160120
	'brwikimedia' => true, // T46054
	'cawikimedia' => true, // T75394
	'commonswiki' => true, // T126901
	'testcommonswiki' => true, // T126901
	'frwiktionary' => true, // T138972
	'gewikimedia' => true, // T236389
	'incubatorwiki' => true,
	'mediawikiwiki' => true,
	'metawiki' => true,
	'nowikimedia' => true, // T152490
	'ruwikimedia' => true, // T121766
	'uawikimedia' => true, // T131731
	'wbwikimedia' => true, // T162510
	'wikidata' => true, // T43585
	'wikimania2012wiki' => true,
	'wikimania2013wiki' => true,
	'wikimania2014wiki' => true,
	'wikimania2015wiki' => true,
	'wikimania2016wiki' => true,
	'wikimania2017wiki' => true,
	'wikimania2018wiki' => true,
	'wikimaniawiki' => true,
],

(source: rOMWC wmf-config/InitialiseSettings.php:7368-7394 (at bd3f27e53)).

@abi_ I think that one year is the best place to start. It will solve most of the problem, with very little risk of unsubscribing an occasional translator. I would even be happy to see 18 months or 2 years of inactivity as the cutoff.

BTW, if this runs only 1 to 4 times per year, then that should reduce the risk of someone re-subscribing and the script immediately removing them for inactivity. Once we get through the initial purge of long-dead and never-used accounts (~60% in these stats), we should see a much smaller percentage removed each round.

In T323192#8917068, @Tacsipacsi wrote:

Wow, 2150 out of 3456 is a whole lot. I have no concerns about the inactivity period of one year, but what about other wikis?

Yea, it was higher than what I was expecting. I'll check a few other wikis and post the results here.

In T323192#8920039, @Whatamidoing-WMF wrote:

BTW, if this runs only 1 to 4 times per year, then that should reduce the risk of someone re-subscribing and the script immediately removing them for inactivity. Once we get through the initial purge of long-dead and never-used accounts (~60% in these stats), we should see a much smaller percentage removed each round.

The script ensures that it does not remove translators who have re-subscribed recently. Planning to run it once every couple of months.

I've updated this comment with information from more wikis.

Change 927977 merged by jenkins-bot:

[mediawiki/extensions/TranslationNotifications@master] UnsubscribeInactiveUsers: Add more verbose logs

https://gerrit.wikimedia.org/r/927977

I ran the script with the --really option on testwiki with a period of 365 days, and the script works as expected. The --really option actually removes the inactive and blocked subscribers.

There were a total of 13 subscribers, 2 in-active, and 1 blocked. After running the script, there are 10 subscribers, 0 in-active, 0 blocked. I sent a test notification that has been delivered to existing subscribers successfully.

I have run the script on the following sites:

1. Incubator: 15 out of 40 subscribers were inactive (since 2022-06-19T09:07:52Z) and have been removed. No blocked subscribers.
2. Wikimania: 2 out of 7 subscribers were inactive (since 2022-06-19T09:14:23Z) and have been removed. No blocked subscribers.
3. Commons: 48 out of 161 subscribers were inactive (since 2022-06-19T12:29:20Z) and have been removed. 10 blocked subscribers were also removed. Remaining subscribers: 103

I ran the script on the following sites:

1. Meta-Wiki: 2151 out of 3464 subscribers were inactive (since 2022-07-03T05:18:19Z) and have been removed. 18 blocked subscribers. Remaining subscribers: 1295
2. MediaWiki: 188 out of 326 subscribers were inactive (since 2022-07-03T05:55:33Z) and have been removed. 8 blocked subscribers. Remaining subscribers: 130

Change 928159 merged by Giuseppe Lavagetto:

[operations/puppet@production] TranslationNotifications: Run UnsubscribeInactiveUsers periodically

https://gerrit.wikimedia.org/r/928159

The cronjobs are correctly installed:

mwmaint1002:~$ systemctl list-timers | grep translationnotifications-unsubscribeinactiveusers
Mon 2023-10-02 02:00:00 UTC  2 months 13 days left n/a                          n/a                mediawiki_job_translationnotifications-unsubscribeinactiveusers-metawiki.timer         mediawiki_job_translationnotifications-unsubscribeinactiveusers-metawiki.service
Mon 2023-10-02 03:30:00 UTC  2 months 13 days left n/a                          n/a                mediawiki_job_translationnotifications-unsubscribeinactiveusers-mediawikiwiki.timer    mediawiki_job_translationnotifications-unsubscribeinactiveusers-mediawikiwiki.service
Mon 2023-10-02 04:15:00 UTC  2 months 13 days left n/a                          n/a                mediawiki_job_translationnotifications-unsubscribeinactiveusers-incubator.timer        mediawiki_job_translationnotifications-unsubscribeinactiveusers-incubator.service
Mon 2023-10-02 05:00:00 UTC  2 months 13 days left n/a                          n/a                mediawiki_job_translationnotifications-unsubscribeinactiveusers-wikimania.timer        mediawiki_job_translationnotifications-unsubscribeinactiveusers-wikimania.service
Mon 2023-10-02 05:30:00 UTC  2 months 14 days left n/a                          n/a                mediawiki_job_translationnotifications-unsubscribeinactiveusers-commons.timer          mediawiki_job_translationnotifications-unsubscribeinactiveusers-commons.service

They will run next on October 2nd

This script would have run on Oct, 2nd, 2023. Here are the current subscriber counts on various wikis:

1. Metawiki: 1,279 subscribers, 25 inactive. Previously: 1295 subscribers on July 3rd, 2023.
2. MediaWiki: 127 subscribers, 1 inactive. Previously: 130 subscribers on July 3rd, 2023.
3. Incubator wiki: 25 subscribers, 0 inactive. Previously: 25 subscribers on Jun 19th, 2023. No change.
4. Wikimania wiki: 5 subscribers, 0 inactive. Previously: 5 subscribers on Jun 19th, 2023. No change.
5. Commons wiki: 100 subscribers, 1 inactive. Previously: 113 subscribers on Jun 19th, 2023.