Page MenuHomePhabricator

Translation Notification Bot sending the same message multiple times to every translator
Open, HighPublic

Description

Hello. It seems that Translation Notification Bot is broken again. It is sending the same notification up to 4 times to the translators. I've picked a random example at: https://meta.wikimedia.org/w/index.php?title=User_talk:MrLeopold&action=history. I've globally locked the account to prevent the bot from editting until this can be sorted out. Thanks.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 5 2016, 11:34 PM
MarcoAurelio triaged this task as High priority.Sep 5 2016, 11:34 PM

FWIW, the email version also went out more than once: I received two identical email notifications at 22:26 and 22:28 UTC.
On the other hand the log shows only one belated entry at 22:28. (Translation Notification Bot sent out messages at 22:26 already, e.g. here.)

I see lots of errors in Logstash between 2016-09-05T22:26:44 and 2016-09-06T00:29:32. All the errors only say Error editing the page. When exactly did you block the bot?

I see lots of errors in Logstash between 2016-09-05T22:26:44 and 2016-09-06T00:29:32. All the errors only say Error editing the page. When exactly did you block the bot?

  • (change visibility) 23:29, 5 September 2016 MarcoAurelio (talk | contribs | block) blocked Translation Notification Bot (talk | contribs) with an expiration time of infinite (autoblock disabled) (Bot malfunctioning: sending the same translation notification 4 times to every user) (unblock | change block)
  • (change visibility) 23:30, 5 September 2016 MarcoAurelio (talk | contribs | block) changed status for global account "User:Translation Notification Bot@global": set locked; unset (none) (Bot malfunctioning: sending the same notification 4 times to every user)

If you see the history page I linked above, it seems this has been happening for some time.

So assuming those are UTC as well, most of the errors could be caused by the block. But the actual reason for duplicates is still unclear.

159-            // TODO: Is this really the best way to test success?
160-            if ( $response['edit']['result'] !== 'Success' ) {
161:                    return "Error editing the page";
162-            }

More information needs to be collected here, why are we seeing an error even though message posting has worked.

  • 2016-09-05T22:26:44 "Error editing the page" errors started to occur.
  • 2016-09-05T22:28:37Z First notification for Template:Usurpation logged
  • 2016-09-05T22:58:29Z Second notification for Support and Safety logged
  • 2016-09-05T23:29:16Z Bot blocked
  • 2016-09-05T23:30:34Z Global account locked
  • 2016-09-06T00:29:32 Errors ended.

In the case of https://meta.wikimedia.org/w/index.php?title=User_talk:MrLeopold&action=history, submission would have queued two jobs: one for a local talk page, another for the foreign talk page but the foreign wiki was selected by the user as Meta. See the target link: in two messages, it's :metawikipedia:Template:Usurpation requested (foreign) and in the other two it's Template:Usurpation requested (local). This can be easily fixed in the code but the other issue is harder to debug without knowing the details of the error.

On Wikimedia, failed jobs are re-attempted an hour later. This can be easily noticed in the history of that talk page. So my guess is somehow our success checking code is not catching all cases of success and thus jobs are marked as failed, causing them to be re-attempted again one hour later even if an edit was successfully done. It might be possible that somewhere in the edit API it's actually not sending a 'Success' response even in actual successful cases. It's very hard to debug this without knowing the exact response so we should probably add more details to logging.

As expected, global block do not prevent the account from sending emails. As expected, I'm receiving duplicate or triplicate translation notifications. The last one at 20 September about translating the CC 4.0 license.

Sending several emails duplicated or n-ated might also lead to @wikimedia addresses to be spam blacklisted as well.

Putting up for consideration for a sprint.

Any progress here? Thanks.

This task is not currently classified as UBN!, and per Language-Team's repository maintenance list we only look at UBN!s for this extension. This task will be considered when there is a focus on maintenance.

MarcoAurelio raised the priority of this task from High to Unbreak Now!.Feb 28 2017, 12:04 PM

In that case I'm changing this as UBN priority since this is something that worked in the past correctly and is now working with mistakes. Also because it's hindering the translation work as translators are not being aware of new translation requests, even for important events. Thanks for your understanding. Best regards.

Restricted Application added subscribers: Jay8g, TerraCodes. · View Herald TranscriptFeb 28 2017, 12:04 PM
Nemo_bis lowered the priority of this task from Unbreak Now! to High.Feb 28 2017, 1:48 PM
Nemo_bis added a subscriber: Nemo_bis.

I would like the Language-Team policy to change, but this task doesn't qualify for "unbreak now": I'm not seeing any critical disruption right now. For instance, https://meta.wikimedia.org/wiki/Special:Contributions/Translation_Notification_Bot doesn't show any flood of duplicate messages and translation notifications can be sent normally, although a minority of translators might suffer an annoyance.

I tested it today after a year of block. The problem seems to be when several translators are to be notified, and translation admins, after the page erroring, reload. The bot then resends the same notifications again. I think that after you hit "send", the page that says that the notifications have been queued should appear and let the background jobs run w/o user awareness, so we can avoid the issue.

Yes, that's a common mistake that adds noise to our data points. However in some cases the bug is real and goes along the lines described by T144780#2611657.

I've seen the sign on the special page AFTER clicking on the "send button" (and of course, after previewed my message multiple times). Instead of leaving a banner that no one notices (banners are like the logo ot the left bar: you mentally skip them), would it be possible to temporary disable the feature?

@Trizek-WMF Disabling is possible, absent a better method, setting this it to false on InitialiseSettings.php. However that does not resolve the issues mentioned here. I wonder how MassMessage works for this kind of mass sending jobs and maybe we can copy from there? Regards.

Indeed I don't see how disabling the feature would help the goal, i.e. have the translators receive the information they need asked for with minimum collateral damage.

Maybe it's a silly idea, but how if translation notifications, instead of leaving TP/Email, triggered a notification?

I was suggesting disabling, because the message explicitly says "don't use the form to inform translators".

I see it as the same thing as put a not a a button expecting people not to use it versus disconnecting the wires to avoid the button to work, or, better, remove the button (and leave a note). But again, IANAD (I'm not a developer). ;)

Misleading messages should be removed but they're not a coding matter.

What is misleading in Please do not use this feature until T144780 is resolved. Thank you!, please? https://meta.wikimedia.org/wiki/MediaWiki:Translationnotifications-text-fetchi18n

That box should always appear in Special:NotifyTranslators. If it was not in your case, then we should investigate why.

Again, disabling is an option, but it's not going to resolve the underlying issues.

I'll file a task to convert the system into using Echo notifications, at least let it be investigated.

What is misleading in Please do not use this feature until T144780 is resolved. Thank you!, please? https://meta.wikimedia.org/wiki/MediaWiki:Translationnotifications-text-fetchi18n
That box should always appear in Special:NotifyTranslators. If it was not in your case, then we should investigate why.

As reported, I've seen it, but too late. People tend to skip warnings as they are to common on the wikis.

Again, disabling is an option, but it's not going to resolve the underlying issues.

Sure. But if it is done temporary, it would prevent possible problems.

Billinghurst added a subscriber: Billinghurst.EditedFeb 2 2018, 7:42 AM

This task is not currently classified as UBN!, and per Language-Team's repository maintenance list we only look at UBN!s for this extension. This task will be considered when there is a focus on maintenance.

Happy 1 year birthday to comment that we do not UBN. Is this any closer to being fixed?

Seems unusual to have a notification that isn't used as it somewhat sends duplicate messages. So there is a backlog of untranslated pages that need translating. Is the implicit decision that no prodding for translations is better than the occasional duplication of messages?

Does this multiple sending problem exist on other wikis with the same translation system, or is it just meta?

Hi @Nikerabbit ! I was wondering if you could take a look at this ticket and work on the notify translators tool? At the moment, it sends 4 messages to each translator, for every message you send. Since we are working on the second iteration of Wikimedia communities and contributors survey, and we need to have some messages translated for the launch. It would be great to be able to use this tool for that.

Thanks for all the work!

@mcruzWMF - Where did the bot sent 4 messages? Was this recently? If so, on which wiki? Thanks.

Hi @MarcoAurelio sorry for the late response. This was on Meta-Wiki, I believe. I haven't tried the tool recently, this was at the end of February, beginning of March.

Tomybrz added a subscriber: Tomybrz.

Please note that, specially on Meta-Wiki, using Special:NotifyTranslators usually results on DB timeouts for languages where a large number of users signed up to be notified via Special:TranslatorSignup. I guess we need to batch the requests/queries. Thanks.

Change 536738 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/TranslationNotifications@master] Adding logging to the TranslationNotificationJob

https://gerrit.wikimedia.org/r/536738

Change 537250 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/TranslationNotifications@master] Breakout notification sending logic into a separate class

https://gerrit.wikimedia.org/r/537250

Change 537363 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[operations/mediawiki-config@master] Add channels for the Translate and TranslationsNotification extension

https://gerrit.wikimedia.org/r/537363

Change 537363 merged by jenkins-bot:
[operations/mediawiki-config@master] Add channels for the Translate and TranslationsNotification extension

https://gerrit.wikimedia.org/r/537363

Mentioned in SAL (#wikimedia-operations) [2019-09-17T11:31:17Z] <urbanecm@deploy1001> Synchronized wmf-config/VariantSettings.php: SWAT: 290e207: Add channels for the Translate and TranslationsNotification extension (T221119, T144780, T143073) (duration: 00m 56s)

Change 537575 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/TranslationNotifications@master] Move code for creating notification jobs to a seperate job

https://gerrit.wikimedia.org/r/537575

Change 537628 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[operations/mediawiki-config@master] Fix incorrect channel name for TranslationNotifications extension

https://gerrit.wikimedia.org/r/537628

Change 536738 merged by jenkins-bot:
[mediawiki/extensions/TranslationNotifications@master] Add logging to the TranslationNotificationJob

https://gerrit.wikimedia.org/r/536738

Change 537628 merged by jenkins-bot:
[operations/mediawiki-config@master] Fix incorrect channel name for TranslationNotifications extension

https://gerrit.wikimedia.org/r/537628

Mentioned in SAL (#wikimedia-operations) [2019-09-25T16:06:40Z] <kartik@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: [[gerrit|537628|Fix incorrect channel name for TranslationNotifications extension (T144780)]] (duration: 01m 06s)

Change 539490 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/TranslationNotifications@master] [WIP] Use MassMessage to send notification to user's talkpage

https://gerrit.wikimedia.org/r/539490

Change 539547 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[integration/config@master] Add MassMessage as a dependency for TranslationNotifications

https://gerrit.wikimedia.org/r/539547

Change 539547 merged by jenkins-bot:
[integration/config@master] Add MassMessage as a dependency for TranslationNotifications

https://gerrit.wikimedia.org/r/539547