Page MenuHomePhabricator

MassMessage not consistently delivering messages
Closed, DuplicatePublic

Description

The last few days, we've had reports of MassMessage not delivering messages to everyone on the target list.

For T213864, two hours after the message was sent out (14:47 UTC, 16 January 2019) to https://meta.wikimedia.org/w/index.php?oldid=18788945 a manual check shows that some of the targets have received the message, but multiple wikis have not. The queue is said to be empty.

At https://meta.wikimedia.org/wiki/Talk:Tech/News/2019/03 @IKhitron has reported messages not being delivered (seen on he.wp, where 1 user over 11 received the issue).

So far this means that some communities do not get the message that they can't edit the wikis for a short period of time tomorrow, and that others do not get the updates of technical changes, so it's a somewhat urgent problem.

Event Timeline

Johan created this task.Jan 16 2019, 4:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 16 2019, 4:46 PM
Trizek-WMF updated the task description. (Show Details)Jan 16 2019, 4:49 PM
Johan updated the task description. (Show Details)Jan 16 2019, 5:06 PM
Tgr added a subscriber: Tgr.Jan 16 2019, 5:22 PM

Probably a case of T139380: MassMessage failed delivery claiming "readonly" although the page is not protected? You should probably see the error in the local wiki's massmessage log.
(Yes, that's not super useful.)

In any case, for something as severe and sudden as a DB outage, I'd go with a centralnotice message to all logged-in editors on the affected wikis.

Johan added a subscriber: Jseddon.Jan 16 2019, 5:41 PM

I've pinged @Jseddon for a CentralNotice banner, but I don't have CentralNotice admin rights myself, unfortunately.

Elitre added a subscriber: Elitre.Jan 16 2019, 6:06 PM

One of the wikis reports:
14:47, 16 January 2019 Delivery of "No editing for 30 minutes 17 January" to Wikibooks:Reading room/General failed with an error code of readonly .

Johan added a comment.Jan 16 2019, 6:22 PM

Yeah, we had about 440 failed deliveries.

Elitre added a comment.EditedJan 16 2019, 6:33 PM

I wonder if there's something in our code?

The only other message that failed there was again from us,
12:03, 4 October 2018 Delivery of "Reminder: No editing for up to an hour on 10 October" to Wikibooks:Reading room/General failed with an error code of readonly .

The page is getting other MMs: https://en.wikibooks.org/w/index.php?title=Wikibooks:Reading_room/General&action=history .

(This is also true for https://en.wikiversity.org/w/index.php?title=Special:Log&page=Wikiversity%3AColloquium , FWIW.)