Page MenuHomePhabricator

MassMessage not delivering
Open, MediumPublic

Description

MassMessage not delivering:
https://meta.wikimedia.org/wiki/Special:Log?type=massmessage&user=&page=&wpdate=&tagfilter=&subtype=

Tried an hour ago (19:40 UTC), and now again (20:40 UTC).

At 14:04 UTC, an earlier message was successfully delivered.

Event Timeline

Ok, so I found logs about the submission:

2021-02-22 21:15:47 [ece701b877f74cd12971482f] mw1310 metawiki 1.36.0-wmf.31 JobExecutor DEBUG: Beginning job execution {"job":"MassMessageSubmitJob User:Johan_(WMF)/Tech_News_test_list data=array(11) pages=[{\"title\":\"User talk:Johan (WMF)\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"ar.wikipedia.org\",\"wiki\":\"arwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"fr.wikipedia.org\",\"wiki\":\"frwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"en.wikipedia.org\",\"wiki\":\"enwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"mai.wikipedia.org\",\"wiki\":\"maiwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"commons.wikimedia.org\",\"wiki\":\"commonswiki\"},{\"title\":\"Wikipedia:MassMessage test\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"}] class=MediaWiki\\MassMessage\\Job\\MassMessageJob timestamp=20210222194328 namespace=2 title=Johan_(WMF)/Tech_News_test_list requestId=YDQJYOMBefm3LhMI83WUZAAAAMs","job_type":"MassMessageSubmitJob"} 
2021-02-22 21:15:47 [e12f362b08eae63768c70675] mw1296 metawiki 1.36.0-wmf.31 JobExecutor DEBUG: Beginning job execution {"job":"MassMessageSubmitJob User:Johan_(WMF)/Tech_News_test_list data=array(11) pages=[{\"title\":\"User talk:Johan (WMF)\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"ar.wikipedia.org\",\"wiki\":\"arwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"fr.wikipedia.org\",\"wiki\":\"frwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"en.wikipedia.org\",\"wiki\":\"enwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"mai.wikipedia.org\",\"wiki\":\"maiwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"commons.wikimedia.org\",\"wiki\":\"commonswiki\"},{\"title\":\"Wikipedia:MassMessage test\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"}] class=MediaWiki\\MassMessage\\Job\\MassMessageJob timestamp=20210222194143 namespace=2 title=Johan_(WMF)/Tech_News_test_list requestId=YDQI9@@PoO@t1zP6hU1GWgAAAAw","job_type":"MassMessageSubmitJob"} 
2021-02-22 21:15:49 [YDQI9@@PoO@t1zP6hU1GWgAAAAw] mw1296 metawiki 1.36.0-wmf.31 JobExecutor INFO: Finished job execution {"job":"MassMessageSubmitJob User:Johan_(WMF)/Tech_News_test_list data=array(11) pages=[{\"title\":\"User talk:Johan (WMF)\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"ar.wikipedia.org\",\"wiki\":\"arwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"fr.wikipedia.org\",\"wiki\":\"frwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"en.wikipedia.org\",\"wiki\":\"enwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"mai.wikipedia.org\",\"wiki\":\"maiwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"commons.wikimedia.org\",\"wiki\":\"commonswiki\"},{\"title\":\"Wikipedia:MassMessage test\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"}] class=MediaWiki\\MassMessage\\Job\\MassMessageJob timestamp=20210222194143 namespace=2 title=Johan_(WMF)/Tech_News_test_list requestId=YDQI9@@PoO@t1zP6hU1GWgAAAAw","job_type":"MassMessageSubmitJob","job_status":true,"job_duration":2.1155190467834473} 
2021-02-22 21:15:50 [YDQJYOMBefm3LhMI83WUZAAAAMs] mw1310 metawiki 1.36.0-wmf.31 JobExecutor INFO: Finished job execution {"job":"MassMessageSubmitJob User:Johan_(WMF)/Tech_News_test_list data=array(11) pages=[{\"title\":\"User talk:Johan (WMF)\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"ar.wikipedia.org\",\"wiki\":\"arwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"fr.wikipedia.org\",\"wiki\":\"frwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"en.wikipedia.org\",\"wiki\":\"enwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"mai.wikipedia.org\",\"wiki\":\"maiwiki\"},{\"title\":\"User talk:Johan (WMF)\",\"site\":\"commons.wikimedia.org\",\"wiki\":\"commonswiki\"},{\"title\":\"Wikipedia:MassMessage test\",\"site\":\"sv.wikipedia.org\",\"wiki\":\"svwiki\"}] class=MediaWiki\\MassMessage\\Job\\MassMessageJob timestamp=20210222194328 namespace=2 title=Johan_(WMF)/Tech_News_test_list requestId=YDQJYOMBefm3LhMI83WUZAAAAMs","job_type":"MassMessageSubmitJob","job_status":true,"job_duration":2.836076021194458}

As you can see, both attempts were actually processed, but at 21.15 UTC, while the job creation (and enqueueing) timestamps were 20210222194143 and 20210222194328. And two submit jobs executed simultaneously.

This indicates the jobs have been sitting in the queue for quite a while..

As a workaround, I suggest trying again, but changing the message content a little bit. Even a whitespace-only change will be sufficient.

I'm adding PET clinic duty, cause this is reappearance of the old series of MassMessage bugs. We've added more logging into job queue change-prop controller which might show what happens, now the logs need to be analyzed.

@Pchelolo OK, tried again little over an hour ago, adding a whitespace. No success.

(None of the executed jobs resulted in messages actually being delivered – the target pages have no new messages since last week.)

Got one of them now. Have tried to deliver my message, hoping it will be delivered when I wake up. (: