Page MenuHomePhabricator

Special:ManageMessageGroups failing when a group has invalid YAML
Closed, ResolvedPublic

Description

Currently due to Ajapaik having issues, ManageMessageGroups is failing to import. We should not stop the export process for other projects / languages if one of the projects has issues.

Log

","exception_id":"abd6701869651ee3857074f1","exception_url":"/w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-a6a386-Invert%20colors%2Fth","caught_by":"mwe_handler"} []
[2019-10-01 00:22:23] exception.ERROR: [7be5f2990d147877f638b67f] /wiki/Special:ManageMessageGroups/unattended   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/messagegroups/FileBasedMessageGroup.php(52): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/utils/MessageGroupCache.php(118): FileBasedMessageGroup->load('en')
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(327): MessageGroupCache->create()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(85): SpecialManageGroups->processSubmit()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPage.php(575): SpecialManageGroups->execute('unattended')
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run('unattended')
#6 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(296): MediaWiki\\Special\\SpecialPageFactory->executePath(Object(Title), Object(RequestContext))
#7 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(896): MediaWiki->performRequest()
#8 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(527): MediaWiki->main()
#9 /srv/mediawiki/tags/2019-09-26_14:59:29/index.php(44): MediaWiki->run()
#10 {main}

Event Timeline

abi_ created this task.Oct 1 2019, 6:22 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2019, 6:22 AM

Change 540081 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/Translate@master] Continue processing other groups if a language under a group fails

https://gerrit.wikimedia.org/r/540081

abi_ added a subscriber: Pginer-WMF.

@Pginer-WMF - Started working on this since exports were blocked on Twn.

Even though concerned project has fixed the issue with their source translation, since I've already submitted a patch I'd like to add this to the current board.

abi_ added a comment.Oct 12 2019, 6:51 PM

@Nikerabbit - Usually when an incoming file has formatting issues it is caught during the import itself when processMessageChanges.php runs. Not sure how in this case the error occurred after submitting the changes on Message group management page page.

The patch that I've submitted should handle both scenarios, but I've only been able to test error handling when during processMessageChanges.php and not on the Message group management page.

Patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/540081

Yay, Phabricator ate my reply. The pastes strace is actually two different requests.

Here is one in translation aids:

[30-Sep-2019 10:30:52 UTC] AggregateMessageGroup dissemin cannot be primary owner of key e24696-To-do_list
[2019-09-30 10:33:29] exception.ERROR: [d60ef67895bd6139698cf191] /w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-653988-SubjectPhoto%2Fet   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/translationaids/GettextDocumentationAid.php(36): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/api/ApiQueryTranslationAids.php(88): GettextDocumentationAid->getData()
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(1599): ApiTranslationAids->execute()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(537): ApiMain->executeAction()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(508): ApiMain->executeActionWithErrorHandling()
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/api.php(87): ApiMain->execute()
#6 {main}
","exception_id":"d60ef67895bd6139698cf191","exception_url":"/w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-653988-SubjectPhoto%2Fet","caught_by":"mwe_handler"} []

It reads the files to read message comments to display in message documentation section. IMHO those should be cached in message group definitions cache.

Here is the one from Special:ManageMessageGroups:

[2019-09-30 18:59:22] exception.ERROR: [3e2ff5eab4d50dc8a574a3b9] /wiki/Special:ManageMessageGroups/unattended   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/messagegroups/FileBasedMessageGroup.php(52): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/utils/MessageGroupCache.php(118): FileBasedMessageGroup->load('en')
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(327): MessageGroupCache->create()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(85): SpecialManageGroups->processSubmit()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPage.php(575): SpecialManageGroups->execute('unattended')
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run('unattended')
#6 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(296): MediaWiki\\Special\\SpecialPageFactory->executePath(Object(Title), Object(RequestContext))
#7 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(896): MediaWiki->performRequest()
#8 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(527): MediaWiki->main()
#9 /srv/mediawiki/tags/2019-09-26_14:59:29/index.php(44): MediaWiki->run()
#10 {main}
","exception_id":"3e2ff5eab4d50dc8a574a3b9","exception_url":"/wiki/Special:ManageMessageGroups/unattended","caught_by":"mwe_handler"} []

The latter is reading the file to create a message group definitions cache. Maybe the following scenario happened:

  • Some changes happened in ajapaik-web, processMessageChanges.php is run and creates a changediff to process.
  • The file gets broken, processMessageChanges.php fails (though I don't see anything about this in error logs, so not sure)
  • One tries to execute Special:ManageMessageGroups, which fails because the file is now broken

Change 543147 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/Translate@master] Continue import for other groups, if one group import fails

https://gerrit.wikimedia.org/r/543147

Change 543147 abandoned by Abijeet Patro:
Continue import for other groups, if one group import fails

Reason:
Changes should be added as part of Iafa86879d7ea5fe6fc060f5b0a529e9536698ef7

https://gerrit.wikimedia.org/r/543147

Pginer-WMF triaged this task as Medium priority.Oct 30 2019, 8:27 AM

Change 540081 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Continue import for other groups, if one group import fails

https://gerrit.wikimedia.org/r/540081

The related patch has been merged. We should keep an eye on Special:ManageMessageGroups page and ensure that, that is working fine.

This can be tested by introducing an error in the file for a group, and then ensuring that we can still process incoming changes.

abi_ closed this task as Resolved.Nov 28 2019, 3:46 PM

We've been using Special:ManageMessageGroups since this was deployed and haven't noticed any issues but this specific test still needs to be done.

abi_ added a comment.Nov 28 2019, 4:35 PM

We've been using Special:ManageMessageGroups since this was deployed and haven't noticed any issues but those specific tests still need to be run.