Page MenuHomePhabricator
Create Task
Maniphest T234306

Special:ManageMessageGroups failing when a group has invalid YAML
Closed, ResolvedPublic
Actions

Description

Currently due to Ajapaik having issues, ManageMessageGroups is failing to import. We should not stop the export process for other projects / languages if one of the projects has issues.

Log

","exception_id":"abd6701869651ee3857074f1","exception_url":"/w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-a6a386-Invert%20colors%2Fth","caught_by":"mwe_handler"} []
[2019-10-01 00:22:23] exception.ERROR: [7be5f2990d147877f638b67f] /wiki/Special:ManageMessageGroups/unattended   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/messagegroups/FileBasedMessageGroup.php(52): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/utils/MessageGroupCache.php(118): FileBasedMessageGroup->load('en')
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(327): MessageGroupCache->create()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(85): SpecialManageGroups->processSubmit()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPage.php(575): SpecialManageGroups->execute('unattended')
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run('unattended')
#6 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(296): MediaWiki\\Special\\SpecialPageFactory->executePath(Object(Title), Object(RequestContext))
#7 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(896): MediaWiki->performRequest()
#8 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(527): MediaWiki->main()
#9 /srv/mediawiki/tags/2019-09-26_14:59:29/index.php(44): MediaWiki->run()
#10 {main}

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Continue import for other groups, if one group import failsAbijeet Patromediawiki/extensions/Translatemaster+78 -17
Continue import for other groups, if one group import failsAbijeet Patromediawiki/extensions/Translatemaster+15 -21
Customize query in gerrit

Event Timeline

abi_ created this task.Oct 1 2019, 6:22 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 1 2019, 6:22 AM
gerritbot added a comment.Oct 1 2019, 7:27 AM

Change 540081 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/Translate@master] Continue processing other groups if a language under a group fails

https://gerrit.wikimedia.org/r/540081

gerritbot added a project: Patch-For-Review.Oct 1 2019, 7:27 AM
abi_ added a project: Language-Team (Language-2019-July-September).Oct 1 2019, 7:57 AM
abi_ added a subscriber: Pginer-WMF.

@Pginer-WMF - Started working on this since exports were blocked on Twn.

Even though concerned project has fixed the issue with their source translation, since I've already submitted a patch I'd like to add this to the current board.

abi_ moved this task from Backlog to In Review on the Language-Team (Language-2019-July-September) board.Oct 1 2019, 7:58 AM
Pginer-WMF edited projects, added: Language-Team (Language-2019-October-December); removed: Language-Team (Language-2019-July-September).Oct 9 2019, 11:51 AM
Pginer-WMF moved this task from Backlog to In Review on the Language-Team (Language-2019-October-December) board.
abi_ added a comment.Oct 12 2019, 6:51 PM

@Nikerabbit - Usually when an incoming file has formatting issues it is caught during the import itself when processMessageChanges.php runs. Not sure how in this case the error occurred after submitting the changes on Message group management page page.

The patch that I've submitted should handle both scenarios, but I've only been able to test error handling when during processMessageChanges.php and not on the Message group management page.

Patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/540081

Nikerabbit added a comment.Oct 13 2019, 9:21 AM

Yay, Phabricator ate my reply. The pastes strace is actually two different requests.

Here is one in translation aids:

[30-Sep-2019 10:30:52 UTC] AggregateMessageGroup dissemin cannot be primary owner of key e24696-To-do_list
[2019-09-30 10:33:29] exception.ERROR: [d60ef67895bd6139698cf191] /w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-653988-SubjectPhoto%2Fet   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/translationaids/GettextDocumentationAid.php(36): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/api/ApiQueryTranslationAids.php(88): GettextDocumentationAid->getData()
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(1599): ApiTranslationAids->execute()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(537): ApiMain->executeAction()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/api/ApiMain.php(508): ApiMain->executeActionWithErrorHandling()
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/api.php(87): ApiMain->execute()
#6 {main}
","exception_id":"d60ef67895bd6139698cf191","exception_url":"/w/api.php?action=translationaids&format=json&title=Ajapaik%3Aweb-653988-SubjectPhoto%2Fet","caught_by":"mwe_handler"} []

It reads the files to read message comments to display in message documentation section. IMHO those should be cached in message group definitions cache.

Here is the one from Special:ManageMessageGroups:

[2019-09-30 18:59:22] exception.ERROR: [3e2ff5eab4d50dc8a574a3b9] /wiki/Special:ManageMessageGroups/unattended   MWException from line 128 of /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php: Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid "Annotation"
msgstr  "Annotation" {"exception":"[object] (MWException(code: 0): Parsing /resources/projects/ajapaik-web/ajapaik/ajapaik/locale/en/LC_MESSAGES/django.po failed: Unable to parse msgstr:

#: ajapaik/ajapaik/models.py:1028
msgid \"Annotation\"
msgstr  \"Annotation\" at /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/ffs/SimpleFFS.php:128)
[stacktrace]
#0 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/messagegroups/FileBasedMessageGroup.php(52): SimpleFFS->read('en')
#1 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/utils/MessageGroupCache.php(118): FileBasedMessageGroup->load('en')
#2 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(327): MessageGroupCache->create()
#3 /srv/mediawiki/tags/2019-09-26_14:59:29/extensions/Translate/specials/SpecialManageGroups.php(85): SpecialManageGroups->processSubmit()
#4 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPage.php(575): SpecialManageGroups->execute('unattended')
#5 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/specialpage/SpecialPageFactory.php(611): SpecialPage->run('unattended')
#6 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(296): MediaWiki\\Special\\SpecialPageFactory->executePath(Object(Title), Object(RequestContext))
#7 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(896): MediaWiki->performRequest()
#8 /srv/mediawiki/tags/2019-09-26_14:59:29/includes/MediaWiki.php(527): MediaWiki->main()
#9 /srv/mediawiki/tags/2019-09-26_14:59:29/index.php(44): MediaWiki->run()
#10 {main}
","exception_id":"3e2ff5eab4d50dc8a574a3b9","exception_url":"/wiki/Special:ManageMessageGroups/unattended","caught_by":"mwe_handler"} []

The latter is reading the file to create a message group definitions cache. Maybe the following scenario happened:

  • Some changes happened in ajapaik-web, processMessageChanges.php is run and creates a changediff to process.
  • The file gets broken, processMessageChanges.php fails (though I don't see anything about this in error logs, so not sure)
  • One tries to execute Special:ManageMessageGroups, which fails because the file is now broken
gerritbot added a comment.Oct 15 2019, 1:42 PM

Change 543147 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/Translate@master] Continue import for other groups, if one group import fails

https://gerrit.wikimedia.org/r/543147

gerritbot added a comment.Oct 15 2019, 1:49 PM

Change 543147 abandoned by Abijeet Patro:
Continue import for other groups, if one group import fails

Reason:
Changes should be added as part of Iafa868

https://gerrit.wikimedia.org/r/543147

Pginer-WMF triaged this task as Medium priority.Oct 30 2019, 8:27 AM
gerritbot added a comment.Nov 4 2019, 6:31 PM

Change 540081 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Continue import for other groups, if one group import fails

https://gerrit.wikimedia.org/r/540081

ReleaseTaggerBot added a project: MW-1.35-notes (1.35.0-wmf.5; 2019-11-05).Nov 4 2019, 7:01 PM
Maintenance_bot removed a project: Patch-For-Review.Nov 4 2019, 7:11 PM
abi_ moved this task from In Review to Recheck after deployment on the Language-Team (Language-2019-October-December) board.Nov 5 2019, 4:01 AM

The related patch has been merged. We should keep an eye on Special:ManageMessageGroups page and ensure that, that is working fine.

This can be tested by introducing an error in the file for a group, and then ensuring that we can still process incoming changes.

abi_ closed this task as Resolved.Nov 28 2019, 3:46 PM
abi_ moved this task from Recheck after deployment to Done on the Language-Team (Language-2019-October-December) board.

We've been using Special:ManageMessageGroups since this was deployed and haven't noticed any issues but this specific test still needs to be done.

abi_ added a comment.Nov 28 2019, 4:35 PM

We've been using Special:ManageMessageGroups since this was deployed and haven't noticed any issues but those specific tests still need to be run.

Nemo_bis added a project: MediaWiki-extensions-Translate.Nov 28 2019, 5:11 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits