Page MenuHomePhabricator

"This namespace is reserved for content page translations" when trying to translate a recently created translation unit
Open, HighPublic

Description

Sometimes translators are unable to save translations due to error "This namespace is reserved for content page translations". This error should not happen in regular use, only in edge cases such as someone keeping a page open so long that the translatable page has changed in meanwhile, or manually editing URLs.

General causes

The underlying cause is a mismatch of the state in MessageIndex versus MessageCollection. MessageIndex is a reverse map of translation unit names to message groups. MessageCollection asks the message group directly what are the names of translation units in that group.

Every time there is a change to translation units, the code is supposed to run a MessageIndexRebuildJob. If this job is not run, or is delayed, or fails, then we get a mismatch of state.

There is additional check on the Translations namespace that prevents editing of translation units that do not correspond to any translatable pages. This is just to keep things tidy to avoid mistakes when people manipulate URLs by hand. Nothing bad would happen if this check was not there.

This issue happens on newly created translation units, or pages newly added for translations.

Current causes

We believe that currently this issue manifests when MessageIndexRebuildJob fails due to Exception: MessageIndex: unable to acquire lock. This means that some other MessageIndexRebuildJob is keeping the table locked for a long time. https://grafana.wikimedia.org/d/000000400/jobqueue-eventbus?orgId=1&from=now%2FM&to=now%2FM&var-site=All&var-type=MessageIndexRebuildJob provides some insight how long the job takes when run in JobQueue (but it is often run as part of other jobs as well).

Workaround

Any other activity that makes the MessageIndexRebuildJob run again will "heal" the MessageIndex by making it up to date again. This is most easily accomplished by doing a dummy edit on any translatable page and marking it for translation. Sometimes just waiting a bit can help if the job is just slow or delayed.

Details

Related Gerrit Patches:
mediawiki/extensions/Translate : masterAllow editing of orphaned translation units
mediawiki/extensions/Translate : masterDisplay an error message if translation aids fail to load
mediawiki/extensions/Translate : masterAdd secondary check before displaying tpt-unknown-page error
operations/mediawiki-config : masterAdd Translate channel for the Translate extension
mediawiki/extensions/Translate : masterLog tpt-unknown-page errors
operations/mediawiki-config : masterAdd channels for the Translate and TranslationsNotification extension
mediawiki/extensions/Translate : masterDefer message group stats update in message index rebuild
mediawiki/extensions/Translate : masterAdd logs to the jobs involved with marking a page for translation

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Still occured today on mediawiki. Then I made a dummy modification in a EN message of the page (delete one 'a' and reinsert the 'a') before resending 'for translation' . It worked. And I noticed the log only register one action 'sent for translation'.

Eihel added a comment.Jul 28 2019, 9:34 PM

Hello,
Yes @Wladek92, it works. It seems to me that Nikerabbit mentioned this: "For urgent cases, to try to make new content translatable, one can (re-) mark any page for translation...". Au plaisir.

Justly, @Nikerabbit, if it's a queue problem, how to explain that in re-mark a page, it re-works immediately? Because the new marking puts the translations back in the queue. Is it possible that bufferization is the cause of the problem? (a telescoping (crash) of information, a concatenated surplus of information, the occurrence of other information denatures the first request, etc.).

The right to mark in translation has brought more demand by increasing the number of translation administrators, QED. I noticed other things. Do you want me to tell you about these experiences or does it not seem useful to you? (This is when and how I see this problem)

Change 526589 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[mediawiki/extensions/Translate@master] WIP: Add a generic job class with helpers to log data

https://gerrit.wikimedia.org/r/526589

Estimation: about 50 % of the translation requests I do must be requested again.

The problem seems to reach also other wikis : similar behaviour (cannot save the translations) has been observed on KDE/userbase (see https://userbase.kde.org/Thread:User_talk:Yurchor/help:_cannot_save_the_FR_translations)

abi_ added a comment.EditedAug 11 2019, 3:49 PM

Thanks @Wladek92 and @Eihel for your inputs.

I've started working on this issue based on Niklas' previous comment and my discussions with him, and here's an update,

Adding checks during translation saving

When a page is marked for translations,

  1. The sections are inserted into the translate_sections table. This happens in SpecialPageTranslation::markForTranslation function itself.
  2. In TranslationUpdateJob, each section is created / updated as a page in the wiki by the MessageUpdateJob
  3. In TranslationUpdateJob, the MessageIndex is updated.
  4. Although not relevant in this context, In TranslationUpdateJob, TranslateRenderJobs are also added to update the translation pages.

Problems

Imagine a page - T221119, with 2 translation units currently. An editor adds a third translation unit ie T221119/3. The existing translation units will be T221119/1 and T221119/2.

MessageHandle::isValid() is called when saving a translation to check if an invalid message that does not belong to a translatable page is being saved..

It checks if the MessageIndex has that key (T221119/3), which will not be present if MessageIndex::singleton()->rebuild(); is not called on the TranslationUpdateJob when the new section is added.

Solution

Updating the MessageIndex should fix this issue but since that is a fairly large intensive process it will have to be done with care. It might be possible to add a method that rebuilds a specific MessageGroup into the cache.

I'll discuss with Niklas and see if some changes can be made.

Adding further logs to increase visibility with failures

I've update the code and submitted a patch to add further logs. Currently waiting for that patch to be merged, and to get access to LogStash.

abi_ added a comment.Aug 11 2019, 6:03 PM

(Updated my previous comment with latest findings)

abi_ added a comment.Aug 17 2019, 3:22 PM

I've now received LogStash access, and reviewing the logs from there for this particular job (TranslationsUpdateJob) reveals that we've been having issues with rebuilding the MessageIndex. The following exception message is present 381 times in the last 30 days - MessageIndex: unable to acquire lock.

Building the page markup, I observe from a practical point of view that it fails each time. But if you have modified tanslation units already existing at step n-1, the translated text you will enter will be accepted. Failure appears with new created translations unit in the EN text.
When you redo what people call a dummy update of the EN text, or if you go further creating new units, you will then validate the pending new previous one (of step n-1) and push the problem one step forward.

I observe also that if you wait long enough (coming back tomorrow) you do not see the problem.

Change 534571 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Defer message group stats update in message index rebuild

https://gerrit.wikimedia.org/r/534571

Change 526589 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Add logs to the jobs involved with marking a page for translation

https://gerrit.wikimedia.org/r/526589

Change 534571 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Defer message group stats update in message index rebuild

https://gerrit.wikimedia.org/r/534571

Change 537363 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[operations/mediawiki-config@master] Add channels for the Translate and TranslationsNotification extension

https://gerrit.wikimedia.org/r/537363

Change 537363 merged by jenkins-bot:
[operations/mediawiki-config@master] Add channels for the Translate and TranslationsNotification extension

https://gerrit.wikimedia.org/r/537363

Mentioned in SAL (#wikimedia-operations) [2019-09-17T11:31:17Z] <urbanecm@deploy1001> Synchronized wmf-config/VariantSettings.php: SWAT: 290e207: Add channels for the Translate and TranslationsNotification extension (T221119, T144780, T143073) (duration: 00m 56s)

Xiplus added a subscriber: Xiplus.Sep 22 2019, 3:17 PM
Quiddity added a comment.EditedOct 10 2019, 8:01 PM

Might this be related to T235027: Translate does not update content page when saving units ?

Also: I encountered this bug again yesterday and today, whilst showing someone how the whole system works from a user-perspective. As usual, a dummy-edit fixed it.

However I did additionally notice that there seems to be a correlation between this bug, and whether or not an entry is added to the page-history and the logs, versus just the logs. E.g.

Possibly irrelevant, but maybe new info.

I often add translation unit markers manually so that I have finer control on how translation units will look like. This means the mark for translation action actually creating the translation unit doesn’t get recorded in the history (as the wikitext doesn’t change), yet, I face this issue in such cases as well (almost always, as usual).

No that is a different, recent regression.

  • Then I made a dummy edit, and marked the page again at 19:44, but that action was only added to the logs, and not to the page-history. And I could successfully save a translation unit afterwards.

Possibly irrelevant, but maybe new info.

That's expected, as you were not creating any new translation units. It's the newly created translation units that fail to get registered sometimes.

I was looking at Logstash, and this issue seems to have become rarer after some fixes were deployed on first half of September:

It is not gone however. Some further things we can do:

  • [high] Add logging for "This namespace is reserved for content page translations" so that we know how often users actually encounter this error
  • [medium] If we detect this error, do a separate check against revtag table using code similar to TranslatablePage::getTranslatablePages()
  • [low] Further instrument MessageIndex to understand what is happening there
Nikerabbit renamed this task from Translate error - "This namespace is reserved for content page translations" to "This namespace is reserved for content page translations" when trying to translate a recently created translation unit.Oct 15 2019, 9:01 AM
Nikerabbit updated the task description. (Show Details)

Change 543102 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Log tpt-unknown-page errors

https://gerrit.wikimedia.org/r/543102

Change 544114 had a related patch set uploaded (by Abijeet Patro; owner: Abijeet Patro):
[operations/mediawiki-config@master] Add Translate channel for the Translate extension

https://gerrit.wikimedia.org/r/544114

Change 543102 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Log tpt-unknown-page errors

https://gerrit.wikimedia.org/r/543102

Change 544114 merged by jenkins-bot:
[operations/mediawiki-config@master] Add Translate channel for the Translate extension

https://gerrit.wikimedia.org/r/544114

Mentioned in SAL (#wikimedia-operations) [2019-10-28T11:12:36Z] <urbanecm@deploy1001> Synchronized wmf-config/InitialiseSettings.php: SWAT: dd2f06c: Add Translate channel for the Translate extension (T221119) (duration: 00m 53s)

seems better performances observed on Translate on mediawiki.org thanks.

The newly adding logging so far shows three entries (of the same translatable page).

Change 546886 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Add secondary check before displaying tpt-unknown-page error

https://gerrit.wikimedia.org/r/546886

Change 546887 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Display an error message if translation aids fail to load

https://gerrit.wikimedia.org/r/546887

Change 546886 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Add secondary check before displaying tpt-unknown-page error

https://gerrit.wikimedia.org/r/546886

Change 546887 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Display an error message if translation aids fail to load

https://gerrit.wikimedia.org/r/546887

We have logging now:

# Succesfull edit
2019-11-04T20:52:01	mw1309	INFO	deleteByQuery against [ttmserver] on cluster [production-search-eqiad] with task id [pCFROq6mRQqF0ir0DG3mfQ:507689153] starting

# Marking page second time
2019-11-04T20:51:53	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Added 4 RenderJobs to the queue
2019-11-04T20:51:53	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Finished TranslationsUpdateJob
2019-11-04T20:51:53	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Finished purging
2019-11-04T20:51:53	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Updated the message group stats
2019-11-04T20:51:52	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Cleared caches
2019-11-04T20:51:45	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Finished running 23 MessageUpdate jobs for 23 sections
2019-11-04T20:51:44	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCPXwpAIC8AAI76gpcAAABR][Title: API:Alldeletedrevisions] Starting TranslationsUpdateJob

# Failed edits
2019-11-04T20:50:27	mw1341	INFO	Unknown translation page: Translations:API:Alldeletedrevisions/22/fr
2019-11-04T20:49:39	mw1347	INFO	Unknown translation page: Translations:API:Alldeletedrevisions/22/fr

# Marking page first time
2019-11-04T20:48:50	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Finished TranslationsUpdateJob
2019-11-04T20:48:50	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Added 4 RenderJobs to the queue
2019-11-04T20:48:50	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Finished purging
2019-11-04T20:48:50	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Updated the message group stats
2019-11-04T20:48:49	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Cleared caches
2019-11-04T20:48:41	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Finished running 23 MessageUpdate jobs for 23 sections
2019-11-04T20:48:40	mw1308	INFO	[Job: TranslationsUpdateJob][Request ID: XcCOpwpAAEMAABm7aTEAAABE][Title: API:Alldeletedrevisions] Starting TranslationsUpdateJob

It's not clear why message index was stale after marking for first time. The code in question is:

// Ensure we are using the latest group definitions. This is needed so
// that in long running scripts we do see the page which was just
// marked for translation. Otherwise getMessageGroup in the next line
// returns null. There is no need to regenerate the global cache.
MessageGroups::singleton()->clearProcessCache();
// Ensure fresh definitions for MessageIndex and stats
$page->getMessageGroup()->clearCaches();

MessageIndex::singleton()->rebuild();

$this->logInfo( 'Cleared caches' );

Maybe there is a still possibility of if using a stale data.

In the last 7 days, we have 265 failures to save a translation due to this error. Most of them are re-attempts, so the number of users facing this issue is smaller. In any case, the error should not happen after my latest patch gets deployed this week (hopefully today for group0).

ok i will tell whether the problem still appears - thanks.

While checking the logs again, I noticed a use case we might want to support: Someone looked at https://meta.wikimedia.org/wiki/Special:WhatLinksHere/Multilingualism/en and probably wanted to update pages linking to it with a bot. However some of those translation units are orphaned. I believe we should allow editing and/or deleting orphaned translation unit pages that exist.

Change 549422 had a related patch set uploaded (by Nikerabbit; owner: Nikerabbit):
[mediawiki/extensions/Translate@master] Allow editing of orphaned translation units

https://gerrit.wikimedia.org/r/549422

Wladek92 added a comment.EditedNov 7 2019, 11:13 AM

Side effect:

  • doc pannel and suggestion have disapeared for this new articles and replaced by an error warning (previous translated items, show the normal way)

(see attached screen shot image)

Wladek92 added a comment.EditedNov 7 2019, 11:43 AM

When the previous side effect appears, translation is accepted but when reloading the page (even with purge action) the translated text does not appear. Returning to the message, translation is present and warning panel 'Title does not correspond to a translatable message' displays again.
see => https://www.mediawiki.org/w/index.php?title=Translations:Skin:Timeless/23/fr&oldid=3499936
Applying Dummy Update on the root EN page unlocks the situation and previously translated text is now present in the generated text without new update of the message.
=>https://www.mediawiki.org/w/index.php?title=Skin:Timeless&diff=3499938&oldid=3499922&diffmode=source

Seems 'random' behaviour since sometimes rendering is ok immediatly

Change 549422 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Allow editing of orphaned translation units

https://gerrit.wikimedia.org/r/549422

You are correct. My change makes it so that saving is possible, but the rest of the system, including translation aids, still depend on the message index being up to date.

According to the logs, it seems more and more likely that T221119#5634547 needs further investigation.

Yes I confirm new articles can now be saved just after 'translation requested' has been executed (which allow to progress in the article list => good) but they are not yet rendered; a 'Dummy update' of the root page is needed to see all translations together.

4nn1l2 added a subscriber: 4nn1l2.Nov 15 2019, 6:24 AM
Eihel added a comment.EditedNov 15 2019, 12:28 PM

When the previous side effect appears, translation is accepted but when reloading the page (even with purge action) the translated text does not appear. Returning to the message, translation is present and warning panel 'Title does not correspond to a translatable message' displays again.
see => https://www.mediawiki.org/w/index.php?title=Translations:Skin:Timeless/23/fr&oldid=3499936
Applying Dummy Update on the root EN page unlocks the situation and previously translated text is now present in the generated text without new update of the message.
=>https://www.mediawiki.org/w/index.php?title=Skin:Timeless&diff=3499938&oldid=3499922&diffmode=source
Seems 'random' behaviour since sometimes rendering is ok immediatly

Hello,
On WD too, nulledit on original en version and that works: https://www.wikidata.org/w/index.php?title=Wikidata:Requests_for_permissions/Administrator&diff=prev&oldid=1053175618&diffmode=source for french translation: https://www.wikidata.org/wiki/Wikidata:Requests_for_permissions/Administrator/Header/fr.
Sincerely.

I observe back now 'normal' behaviour of the functionality; situation is stable in a positive way and has improved for these last few days. : generation of articles to translate is done in one shot operation, immediate integration of translated articles just after translation and reload. Thanks.

Pols12 added a subscriber: Pols12.Nov 27 2019, 7:11 PM

I observe back now 'normal' behaviour of the functionality; situation is stable in a positive way and has improved for these last few days. : generation of articles to translate is done in one shot operation, immediate integration of translated articles just after translation and reload. Thanks.

Oh, unfortunately, I just encountered “Title does not correspond to a translatable message”, trying to translate meta:Community Tech/Who Wrote That tool.

Wladek92 added a comment.EditedNov 28 2019, 11:52 AM

A lot of translated messages do not appear after page purge and reload (but problem of title message has not appear since a long time) . Workaround: doing the dummy update on root EN page to unlock all at once

12:49, 28 November 2019 Wladek92 talk contribs marked Help:RevisionDelete for translation
Wladek92 added a comment.EditedNov 28 2019, 12:03 PM

Response to Pols12 => I can open the page and access to translations (it is long) and have translated a few dates successfully as => === 20 février 2019 === on a french translation => https://meta.wikimedia.org/wiki/Community_Tech/Who_Wrote_That_tool/fr . So your issue has not appeared on my side.

28 novembre 2019 à 12:57 diff hist +1‎ Community Tech/Who Wrote That tool/fr ‎ Created page with "=== 20 février 2019 ===" actuelle

Response to Pols12 => I can open the page and access to translations (it is long) and have translated a few dates successfully as => === 20 février 2019 === on a french translation => https://meta.wikimedia.org/wiki/Community_Tech/Who_Wrote_That_tool/fr . So your issue has not appeared on my side.

It does not appear on this page anymore since I’ve re-marked the page for translation, as Eihel’s workaround.

Arggr even consecutive dummy updates cannot make the translated messages appear (messages are translated but the /FR subpage is not present; ref. => https://www.mediawiki.org/w/index.php?title=Extension:GeoData/geo_tags_table&diff=3566157&oldid=3566151&diffmode=source

Eihel added a comment.EditedDec 13 2019, 6:35 PM

Arggr even consecutive dummy updates cannot make the translated messages appear (messages are translated but the /FR subpage is not present; ref. => https://www.mediawiki.org/w/index.php?title=Extension:GeoData/geo_tags_table&diff=3566157&oldid=3566151&diffmode=source

Hello, see T235027 (not the same Task) and perhaps my comment on this Task with the explanation. That works with a nulledit on the wiki. For you (and for me 'fr'):
Phase: Afficher dans l’éditeur wiki (Click on the link of the corresponding page in the wiki in the Translate tool). Publishing to the wiki brings up the main page. A+

I am not sure if T240518: Some jobs are not being processed / are processed slowly affected Translate jobs, but if it did, marking for translation should work fast now.

Restricted Application added a subscriber: RhinosF1. · View Herald TranscriptDec 18 2019, 12:03 PM

(In reply to the “In Review” column) It still happens, see my recent edit summary.

Pols12 awarded a token.Sun, Jan 5, 2:35 PM

I don’t know if anything changed recently, but now I got a “Service Temporarily Unavailable” message (without any further details) when trying to mark https://commons.wikimedia.org/wiki/Template:Wikidata_Infobox/i18n without any changes (diff to be marked for translation), hoping it will finally become translatable (as new translation units are still untranslatable). It became translatable (see my successful translation), but Special:PageTranslation returned the Service Temporarily Unavailable message, and the /i18n page still had the note about changes pending for translation. Reloading Special:PageTranslation (re-POSTing the form) fixed the issue.