Page MenuHomePhabricator

Async and non-atomic saving of page translation edits?
Closed, ResolvedPublic


NOTE: For a summary on related issues, patches and upgrade avenues, see T48716: Translation page does not contain the latest translations/last translation.

Two edits to two translation units were collated in a single edit to the translation page:
Is this intentional? If yes we should document how the edit summary is chosen, so that the user knows what to write.


In T53731#2151432, Glaisher wrote:

In SpecialPageTranslation::markForTranslation(), TranslateRenderJobs for updating the translation pages are pushed to the queue first and then MessageUpdateJobs are pushed to the queue to update the translation units. These jobs can be run in any order but usually it's sequentially (I think). However, TranslateRenderJob depends on the translation units to be updated through MessageUpdateJobs before it is run because TranslateRenderJob retrieves the content for the translation page edit through the updates from that job.

To confirm this and to make it easier to debug, I locally hacked markForTranslation() to make the jobs be run() immediately instead of pushing them to the queue. If the order is changed so that getTranslationUnitJobs() is run before getRenderJobs(), this bug cannot be reproduced at all but is present if it's in the current order. Note that the /en page would be updated but this is because of the edits to the unit pages which trigger the hook making the TranslateRenderJob be run repeatedly on that page. We should probably stop doing that.

Version: unspecified
Severity: minor
See Also:



Event Timeline

bzimport raised the priority of this task from to Low.Nov 22 2014, 1:30 AM
bzimport set Reference to bz45894.
bzimport added a subscriber: Unknown Object (MLST).

Each save triggers complete refresh of the translation page. There is a race condition here and the order is essentially random.

  1. Multiple translations are updated about the same time
  2. Some of them can complete faster
  3. A hook runs after each save is complete, loading the most recent translation units from master. It saves the updated translation page using the summary of the translation unit which triggered the hook.
  4. For the other saves, they also load the most recent translations, but as the content is the same, no revision is recorded in the history.

The harm is mostly only a confusion to the users.


(In reply to comment #1)

The harm is mostly only a confusion to the users.

What's strange is that I didn't save the two messages so quickly, several seconds passed.

Here a manual edit (translation) was attributed to FuzzyBot:
If you check the previous edits you'll see that no diff is associated to the correct timestamp or summary.

Here it took three dummy edits to get all the updates (confirming outdated translations) applied to the translation page:

Curiously, it's possible to trigger an update of a translation page even with a null (API) edit to a translation unit page: doesn't have any corresponding edit in Translations namespace because content is the same as a (2 months old) previous edit.

(In reply to Nemo from comment #5)

Curiously, it's possible to trigger an update of a translation page even
with a null (API) edit to a translation unit page.

I think this is intentional. For instance, I used my bot to clear some usages of obsolete images:

It would be nice to hear whether this asynchronousness is really required. When the edits were still synchronous, we had no issues like T48716.

It is not async. It is a race condition like I said in my first comment.

I may have an issue with terminology, I can't understand the difference/opposition between async and race condition. I'll try to rephrase the question: is there a way to make these edits happen in a sequential, deterministic way?

Yes, with some locking. But locking is expensive and cause performance issues, so it is usually used only when necessary.

(A)synchronous is mostly about how some code is written.

Simple synchronous example (in JavaScript):

var users = getUsers();
// Code execution stops here until the function call returns.
doSomethingWith( users );

Simple asynchronous example:

getUsers().done( function ( users ) {
  // Code starts executing here when getUsers is finished
  doSomethingWith( users );
} );
// Code here keeps executed immediately

Race condition usually means that some data changes value after it or something it depends on have been read. For example, let's assume there are two processed that update the value of foo, which is initially set to 2.

Process A does foo = foo + 5;
Process B does foo = foo * 3;

If there is no locking, any of these outcomes is possible:

  • foo = 21 (A reads and writes, B reads and writes)
  • foo = 6 (A and B reads in any order, A writes, B writes)
  • foo = 7 (A and B reads in any order, B writes, A writes)
  • foo = 11 (B reads and writes, A reads and writes)

What is happening here is that

  1. a translation page unit is updated by web request C.
  2. a unit (can be same or different) is updated by web request D.
  3. translatable page is updated by fetching all latest units (including the one updated by D) by web request C.
  4. translatable page is updated by fetching all latest units by web request D - there is no change as the content is identical to the previous update.

So, should the summary be rephrased to "Introduce locking in saving of page translation edits"?

I prefer problem statements rather than explicit actions. Here the problem is that translation page updates are not atomic.

Yes, that's one part of the issue. The other one is that they are non-deterministic. It would be ok to conflate multiple edits into one, if only the user could understand the logic. Getting random outputs from non-random inputs confuses the user about the feature and makes the wiki (e.g. page history) less transparent.

So, what can be done to make the updates atomic?

Nemo_bis renamed this task from Async saving of page translation edits? to Async and non-atomic saving of page translation edits?.Mar 24 2016, 4:08 PM

A non-Wikimedia translator was confused by this bug on ; the most frequent cause is FuzzyBot edits not being applied, so I instructed them to upgrade to fix T48716: Translation page does not contain the latest translations/last translation. I also added a note to this purpose on the task description.

Nikerabbit claimed this task.

The jobs are again using jobqueue, which means they are again async, but I haven't seen complaints about that.