Page MenuHomePhabricator

UploadWizard does not preserve the order of the images between upload and later steps
Closed, ResolvedPublic

Description

From T92734:

On the screen for naming the files, they are not arranged in alphabetical order which makes it inconvenient to use the automatic numbering function. Files from digital cameras are often numbered sequentially (eg, "IM000213", "IM000214", "IM000215" and so on). If the files appeared on the screen in alphabetical order, then they would also be sequential according to their numbers, and the automatic numbering function would be able to number them according to the order in which they were taken.

Event Timeline

Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added projects: UploadWizard, Commons.
Tgr subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

If I remember correctly, they currently end up ordered on the describe step by how soon each upload ended. The file that finished uploading the soonest will be first in the list, etc. At least that was my experience with flickr uploads when I wrote the browser test for it.

While some people might prefer to have them automatically sorted for them at every step, the more cautious first step would be to at least respect the order shown at the stage where the files have been picked. It will still achieve the same result as this task's description when the user picks consecutive file from a folder at once to upload.

I say that because I anticipate that if we automatically sort them at the file picking stage it will break some people's workflow, whereby they want to pick them in a particular order. Granted that anybody trying to do that at the moment must be frustrated because they are inevitably shuffled once they get to the describe stage. Unless the user has set upload concurrency to 1 (in theory, because I've just checked and it seems like this preference doesn't work).

I tried altering concurrent uploads to 1 but that doesn't seem to help in getting files to upload in a particular order.

Yep, that preference doesn't do what it's supposed to: T92809

So can anything be done about this?

matmarex added subscribers: Ricordisamoa, Pigsonthewing.
matmarex subscribed.

If I remember correctly, they currently end up ordered on the describe step by how soon each upload ended. The file that finished uploading the soonest will be first in the list, etc. At least that was my experience with flickr uploads when I wrote the browser test for it.

Yeah, I can reproduce this.

Change 234530 had a related patch set uploaded (by Bartosz Dziewoński):
Keep the uploads sorted in the order they were created in initially

https://gerrit.wikimedia.org/r/234530

Change 234530 merged by jenkins-bot:
Keep the uploads sorted in the order they were created in initially

https://gerrit.wikimedia.org/r/234530

Change 234553 had a related patch set uploaded (by Bartosz Dziewoński):
Keep the uploads sorted in the order they were created in initially

https://gerrit.wikimedia.org/r/234553

I scheduled the patch fixing this bug (above) to be deployed on Monday morning.

Change 234553 merged by jenkins-bot:
Keep the uploads sorted in the order they were created in initially

https://gerrit.wikimedia.org/r/234553

Indeed I didn't reproduce the bug any longer. I experienced the regression on August 29 but not on September 16.

intracer updated the task description. (Show Details)
intracer set Security to None.
matmarex removed a project: Patch-For-Review.
@intracer wrote:

Still reproduced.
https://commons.wikimedia.org/?curid=44662242 File:Monuments of Ukraine - Crimea Awards 30.JPG is uploaded before
https://commons.wikimedia.org/?curid=44662243
File:Monuments of Ukraine - Crimea Awards 29.JPG

@intracer, thanks for reporting this, but I think it is the expected behavior.

Unless you changed the preference "Maximum number of concurrent uploads" at https://commons.wikimedia.org/wiki/Special:Preferences?uselang=en#mw-prefsection-uploads to 1 (the default is 3), UploadWizard will be publishing three of the uploads you filled in at once. It starts doing this in the correct order, but we have no control of when the requests reach the backend software, or how long they take, or in which order they finish. If you want your uploads to always be submitted in the order you filled them, set this preference to 1 (but be aware that this will probably make uploading and publishing a bit slower for you).

I have Maximum number of concurrent uploads set to 0. What does it mean? some default or unlimited?
If i'm not mistaken I believe I specifically set it to zero because some WMF developer told that this will resolve upload reordering before this issue was resolved (which not happened).

Mutual exclusion of reordering and parallel uploads is not the only possible way.
Maybe the simplest to assume, but why close this issue then?

Yes, it's the default. Setting it to 0 or 3 is exactly equivalent. Don't ask me why :)

Whoever told you that either made a mistake, because that isn't true now and hasn't been true for a while, or did so a very long time ago when it could have been true.

File uploads are already parallel (and don't even respect this preference, see T92809) and their order already does not matter (since I fixed this bug). In the last step, which we're discussing, they are only "published" and their descriptions are submitted. I am afraid that it is indeed not possible to force parallel tasks to end in a given order if we don't know how long they'll take before starting them.

Why not possible? Not trivial but possible. For example - reserve all ids in batch at once, and then give them to the files when they are uploaded in order.

Okay, let me rephrase. It is impossible to fix in UploadWizard itself, without a refactoring of the MediaWiki upload backend (and probably page creation too), perhaps something like you describe, which unfortunately is not something any of us UploadWizard developers would be comfortable doing.

Actually, at least for me, UploadWizard currently seems to be keeping the files in the order they are selected for upload. The bug thus appears to have been fixed.

Okay, let me rephrase. It is impossible to fix in UploadWizard itself, without a refactoring of the MediaWiki upload backend (and probably page creation too), perhaps something like you describe, which unfortunately is not something any of us UploadWizard developers would be comfortable doing.

OK, lets assume that a large part of the bug is fixed. Maybe its better to track the remaining behaviour in another issue

Actually, at least for me, UploadWizard currently seems to be keeping the files in the order they are selected for upload. The bug thus appears to have been fixed.

Well, WORKSFORME argument is anecdotal regradless of whether it's fixed or not. I provided upload example that happened today and reordered the files.

OK, lets assume that a large part of the bug is fixed. Maybe its better to track the remaining behaviour in another issue

The bug is wholly fixed (thanks MM team!). You seem to have a separate issue in mind, which is that MediaWiki should, in its various upload lists (recent files, personal gallery etc) display the files in the same order as they were added in UploadWizard. Feel free to open a new task about it, but I'm pretty sure it's going to be wontfixed - it's just not worth the effort it would take. By pre-reserving ids you violate the assumption that every row in the image table identifies an image, and all code depending on it would have to be updated (including MediaWiki, extensions and Tool Labs tools) while the benefit from it is minimal. There are many bugs with serious negative effects where that effort would be better spent.