Images permanently disappear when renaming files on Commons
Closed, ResolvedPublic

Description

Author: material.scientist

Description:
At least three images have disappeared from Commons as a result of file renaming over the last few weeks. All renames were carried out using the

http://commons.wikimedia.org/wiki/Template:Rename

template. Nothing abnormal was seen during these renames, except for the "usual" general red error box (something like API error - retry/cancel/abort).

  1. http://commons.wikimedia.org/wiki/File:Capuchin_monastery_in_Lubart%C3%B3w,_Poland.jpg
  1. http://commons.wikimedia.org/wiki/Commons:Administrators%27_noticeboard/Archive_35#Botched_file_move
  1. http://commons.wikimedia.org/wiki/Commons:Deletion_requests/File:Vail_Hotel.jpg

(there were more, but can't find links)


Version: unspecified
Severity: critical

bzimport set Reference to bz36440.
bzimport created this task.Via LegacyMay 2 2012, 1:52 PM
aaron added a comment.Via ConduitMay 4 2012, 3:00 PM

You should try moving the page back and seeing if the file looks OK again. This is typically how move failures are handled (or delete/restore for that matter).

Does this only happen via API?

aaron added a comment.Via ConduitMay 4 2012, 11:00 PM

(In reply to comment #1)

You should try moving the page back and seeing if the file looks OK again. This
is typically how move failures are handled (or delete/restore for that matter).

Looking around via shell it looks like the old name copies were deleted too :(

https://gerrit.wikimedia.org/r/6638 deployed in response. That is a confirmed issue, though I don't no for sure it it caused this.

bzimport added a comment.Via ConduitMay 6 2012, 10:42 PM

material.scientist wrote:

Same thing happened since with
http://commons.wikimedia.org/wiki/File:Crypte_de_l%27%C3%A9glise_de_Gargilesse-Resurection_des_morts_G-20120429.jpg

Here, there might have been a move conflict: I've tried to move that file and http://commons.wikimedia.org/wiki/File:Crypte_de_l%27%C3%A9glise_de_Gargilesse-R%C3%A9surection_des_morts_D-20120429.jpg, and the final move might be same (uploader requested, and it was hard to see what he/she wants)

aaron added a comment.Via ConduitMay 7 2012, 1:19 AM

https://gerrit.wikimedia.org/r/6638 has caught several errors according to the logs, but those were just for uploads. So something else is also going on.

Looking at LocalFileMoveBatch, I can see that the begin() calls break the file locking (which uses FOR UPDATE). Note that the file locking hasn't worked at for anything until the last few months, so I'm not sure that's the cause.

aaron added a comment.Via ConduitMay 7 2012, 1:28 AM

(In reply to comment #4)

Looking at LocalFileMoveBatch, I can see that the begin() calls break the file
locking (which uses FOR UPDATE). Note that the file locking hasn't worked at
for anything until the last few months, so I'm not sure that's the cause.

*hasn't worked at all

bzimport added a comment.Via ConduitMay 7 2012, 3:16 AM

material.scientist wrote:

Naive comments: (i) upload mistakes do block file moves - we have to delete failed uploads in a file page to unlock the move (or we get that red API error box, without telling the problem). (ii) I thought maybe move conflicts are a factor? To clarify: in this case I tried to move two files into the same destination from 2 firefox windows (one got "moved" one got API error). Often two editors try to move one file simultaneously, and I see that only indirectly (as a glitch with another mover).

bzimport added a comment.Via ConduitMay 7 2012, 10:26 PM

material.scientist wrote:

One more from minutes ago
http://commons.wikimedia.org/wiki/File:Mrpfpindiraghandi.jpg
This happened before: an image disappears from the file "after clicking on the button in the rename template" without a log (well, at least on my screen).

aaron added a comment.Via ConduitMay 8 2012, 9:49 PM

(In reply to comment #7)

See also: https://gerrit.wikimedia.org/r/#/c/6783/

Deployed now.

aaron added a comment.Via ConduitMay 10 2012, 10:57 PM

(In reply to comment #9)

(In reply to comment #7)
> See also: https://gerrit.wikimedia.org/r/#/c/6783/

Deployed now.

Have there been instances since that comment?

MarkAHershberger added a comment.Via ConduitMay 11 2012, 1:49 PM

I've asked for input http://commons.wikimedia.org/w/index.php?diff=71016664&oldid=71005678 and we'll see if we get any.

bzimport added a comment.Via ConduitMay 12 2012, 5:21 AM

material.scientist wrote:

API errors are common with me these days, i.e. renaming scrip fails with 50/50 chances, but the files are not lost (just have to manually complete the rename). That said, have a look at
http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg
I haven't touched that file and was only involved with commonsdelinker - looks like a double move (Hypselodoris fontandraui (Pruvot-Fol, 1951) 0 .jpg -> Felimare fontandraui (Pruvot-Fol, 1951).jpg -> Felimare_fontandraui.jpg) with the same image disappearance as when I started this thread. The image thumbnail can be found by ImageGoogling "Hypselodoris fontandraui (Pruvot-Fol, 1951) 0 .jpg" - a violet horizontal slag.

aaron added a comment.Via ConduitMay 19 2012, 8:36 AM

Maybe a list of affected files can be kept somewhere?

aaron added a comment.Via ConduitMay 25 2012, 7:59 PM

(In reply to comment #12)

API errors are common with me these days, i.e. renaming scrip fails with 50/50
chances, but the files are not lost (just have to manually complete the
rename). That said, have a look at
http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg

Have you ran into any new instances?

bzimport added a comment.Via ConduitJun 2 2012, 11:14 PM

material.scientist wrote:

(In reply to comment #14)

(In reply to comment #12)
> API errors are common with me these days, i.e. renaming scrip fails with 50/50
> chances, but the files are not lost (just have to manually complete the
> rename). That said, have a look at
> http://commons.wikimedia.org/wiki/File:Felimare_fontandraui.jpg

Have you ran into any new instances?

http://commons.wikimedia.org/wiki/File:Arjuna_and_His_Charioteer_Krishna_Confront_Karna,_crop.jpg
Images (being renamed) temporarily disappear during database delays (during which the renaming script also fails) but the above seems permanent.

bzimport added a comment.Via ConduitJun 5 2012, 4:57 AM

ladarabara wrote:

(In reply to comment #17)

Two fresh losses (looks like double moves have a high chance of failure)
http://commons.wikimedia.org/wiki/File:2010-08-04_%2827%29_Purpur-Fruchtwanze,_Carpocoris_purpureipennis.jpg
http://commons.wikimedia.org/wiki/File:BSicon_kSTRq%2Br.svg

:Sorry, the BSicon file (2nd) was not a loss [a file mover managed to manually overwrite a redirect with a page that looked like a lost image image to me :-].

aaron added a comment.Via ConduitJun 5 2012, 10:35 PM

(In reply to comment #16)

See https://gerrit.wikimedia.org/r/9878

Deployed just now.

Dereckson added a comment.Via ConduitJun 6 2012, 8:35 AM

Bug status set to ASSIGNED (previously: NEW).

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.