[Regression] Redirects on file repository no longer work on local wiki
Closed, ResolvedPublic

Description

[[commons:File:MaintenanceShell-Screenshot.png]] is a redirect to
[[commons:File:MaintenanceShell-v0.4.0-screenshot.png]].


Version: 1.22.0
Severity: major

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz52200.
Krinkle created this task.Via LegacyJul 28 2013, 7:25 PM
Bawolff added a comment.Via ConduitJul 28 2013, 7:50 PM

Hmm. The file on commons isn't supressing the "no file by this name text". Its almost as if its being treated as a normal redirect instead of a file redirect.

greg added a comment.Via ConduitJul 29 2013, 7:37 PM

Confirming request for backport_to_WMF on this as it is pretty high priority. Let's try to get this done before the end of the week so we don't run into Wikimania mania.

Bawolff added a comment.Via ConduitJul 29 2013, 7:47 PM

(In reply to comment #2)

Confirming request for backport_to_WMF on this as it is pretty high priority.
Let's try to get this done before the end of the week so we don't run into
Wikimania mania.

Note, this appears to be an isolated incident. Most file redirects still appear to work.

Additionally, it appears some sort of cache expired, and the example above is no longer broken.

greg added a comment.Via ConduitJul 29 2013, 7:49 PM

Oh.... well then.... cleared the request. :)

Bawolff added a comment.Via ConduitJul 29 2013, 8:35 PM

(In reply to comment #3)

(In reply to comment #2)
> Confirming request for backport_to_WMF on this as it is pretty high priority.
> Let's try to get this done before the end of the week so we don't run into
> Wikimania mania.

Note, this appears to be an isolated incident. Most file redirects still
appear
to work.

Meh, appears to be common on newer redirects (example: https://commons.wikimedia.org/w/index.php?title=File:Baku9.jpg&redirect=no ). Some sort of cache isn't being purged properly.

Bawolff added a comment.Via ConduitJul 29 2013, 9:03 PM

Current suspicion: File objects get cached in memcached. Perhaps when moving file, file isn't cleared from memcached. Person goes to view redirect, various code loads the file from cache. However the file object loaded from cached is the one that used to be associated with this title, but isn't anymore. Thus $file->getRedirected() returns false, because its still the old version of the file before it was a redirect. Eventually file falls out of memcached, and everything works.

(I haven't tested this theory yet. may be wrong).

Anomie added a comment.Via ConduitJul 29 2013, 9:36 PM

I took a few minutes to do some debugging on this. It appears that the problem is that a memc key is being set in LocalRepo::checkRedirect() with an expiry of 86400s. This key is supposed to be invalidated by a call to LocalRepo::invalidateImageRedirect() when necessary, but apparently that is getting missed somehow.

Bawolff added a comment.Via ConduitAug 4 2013, 2:03 AM

Perhaps a more robust thing to do, would be instead of clearing that key, explicitly populate it on page move. That way, it shouldn't fallback to reading the (lagged) slave.

Interestingly enough, problem is not reproducible on test.wikipedia.org. ( https://test.wikipedia.org/w/index.php?title=File:Ex_1%25_ample.jpg&redirect=no ). I wonder if possibly there's some anti-vandalism bot that auto-loads pages on edit at commons, which is causing the cache to be repopulated in a different request before the change replicates to the slave db. (That theory might be stretching it though).

Bawolff added a comment.Via ConduitAug 4 2013, 2:12 AM

Also does not appear to happen on enwikipedia either ( example: https://en.wikipedia.org/w/index.php?title=File:ALANIS-FOE-FINAL-COVER.JPG&redirect=no )

Bawolff added a comment.Via ConduitAug 4 2013, 3:55 AM

I made a patch at https://gerrit.wikimedia.org/r/77562 which would instead of clearing the image redirect cache on page move, simply repopulate it (and hopefully avoid issues with slave db replag). I'm unsure if that would solve the issue here (Since the issue here seems only reproducible on commons, I'm a bit confused as to what's different between commons and enwikipedia - aka I have no idea what's causing this). Nonetheless I think that patch is a step in the right direction.

I also did https://gerrit.wikimedia.org/r/77563 to make this cache be cleared on ?action=purge, since it seems like something that should be.

Bawolff added a comment.Via ConduitAug 4 2013, 3:57 AM

(In reply to comment #4)

Oh.... well then.... cleared the request. :)

My comment 3 above was in error (It happens to all recent moves on commons < 24 hours old), so I'm resetting that flag.

greg added a comment.Via ConduitAug 4 2013, 4:50 AM

(In reply to comment #11)

(In reply to comment #4)
> Oh.... well then.... cleared the request. :)

My comment 3 above was in error (It happens to all recent moves on commons <
24
hours old), so I'm resetting that flag.

OK, I'll wait until Gerrit tells me it is merged and we'll go from there.

Bawolff added a comment.Via ConduitAug 5 2013, 9:21 PM

(In reply to comment #10)

I made a patch at https://gerrit.wikimedia.org/r/77562 which would instead of
clearing the image redirect cache on page move, simply repopulate it (and
hopefully avoid issues with slave db replag). I'm unsure if that would solve
the issue here (Since the issue here seems only reproducible on commons, I'm
a
bit confused as to what's different between commons and enwikipedia - aka I
have no idea what's causing this). Nonetheless I think that patch is a step
in
the right direction.

Ok. So there is a way for the user to trigger invalidateImageRedirectCache - namely, making an edit to the redirect (real edit, not a null edit). I tested that with https://commons.wikimedia.org/w/index.php?title=File:Shaik_Mydeen-3.jpg&redirect=no, and my edit seemed to fix the issue, which seems to confirm that it is indeed a problem with that cache.

gerritbot added a comment.Via ConduitAug 7 2013, 8:31 PM

Change 77562 had a related patch set uploaded by Brian Wolff:
More rigorous clearing of image redirect cache

https://gerrit.wikimedia.org/r/77562

gerritbot added a comment.Via ConduitAug 9 2013, 9:29 PM

Change 77562 merged by jenkins-bot:
More rigorous clearing of image redirect cache

https://gerrit.wikimedia.org/r/77562

Bawolff added a comment.Via ConduitAug 19 2013, 10:41 PM

Confirmed fixed (This is now deployed to commons. Can confirm new file redirects do not suffer from this issue)

Gilles added a project: Multimedia.Via WebDec 4 2014, 9:35 AM
Gilles raised the priority of this task from "High" to "Unbreak Now!".Via WebDec 4 2014, 10:11 AM
Gilles moved this task to Closed on the Multimedia workboard.
Gilles lowered the priority of this task from "Unbreak Now!" to "High".Via ConduitDec 4 2014, 11:22 AM

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.