[Regression] Thumbnail cache should be invalidated on re-upload
Closed, DeclinedPublic

Description

I'm not entirely sure if it's a regression. Marking as such for now.

https://commons.wikimedia.org/wiki/File:Station-Leeuwarden-WLM.jpg has had a new upload, but the 800px thumbnail cache is not being invalided/purged/cleared.

A recent reupload took place removing the watermark, but the 800px thumbnail still has it. Any other random size that I come up with that wasn't cached before does show the correct latest version.

800px: https://upload.wikimedia.org/wikipedia/commons/thumb/0/0e/Station-Leeuwarden-WLM.jpg/800px-Station-Leeuwarden-WLM.jpg

651px: https://upload.wikimedia.org/wikipedia/commons/thumb/0/0e/Station-Leeuwarden-WLM.jpg/651px-Station-Leeuwarden-WLM.jpg

action=purge or null-edit on the description page didn't fix it. Neither did clearing complete browser cache.

Even if action=purge did work, imho cache should automatically be marked invalid/cleared/whatever when a file has been (re)uploaded, just like is done for wikitext cache when an article is edited.


Version: unspecified
Severity: major
URL: https://commons.wikimedia.org/wiki/Commons:Village_pump/Archive/2011/11#Pissed_off_.28updating_the_thumbnails_after_a_new_file_version_upload.29
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=41130
https://bugzilla.wikimedia.org/show_bug.cgi?id=48927

bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz31680.
Krinkle created this task.Via LegacyOct 13 2011, 10:10 PM
MarkAHershberger added a comment.Via ConduitOct 15 2011, 8:38 PM

I'm not seeing the watermark now. Can you give us an example?

MarkAHershberger added a comment.Via ConduitOct 15 2011, 10:03 PM

tagging bugs for Marcus to look at

bzimport added a comment.Via ConduitNov 15 2011, 2:39 PM

ecmporter wrote:

See discussion on Wikimedia Commons on this page (Pissed off):
http://commons.wikimedia.org/wiki/Commons:Village_pump

bzimport added a comment.Via ConduitNov 15 2011, 4:13 PM

saibotrash wrote:

Merged to here from (my) Bug 32430

Current thumbs fail to update on new file version. The most current thumb shows
an older version.

Bug 28613 apparently reoccurs

Example:
https://commons.wikimedia.org/wiki/File:Introduction_to_the_RepRap_Self_Replicating_3D_Printer.ogv
You should not see the Google Earth screenshot - it is only in the oldest
version. Both newer versions are blanked out. This behaviour did also occur
before I had hidden the oldest file version's content.

More examples: [[commons:Commons:Village_pump#Pissed_off]]

bzimport added a comment.Via ConduitNov 15 2011, 4:14 PM

saibotrash wrote:

*** Bug 32430 has been marked as a duplicate of this bug. ***

Bawolff added a comment.Via ConduitNov 16 2011, 12:34 AM

bug 32388 is borderline a dupe of this too. There have been intermittent examples of this bug for several months now...

bzimport added a comment.Via ConduitNov 16 2011, 1:26 PM

saibotrash wrote:

This bug messes up file histories if users are not aware of this bug. E.g. when using the Rotatebot to reupload a rotated version of a image (needed due to the sloppy implementation of the EXIF rotation at MediaWiki). Please do not include too much bugs in MediaWiki - otherwise it gets usuable (even with workarounds like Rotatebot). ;-)

Bawolff added a comment.Via ConduitNov 16 2011, 1:44 PM

(In reply to comment #7)

This bug messes up file histories if users are not aware of this bug. E.g. when
using the Rotatebot to reupload a rotated version of a image (needed due to the
sloppy implementation of the EXIF rotation at MediaWiki). Please do not include
too much bugs in MediaWiki - otherwise it gets usuable (even with workarounds
like Rotatebot). ;-)

How prevalent is old thumbnails not updating in your opinion (like very rough estimate - is it 1 out of 100 000 images, or 1 out of every 10 type deal).

Please do not include too much bugs in MediaWiki

lol, we try to avoid that.

Peachey88 added a comment.Via ConduitNov 16 2011, 1:49 PM

(In reply to comment #7)

This bug messes up file histories if users are not aware of this bug. E.g. when
using the Rotatebot to reupload a rotated version of a image (needed due to the
sloppy implementation of the EXIF rotation at MediaWiki).

(In regards to the comment about the "sloppy" implementation of rotations)
If there are issues, Please report them in bugzilla.

But the issue I believe you are talking about presenting is images being rotated in third party image editing programs (for example: paint.net, gimp, photoshop, etc) and the program subsequently not updating the images metadata ([[Exif]] data) to reflect the rotational change in the file, which is what MediaWiki reads to detect how the image should be displayed.

Although this is a issue, There isn't really anything much more on the MediaWiki end that can be done to combat this I believe.

bzimport added a comment.Via ConduitNov 16 2011, 2:25 PM

saibotrash wrote:

(In reply to comment #8)

How prevalent is old thumbnails not updating in your opinion (like very rough
estimate - is it 1 out of 100 000 images, or 1 out of every 10 type deal).

Well, you can click through https://commons.wikimedia.org/wiki/User:Rotatebot/Log
30 checked - nonupdating 4 (but just the 800px thumbs):

>Please do not include too much bugs in MediaWiki
lol, we try to avoid that.

Just wanted to warn just in case... ;-)

(In reply to comment #9)

(In regards to the comment about the "sloppy" implementation of rotations)
If there are issues, Please report them in bugzilla.

there: Bug 31504

But the issue I believe you are talking about presenting is images being
rotated in third party image editing programs (for example: paint.net, gimp,
photoshop, etc) and the program subsequently not updating the images metadata
([[Exif]] data) to reflect the rotational change in the file, which is what
MediaWiki reads to detect how the image should be displayed.

Yes, that is a problem too. Anyway: Off topic here.

Bawolff added a comment.Via ConduitNov 16 2011, 2:54 PM

(In reply to comment #10)

(In reply to comment #8)
> How prevalent is old thumbnails not updating in your opinion (like very rough
> estimate - is it 1 out of 100 000 images, or 1 out of every 10 type deal).
Well, you can click through
https://commons.wikimedia.org/wiki/User:Rotatebot/Log
30 checked - nonupdating 4 (but just the 800px thumbs):

I just had a conversation with Saibo on irc. Apparently these thumbs are only wrong if you're browsing from europe. Us north american folks can simulate by putting
91.198.174.234 upload.wikimedia.org
in our etc/hosts.

So my first guess is that HTCP purge packets aren't getting to european squids, but it gets weirder. If you mangle the url of the thumb so that it should be a cache miss (aka put ?foo at the end of the thumb url) I still get the wrong thumb. My understanding is that an image squid cache miss should come from some server full of image thumbs, so this should be the same regardless from where you're accessing. (But I could be mis-interpreting something, I'm not super familar with how wmf works its servers in different places of the world set up).

Anyways, here is the HTTP headers of the request that returned the wrong thumb.

Server nginx/0.7.65
Date Wed, 16 Nov 2011 14:43:00 GMT
Content-Type image/jpeg
Connection keep-alive
Content-Length 105454
Accept-Ranges bytes
X-Cache MISS from amssq51.esams.wikimedia.org, MISS from amssq55.esams.wikimedia.org
X-Cache-Lookup MISS from amssq51.esams.wikimedia.org:3128, MISS from amssq55.esams.wikimedia.org:80
Via 1.1 amssq51.esams.wikimedia.org:3128 (squid/2.7.STABLE9), 1.0 amssq55.esams.wikimedia.org:80 (squid/2.7.STABLE9)

RobLa-WMF added a comment.Via ConduitNov 16 2011, 9:33 PM

Aaron Schulz and I did a fair amount of work trying to repro and investigate this last night. We didn't think to do the /etc/hosts trick, but one of our theories was a Europe-only problem, so that makes sense. We *did* repro a thumbnailing problem, though it was only temporary and easily fixed with ?action=purge on the image description page. I'm not sure we actually reproduced the problem that's causing the most grief, but we did at least prove it's possible (if not as easy) to see the problem on one of the North American squid servers.

One theory we had: maybe it's a race condition, where the thumb is requested after the image is updated, but before the thumb is generated, since the "Age:" http header for the image was clearly younger than the upstream thumbnail should have been.

RobLa-WMF added a comment.Via ConduitNov 16 2011, 10:25 PM

Mark Bergsma is reporting that the HTCP purge daemon hasn't been running for a while, and will only restart manually, which is the likely cause. He's manually restarted this (so the symptoms should go away...please test), but let's not close this until he's confirmed that the daemon can be restarted automatically.

Bawolff added a comment.Via ConduitNov 16 2011, 11:42 PM

I just tested on the djvu file listed at bug 32388. Earlier that file had a broken thumb (When viewed from europe) that action=purging on the image desc page wasn't fixing. I just viewed it again, and thumb was still broken, but i purged, and this time the thumb got purged.

Additionally Saibo said on irc he hasn't seen any new instances recently

Therefore I'm calling this fixed.

p.s.
One thing I am rather confused about is how the wrong thumb was being shown on squid cache *misses* even if it was just htcp that was broken. (Perhaps there's just more caching layers that work in complicated ways I'm not familiar with)

Bawolff added a comment.Via ConduitNov 16 2011, 11:46 PM
  • Bug 32388 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitNov 16 2011, 11:55 PM

saibotrash wrote:

[[:File:White Storks Migrating Northwards Over Bental Mountain DSC00695.JPG]] and [[:File:Introduction_to_the_RepRap_Self_Replicating_3D_Printer.ogv]] could be fixed now by a purge (which didn't help before - at least at the video and I am sure someone tried to purge the Storks, too).

MarkAHershberger added a comment.Via ConduitNov 17 2011, 2:47 PM

(In reply to comment #16)

[[:File:White Storks Migrating Northwards Over Bental Mountain DSC00695.JPG]]
and [[:File:Introduction_to_the_RepRap_Self_Replicating_3D_Printer.ogv]] could
be fixed now by a purge (which didn't help before - at least at the video and I
am sure someone tried to purge the Storks, too).

Could you clarify why you re-opened this?

bzimport added a comment.Via ConduitNov 17 2011, 4:39 PM

saibotrash wrote:

(In reply to comment #17)

Could you clarify why you re-opened this?

I did not know that it was closed. Blame bugzilla. Maybe I didn't reload the page before typing my comment and it was closed in the meantime?

bzimport added a comment.Via ConduitNov 21 2011, 6:32 PM

saibotrash wrote:

It seems to reoccur:
75px-NYCS-bull-trans-3.svg.png and 76px-NYCS-bull-trans-3.svg.png differ in color since a new version with slightly different color was uploaded.
Purge https://commons.wikimedia.org/w/index.php?title=File:NYCS-bull-trans-3.svg&action=purge doesn't help.

wget https://upload.wikimedia.org/wikipedia/commons/thumb/
2/25/NYCS-bull-trans-3.svg/75px-NYCS-bull-trans-3.svg.png
--2011-11-21 19:24:43-- https://upload.wikimedia.org/wikipedia/commons/thumb/2/
25/NYCS-bull-trans-3.svg/75px-NYCS-bull-trans-3.svg.png
Auflösen des Hostnamen »upload.wikimedia.org«.... 91.198.174.234
Verbindungsaufbau zu upload.wikimedia.org|91.198.174.234|:443... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: 2172 (2,1K) [image/png]
In »75px-NYCS-bull-trans-3.svg.png« speichern.

100%[======================================>] 2.172 --.-K/s in 0s

2011-11-21 19:24:44 (148 MB/s) - »75px-NYCS-bull-trans-3.svg.png« gespeichert [2 172/2172]

wget https://upload.wikimedia.org/wikipedia/commons/thumb/ 2/25/NYCS-bull-trans-3.svg/76px-NYCS-bull-trans-3.svg.png
--2011-11-21 19:24:48-- https://upload.wikimedia.org/wikipedia/commons/thumb/2/ 25/NYCS-bull-trans-3.svg/76px-NYCS-bull-trans-3.svg.png
Auflösen des Hostnamen »upload.wikimedia.org«.... 91.198.174.234
Verbindungsaufbau zu upload.wikimedia.org|91.198.174.234|:443... verbunden.
HTTP Anforderung gesendet, warte auf Antwort... 200 OK
Länge: nicht spezifiziert [image/png]
In »76px-NYCS-bull-trans-3.svg.png« speichern.

[ <=>                                   ] 2.297       --.-K/s   in 0s

2011-11-21 19:24:58 (356 MB/s) - »76px-NYCS-bull-trans-3.svg.png« gespeichert [2 297]

md5sum *NYC*
46ec1ad2dac00fc86aa5f670ef363ba2 75px-NYCS-bull-trans-3.svg.png
37d2adff21668b9089f303cd39e91e9f 76px-NYCS-bull-trans-3.svg.png

Bawolff added a comment.Via ConduitNov 22 2011, 2:45 PM

md5sum *NYC*
46ec1ad2dac00fc86aa5f670ef363ba2 75px-NYCS-bull-trans-3.svg.png
37d2adff21668b9089f303cd39e91e9f 76px-NYCS-bull-trans-3.svg.png

Umm, comparing md5sum's of different size thumbnails isn't going to tell you very much. Even if they were both correct thumbnails, the md5sum would be different. (However, I can confirm by visual inspection that the wrong thumb was returned for the 75px size).

This seems to have a different root cause then the other issues reported here, as it doesn't differ between europe/non-europe.

bzimport added a comment.Via ConduitNov 22 2011, 6:14 PM

ecmporter wrote:

I am certainly not a computer-guru but I use the Wikipedia-pages and the Wikimedia Commons-page quite often.

And I have experienced the problem described here. I have also experienced something else.

After I have uploaded a picture from which I have removed a watermark, I want to edit the page to remove the template that is on the page.

This problem occurs when I push the edit-button too early (milliseconds of course) to make a quick edit of the page.

Solution: upload the file and then wait till the thumbnail is fully updated. Then edit the page.

Maybe (and I repeat maybe) this is the solution.

bzimport added a comment.Via ConduitDec 12 2011, 8:46 PM

saibotrash wrote:

(In reply to comment #20)

>md5sum *NYC*
>46ec1ad2dac00fc86aa5f670ef363ba2 75px-NYCS-bull-trans-3.svg.png
>37d2adff21668b9089f303cd39e91e9f 76px-NYCS-bull-trans-3.svg.png

Umm, comparing md5sum's of different size thumbnails isn't going to tell you
very much.

I wasn't comparing them. It was just to enable others to see which file I get delivered. ;-)

bzimport added a comment.Via ConduitDec 19 2011, 4:38 AM

mr.heat wrote:

PS: There must be a much bigger problem. All thumbnails are loading very, very slow. Some are not loading at all (timeout). For example, when going to http://commons.wikimedia.org/wiki/Special:NewFiles it takes about a minute to load all thumbnails. A few thumbnails load and are displayed, then all transfers are blocked, then a few more thumbnails load, transfer is stopped again and so on. This does not happen on other web pages, so it's not my network.

bzimport added a comment.Via ConduitDec 19 2011, 6:11 PM

saibotrash wrote:

(In reply to comment #25)

PS: There must be a much bigger problem.

Yes, but it is another problem → Already at [[:bugzilla:32432]].

MarkAHershberger added a comment.Via ConduitApr 19 2012, 4:38 PM

also see bug 36048 which may be a dupe.

bzimport added a comment.Via ConduitMay 16 2012, 9:38 PM

mr.heat wrote:

I re-uploaded the file [[commons:File:Weisswasser0078.png]] but all previously generated thumbnails show the old image. From what I know this problem exists for at least a year. Isn't it possible to fix it?

Bad thumbnails:

http://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/Weisswasser0078.png/640px-Weisswasser0078.png
http://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/Weisswasser0078.png/120px-Weisswasser0078.png

All other sizes show the new image:

http://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/Weisswasser0078.png/220px-Weisswasser0078.png
http://upload.wikimedia.org/wikipedia/commons/thumb/9/9d/Weisswasser0078.png/320px-Weisswasser0078.png

bzimport added a comment.Via ConduitMay 16 2012, 9:53 PM

mr.heat wrote:

PS: I had an idea. Maybe the "invalidate all thumbnails" process is not triggered because of the upload method used or because of some broken JavaScript in one of the upload methods? I'm using the basic upload form.

This is an other bug but I have to explain it a bit. When I click "re-upload this image" an way to complicated and therefor useless upload form shows up, made of a dozen input elements. This doesn't make any sense because most of the information entered there will be lost. The comment will be cropped. Sometimes the basic upload form shows up when I try to re-upload an image but most of the time I have to add "&uploadformstyle=basic" to the URL to force it.

I'm using Opera 11.64.

aaron added a comment.Via ConduitMay 16 2012, 9:53 PM

None of the bad thumbsnails are on ms5, I assume they are at least on squid.

bzimport added a comment.Via ConduitMay 16 2012, 10:01 PM

bhartshorne wrote:

just for the record, the four resolutions in swift are 142, 220, 320, and 800. Neither of the two broken files exist on swift (120px or 640px). (neither in the container listing nor on disk.)

McZusatz added a comment.Via ConduitJun 13 2012, 11:17 AM

(In reply to comment #10)

(In reply to comment #8)
> How prevalent is old thumbnails not updating in your opinion (like very rough
> estimate - is it 1 out of 100 000 images, or 1 out of every 10 type deal).
Well, you can click through
https://commons.wikimedia.org/wiki/User:Rotatebot/Log
30 checked - nonupdating 4 (but just the 800px thumbs):

The 800px thumbs of the last three files are still not updated right now.

Furthermore https://commons.wikimedia.org/wiki/File:Grau.jpg is affected (95px-thumb)!
and https://commons.wikimedia.org/wiki/File:Kkerepresentant.jpg and https://commons.wikimedia.org/wiki/File:PHILIPPE_SUEUR.jpg were affected. (I circumvent by reverting twice to the old version...)

bzimport added a comment.Via ConduitJul 31 2012, 4:33 PM

mr.heat wrote:

I do not upload many files to commons but every time I do it I came across this issue. Today this file (out of four) does not refresh after I uploaded a new version (the old version contains thin white lines between the colors). I tried to add "?action=purge" to both the description page and the thumbnail.

http://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Rgb-colorwheel.svg/480px-Rgb-colorwheel.svg.png

As usual other thumbnail sizes show the new version:

http://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Rgb-colorwheel.svg/481px-Rgb-colorwheel.svg.png

Bawolff added a comment.Via ConduitJul 31 2012, 4:38 PM

(In reply to comment #33)

I do not upload many files to commons but every time I do it I came across this
issue. Today this file (out of four) does not refresh after I uploaded a new
version (the old version contains thin white lines between the colors). I tried
to add "?action=purge" to both the description page and the thumbnail.

http://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Rgb-colorwheel.svg/480px-Rgb-colorwheel.svg.png

As usual other thumbnail sizes show the new version:

http://upload.wikimedia.org/wikipedia/commons/thumb/7/74/Rgb-colorwheel.svg/481px-Rgb-colorwheel.svg.png

Look fine to me....

bzimport added a comment.Via ConduitAug 1 2012, 2:44 PM

mr.heat wrote:

(In reply to comment #34)

Look fine to me....

Yes, because it was finally refreshed and displays the latest version now. Yesterday it showed the old version for hours.

I uploaded several new versions and most (but nor all) of the thumbnails are refreshed immediately. Some showed the old version but I was able to force a refresh by adding "?action=purge" to the thumb URL and the description page. This single thumb always showed the old version no matter what I tried. I consider this a bug. As said, this happens every few uploads. It's not a rare bug.

Bawolff added a comment.Via ConduitAug 2 2012, 1:29 PM

This single thumb always showed the old version no matter what I tried. I
consider this a bug.

So just to clarify, the bug is not that it never gets purged, the bug is that in some cases it takes several hours to get purged?

bzimport added a comment.Via ConduitAug 4 2012, 8:17 PM

mr.heat wrote:

(In reply to comment #36)

So just to clarify, the bug is not that it never gets purged, the bug is that
in some cases it takes several hours to get purged?

Seems so. Does not make a big difference (I know, it may be an important difference for the developers). This bug is wasting our time for many, many months now. It makes people re-upload files again and again. Nobody knows why a thumb is not refreshed and why it is "magically" refreshed the next day.

Also this is no new information. I don't see anybody complaining about a thumb that is still wrong after weeks. It seems in all cases it is refreshed after a while. No idea how long it took in average. I had better things to do.

bzimport added a comment.Via ConduitOct 10 2012, 12:29 PM

mr.heat wrote:

Almost every time I re-upload an image I get old garbage for some thumbnail sizes. At least this is how it feels. I think I posted dozens of examples for over a year now. See bug #28613. Purge never does anything. Some hours or days later the bad thumbnails are "magically" fixed.

Old garbage:

http://upload.wikimedia.org/wikipedia/commons/thumb/7/73/Mediawiki-versionsvergleich.png/640px-Mediawiki-versionsvergleich.png

Correct:

http://upload.wikimedia.org/wikipedia/commons/thumb/7/73/Mediawiki-versionsvergleich.png/641px-Mediawiki-versionsvergleich.png

bzimport added a comment.Via ConduitOct 16 2012, 5:50 PM

mr.heat wrote:

The story continues. This returns a 404 since yesterday. What the ...?

http://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Broom_icon.svg/22px-Broom_icon.svg.png

aaron added a comment.Via ConduitOct 16 2012, 5:54 PM

The "magic" fixing might just be the files getting rotated out of squid/varnish via LRU or expiry (cache time depends on last-modified date). I doubt the actual purge actions were just delayed for that long.

Either this is systemic (certain file purge actions always give bogus purge urls to cache) or there is some intermittent network problem.

bzimport added a comment.Via ConduitOct 16 2012, 7:46 PM

mr.heat wrote:

I'm spending $100 to the foundation if somebody please, please fix the broken squid or whatever causes this issue. It's more than a year now!

Aklapper added a comment.Via ConduitOct 16 2012, 8:47 PM

(In reply to comment #39)

The story continues. This returns a 404 since yesterday. What the ...?
http://upload.wikimedia.org/wikipedia/commons/thumb/2/2c/Broom_icon.svg/22px-Broom_icon.svg.png

I get the suspicion that bug 40980 is the same issue.

Platonides added a comment.Via ConduitOct 16 2012, 10:07 PM

(In reply to comment #40)

Either this is systemic (certain file purge actions always give bogus purge
urls to cache) or there is some intermittent network problem.

If it was an intermittent network problem then after purging several times one of them would succeed. Maybe an squid ignores all purges (under certain load conditions?) and when a thumbnail gets assigned to it, it never gets out? (until expiry)

Or maybe if the network is overloaded all the purges gets lost in some hour frame...

bzimport added a comment.Via ConduitOct 17 2012, 8:10 PM

mr.heat wrote:

I'm not sure but I think we reached a new record. The file

http://upload.wikimedia.org/wikipedia/commons/thumb/7/73/Mediawiki-versionsvergleich.png/640px-Mediawiki-versionsvergleich.png
http://commons.wikimedia.org/wiki/File:Mediawiki-versionsvergleich.png

is showing old garbage for a total of 8 days now! Time to call Guinness? Did somebody turned the "purge" feature off finally? Is the foundation out of money and stopped the thumbnail servers? Are the "Wiki Love Monuments" mass uploads to blame?

Aklapper added a comment.Via ConduitOct 23 2012, 1:00 AM

(In reply to comment #40)

The "magic" fixing might just be the files getting rotated out of squid/varnish
via LRU or expiry (cache time depends on last-modified date). I doubt the
actual purge actions were just delayed for that long.

Either this is systemic (certain file purge actions always give bogus purge
urls to cache) or there is some intermittent network problem.

Aaron: Any idea how to investigate further in order to track this down?

aaron added a comment.Via ConduitOct 26 2012, 12:19 AM

(In reply to comment #43)

(In reply to comment #40)
> Either this is systemic (certain file purge actions always give bogus purge
> urls to cache) or there is some intermittent network problem.

If it was an intermittent network problem then after purging several times one
of them would succeed. Maybe an squid ignores all purges (under certain load
conditions?) and when a thumbnail gets assigned to it, it never gets out?
(until expiry)

Or maybe if the network is overloaded all the purges gets lost in some hour
frame...

An intermittent problem could become permanent. See bug 41130.

(In reply to comment #45)

Aaron: Any idea how to investigate further in order to track this down?

The cause is likely the same as 41130.

Nemo_bis added a comment.Via ConduitNov 7 2012, 8:46 AM

Created attachment 11323
180px thumb

LOL
To clarify, the 180px thumb is from the previous image, which got deleted https://commons.wikimedia.org/wiki/Commons:Deletion_requests/File:Faedo.jpg
Is cache invalidation on deletion also covered by this bug?

Attached:

TheDJ added a comment.Via ConduitNov 8 2012, 11:25 AM

@Nemo Hard to tell, since bug 41130 in theory affects each possible cache invalidation, this can also be the case for invalidations after a deletion.

However, if every deletion shows this problem all the time (instead of irregularly) then it's more likely to be a separate issue then bug 41130.

Note I just fixed this particular case, by applying a workaround for bug 41130.

McZusatz added a comment.Via ConduitNov 8 2012, 1:35 PM

(In reply to comment #49)

Note I just fixed this particular case, by applying a workaround for bug 41130.

the 180px thumb I mentioned above is now fixed.

Nemo_bis added a comment.Via ConduitJan 23 2013, 1:01 PM

https://commons.wikimedia.org/wiki/File:InterfaceMessageGroup_inherit_graph.png after reupload showed a wrong "thumb" with |frame (it's actually the original).

Compare:
https://upload.wikimedia.org/wikipedia/commons/c/c9/InterfaceMessageGroup_inherit_graph.png (old)
https://upload.wikimedia.org/wikipedia/commons/c/c9/InterfaceMessageGroup_inherit_graph.png?foo=bar (correct)

wget -S https://upload.wikimedia.org/wikipedia/commons/c/c9/InterfaceMessageGroup_inherit_graph.png
--2013-01-23 14:01:12-- https://upload.wikimedia.org/wikipedia/commons/c/c9/InterfaceMessageGroup_inherit_graph.png
Risoluzione di upload.wikimedia.org (upload.wikimedia.org)... 91.198.174.234, 2620:0:862:ed1a::b
Connessione a upload.wikimedia.org (upload.wikimedia.org)|91.198.174.234|:443... connesso.
Richiesta HTTP inviata, in attesa di risposta...

HTTP/1.1 200 OK
Server: nginx/1.1.19
Date: Wed, 23 Jan 2013 13:01:13 GMT
Content-Type: image/png
Content-Length: 49315
Connection: keep-alive
X-Object-Meta-Sha1base36: nodvjwj8x4qo47cmfuoaze2mf34ynbo
Last-Modified: Tue, 02 Oct 2012 11:16:26 GMT
ETag: efa32df0a11e6b3866ef4648faae69fd
X-Timestamp: 1349176586.61961
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
X-Cache: HIT from sq44.wikimedia.org
X-Cache-Lookup: HIT from sq44.wikimedia.org:3128
Age: 533511
X-Cache: HIT from knsq20.knams.wikimedia.org
X-Cache-Lookup: HIT from knsq20.knams.wikimedia.org:3128
X-Cache: MISS from knsq20.knams.wikimedia.org
X-Cache-Lookup: MISS from knsq20.knams.wikimedia.org:80
Via: 1.1 sq44.wikimedia.org:3128 (squid/2.7.STABLE9), 1.0 knsq20.knams.wikimedia.org:3128 (squid/2.7.STABLE9), 1.0 knsq20.knams.wikimedia.org:80 (squid/2.7.STABLE9)

Lunghezza: 49315 (48K) [image/png]

bzimport added a comment.Via ConduitJan 25 2013, 8:47 PM

mr.heat wrote:

This bug is like a virus. Also local file uploads in the German Wikipedia are infected.

File description page:
http://de.wikipedia.org/wiki/Datei:Radiobuttons.gif

Broken garbage:
http://upload.wikimedia.org/wikipedia/de/d/dc/Radiobuttons.gif
http://upload.wikimedia.org/wikipedia/de/thumb/d/dc/Radiobuttons.gif/120px-Radiobuttons.gif

How it should look:
http://upload.wikimedia.org/wikipedia/de/d/dc/Radiobuttons.gif?dummy

Tracert:
Routenverfolgung zu upload-lb.esams.wikimedia.org [91.198.174.234] über maximal 30 Abschnitte:
[I removed the first few steps]

8    53 ms    52 ms    51 ms  ge0-1-0-cr0.ixf.de.as6908.net [80.81.192.244]
9    58 ms    57 ms    56 ms  te3-4-3502-cr0.nik.nl.as6908.net [78.41.154.17]

10 56 ms 54 ms 56 ms ge-2-2.br1-knams.wikimedia.org [78.41.155.38]
11 56 ms 56 ms 56 ms ve7.te-8-1.csw1-esams.wikimedia.org [91.198.174.250]
12 56 ms 58 ms 64 ms upload-lb.esams.wikimedia.org [91.198.174.234]

McZusatz added a comment.Via ConduitJan 28 2013, 6:24 PM

(In reply to comment #51)

(In reply to comment #52)

Both occureces seem to be fixed. Thank goes to Leslie I assume?

McZusatz added a comment.Via ConduitJan 30 2013, 1:16 PM

(In reply to comment #54)

And I'm in Europe.

Me too.
Still working for me since 28.1.

Aklapper added a comment.Via ConduitJan 30 2013, 1:36 PM

(In reply to comment #55)

Still working for me since 28.1.

Today it also works for me, finally.

Bawolff added a comment.Via ConduitJul 8 2013, 3:35 PM

Resolving worksforme. Seems to work now (presumably due to various varnish htcp fixes).

If this acts up again, probably best to open a new bug instead of reopening this one.

Gilles added a project: Multimedia.Via WebDec 4 2014, 10:15 AM
Gilles raised the priority of this task from "High" to "Unbreak Now!".Via WebDec 4 2014, 10:21 AM
Gilles moved this task to Closed on the Multimedia workboard.
Gilles lowered the priority of this task from "Unbreak Now!" to "High".Via ConduitDec 4 2014, 11:20 AM

Add Comment