Page MenuHomePhabricator

Thumbnail of deleted image shown in "File history" after new image with same filename got uploaded
Open, MediumPublic

Description

File:Benjamin Ola Akande.jpg was deleted in 2019. Today, a different user uploaded a file with the same name, which results in the already deleted image being shown in the File history section.

Event Timeline

Aklapper renamed this task from Deleted images still visible in file history to Thumbnail of deleted image shown in "File history" after new image with same filename got uploaded.May 4 2021, 8:38 AM
Aklapper edited projects, added Thumbor, SRE-swift-storage; removed Commons.

Attaching a screenshot:

Screenshot from 2021-05-04 10-40-58.png (857×660 px, 328 KB)

Could be a simple caching issue...

https://upload.wikimedia.org/wikipedia/commons/thumb/8/81/Benjamin_Ola_Akande.jpg/120px-Benjamin_Ola_Akande.jpg

HTTP/2 200 OK 
date: Wed, 05 May 2021 15:00:58 GMT
content-type: image/jpeg
content-length: 7472
x-object-meta-sha1base36: pdif0f0ipygkgl5z3li416kx4x5v9bj
content-disposition: inline;filename*=UTF-8''Benjamin_Ola_Akande.jpg
last-modified: Sat, 08 Apr 2017 20:01:11 GMT
etag: 381e0ba55afc473e10f27a58e5f2004e
x-timestamp: 1491681670.44952
server: ATS/8.0.8
age: 22
x-cache: cp1088 miss, cp1076 hit/3
x-cache-status: hit-front
server-timing: cache;desc="hit-front", host;desc="cp1076"
strict-transport-security: max-age=106384710; includeSubDomains; preload
report-to: { "group": "wm_nel", "max_age": 86400, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 86400, "failure_fraction": 0.05, "success_fraction": 0.0}
permissions-policy: interest-cohort=()
x-client-ip: ***
access-control-allow-origin: *
access-control-expose-headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
timing-allow-origin: *
accept-ranges: bytes
X-Firefox-Spdy: h2

x-timestamp evaluates to 2017-04-08 20:01:10 UTC.
This sort of issue has happened before, I believe it's related to T206190. These sorts of tasks usually get closed as "fixed itself eventually", but it would be nice to actually get this cache invalidation problem fixed.

For the record, purging the file description page, null-editing the file description page, and adding text to the path had no effect.

File:Benjamin Ola Akande.jpg was deleted after this task was filed, but the offending thumbnail is still accessible.

HTTP/2 200 OK
date: Wed, 25 Aug 2021 18:58:11 GMT
content-type: image/jpeg
content-length: 7472
x-object-meta-sha1base36: pdif0f0ipygkgl5z3li416kx4x5v9bj
content-disposition: inline;filename*=UTF-8''Benjamin_Ola_Akande.jpg
last-modified: Sat, 08 Apr 2017 20:01:11 GMT
etag: 381e0ba55afc473e10f27a58e5f2004e
x-timestamp: 1491681670.44952
server: ATS/8.0.8
age: 241
x-cache: cp1088 miss, cp1082 hit/1
x-cache-status: hit-front
server-timing: cache;desc="hit-front", host;desc="cp1082"
strict-transport-security: max-age=106384710; includeSubDomains; preload
report-to: { "group": "wm_nel", "max_age": 86400, "endpoints": [{ "url": "https://intake-logging.wikimedia.org/v1/events?stream=w3c.reportingapi.network_error&schema_uri=/w3c/reportingapi/network_error/1.0.0" }] }
nel: { "report_to": "wm_nel", "max_age": 86400, "failure_fraction": 0.05, "success_fraction": 0.0}
permissions-policy: interest-cohort=()
x-client-ip: [...]
access-control-allow-origin: *
access-control-expose-headers: Age, Date, Content-Length, Content-Range, X-Content-Duration, X-Cache
timing-allow-origin: *
accept-ranges: bytes
X-Firefox-Spdy: h2

@AntiCompositeNumber I tried purging it with no success- my guess would be that it is not on cache, but on persistence layer for some reason. Let me make sure I can find it on swift and I delete the thumb.

The thumb is not on swift, so it is a cache issue.

I ran the following:

$ mwscript purgeChangedFiles.php --wiki=commonswiki --starttime=20210511122541 --endtime=20210511122541 -v
Purging files that were: created, deleted, modified
Checking for created files...
Scanning for upload/upload
Scanning for import/upload,interwiki
...created files purged.

Checking for deleted files...
Scanning for delete/delete,revision
Purged file Benjamin_Ola_Akande.jpg; deleted @20210511122541.
Scanning for suppress/delete,revision
...deleted files purged.

Checking for modified files...
Scanning for upload/overwrite,revert
Scanning for move/move,move_redir
...modified files purged.

$ mwscript purgeChangedFiles.php --wiki=commonswiki --starttime=20190520124408 --endtime=20190520124408 -v
Purging files that were: created, deleted, modified
Checking for created files...
Scanning for upload/upload
Scanning for import/upload,interwiki
...created files purged.

Checking for deleted files...
Scanning for delete/delete,revision
Purged file Benjamin_Ola_Akande.jpg; deleted @20190520124408.
Scanning for suppress/delete,revision
...deleted files purged.

Checking for modified files...
Scanning for upload/overwrite,revert
Scanning for move/move,move_redir
...modified files purged.

Which in theory should have purged all old versions of the file, but didn't seem to work.

I'm silly, the issue was on swift, not on cache:

curl -I http://ms-fe.svc.eqiad.wmnet/wikipedia/commons/thumb/8/81/Benjamin_Ola_Akande.jpg/120px-Benjamin_Ola_Akande.jpg
HTTP/1.1 200 OK

But I cannot find the thumbnail on swift:

ms-fe1005:~$ swift list wikipedia-commons-local-thumb.81 -p '8/81/Benjamin_Ola_Akande.jpg/120px-Benjamin_Ola_Akande.jpg'
ms-fe1005:~$ swift list wikipedia-commons-local-thumb.81 -p '8/81/' | grep Akande
(no results)

, so to me this looks like a weird thumbor issue.

fwiw this issue was recently reported on https://commons.wikimedia.org/wiki/File:Anshu_Dikshant.jpg but I didn't get a chance to look at it before it was deleted.

The thumbnail not existing in Swift is certainly weird. https://wikitech.wikimedia.org/wiki/Swift/How_To#List_containers_and_contents does say that -p/--prefix must be before the container name in the command though.

fgiunchedi triaged this task as Medium priority.Aug 30 2021, 8:07 AM

The thumbnail not existing in Swift is certainly weird. https://wikitech.wikimedia.org/wiki/Swift/How_To#List_containers_and_contents does say that -p/--prefix must be before the container name in the command though.

That's not it, see we get the same output for a random file on the same container (I will update the doc to reflect this issue, it could be behavior of a previous client version):

ms-fe1005:~$ swift list -p '8/81/!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf/page1-106px-!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf.jpg' wikipedia-commons-local-thumb.81
8/81/!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf/page1-106px-!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf.jpg

ms-fe1005:~$ swift list wikipedia-commons-local-thumb.81 -p '8/81/!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf/page1-106px-!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf.jpg'
8/81/!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf/page1-106px-!Al_asalto!_-_comedia_en_dos_actos_(IA_alasaltocomediae00mont).pdf.jpg

And I checked that's the case for the relevant file, too:

ms-fe1005:~$ swift list -p '8/81/Benjamin_Ola_Akande.jpg/120px-Benjamin_Ola_Akande.jpg' wikipedia-commons-local-thumb.81
(no output)