Missing file after upload - unknown error occurred in storage backend "local-swift-eqiad"
Closed, ResolvedPublic

Description

User uploads file, gets

An unknown error occurred in storage backend "local-swift-eqiad".

File page created normally. image table has all info about image (size, metadata, sha1, etc). However url to image is 404 ( https://upload.wikimedia.org/wikipedia/commons/a/a3/Disqus_logo_official_-_white_on_blue_background.png ):

The resource could not be found.

File not found: /v1/AUTH_mw/wikipedia-commons-local-public.a3/a/a3/Disqus_logo_official_-_white_on_blue_background.png


Image description page is:
https://commons.wikimedia.org/wiki/File:Disqus_logo_official_-_white_on_blue_background.png

Image uploaded at 2014-04-17T20:20:34

Ideally we should figure out what happened, and try and prevent other upload from failing in a similar way


Version: unspecified
Severity: normal

bzimport added a project: Wikimedia-Media-storage.Via ConduitNov 22 2014, 3:15 AM
bzimport set Reference to bz64071.
Bawolff created this task.Via LegacyApr 17 2014, 8:36 PM
Bawolff added a comment.Via ConduitApr 18 2014, 8:11 AM

This is probably not really "high" priority as it appears to be an isolated incident. Would still be nice if we could figure out what happened though.

Aklapper added a comment.Via ConduitApr 18 2014, 8:21 AM

To me it's high as long as it's fresh and there's logs.
It'll get lower priority if nobody investigates soon'ish.

Bawolff added a comment.Via ConduitApr 18 2014, 3:13 PM

So much for being isolated. From commons:vp

*[[:File:Tibi Fall-Winter 2010 117.jpg]] (file description page missing too in this case)
*[[:File:Tibi Fall-Winter 2010 121.jpg]]
*[[:File:Tibi Fall-Winter 2010 122.jpg]]
*[[:File:Tibi Fall-Winter 2010 123.jpg]]
*[[:File:Tibi Fall-Winter 2010 124.jpg]]
*[[:File:Tibi Fall-Winter 2010 125.jpg]]
*[[:File:Tibi Fall-Winter 2010 126.jpg]]
*[[:File:Tibi Fall-Winter 2010 128.jpg]]
*[[:File:Tibi Fall-Winter 2010 129.jpg]]
*[[:File:Tibi Fall-Winter 2010 131.jpg]]
*[[:File:Tibi Fall-Winter 2010 134.jpg]]
*[[:File:Tibi Fall-Winter 2010 135.jpg]]
*[[:File:Tibi Fall-Winter 2010 136.jpg]]
*[[:File:Tibi Fall-Winter 2010 137.jpg]]
*[[:File:Tibi Fall-Winter 2010 139.jpg]]
*[[:File:Tibi Fall-Winter 2010 140.jpg]]
*[[:File:Tibi Fall-Winter 2010 141.jpg]]
*[[:File:Tibi Fall-Winter 2010 142.jpg]]
*[[:File:Tibi Fall-Winter 2010 145.jpg]]
*[[:File:Tibi Fall-Winter 2010 146.jpg]]
*[[:File:Tibi Fall-Winter 2010 147.jpg]]
*[[:File:Tibi Fall-Winter 2010 148.jpg]]
*[[:File:Tibi Fall-Winter 2010 149.jpg]]
*[[:File:Tibi Fall-Winter 2010 151.jpg]]
*[[:File:Tibi Fall-Winter 2010 152.jpg]]
*[[:File:Tibi Fall-Winter 2010 153.jpg]]
*[[:File:Tibi Fall-Winter 2010 154.jpg]]
*[[:File:Tibi Fall-Winter 2010 157.jpg]]
*[[:File:Tibi Fall-Winter 2010 161.jpg]]

gerritbot added a comment.Via ConduitApr 18 2014, 7:37 PM

Change 127326 had a related patch set uploaded by Aaron Schulz:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127326

gerritbot added a comment.Via ConduitApr 21 2014, 4:54 PM

Change 127326 merged by jenkins-bot:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127326

gerritbot added a comment.Via ConduitApr 21 2014, 5:03 PM

Change 127650 had a related patch set uploaded by Aaron Schulz:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127650

gerritbot added a comment.Via ConduitApr 21 2014, 5:03 PM

Change 127652 had a related patch set uploaded by Aaron Schulz:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127652

gerritbot added a comment.Via ConduitApr 21 2014, 5:11 PM

Change 127650 merged by jenkins-bot:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127650

gerritbot added a comment.Via ConduitApr 21 2014, 5:23 PM

Change 127652 merged by jenkins-bot:
Avoid image table updates on file upload failure

https://gerrit.wikimedia.org/r/127652

aaron added a comment.Via ConduitApr 21 2014, 5:26 PM

Description pages will no longer be created on backed errors.

The underlying errors are sporatic HTTP 401s that happen (due to very premature token expiration or validation errors for some reason...possibly internal memcached problems in Swift).

Ankry added a comment.Via ConduitAug 21 2014, 9:24 AM

I don't know whether it is useful information, but deleting and restoring the files below (performed tonight) seem to retrieve their initial version also. (Their initial versions were shown as broken till yesterday).

(In reply to Bawolff (Brian Wolff) from comment #3)

So much for being isolated. From commons:vp

*[[:File:Tibi Fall-Winter 2010 117.jpg]] (file description page missing too
in this case)
*[[:File:Tibi Fall-Winter 2010 121.jpg]]
*[[:File:Tibi Fall-Winter 2010 122.jpg]]
*[[:File:Tibi Fall-Winter 2010 123.jpg]]
*[[:File:Tibi Fall-Winter 2010 124.jpg]]
*[[:File:Tibi Fall-Winter 2010 125.jpg]]
*[[:File:Tibi Fall-Winter 2010 126.jpg]]
*[[:File:Tibi Fall-Winter 2010 128.jpg]]
*[[:File:Tibi Fall-Winter 2010 129.jpg]]
*[[:File:Tibi Fall-Winter 2010 131.jpg]]
*[[:File:Tibi Fall-Winter 2010 134.jpg]]
*[[:File:Tibi Fall-Winter 2010 135.jpg]]
*[[:File:Tibi Fall-Winter 2010 136.jpg]]
*[[:File:Tibi Fall-Winter 2010 137.jpg]]
*[[:File:Tibi Fall-Winter 2010 139.jpg]]
*[[:File:Tibi Fall-Winter 2010 140.jpg]]
*[[:File:Tibi Fall-Winter 2010 141.jpg]]
*[[:File:Tibi Fall-Winter 2010 142.jpg]]
*[[:File:Tibi Fall-Winter 2010 145.jpg]]
*[[:File:Tibi Fall-Winter 2010 146.jpg]]
*[[:File:Tibi Fall-Winter 2010 147.jpg]]
*[[:File:Tibi Fall-Winter 2010 148.jpg]]
*[[:File:Tibi Fall-Winter 2010 149.jpg]]
*[[:File:Tibi Fall-Winter 2010 151.jpg]]
*[[:File:Tibi Fall-Winter 2010 152.jpg]]
*[[:File:Tibi Fall-Winter 2010 153.jpg]]
*[[:File:Tibi Fall-Winter 2010 154.jpg]]
*[[:File:Tibi Fall-Winter 2010 157.jpg]]
*[[:File:Tibi Fall-Winter 2010 161.jpg]]

Gilles added a project: Multimedia.Via WebDec 4 2014, 9:24 AM
Gilles raised the priority of this task from "High" to "Unbreak Now!".Via WebDec 4 2014, 10:11 AM
Gilles moved this task to Closed on the Multimedia workboard.
Gilles lowered the priority of this task from "Unbreak Now!" to "High".Via ConduitDec 4 2014, 11:23 AM

Add Comment