Page MenuHomePhabricator

Uploads fail with verification-error due to incorrect timeout handling
Closed, ResolvedPublic

Description

GWT returns the following error:
10:13, 19 November 2015 Swiss National Library (talk | contribs) mediafile job failed. Metadata record 2. <verification-error> original URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif evaluated URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif.

tried it twice with the same xml-file, today and yesterday

Yesterday it returned the following error:
15:20, 18 November 2015 Swiss National Library (talk | contribs) mediafile job failed. Metadata record 3. This file did not pass file verification. original URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif evaluated URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif.

It returns the same error for all pictures linked in the xml.
Uesed xml ist the folowing:

Pictures are uploaded here: http://upload-glam.wikimedia.ch/Gugelmann

Event Timeline

Swiss-National-Library raised the priority of this task from to Needs Triage.
Swiss-National-Library updated the task description. (Show Details)
Restricted Application added projects: Multimedia, Commons. · View Herald TranscriptNov 19 2015, 10:33 AM
Restricted Application added subscribers: StudiesWorld, Steinsplitter, Aklapper. · View Herald Transcript
Swiss-National-Library set Security to None.
Restricted Application added a subscriber: Matanya. · View Herald TranscriptNov 19 2015, 10:41 AM
zhuyifei1999 moved this task from Uploading (Special:Upload) to Uploading on the Commons board.
zhuyifei1999 added a subscriber: zhuyifei1999.
MarkTraceur triaged this task as Normal priority.Dec 21 2015, 9:35 PM
MarkTraceur added a subscriber: MarkTraceur.

I have tried to reproduce that bug with an ever simplified version of the XML including only the row with "GS-GUGE-FREUDENBERGER-A-1". In my case it fails too with an other error:

17:46, 23 December 2015 Kelson (talk | contribs) mediafile job failed. Metadata record 1. HTTP request timed out. original URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif evaluated URL: http://upload-glam.wikimedia.ch/Gugelmann/GS-GUGE-FREUDENBERGER-A-1.tif.

I have reproduced the problem many times and I can see server side that everything goes right up to 90s after the beginning of the GET request. Then the transfer is stopped, probably by Mediawiki. It looks like that the overall request max lifetime for this download-by-url seems to be set to 90s and is obviously too short. We are as a consequence impacted by T86024.

Kelson added a subscriber: Bawolff.Dec 23 2015, 5:19 PM

We should really try to fix this in the next times because we have a whole GLAM upload/GLAM partner stuck because of that bug.

Bawolff-alt added a subscriber: Bawolff-alt.

There's a bug in WikiChecks::increaseHTTPTimeout().

It increases the timeout incorrectly. As a result the wrong timeout is applied to gwtoolset. Thus when we set the other timeouts to 90 seconds, it started overriding gwtoolset's limit, which its not supposed to.

Instead gwtoolset should pass its custom limit to fetchFile() method of UploadFromUrl (Which it calls from UploadHandler::uploadMediaFileViaUploadFromUrl() ). GWToolset should not try to override global variables.

@Bawolff
Thank you very much for your feedback and your analysis on that bug. Do you think you or someone else will get a chance to fix it within a few weeks? Would that be complicated?

Nemo_bis renamed this task from GWT <verification-error> to Uploads fail with verification-error due to incorrect timeout handling.Jan 8 2016, 7:15 AM
Nemo_bis moved this task from Proposed to Imported into GCI site on the Google-Code-In-2015 board.
Mhutti1 claimed this task.Jan 8 2016, 10:37 AM

@mhutti

Really surprising to see you there too! But would be really really helpful if you achieved to deal with that we have at least two GLAM institutions currently stuck/braked in their upload project due to that bug. Feel free to contact me if you need a tester.

@Kelson yeah I like mixing up the tasks I do :). Once there are some non-beginner kiwix tasks I will do some more of those :)

Change 263142 had a related patch set uploaded (by Mhutti1):
Fixed verification-error due to incorrect timeout handling

https://gerrit.wikimedia.org/r/263142

Change 263142 merged by jenkins-bot:
Fixed verification-error due to incorrect timeout handling

https://gerrit.wikimedia.org/r/263142

Mhutti1 closed this task as Resolved.Jan 9 2016, 4:20 PM