Page MenuHomePhabricator
Create Task
Maniphest T280926

Various errors when trying to upload large files (Could not acquire lock, Service Temporarily Unavailable, 503 Backend fetch failed, 502 Next Hop Connection Failed)
Closed, ResolvedPublic
Actions

Description

I'm trying to overwrite https://en.wikipedia.org/wiki/File:Jefferson_Davis_High_School_band_2020.ogv. Anything over 100MB is no bueno due to T278389. Anything under 100MB is no bueno either. I randomly get one of the following:

Could not acquire lock for "mwstore://local-multiwrite/local-public/7/78/Jefferson_Davis_High_School_band_2020.ogv".
Return to Main Page.

Or:

Error
Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes.
See the error message at the bottom of this page for more information.
Request from (my ip) via cp3052 frontend, Varnish XID 690378672
Error: 503, Backend fetch failed at Thu, 22 Apr 2021 16:35:19 GMT

Or:

Service Temporarily Unavailable
Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes.

Edit: eventually one of the uploads made it through. Oddly, I found out because I got an upload warning for a duplicate file. So despite throwing an error, one of the uploads did succeed.

Edit 2: The 70MB and 50MB overwrites that previously failed I now managed to do. The 100MB overwrite just rendered me another "Service Temporarily Unavailable Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes."

Edit 3: Tried the 100MB overwrite again: "Action failed Could not acquire lock for "mwstore://local-multiwrite/local-public/7/78/Jefferson_Davis_High_School_band_2020.ogv". "

Edit 4: Tried again, "Service Temporarily Unavailable"

Edit 5: Tried again, "Action failed Could not acquire lock for "mwstore://local-multiwrite/local-public/7/78/Jefferson_Davis_High_School_band_2020.ogv". "

Edit 6: Tried to upload the 100MB file to betacommons: "Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes. Request from - via deployment-cache-text06.deployment-prep.eqiad.wmflabs, ATS/8.0.8 Error: 502, Next Hop Connection Failed at 2021-04-22 17:25:23 GMT"

Edit 7: Tried to upload the 100MB file to betacommons again, "Request from - via deployment-cache-text06.deployment-prep.eqiad.wmflabs, ATS/8.0.8
Error: 502, Next Hop Connection Failed at 2021-04-22 17:32:38 GMT"

The 100MB file is an OGV file of 99,486,646 bytes.

Related Objects
Search...

Event Timeline

AlexisJazz created this task.Apr 22 2021, 4:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 22 2021, 4:46 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 4:49 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:01 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:08 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:14 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:18 PM
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:31 PM
Maintenance_bot added a project: Commons.Apr 22 2021, 5:45 PM
AlexisJazz renamed this task from 503, Backend fetch failed when trying to overwrite a file to Various errors when trying to upload large files (Could not acquire lock, Service Temporarily Unavailable, 503 Backend fetch failed, 502 Next Hop Connection Failed).Apr 22 2021, 5:53 PM
AlexisJazz updated the task description. (Show Details)
AlexisJazz updated the task description. (Show Details)Apr 22 2021, 5:57 PM
Krinkle edited projects, added MediaWiki-Uploading, Traffic, Structured Data Engineering; removed MediaWiki-File-management.Apr 22 2021, 6:10 PM
Restricted Application added a project: Structured-Data-Backlog. · View Herald TranscriptApr 22 2021, 6:10 PM
Maintenance_bot added a project: SRE.Apr 22 2021, 6:45 PM
akosiaris triaged this task as Medium priority.Apr 23 2021, 12:05 PM
Krinkle moved this task from Untriaged to Apr 2021 on the Wikimedia-production-error board.Apr 28 2021, 6:54 PM
Umherirrender mentioned this in T282173: Possible uptick in "DBTransactionSizeError: Transaction spent [n] second(s) in writes, exceeding the limit of 3".May 7 2021, 8:42 PM
Xover subscribed.May 11 2021, 6:34 AM
CBogen edited projects, added Structured-Data-Backlog (Current Work); removed Structured-Data-Backlog.May 12 2021, 2:14 PM
CBogen moved this task from Incoming to Ready for Estimation on the Structured-Data-Backlog (Current Work) board.
CBogen edited projects, added Structured-Data-Backlog; removed Structured-Data-Backlog (Current Work).Jun 30 2021, 4:50 PM
CBogen moved this task from Triage to Multimedia on the Structured-Data-Backlog board.
Meow moved this task from Incoming to Thumbnail and file renderings on the Commons board.Jul 28 2021, 9:02 AM
Meow moved this task from Thumbnail and file renderings to Uploading on the Commons board.Jul 28 2021, 9:16 AM
Yann subscribed.Oct 3 2021, 6:58 PM
Fastily subscribed.Oct 6 2021, 6:12 AM
aborrero added a parent task: T292954: [epic] large file uploads to commons.Oct 11 2021, 10:10 AM
Legoktm subscribed.Nov 4 2021, 8:11 AM

@AlexisJazz is it OK if we close this task? I believe all the issues in this ticket are covered by either T274589: No atomic section is open (got LocalFile::lockingTransaction) or T295009: Improve download speed from archive.org on appservers or T295007: Upload by URL should use the job queue, possibly chunked with range requests.

AlexisJazz added a comment.Nov 4 2021, 12:20 PM
In T280926#7480672, @Legoktm wrote:

@AlexisJazz is it OK if we close this task? I believe all the issues in this ticket are covered by either T274589: No atomic section is open (got LocalFile::lockingTransaction) or T295009: Improve download speed from archive.org on appservers or T295007: Upload by URL should use the job queue, possibly chunked with range requests.

Sure, tickets for issues with an identified cause are easier to work on.

Legoktm closed this task as Resolved.Nov 4 2021, 4:16 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL