Page MenuHomePhabricator

Switch to chunked uploading for large DjVus
Closed, ResolvedPublic5 Estimated Story Points

Description

Reported on the Wikisource mailing list.

Commons is rejecting uploads over 100 MB.

[2017-08-02 08:00:49] LOG.CRITICAL: Client error: POST https://commons.wikimedia.org/w/api.php resulted in a 413 Request Entity Too Large response: <html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity (truncated...)  [] []

Happens with quite a few items, e.g.:

010OnceAWeek10londuoftBwDec1863ToJun64/log.txt:[2017-08-02 08:00:49] LOG.CRITICAL: Client error: POST https://commons.wikimedia.org/w/api.php resulted in a 413 Request Entity Too Large response: <html> <head><title>413 Request Entity Too Large</title></head> <body bgcolor="white"> <center><h1>413 Request Entity (truncated...) [] []

where the resultant 010OnceAWeek10londuoftBwDec1863ToJun64.djvu is 279 MB.

The Commons upload limit is 4 GB, but these must be done in multiple parts.

Event Timeline

Chunked uploading docs: https://www.mediawiki.org/wiki/API:Upload#Chunked_uploading

There are currently 20 jobs failed with "413 Request Entity Too Large" in their logs.

Samwilson renamed this task from Commons says "Request Entity Too Large" to Switch to chunked uploading for large DjVus.Sep 12 2017, 12:31 PM
Samwilson added a project: Community-Tech.
Samwilson updated the task description. (Show Details)
DannyH set the point value for this task to 5.
DannyH moved this task from New & TBD Tickets to Up Next (June 3-21) on the Community-Tech board.