Page MenuHomePhabricator

Article concatenation fails on large books
Closed, InvalidPublic

Description

Reducing the number of articles in the book using the book creator from 60 to 25 seems to allow the book to render.
This raises serious questions about whether our concatenation method will work at scale.

Note, that the limit seems to be on size of articles not number.
A similar book with 26 pages can be rendered:
https://en.wikipedia.beta.wmflabs.org/wiki/User:Jdlrobson/Books/The_big_book

Gergo has unanswered comments on the following patches (which maybe helpful for this task, but we should get to them in eventually):

  1. https://gerrit.wikimedia.org/r/#/c/381277/
  2. https://gerrit.wikimedia.org/r/#/c/381133/

Related Objects

StatusAssignedTask
OpenJKatzWMF
InvalidNone
InvalidNone
Resolvedphuedx
InvalidJdlrobson
ResolvedNone
InvalidJdlrobson
ResolvedJdlrobson
StalledNone
InvalidNone
DuplicateNone
DeclinedNone
InvalidNone
InvalidNone
Resolvedbmansurov
Invalidovasileva
Resolvedovasileva
ResolvedJdlrobson
Resolvedpmiazga
Resolvedovasileva
InvalidNone
Resolvedovasileva
InvalidNone
InvalidNone
Resolved dpatrick

Event Timeline

Jdlrobson removed the point value for this task.Oct 11 2017, 8:06 PM
Tgr added a subscriber: Tgr.Oct 11 2017, 8:57 PM

Looking at logstash, there is a too-many-titles-for-query API error (I did point that out in code review but it got ignored), a bunch of other errors caused by that, and a (probably unrelated) memcached error (which helpfully just says "SERVER ERROR"). I did not think of memcached size limits which is probably what we are hitting here; the HTML part of the book might need to be split up into many cache entries or use something other than memcached.

ovasileva renamed this task from [subtask] Article concatenation fails on large books to Article concatenation fails on large books.Oct 12 2017, 5:28 PM

@Tgr fyi I saw your comments on my 2 patches, but the patches got merged before I could follow up on them. I think it's important I should follow up on them. Not sure if doing that work fits our priorities at this time, however.

@bmansurov: Could you add details about those comments and what the suggested fixes are to the description of this task?

bmansurov updated the task description. (Show Details)Oct 13 2017, 1:23 PM
Jdlrobson changed the task status from Open to Stalled.Oct 30 2017, 11:27 PM

Until we have time to talk through T178095

ovasileva closed this task as Invalid.Apr 9 2018, 2:42 PM

Closing as per T184772#4116906. Pediapress will be taking on rendering of PDF books.