Page MenuHomePhabricator

Bulk OCR Image data fetch times out on books with many pages
Closed, ResolvedPublicBUG REPORT

Description

Image data fetch times out on books with many pages.png (1,895×556 px, 27 KB)

Description

When Bulk OCR runs on a book with many untranscribed pages on larger books this request times out no images are returned and OCR produces no results.Tested on more 15pages book the request was canceled after the second Api timeout. A 9page book works correctly.

Details and specifics

  • server generates image data that is thrown away.
  • proofread pages in index has no limit or continuation parameter so the generator call cannot be paged.
  • image for page works with an explicit titles list, which returns quickly.
  • Suggested fix:**
    • request image data only for the pages that need OCR in batches, matching the existing batch size.
    • The batches should merge into a single pageImageMap and trigger ocr-images-loaded once after the last batch.

Event Timeline

Ssein updated the task description. (Show Details)

Change #1337320 had a related patch set uploaded (by Ssein; author: Ssein):

[mediawiki/extensions/Wikisource@master] Fetch Bulk OCR image data in batches instead of the whole index

https://gerrit.wikimedia.org/r/1337320

Change #1337320 merged by jenkins-bot:

[mediawiki/extensions/Wikisource@master] Fetch Bulk OCR image data in batches instead of the whole index

https://gerrit.wikimedia.org/r/1337320