Page MenuHomePhabricator

reflinks should fetch only 1Mb of each linked document
Open, HighPublic

Description

T111300: Convert reflinks to requests regressed performance slightly, as the previous code would only request ~1Mb. This fetch limit should be re-added to reflinks.

Event Timeline

jayvdb raised the priority of this task from to High.
jayvdb updated the task description. (Show Details)
MtDu added a subscriber: MtDu.

I'll fix this.

Change 265769 had a related patch set uploaded (by MtDu):
Make reflinks only fetch 1 Mb of each linked document

https://gerrit.wikimedia.org/r/265769

@jayvdb
Could you take a look at the patch and offer some comments?

@jayvdb
Could you take a look at the patch and offer some comments?

Done.

@jayvdb
Thanks for reviewing the panoramiopicker patch. I recently uploaded a new patch here. Could you please give some insight?
Thanks,
MtDu

sorry I missed this; could you rebase the patch and submit to travis so we can see if it doesnt break anything (reflinks has good tests, so that is sufficient to verify nothing is broken). Then make sure that one of the tests tries to fetch a 2 Mb PDF, and emit a warning to indicate that the document was trimmed.

MtDu removed MtDu as the assignee of this task.Mar 18 2020, 6:01 PM