T111300: Convert reflinks to requests regressed performance slightly, as the previous code would only request ~1Mb. This fetch limit should be re-added to reflinks.
Description
Description
Details
Details
| Project | Branch | Lines +/- | Subject | |
|---|---|---|---|---|
| pywikibot/core | master | +15 -2 | Make reflinks only fetch 1 Mb of each linked document |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | Xqt | T68102 use one library for all http requests | |||
| Resolved | MtDu | T111300 Convert reflinks to requests | |||
| Open | None | T124138 reflinks should fetch only 1Mb of each linked document |
Event Timeline
Comment Actions
Change 265769 had a related patch set uploaded (by MtDu):
Make reflinks only fetch 1 Mb of each linked document
Comment Actions
@jayvdb
Thanks for reviewing the panoramiopicker patch. I recently uploaded a new patch here. Could you please give some insight?
Thanks,
MtDu
Comment Actions
sorry I missed this; could you rebase the patch and submit to travis so we can see if it doesnt break anything (reflinks has good tests, so that is sufficient to verify nothing is broken). Then make sure that one of the tests tries to fetch a 2 Mb PDF, and emit a warning to indicate that the document was trimmed.