preloading using multiple properties does not work
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	jayvdb
	Nov 15 2014, 11:40 AM

Description

interwiki uses the templates=True and langlinks=True functionality provided by APISite.preloadpages

site.preloadpages(pageGroup, templates=True, langlinks=True)

However, it does not work. Continuation for langlinks & templates fails. Only the revisions continuation is performed.

I presume it has never worked, as the commit that added it did not do anything related to handling of multiple continuations

http://git.wikimedia.org/commit/pywikibot%2Fcore.git/aa8739dc5ff876b71e42cc8158df42f2860261c8

Version: core-(2.0)
Severity: major

Details

Reference: bz73461

Related Objects
Search...

Status	Subtype	Assigned	Task
Invalid		None	T72936 Important tasks to be solved (tracking)
Open	Feature	None	T67161 PreloadingPageGenerator should allow specifying parameters to preload
Resolved		Xqt	T75461 preloading using multiple properties does not work
Open		None	T67164 site.preloadpages / data.api.update_page should set preloaded parameters to 'empty' if no data is returned

Event Timeline

• bzimport raised the priority of this task from to High.Nov 22 2014, 4:00 AM

• bzimport added projects: Pywikibot-network, Performance Issue.

• bzimport set Reference to bz73461.

• bzimport added a subscriber: Unknown Object (????).

jayvdb created this task.Nov 15 2014, 11:40 AM

Change 173499 had a related patch set uploaded by John Vandenberg:
Preloading tests

https://gerrit.wikimedia.org/r/173499

As far as I can see, only interwiki uses this.
This bug is not causing errors, but langlinks & templates are not being preloaded, so interwiki is operating much slower than it would in compat.

Support for multiple continuation has been added here:
https://bugzilla.wikimedia.org/show_bug.cgi?id=55193

Change 173499 merged by jenkins-bot:
Preloading tests

https://gerrit.wikimedia.org/r/173499

Aklapper added a project: Pywikibot.Nov 27 2014, 4:10 PM

A quick update: https://gerrit.wikimedia.org/r/#/c/173630/ improved the efficiency of the query generator underneath preloading , so much that it actually removed (hid) some of the preloading bugs.

test_preload_templates is consistently passing on en.wp, test.wd, fr.wikt, however it occasionally fails on ar.wp; usually all platforms fail together. e.g.
https://travis-ci.org/wikimedia/pywikibot-core/builds/42379426
https://travis-ci.org/wikimedia/pywikibot-core/builds/42379426

I'm not able to deduce what might be causing those irregular failures; my only (and probably wrong) guess is that the test is finding pages on ar.wp that dont have any templates.

jayvdb added a subtask: T67164: site.preloadpages / data.api.update_page should set preloaded parameters to 'empty' if no data is returned.Dec 13 2014, 1:25 AM

Tests passes

preloading using multiple properties does not workClosed, ResolvedPublicActions

Description

Details

Related ObjectsSearch...

Event Timeline

preloading using multiple properties does not work
Closed, ResolvedPublic
Actions

Related Objects
Search...