Page MenuHomePhabricator

-step parameter doesn't taken into account when using -file generator
Closed, DeclinedPublic

Description

I've noticed that -step isn't taken into account when using -file as a generator.

For example, I've generated a file with a list of pages to work on. And I want Pywikibot to run them in that specific order, because that will improve the review process later.

Pywikibot by default tries to retrieve information of 50 pages by default, but the api returns them in alphabetic (pagename) order rather than the order in which we've requested it. Hence, I found that -step will solve the issue by querying pages one by one, following the same order.

The problem is that pywikibot ignores -step for -file, so it keeps returning 50 pages at a time.

Event Timeline

Ciencia_Al_Poder raised the priority of this task from to Needs Triage.
Ciencia_Al_Poder updated the task description. (Show Details)
Ciencia_Al_Poder added a project: Pywikibot.
Ciencia_Al_Poder subscribed.
Restricted Application added subscribers: Aklapper, Unknown Object (MLST). · View Herald TranscriptApr 1 2015, 8:04 PM

I've found that the fix is, in each script that use generators, change

gen = pagegenerators.PreloadingGenerator(gen)

with

gen = pagegenerators.PreloadingGenerator(gen, genFactory.step)

And that works for me.

That's because PreloadingGenerator has a step parameter that defaults to 50, and since none of the scripts specifies it, then it takes it as the current step value

I've actually written a patch which will return the pages in the requested order when using PreloadingGenerator: https://gerrit.wikimedia.org/r/#/c/199631/ You might give it a spin, though this bug here might still be relevant.

Hi @Ciencia_Al_Poder, it looks that -step parameter were deprecated (see: T109208). Also in PreloadingGenerator parameter was renamed fro step into groupsize (see: 1), which defaults to 50.

As I understand You problem was not particularly about step parameter, rather about correct order of returned pages, so I believe these should be resolved in independend change (see: 2).

Could You please check if Your problem occurs at current tip of master?

CC: @Xqt, @Mpaa

Yeah, order is now respected in -file generator. For me it's fine to close this task as declined/invalid