Page MenuHomePhabricator

PreloadingGenerator does not follow the groupsize parameter when preloading pages
Closed, ResolvedPublicBUG REPORT

Description

List of steps to reproduce (step by step, including full links if applicable):

  • use with groupsize different from 50 e.g.:

pagegenerators.PreloadingGenerator(generator, groupsize=60)

What happens?:
Pages are preloaded in two steps:

Retrieving 50 pages from wikipedia:fr.
Retrieving 10 pages from wikipedia:fr.
Retrieving 50 pages from wikipedia:fr.
Retrieving 10 pages from wikipedia:fr.

What should have happened instead?:
Bulk preload of the given groupsize:
Retrieving 60 pages from wikipedia:fr.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc:

C:\pwb\GIT\core>pwb version
Pywikibot: [ssh] pywikibot-core (2cf5b09, g15424, 2021/09/25, 12:31:29, master)
Release version: 7.0.0.dev0
setuptools version: 57.0.0
mwparserfromhell version: 0.6.2
wikitextparser version: n/a
requests version: 2.20.1
  cacerts: C:\Python39\lib\site-packages\certifi\cacert.pem
    certificate test: ok
Python: 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)]
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB: C:\pwb\GIT\core
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: C:\pwb\GIT\core

Event Timeline

Xqt triaged this task as Medium priority.Sep 26 2021, 8:52 AM

Change 724028 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Adjust groupsize with pagegenerators.PreloadingGenerator

https://gerrit.wikimedia.org/r/724028

Xqt claimed this task.

Change 724028 merged by jenkins-bot:

[pywikibot/core@master] [IMPR] Adjust groupsize with pagegenerators.PreloadingGenerator

https://gerrit.wikimedia.org/r/724028