Change 561697 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [FEAT] Add ability to preload categories

https://gerrit.wikimedia.org/r/561697

My script seems no faster with the patch, even with categories = True in the call to APISite.preloadpages. Commented on the part of api.py where separate requests are being sent for each page's categories.

It does not preload the category content itself but the category titles related to the pages. A second request to this information is about 30 times faster. You could create a generator of all categories and preload them in a separate step or combined with the preloading iterator.

Okay, so an additional request must be sent to actually construct the Category objects.

I guess for my script I only need the titles, not the Category objects. It's not quite what the task title asks for, but is there a way to access the preloaded category titles when using APISite.preloadpages and not send the additional request per page to get the category information that I'm not using? This doesn't seem possible in the patch because update_page calls BasePage.categories if categories have been preloaded.

This is the general idea: a function get_category_titles in the following code that does not send an additional request beyond the requests sent by APISite.preloadpages.

def filter_pages_by_categories(pages, category_titles_to_find):
    for page in site.preloadpages(pages, categories = True):
        if any(category_title in category_titles_to_find for category_title in get_category_titles(page)):
            yield page

See T67163

Change 561697 merged by jenkins-bot:

[pywikibot/core@master] [FEAT] Add ability to preload categories

https://gerrit.wikimedia.org/r/561697