Page MenuHomePhabricator

Error parsing the list of pages in the category containing files
Closed, InvalidPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):

What happens?:

Script terminated by exception:

ERROR: APISite instance has no attribute 'file_extensions' (AttributeError)
Traceback (most recent call last):
  File "C:\Users\DragonMsi\Documents\pywikibot\pwb.py", line 58, in <module>
    sys.exit(main())
             ~~~~^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pwb.py", line 54, in main
    runpy.run_path(str(path), run_name='__main__')
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen runpy>", line 287, in run_path
  File "<frozen runpy>", line 98, in _run_module_code
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\scripts\wrapper.py", line 558, in <module>
    main()
    ~~~~^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\scripts\wrapper.py", line 542, in main
    if not execute():
           ~~~~~~~^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\scripts\wrapper.py", line 529, in execute
    run_python_file(filename, script_args, module)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\scripts\wrapper.py", line 154, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
    ~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         main_mod.__dict__)
         ^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\scripts\replace.py", line 1148, in <module>
    main()
    ~~~~^^
  File "C:\Users\DragonMsi\Documents\pywikibot\scripts\replace.py", line 1144, in main
    bot.run()
    ~~~~~~~^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\bot.py", line 1579, in run
    for item in self.generator:
                ^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\pagegenerators\__init__.py", line 672, in PreloadingGenerator
    for page in generator:
                ^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\pagegenerators\_filters.py", line 84, in <genexpr>
    return (page for page in generator if page.namespace() in namespaces)
                             ^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\pagegenerators\_generators.py", line 358, in CategorizedPageGenerator
    yield from category.articles(
    ...<5 lines>...
    )
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_category.py", line 190, in articles
    for member in self.members(
                  ~~~~~~~~~~~~^
            member_type=member_type, recurse=recurse, **kwargs):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_category.py", line 259, in members
    for member in subcat.members(
                  ~~~~~~~~~~~~~~^
            recurse=recurse, total=total, **kwargs):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_category.py", line 259, in members
    for member in subcat.members(
                  ~~~~~~~~~~~~~~^
            recurse=recurse, total=total, **kwargs):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_category.py", line 259, in members
    for member in subcat.members(
                  ~~~~~~~~~~~~~~^
            recurse=recurse, total=total, **kwargs):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_category.py", line 245, in members
    for member in self.site.categorymembers(self, total=total, **kwargs):
                  ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen _collections_abc>", line 360, in __next__
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\tools\collections.py", line 285, in send
    return next(self._started_gen)
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\data\api\_generators.py", line 723, in generator
    yield from self._extract_results(resultdata)
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\data\api\_generators.py", line 654, in _extract_results
    result = self.result(item)
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\data\api\_generators.py", line 838, in result
    p = pywikibot.FilePage(
        p, ignore_extension='imageinfo' in pagedata)
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\page\_filepage.py", line 73, in __init__
    if not sep or extension.lower() not in self.site.file_extensions:
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\site\_basesite.py", line 218, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
                         f'attribute {name!r}') from None
AttributeError: APISite instance has no attribute 'file_extensions'. Did you mean: 'has_extension'?
CRITICAL: Exiting due to uncaught exception AttributeError: APISite instance has no attribute 'file_extensions'

I see that the drop occurs when trying to process a category containing a file. For example this one: https://ru.minecraft.wiki/w/Категория:Спрайтовые_изображения_Applied_Energistics
But I don't see why this should cause problems if all the files are in a namespace that was excluded by the -ns:0 key.

Related Objects

Event Timeline

Very strange because file_extension property was introduced with Pywikibot 8.4 already. Could you please try the following code snippet (exprecting you have a minecraft_family file:

py -m pwb shell
>>> site = pywikibot.Site('minecraft:ru')
>>> site.file_extensions

The following is expected then:

['flac', 'gif', 'ico', 'jpeg', 'jpg', 'mkv', 'mov', 'mp3', 'mp4', 'odm', 'oga', 'ogg', 'pdf', 'png', 'svg', 'wav', 'webm', 'webp']

btw. the namespace is filtered later in (page for page in generator if page.namespace() in namespaces).

Very strange because file_extension property was introduced with Pywikibot 8.4 already. Could you please try the following code snippet (exprecting you have a minecraft_family file:

py -m pwb shell
>>> site = pywikibot.Site('minecraft:ru')
>>> site.file_extensions

The following is expected then:

['flac', 'gif', 'ico', 'jpeg', 'jpg', 'mkv', 'mov', 'mp3', 'mp4', 'odm', 'oga', 'ogg', 'pdf', 'png', 'svg', 'wav', 'webm', 'webp']
>>> site = pywikibot.Site('mcw:ru')
>>> site.file_extensions
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\site\_basesite.py", line 218, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
                         f'attribute {name!r}') from None
AttributeError: APISite instance has no attribute 'file_extensions'. Did you mean: 'has_extension'?

Okay, I updated the bot, now the error is different:

>>> site.file_extensions
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\Users\DragonMsi\Documents\pywikibot\pywikibot\site\_basesite.py", line 218, in __getattr__


    raise AttributeError(f'{type(self).__name__} instance has no '
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: APISite instance has no attribute 'file_extensions'. Did you mean: 'has_extension'?

Okay, I updated the bot, now the error is different:

Pretty sure your bot is either modified or not up to date. Trying to use a unknown property would fail in line 219 for stable release 11.0 and current master 11.1 but not at line 218.

>>> import pywikibot
>>> site = pywikibot.Site('minecraft:ru')
>>> site.unknown_property
Traceback (most recent call last):
  File "<pyshell#19>", line 1, in <module>
    site.unknown_property
  File "C:\pwb\GIT\core\pywikibot\site\_basesite.py", line 219, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
AttributeError: APISite instance has no attribute 'unknown_property'

I get a similar traceback for older Pywikibot, like 8.3.3:

site = pywikibot.Site('minecraft:ru')
site.file_extension
Traceback (most recent call last):
  File "<pyshell#22>", line 1, in <module>
    site.file_extensions
  File "C:\pwb\GIT\core\pywikibot\site\_basesite.py", line 197, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
AttributeError: APISite instance has no attribute 'file_extension'. Did you mean: 'has_extension'?

because file_extensions was introduced with PWB 8.4.

You may check the APISite in file pywikibot/site/_apisite. The current master and the stable release has this file_extensions property at line 485:
https://phabricator.wikimedia.org/diffusion/PWBC/browse/master/pywikibot/site/_apisite.py#L485
https://phabricator.wikimedia.org/diffusion/PWBC/browse/stable/pywikibot/site/_apisite.py#L485

Could you run py -m pwb version -nouser and tag the result here please.

Change #1240880 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] pagegenerators: Use namespaces parameter with any category option

https://gerrit.wikimedia.org/r/1240880

This comment was removed by Ivan-r.

Could you run py -m pwb version -nouser and tag the result here please.

C:\Users\DragonMsi\Documents\pywikibot>py -m pwb version -nouser
Pywikibot: pywikibot/__init__.py (, -1 (unknown), 2025/11/03, 04:02:03, UNKNOWN)
Release version: 10.7.0
packaging version: 25.0
mwparserfromhell version: 0.6.5
wikitextparser version: 0.56.4
requests version: 2.32.5
  cacerts: C:\Program Files\Python314\Lib\site-packages\certifi\cacert.pem
    certificate test: ok
Python: 3.14.0 (tags/v3.14.0:ebf955d, Oct  7 2025, 10:15:03) [MSC v.1944 64 bit (AMD64)]
User-Agent: version (mcw:ru; User:ProBot) Pywikibot/10.7.0 (-1 (unknown)) requests/2.32.5 Python/3.14.0.final.0
PYWIKIBOT_DIR: Not set
PYWIKIBOT_DIR_PWB: C:\Users\DragonMsi\Documents\pywikibot\pywikibot\scripts
PYWIKIBOT_NO_USER_CONFIG: Not set
Config base dir: C:\Users\DragonMsi\Documents\pywikibot

изображение.png (427×817 px, 55 KB)

I did a completely clean installation of the latest stable version, copying only family and user-config file. Here is the result:

C:\Users\DragonMsi\Documents\pywikibot>cd %userprofile%/Documents/pywikibot2

C:\Users\DragonMsi\Documents\pywikibot2>py -m pwb shell
Welcome to the Pywikibot interactive shell!
>>> site = pywikibot.Site('mcw:ru')
>>> site.file_extensions
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\Users\DragonMsi\Documents\pywikibot2\pywikibot\site\_basesite.py", line 219, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
                         f'attribute {name!r}') from None
AttributeError: APISite instance has no attribute 'file_extensions'. Did you mean: 'has_extension'?
>>>

Can't the problem be on the site side?

No glue yet. Could you please tag the results of the following tests:

py -m pwb shell
>>> site = pywikibot.Site('mcw:ru')
>>> type(site)
<class 'pywikibot.site._apisite.APISite'>
>>> site.__class__.__module__
pywikibot.site._apisite
>>> from pprint import pprint
>>> pprint(site.__class__.__mro__)
(<class 'pywikibot.site._apisite.APISite'>,
 <class 'pywikibot.site._basesite.BaseSite'>,
 <class 'pywikibot.tools.ComparableMixin'>,
 <class 'abc.ABC'>,
 <class 'pywikibot.site._extensions.EchoMixin'>,
 <class 'pywikibot.site._generators.GeneratorsMixin'>,
 <class 'pywikibot.site._extensions.GeoDataMixin'>,
 <class 'pywikibot.site._extensions.GlobalUsageMixin'>,
 <class 'pywikibot.site._extensions.LinterMixin'>,
 <class 'pywikibot.site._extensions.PageImagesMixin'>,
 <class 'pywikibot.site._extensions.ProofreadPageMixin'>,
 <class 'pywikibot.site._extensions.TextExtractsMixin'>,
 <class 'pywikibot.site._extensions.ThanksMixin'>,
 <class 'pywikibot.site._extensions.UrlShortenerMixin'>,
 <class 'pywikibot.site._extensions.WikibaseClientMixin'>,
 <class 'object'>)
>>> page = pywikibot.FilePage(site, 'File:Example.jpg')

To be sure please also delete all __pycache__ folders , especially within pywikibot/site folder.

Can't the problem be on the site side?

No, this is clearly an issue in the code, the site provides the extensions as expected. I suspect this is a problem with the family file (cannot find it anywhere in the task), possibly referencing the wrong APISite / old installation. Or some pycache corruption, but I have never seen a case of it.

Can't the problem be on the site side?

No, this is clearly an issue in the code, the site provides the extensions as expected. I suspect this is a problem with the family file (cannot find it anywhere in the task), possibly referencing the wrong APISite / old installation. Or some pycache corruption, but I have never seen a case of it.

https://ru.minecraft.wiki/w/Справка:PyWikiBot#mcw_family.py

@Ivan-r could you please tag the additional information I asked for.

@Ivan-r could you please tag the additional information I asked for.

C:\Users\DragonMsi\Documents\pywikibot2>py -m pwb shell
Welcome to the Pywikibot interactive shell!
>>> site = pywikibot.Site('mcw:ru')
>>> type(site)
<class 'pywikibot.site._apisite.APISite'>
>>> site.__class__.__module__
'pywikibot.site._apisite'
>>> from pprint import pprint
>>> pprint(site.__class__.__mro__)
(<class 'pywikibot.site._apisite.APISite'>,
 <class 'pywikibot.site._basesite.BaseSite'>,
 <class 'pywikibot.tools.ComparableMixin'>,
 <class 'abc.ABC'>,
 <class 'pywikibot.site._extensions.EchoMixin'>,
 <class 'pywikibot.site._generators.GeneratorsMixin'>,
 <class 'pywikibot.site._extensions.GeoDataMixin'>,
 <class 'pywikibot.site._extensions.GlobalUsageMixin'>,
 <class 'pywikibot.site._extensions.LinterMixin'>,
 <class 'pywikibot.site._extensions.PageImagesMixin'>,
 <class 'pywikibot.site._extensions.ProofreadPageMixin'>,
 <class 'pywikibot.site._extensions.TextExtractsMixin'>,
 <class 'pywikibot.site._extensions.ThanksMixin'>,
 <class 'pywikibot.site._extensions.UrlShortenerMixin'>,
 <class 'pywikibot.site._extensions.WikibaseClientMixin'>,
 <class 'object'>)
>>> page = pywikibot.FilePage(site, 'File:Example.jpg')
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\Users\DragonMsi\Documents\pywikibot2\pywikibot\page\_filepage.py", line 73, in __init__
    if not sep or extension.lower() not in self.site.file_extensions:
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\DragonMsi\Documents\pywikibot2\pywikibot\site\_basesite.py", line 219, in __getattr__
    raise AttributeError(f'{type(self).__name__} instance has no '
                         f'attribute {name!r}') from None
AttributeError: APISite instance has no attribute 'file_extensions'. Did you mean: 'has_extension'?
>>>

(unfortunately) that looks good. Please confirm that the __pycache__ files were deleted previously. I have no further ideas except:

  • check pip freeze (py -m pip freeze) whether any Pywikibot is installed as side package
  • tag the whole pywikibot repository as zipfile here for further investigations

Change #1240880 merged by Xqt:

[pywikibot/core@master] pagegenerators: Support namespaces parameter for all category generators

https://gerrit.wikimedia.org/r/1240880

(unfortunately) that looks good. Please confirm that the __pycache__ files were deleted previously. I have no further ideas except:

I have checked for all pycache, and delete them.

  • check pip freeze (py -m pip freeze) whether any Pywikibot is installed as side package
C:\Users\DragonMsi>py -m pip freeze
beautifulsoup4==4.14.2
certifi==2025.10.5
charset-normalizer==3.4.4
colorama==0.4.6
googlesearch-python==1.3.0
idna==3.11
iniconfig==2.3.0
memento-client==0.6.1
mwoauth==0.4.0
mwparserfromhell==0.7.2
oauthlib==3.3.1
packaging==25.0
pillow==12.0.0
pluggy==1.6.0
pydot==4.0.1
Pygments==2.19.2
PyJWT==2.9.0
PyMySQL==1.1.2
pyparsing==3.2.5
pytest==9.0.2
python-stdnum==2.1
regex==2025.11.3
requests==2.32.5
requests-oauthlib==2.0.0
requests-sse==0.5.2
soupsieve==2.8
typing_extensions==4.15.0
urllib3==2.5.0
wcwidth==0.2.14
wikitextparser==0.56.4
yt-dlp-ejs==0.3.1
  • tag the whole pywikibot repository as zipfile here for further investigations

What do you mean?

Change #1240880 merged by Xqt:

[pywikibot/core@master] pagegenerators: Support namespaces parameter for all category generators

https://gerrit.wikimedia.org/r/1240880

Yes, it solved the problem.

What do you mean?

For further investigations I need your local repository. Try to upload it here as zip file.

What do you mean?

For further investigations I need your local repository. Try to upload it here as zip file.

https://disk.yandex.ru/d/shNsCeNfaiQF8Q

What do you mean?

For further investigations I need your local repository. Try to upload it here as zip file.

https://disk.yandex.ru/d/shNsCeNfaiQF8Q

Thank you. I have an idea.

Xqt changed the task status from Open to In Progress.Mar 1 2026, 7:47 AM
Xqt claimed this task.
Xqt triaged this task as High priority.
Xqt removed a project: TestMe.
Xqt changed the task status from In Progress to Open.Mar 1 2026, 9:00 AM

@Ivan-r: Your family file is incorrect and causes this issue.

I.
Please replace

crossnamespace: family.CrossnamespaceType = collections.defaultdict(dict)

by

crossnamespace = collections.defaultdict(dict)

(removing type hints)

II.
Please replace this line

shared_image_repository = ('mcw', 'en')

and instead add the proper method

def shared_image_repository(self, code):
    """Return the shared image repository, if any."""
    return ('en', 'mcw')

For example, place it at the bottom of the family file.
Please ensure that the indentation is correct.

Reason
Some site methods are delegated from the Site object to the Family object. This delegation only works for Family methods that have code as their first parameter. shared_image_repository() is one such method. If you overwrite it by an attribute instead of defining it as a method, this delegation cannot occur, and the corresponding Site method will fail.

Change #1246751 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] site: raise NotImplementedError in BaseSite.__getattr__

https://gerrit.wikimedia.org/r/1246751

Change #1246751 merged by Xqt:

[pywikibot/core@master] site: raise NotImplementedError in BaseSite.__getattr__

https://gerrit.wikimedia.org/r/1246751