Page MenuHomePhabricator

scripts fail with -v option in python3.8
Closed, ResolvedPublicBUG REPORT

Description

Scripts crash when called with -v option in python 3.8.
Cannot say about other versions, except my comment below in Expected Results:

Release version: 5.0.1.dev0
requests version: 2.24.0
  cacerts: /home/user/anaconda3/lib/python3.8/site-packages/certifi/cacert.pem
    certificate test: ok
Python: 3.8.3 (default, Jul  2 2020, 16:21:59)

Steps to Reproduce:
python scripts/listpages.py -start:A -v

Actual Results:

user@pc:~/python/core$ python scripts/listpages.py -start:A -v

=== Pywikibot framework v5.0.1.dev0 -- Logging header ===
COMMAND: ['scripts/listpages.py', '-start:A', '-v']
DATE: 2020-10-19 20:01:07.414882 UTC
VERSION: [ssh] pywikibot-core.git (4cdf2f0, g13425, 2020/10/19, 17:05:17, n/a)
SYSTEM: posix.uname_result(sysname='Linux', nodename='pc', release='4.15.0-117-generic', version='#118-Ubuntu SMP Fri Sep 4 20:02:41 UTC 2020', machine='x86_64')
CONFIG FILE DIR: /home/user/.pywikibot
PACKAGES:
Traceback (most recent call last):
  File "scripts/listpages.py", line 284, in <module>
    main()
  File "scripts/listpages.py", line 190, in main
    local_args = pywikibot.handle_args(args)
  File "/home/user/python/core/pywikibot/bot.py", line 850, in handle_args
    pywikibot.Site()
  File "/home/user/python/core/pywikibot/tools/__init__.py", line 1450, in wrapper
    return obj(*__args, **__kw)
  File "/home/user/python/core/pywikibot/__init__.py", line 1203, in Site
    debug("Instantiated %s object '%s'"
  File "/home/user/python/core/pywikibot/logging.py", line 178, in debug
    logoutput(text, decoder, newline, DEBUG, layer, **kwargs)
  File "/home/user/python/core/pywikibot/logging.py", line 76, in logoutput
    _init()
  File "/home/user/python/core/pywikibot/logging.py", line 35, in _init
    init_routine()
  File "/home/user/python/core/pywikibot/bot.py", line 346, in init_handlers
    writelogheader()
  File "/home/user/python/core/pywikibot/bot.py", line 392, in writelogheader
    packages = version.package_versions(check_package_list)
  File "/home/user/python/core/pywikibot/version.py", line 519, in package_versions
    if os.path.normcase(package.__file__).startswith(
  File "/home/user/anaconda3/lib/python3.8/posixpath.py", line 54, in normcase
    return os.fspath(s)
TypeError: expected str, bytes or os.PathLike object, not NoneType
Found 1 wikisource:en processes running, including this one.
Dropped throttle(s).
Closing network session.
CRITICAL: Exiting due to uncaught exception <class 'TypeError'>
Network session closed.

The offending package is (there might be others)

{'__name__': 'mpl_toolkits',
'__doc__': None,
'__package__': 'mpl_toolkits', 
'__loader__': <_frozen_importlib_external._NamespaceLoader object at 0x7f03eb523f10>,
'__spec__': ModuleSpec(name='mpl_toolkits', loader=<_frozen_importlib_external._NamespaceLoader object at 0x7f03eb523f10>, submodule_search_locations=_NamespacePath(['/home/user/anaconda3/lib/python3.8/site-packages/mpl_toolkits'])), 
'__file__': None,
'__path__': _NamespacePath(['/home/user/anaconda3/lib/python3.8/site-packages/mpl_toolkits'])}

where:

__file__': None

Expected Results:
The script should work, e.g. in T265928 (python3.6), it is possible to see:

PACKAGES:
 ...
  mpl_toolkits ([path unknown]) = ??
  mwoauth (/srv/paws/lib/python3.6/site-packages/mwoauth/) = 0.3.7
  ...

Event Timeline

In PAWS with python3.6 there is no

__file__

attribute.

@PAWS:~$ python
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> import sys
>>> vars(sys.modules['mpl_toolkits'])
import sys

vars(sys.modules['mpl_toolkits'])

{'__name__': 'mpl_toolkits',
 '__doc__': None,
 '__package__': 'mpl_toolkits',
 '__loader__': <_frozen_importlib_external._NamespaceLoader at 0x7f6e27d83208>,
 '__spec__': ModuleSpec(name='mpl_toolkits', loader=None, origin='namespace', submodule_search_locations=_NamespacePath(['/srv/paws/lib/python3.6/site-packages/mpl_toolkits'])),
 '__path__': _NamespacePath(['/srv/paws/lib/python3.6/site-packages/mpl_toolkits'])}

Cannot reproduce it with Windows.

btw, I am using Anaconda distribution.

file attribute is used several times inside the framework. Is there a better solution?

Change 635637 had a related patch set uploaded (by Mpaa; owner: Mpaa):
[pywikibot/core@master] [bugfix] support Namespace packages in version.py

https://gerrit.wikimedia.org/r/635637

Xqt assigned this task to Mpaa.

Change 635637 merged by jenkins-bot:
[pywikibot/core@master] [bugfix] support Namespace packages in version.py

https://gerrit.wikimedia.org/r/635637