Page MenuHomePhabricator

Cannot login to private wiki using BotPassword
Open, LowPublicBUG REPORT

Description

List of steps to reproduce

  • setup a private (self-hosted) wiki
  • create a bot password
  • configure user-config.py and user-password.py with the botpassword
  • python pwb.py login

What happens?

The login fails, because the login procedure attempt to retrieve the mediawiki version with an api call, but it does not have permission to do this:

WARNING: API error readapidenied: You need read permission to use this module.
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
Logging in to timeseries:timeseries as Chris.blom@Ai
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
ERROR: Login failed (readapidenied).
Traceback (most recent call last):
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/login.py", line 307, in login
    self.login_to_site()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/data/api.py", line 2853, in login_to_site
    below_mw_1_27 = self.site.mw_version < '1.27'
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_apisite.py", line 951, in mw_version
    mw_ver = MediaWikiVersion(self.version())
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_apisite.py", line 927, in version
    version = self.siteinfo.get('generator', expiry=1).split(' ')[1]
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 296, in get
    preloaded = self._get_general(key, expiry)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 237, in _get_general
    default_info = self._get_siteinfo(props, expiry)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 162, in _get_siteinfo
    data = request.submit()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/data/api.py", line 2009, in submit
    self._data = super().submit()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/data/api.py", line 1859, in submit
    raise pywikibot.exceptions.APIError(**result['error'])
pywikibot.exceptions.APIError: readapidenied: You need read permission to use this module.
[help: See https://wiki.timeseries.com/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/chris/virtualenv/foo/bin/navi", line 223, in <module>
    action.execute()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/wiki_bot_navi/actions.py", line 167, in execute
    if not wiki.get_page(self.component).exists():
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/wiki_bot_navi/wiki.py", line 27, in get_page
    return pywikibot.Page(_get_site(), page_name)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/tools/__init__.py", line 1648, in wrapper
    return obj(*__args, **__kw)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/page/__init__.py", line 2061, in __init__
    super().__init__(source, title, ns)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/page/__init__.py", line 171, in __init__
    self._link = Link(title, source=source, default_namespace=ns)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/tools/__init__.py", line 1648, in wrapper
    return obj(*__args, **__kw)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/page/__init__.py", line 5192, in __init__
    self._defaultns = self._source.namespaces[default_namespace]
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_basesite.py", line 253, in namespaces
    self._namespaces = NamespacesDict(self._build_namespaces())
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_apisite.py", line 860, in _build_namespaces
    for nsdata in self.siteinfo.get('namespaces', cache=False).values():
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 296, in get
    preloaded = self._get_general(key, expiry)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 237, in _get_general
    default_info = self._get_siteinfo(props, expiry)
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_siteinfo.py", line 162, in _get_siteinfo
    data = request.submit()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/data/api.py", line 2009, in submit
    self._data = super().submit()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/data/api.py", line 1825, in submit
    self.site.login()
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/site/_apisite.py", line 388, in login
    if login_manager.login(retry=True, autocreate=autocreate):
  File "/home/chris/virtualenv/foo/lib/python3.9/site-packages/pywikibot/login.py", line 316, in login
    raise NoUsernameError(error_msg)
pywikibot.exceptions.NoUsernameError: Username "Chris.blom@Ai" does not have read permissions on timeseries:timeseries
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.NoUsernameError'>

The problem is that LoginManager, login_to_site, attempt to retrieve the version from mediawiki via an api call:
(See https://github.com/wikimedia/pywikibot/blob/heads%2F6.3.0/pywikibot/data/api.py#L2853, via https://github.com/wikimedia/pywikibot/blob/heads%2F6.3.0/pywikibot/data/api.py#L2853)

By adding logging I could inspect the request made to mediawiki:

Request:
uri=/api.php?action=query&meta=siteinfo%7Cuserinfo&siprop=namespaces%7Cnamespacealiases%7Cgeneral&continue=&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json
body=None
headers={'Content-Type': 'application/x-www-form-urlencoded'}

Response:
response body:  {'error': {'code': 'readapidenied', 'info': 'You need read permission to use this module.', '*': 'See https://wiki.timeseries.com/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/

If I modify the method that does the api call (APISite.mw_version) to return a hardcoded version, the login works.
Once logged in, I can revert this change and it all just works. (Once logged in, it has permission to query the version)

What should have happened instead?:

The login procedure should query the mediawiki version in a way that does not result in the readapidenied error, or skip the query when not logged in.

Software version:

I've replicated this problem with all combinations of mediawiki-1.32 and mediawiki-1.35, pywikibot-6.1.0, and pywikibot-6.3.0

Event Timeline

Is this a WMF-hosted private wiki, or a self-hosted private wiki?

Xqt triaged this task as High priority.Jun 8 2021, 4:53 PM
Xqt added subscribers: Dvorapa, Mpaa.

These are the permission settings:

# The following permissions were set based on your choice in the installer
$wgGroupPermissions['*']['createaccount'] = true;
$wgGroupPermissions['*']['autocreateaccount'] = false;
$wgGroupPermissions['*']['edit'] = false;
$wgGroupPermissions['*']['read'] = false;

# Bot users need read access in order to function
$wgGroupPermissions['*']['bot'] = true;

I don't think it's a problem with the permissions: when i comment out the version check before the login as described, the login succeeds.

After some experimentation, i've come up with a possible fix:

  1. Make it possible to specify the mediawiki version in the family file
  2. If the mediawiki version is specified in the familiy file, use that version instead of getting it from the mediawiki server

I've also noticed that isPublic in family files is no longer used.
I found some old documentation that suggest that overriding isPublic in the family file would have a similar effect, but this no longer works: isPublic is no longer used.

I can prepare a pull request for this fix, please let me know if this approach is acceptable.

I've also tried to delay the lookup of the version until after login, but unfortunately the version is also check whenever a request is submitted, so i think it is not feasible to fix it that way.

This comment was removed by ChrisBlomTS.

I think the problem is the api.LoginManager.login_to_site code part. No further api calls should be necessary before login is either successfull or denied. A try/except clause instead of mw_version test should be made here

Ok, i've tried that, but unfortunately there are a few other places in the codebase where calls to mw_version are made:

For example here, in api.py, Request.add_defaults() via Request.submit() see:
https://github.com/wikimedia/pywikibot/blob/master/pywikibot/data/api.py#L1245-L1247

Change 699152 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [bugfix] don't try to retrieve data until logged in

https://gerrit.wikimedia.org/r/699152

Change 699152 seems like a cleaner solution, i'll check if it also works for my private wiki / BotPassword case

It doesn't, the mw_version check in api.Request.add_defaults() (line 1243) still fails.

It doesn't, the mw_version check in api.Request.add_defaults() (line 1243) still fails.

Ah, during get_login_token() I think.

I can confirm it works with the latest changes

Change 699195 had a related patch set uploaded (by Xqt; author: Xqt):

[pywikibot/core@master] [IMPR] Raise a RuntimeError if a MediaWiki version is used during login

https://gerrit.wikimedia.org/r/699195

Change 699152 merged by jenkins-bot:

[pywikibot/core@master] [bugfix] don't try to retrieve data until logged in

https://gerrit.wikimedia.org/r/699152

Xqt lowered the priority of this task from High to Low.Jun 10 2021, 4:16 PM

user-config.py, user-password.py are configured, wiki family class is generated.

Pywikibot is a fresh master 33803553adc3f1afa0e950f20fa6756619cbb459
Some info about Wiki site:

"generator": "MediaWiki 1.39.1",
"phpversion": "8.2.1",
"phpsapi": "apache2handler",
"dbtype": "postgres",
"dbversion": "14.6 (Ubuntu 14.6-201-yandex.52665.7e82983c2c)",

Wiki site is currently configured to allow reading, writing, and making API requests for logged in users only.

When I run "python pwb.py login" I have:

(.virtualenv) c:\work\wiki\bot\Python\git\core>python pwb.py login
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
Logging in to znanierussia:ru as NirvanaPyBot@pywikibot
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes.
WARNING: API warning (login): Fetching a token via "action=login" is deprecated. Use "action=query&meta=tokens&type=login" instead.
ERROR: Received incorrect login token. Forcing re-login.
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
ERROR: Username 'NirvanaPyBot@pywikibot' does not have read permissions on znanierussia:ru
Not logged in on znanierussia:ru.

Execution time: 0 seconds

The bot does "mediawiki/api.php?action=query&meta=siteinfo&siprop=namespaces|namespacealiases|general" request without beeing logged in and fails.
If you visit in browser "https://my_site/mediawiki/api.php?action=query&meta=siteinfo&siprop=namespaces|namespacealiases|general" and not logged in I get

{
    "error": {
        "code": "readapidenied",
        "info": "You need read permission to use this module.",
        "*": "See ...."
    }
}

If I login in browser It works OK.

Running python pwb.py login -v -debug shows that the first request the bot is making is "query siteinfo":

2023-02-05 05:11:00       _requests.py,  672 in _get_request_params: DEBUG    API request to znanierussia:ru (uses get: True):
Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
URI: '/mediawiki/api.php?action=query&meta=siteinfo%7Cuserinfo&siprop=namespaces%7Cnamespacealiases%7Cgeneral&continue=&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'

This is where pywikibot is making request.

Traceback (most recent call last):
  File "c:\work\wiki\bot\Python\git\core\pwb.py", line 39, in <module>
    sys.exit(main())
  File "c:\work\wiki\bot\Python\git\core\pwb.py", line 35, in main
    runpy.run_path(str(path), run_name='__main__')
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2544.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\wrapper.py", line 516, in <module>
    main()
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\wrapper.py", line 500, in main
    if not execute():
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\wrapper.py", line 487, in execute
    run_python_file(filename, script_args, module)
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\wrapper.py", line 147, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\login.py", line 177, in <module>
    main()
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\login.py", line 171, in main
    login_one_site(lang, family_name, *params)
  File "c:\work\wiki\bot\Python\git\core\pywikibot\scripts\login.py", line 113, in login_one_site
    site.login(autocreate=autocreate)
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 375, in login
    if self.userinfo['name'] == self.user():
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 535, in userinfo
    uidata = uirequest.submit()
  File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_requests.py", line 955, in submit
    self._add_defaults()
  File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_requests.py", line 444, in _add_defaults
    and self.site.mw_version >= '1.25wmf5'):
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 1151, in mw_version
    mw_ver = MediaWikiVersion(self.version())
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 1124, in version
    version = self.siteinfo.get('generator', expiry=1).split(' ')[1]
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_siteinfo.py", line 305, in get
    preloaded = self._get_general(key, expiry)
  File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_siteinfo.py", line 228, in _get_general
    raise Exception('HOUHOU')
Exception: HOUHOU

We need some flag in family settings to specify that Wiki API usage is forbidden before login.

Wow. The problem I was writing about is handled already with using try-except.

# May occur if you are not logged in (no API read permissions).
        except APIError:
            pass

So I have another problem actually.

In my case it tries to login without token (may be it relies that cookies may be available?) and fails.

ERROR    Received incorrect login token. Forcing re-login.

After that, it is making again query siteinfo

URI: '/mediawiki/api.php?action=query&meta=siteinfo%7Cuserinfo&siprop=namespaces%7Cnamespacealiases%7Cgeneral&continue=&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'

and fails with API error readapidenied and finishes with login_one_site: INFO Not logged in on znanierussia:ru.

So it crashes while getting new token:

ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 409, in login
    if login_manager.login(retry=True, autocreate=autocreate):
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\login.py", line 293, in login
    self.login_to_site()
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\login.py", line 432, in login_to_site
    self.keyword('token')] = self.site.tokens['login']
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_tokenwallet.py", line 43, in __getitem__
    self._tokens = self.site.get_tokens([])
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 1609, in get_tokens
    pdata = self._paraminfo.parameter('query+tokens', 'type')
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_paraminfo.py", line 485, in parameter
    module = self[module]
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_paraminfo.py", line 444, in __getitem__
    self.fetch({key})
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_paraminfo.py", line 175, in fetch
    self._init()
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\data\api\_paraminfo.py", line 95, in _init
    mw_ver = self.site.mw_version
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 1151, in mw_version
    mw_ver = MediaWikiVersion(self.version())
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_apisite.py", line 1124, in version
    version = self.siteinfo.get('generator', expiry=1).split(' ')[1]
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_siteinfo.py", line 310, in get
    preloaded = self._get_general(key, expiry)
ERROR: File "c:\work\wiki\bot\Python\git\core\pywikibot\site\_siteinfo.py", line 232, in _get_general
    for line in traceback.format_stack():
WARNING: API error readapidenied: You need read permission to use this module.

I found a hotfix (workaround) that helped me to log in:

class TokenWallet(Container):

    def __getitem__(self, key: str) -> str:
       ...

        if not self._tokens:
            # NEW CODE
            if key == 'login':
                _tokens = self.site.get_tokens(['login'])
                return _tokens['login']
            else:
                # OLD CODE
                self._tokens = self.site.get_tokens([])

I can't login either to a private wiki using a botpassword, but in my case it's a private WMF wiki. After the cookie *.lwp data generated during generate_user_files expired, the bot keeps complaining that (a) the API method to retrieve tokens is deprecated and (b) readapidenied.

$ pwb.py login
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
Logging in to xxxx:xxx as xxxx@xxxx
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.
WARNING: API warning (login): Fetching a token via "action=login" is deprecated. Use "action=query&meta=tokens&type=login" instead.
ERROR: Received incorrect login token. Forcing re-login.
WARNING: API error readapidenied: You need read permission to use this module.
ERROR: You have no API read permissions. Seems you are not logged in.
ERROR: Username 'xxxx@xxxx' does not have read permissions on xxxxxx
Not logged in on xxxxx.

Execution time: 1 seconds

Any fix I could put on my custom family file to fix this?

I found a hotfix (workaround) that helped me to log in:

class TokenWallet(Container):

    def __getitem__(self, key: str) -> str:
       ...

        if not self._tokens:
            # NEW CODE
            if key == 'login':
                _tokens = self.site.get_tokens(['login'])
                return _tokens['login']
            else:
                # OLD CODE
                self._tokens = self.site.get_tokens([])

I tested that and it let me log in, but running a script immediately afterwards was not possible with WARNING: API error readapidenied: You need read permission to use this module.

Aklapper removed Xqt as the assignee of this task.May 16 2024, 4:40 PM

@Xqt: Removing task assignee as this open task has been assigned for more than two years - see the email sent to all task assignees on 2024-04-15.
Please assign this task to yourself again if you still realistically [plan to] work on this task - it would be welcome! :)
If this task has been resolved in the meantime, or should not be worked on by anybody ("declined"), please update its task status via "Add Action… 🡒 Change Status".
Also see https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup for tips how to best manage your individual work in Phabricator. Thanks!