Page MenuHomePhabricator
Create Task
Maniphest T343187

Problems with logging in
Open, HighPublicBUG REPORT
Actions

Description

For a few weeks or months, I have constantly been getting errors such as:

Traceback (most recent call last):
  File "/usr/lib/python3.9/threading.py", line 954, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.9/threading.py", line 892, in run
    self._target(*self._args, **self._kwargs)
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/__init__.py", line 1485, in async_manager
    request(*args, **kwargs)
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/page/_decorators.py", line 34, in handle
    func(self, *args, **kwargs)
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/page/_wikibase.py", line 680, in editEntity
    super().editEntity(data, **kwargs)
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/page/_wikibase.py", line 315, in editEntity
    updates = self.repo.editEntity(
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/site/_decorators.py", line 92, in callee
    return fn(self, *args, **kwargs)
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/site/_datasite.py", line 310, in editEntity
    params['token'] = self.tokens['csrf']
  File "/data/project/matsubot/pwbvenv/lib/python3.9/site-packages/pywikibot/site/_tokenwallet.py", line 59, in __getitem__
    raise KeyError(
KeyError: "Invalid token 'csrf' for user 'MatSuBot' on wikidata:wikidata wiki."

or

WARNING: API error badtoken: Invalid CSRF token.

or

ERROR: User assertion failed. Forcing re-login.
Logging in to wikidata:wikidata as MatSuBot@pywikibot
ERROR: Received incorrect login token. Forcing re-login.

for my Toolforge-based bot running on Wikidata. It's happening with and without BotPasswords.

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenBUG REPORTNoneT343187 Problems with logging in
 OpenBUG REPORTNoneT284577 Cannot login to private wiki using BotPassword
 OpenBUG REPORTNoneT328814 Login retrieves token from incorrect location

Event Timeline

matej_suchanek created this task.Aug 1 2023, 7:30 AM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptAug 1 2023, 7:30 AM
Xqt subscribed.Aug 1 2023, 10:13 AM

It would be better to receive the token instead raising a KeyError

Xqt triaged this task as High priority.Aug 1 2023, 10:14 AM
Xqt added a parent task: T342850: Deploy Pywikibot 8.4 stable release.
Xqt moved this task from Backlog to Login on the Pywikibot board.
Xqt added a project: Pywikibot-login.py.
Xqt added a subtask: T284577: Cannot login to private wiki using BotPassword.Aug 1 2023, 10:21 AM
Xqt added a subtask: T328814: Login retrieves token from incorrect location.
dimka subscribed.Sep 26 2023, 3:10 AM
Xqt edited parent tasks, added: T348374: Deploy Pywikibot 8.5 stable release; removed: T342850: Deploy Pywikibot 8.4 stable release.Oct 7 2023, 10:55 AM
matej_suchanek added a comment.Oct 7 2023, 1:43 PM

The problem persists. It always makes the same script fail when it starts. The script updates Wikidata labels. Other Wikidata scripts which edit data do not fail.

Xqt removed a parent task: T348374: Deploy Pywikibot 8.5 stable release.Dec 6 2023, 6:14 PM
matej_suchanek added a comment.Dec 18 2023, 4:20 PM

Found a new exception in the logs:

WARNING: /usr/lib/python3.9/http/cookiejar.py:1971: UserWarning: http.cookiejar bug!
Traceback (most recent call last):
  File "/usr/lib/python3.9/http/cookiejar.py", line 1953, in _really_load
    c = Cookie(h("version"), name, value,
  File "/usr/lib/python3.9/http/cookiejar.py", line 767, in __init__
    if version is not None: version = int(version)
ValueError: invalid literal for int() with base 10: 'None'

  _warn_unhandled_exception()
Xqt added a comment.Jan 8 2024, 12:45 PM
This comment was removed by Xqt.
mykhal subscribed.EditedJan 24 2024, 2:40 PM

Same problem with pwb.py replace (some older version, then also 8.5.1, 8.6.0, master):

WARNING: Http response status 404
WARNING: Http response status 404
Exception in thread Put-Thread:
Traceback (most recent call last):
  File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/m/src/pywikibot/pywikibot/__init__.py", line 399, in async_manager
    request(*args, **kwargs)
  File "/home/m/src/pywikibot/pywikibot/page/_decorators.py", line 35, in handle
    func(self, *args, **kwargs)
  File "/home/m/src/pywikibot/pywikibot/page/_basepage.py", line 1281, in _save
    done = self.site.editpage(self, summary=summary, minor=minor,
  File "/home/m/src/pywikibot/pywikibot/site/_decorators.py", line 86, in callee
    return fn(self, *args, **kwargs)
  File "/home/m/src/pywikibot/pywikibot/site/_apisite.py", line 2057, in editpage
    token = self.tokens['csrf']
  File "/home/m/src/pywikibot/pywikibot/site/_tokenwallet.py", line 60, in __getitem__
    raise KeyError(
KeyError: "Invalid token 'csrf' for user 'MykhalBot' on wikipedia:cs wiki."

on some cswiki page, but did not have problem on some other page.

mykhal added a comment.Jan 24 2024, 2:47 PM
This comment was removed by mykhal.
mykhal added a comment.Jan 24 2024, 3:25 PM

In the log I see queries to doi.org.. ??

2024-01-24 16:12:55       _requests.py,  744 in        _json_loads: DEBUG    Non-JSON response received from server doi:doi for url
https://www.doi.org/w/api.php?action=query&meta=userinfo&uiprop=blockinfo%7Cgroups%7Chasmsg%7Cratelimits%7Crights&formatversion=2&maxlag=5&format=json
The server may be down.
Status code: 404
mykhal added a comment.EditedJan 24 2024, 4:23 PM

So in my case, the problem is {{doi:…}} in the page (probably regardless if the prefix handling exists or not, in my case the syntax was not functional in either cswiki and testwiki).
(I've isolated the problem in https://test.wikipedia.org/wiki/Test_bot_doi_problems (ignore the NBSP typo which the script is designed to fix))

mykhal added a comment.EditedJan 24 2024, 4:28 PM

.. the problem disappears on the second try, but on cleanup apicache-py3/ cleanup it's there again. (but I could not reproduce it on another tries later on)

matej_suchanek added a comment.Jan 25 2024, 9:33 AM

I was suspicious there could be a race condition since my bot uses the asynchronous saving feature. So, I forced it to save synchronously. When I have it log out before running the script, I get messages like

Change summary: add missing labels
Logging in to wikidata:wikidata as MatSuBot@[redacted bot password key]
WARNING: API warning (main): Subscribe to the mediawiki-api-announce mailing list at <https://lists.wikimedia.org/postorius/lists/mediawiki-api-announce.lists.wikimedia.org/> for notice of API deprecations and breaking changes. Use [[Special:ApiFeatureUsage]] to see usage of deprecated features by your application.
WARNING: API warning (login): Fetching a token via "action=login" is deprecated. Use "action=query&meta=tokens&type=login" instead.
ERROR: Received incorrect login token. Forcing re-login.
Change summary: add missing labels
ERROR: User assertion failed. Forcing re-login.
Change summary: add missing labels
ERROR: User assertion failed. Forcing re-login.
Change summary: add missing labels

and so on. The ERROR: lines later occur less often. But the script keeps running, and edits are saved.

So there are two problems:

  • the bot does not handle and retry in asynchronous mode (but it does in synchronous mode)
  • there are still some login (edit token) problems

The doi links problem could be related, but I get errors by running the bot on Wikidata.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits