Page MenuHomePhabricator

login.py crashes on 1.33 Fandom wikis
Open, HighPublic

Description

Pywikibot 3.1.dev0 (Python 3.8.2) on Linux fails to login on new Fandom (Wikia) technology platform called UCP (MediaWiki 1.33.3) with error:

[rail@rail-pc pwb]$ python pwb.py login
Traceback (most recent call last):
  File "pwb.py", line 389, in <module>
    if not main():
  File "pwb.py", line 381, in main
    run_python_file(filename,
  File "pwb.py", line 100, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File "./scripts/login.py", line 188, in <module>
    main()
  File "./scripts/login.py", line 158, in main
    site = pywikibot.Site()
  File "/home/rail/pwb/pywikibot/tools/__init__.py", line 1790, in wrapper
    return obj(*__args, **__kw)
  File "/home/rail/pwb/pywikibot/__init__.py", line 1242, in Site
    fam = Family.load(fam)
  File "/home/rail/pwb/pywikibot/tools/__init__.py", line 1790, in wrapper
    return obj(*__args, **__kw)
  File "/home/rail/pwb/pywikibot/family.py", line 1000, in load
    assert all(x in NAME_CHARACTERS for x in fam), \
AssertionError: Name of family ucp-sandbox must be ASCII characters and digits
CRITICAL: Exiting due to uncaught exception <class 'AssertionError'>

New Fandom platform probably causes more issues.

Tested wikis

Family files generated automatically with generate_family_file.py

user-config.py

# -*- coding: utf-8  -*-
family = # family
mylang = # lang

usernames['ucp-sandbox']['pl'] = # my bot account
usernames['ucp-woodhouse']['en'] = # my bot account

password_file = 'user-password.py'
console_encoding = 'utf-8'

Event Timeline

Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptApr 4 2020, 7:30 AM
Xqt triaged this task as High priority.Apr 4 2020, 8:13 AM
Dvorapa added a subscriber: Dvorapa.EditedApr 4 2020, 8:15 AM

Hello, currently you have to call your family only with alphanumeric characters, hyphens are not allowed. This is only for internal Pywikibot purposes, so it will not affect the communication with your wiki. Change the following lines in your user-config.py and also the corresponding lines in your family files and you should be fine:

family = 'ucpsandbox'
usernames['ucpsandbox']['pl'] = # my bot account
usernames['ucpwoodhouse']['en'] = # my bot account
Xqt claimed this task.Apr 4 2020, 8:15 AM

Change 585860 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [bugfix] add dash to valid NAME_CHARACTERS for Family.name

https://gerrit.wikimedia.org/r/585860

Change 585860 abandoned by Xqt:
[bugfix] add hyphen to valid NAME_CHARACTERS for Family.name

https://gerrit.wikimedia.org/r/585860

Xqt removed Xqt as the assignee of this task.Apr 4 2020, 8:21 AM
Xqt removed a project: Patch-For-Review.
Xqt added a subscriber: Xqt.

Oh well it worked but now throws different error:

[rail@rail-pc pwb]$ python pwb.py login
WARNING: No user is logged in on site ucpsandbox:pl
Logging in to ucpsandbox:pl as RailBot
WARNING: API warning (main): Unrecognized parameters: username, password.
ERROR: Login failed (FAIL).
Traceback (most recent call last):
  File "/home/rail/pwb/pywikibot/login.py", line 300, in login
    cookiedata = self.getCookie()
  File "/home/rail/pwb/pywikibot/tools/__init__.py", line 1875, in wrapper
    return obj(*new_args, **new_kwargs)
  File "/home/rail/pwb/pywikibot/data/api.py", line 3222, in getCookie
    raise APIError(code=status, info=info)
pywikibot.data.api.APIError: FAIL: Podane dane nie mogą być użyte do kontroli poświadczeń.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "pwb.py", line 389, in <module>
    if not main():
  File "pwb.py", line 381, in main
    run_python_file(filename,
  File "pwb.py", line 100, in run_python_file
    exec(compile(source, filename, 'exec', dont_inherit=True),
  File "./scripts/login.py", line 188, in <module>
    main()
  File "./scripts/login.py", line 170, in main
    site.login(autocreate=autocreate)
  File "/home/rail/pwb/pywikibot/tools/__init__.py", line 1790, in wrapper
    return obj(*__args, **__kw)
  File "/home/rail/pwb/pywikibot/site/__init__.py", line 2090, in login
    if login_manager.login(retry=True, autocreate=autocreate):
  File "/home/rail/pwb/pywikibot/login.py", line 309, in login
    raise NoUsername(error_msg)
pywikibot.exceptions.NoUsername: Username "RailBot" does not have read permissions on ucpsandbox:pl
.Podane dane nie mogą być użyte do kontroli poświadczeń.
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.NoUsername'>

Probably it is due to custom Auth system Fandom is using.

Dvorapa claimed this task.EditedApr 4 2020, 9:16 AM

Yes, it looks so. clientlogin on UCP somehow does not require standard credentials like username and password (weird). Old Fandom system was working correctly.

Anyway, thank you for bringing UCP up (I didn't know about it). We should probably test also UCP, not only old Fandom system in our tests.

Dvorapa moved this task from Backlog to Upstream issues on the Pywikibot board.EditedApr 4 2020, 9:22 AM
Dvorapa added a project: Upstream.

Hmm, it seems UCP does not support login through API currently: https://wreckit-woodhouse.fandom.com/api.php?action=query&meta=authmanagerinfo&amirequestsfor=login.

canauthenticatenow response is empty, indicates login action is currently not possible.

Anyway we should provide better informative error message in this case.

Are you sure your UCP wiki is configured correctly? Where could we contact Fandom developers about this issue?

Dvorapa added a comment.EditedApr 4 2020, 9:34 AM

Perhaps they disabled this on purpose as they are still working on it? Neither clientlogin, nor login action works for me through API. (You can test yourself using API Sandbox: https://wreckit-woodhouse.fandom.com/wiki/Special:ApiSandbox)

Dvorapa removed Dvorapa as the assignee of this task.Apr 4 2020, 9:40 AM

(just to allow anyone to fix the error message issue, until I have more time)

Railfail536 added a comment.EditedApr 4 2020, 9:56 AM

You have to use Fandom's custom Service API available through https://services.fandom.com (https://services.wikia.org for wikia.org wikis) I guess.

Services API doesn't have any official documentation but that one might be helpful: click.

Dvorapa added a comment.EditedApr 4 2020, 10:40 AM

But this is not MediaWiki API. Pywikibot is based on MediaWiki API, I'm unsure whether we are capable to support non-standard APIs like this. Sorry to be rejective, but if every wiki family had their own separate API for login, Pywikibot would have 320 GB of specific methods to login to every wiki family all over the internet. We deprecated non-MediaWiki wikis years ago as it was a burden.

Also I don't understand the fact, that for logging in to "https://rail-ucp.fandom.com/pl/api.php" you need to call completely different url "https://services.fandom.com/auth/token" with completely different structure of parameters (missing format, maxlag, rememberMe). Also it is not documented anywhere properly.

Anyway, this would need to rewrite standard login() from Family class specifically in FandomFamily class to use the special url instead of the usual one (unsure if it is even possible without rewriting the whole login process).

Dvorapa added a comment.EditedApr 4 2020, 10:44 AM

MediaWiki API seems to work with that special login url correctly, so I would try to write the specific login method (but without any warranty) in summer if noone will be faster. Pywikibot should then wait until Fandom enables login using standard MediaWiki API again.

Anyway, have you tried Bot passwords or OAuth instead? These might still work regardless password based login issues (but unsure if they do).

AFAIK Fandom is already using custom auth method on old 1.19 platform. I don't know how to solve this problem (I'm not a developer) but it's serious and need to be solved.

Dvorapa added a comment.EditedApr 4 2020, 11:23 AM

AFAIK Fandom is already using custom auth method on old 1.19 platform.

Maybe, but MediaWiki auth methods work too.

I don't know how to solve this problem (I'm not a developer) but it's serious and need to be solved.

Yes, but it is not crucial as Fandom developers admit UCP is work in progress and still full of bugs and regressions yet to be solved.

Anyway, It would be good to see any information from Fandom developers about how the login process will look in the future as I don't want to rewrite something in Pywikibot and then rewrite it again after a month because Fandom developers changed it/fixed it.

I've shared link to this ticket with one of Fandom's contractors. Maybe their Staff member will clarify thing or two here.

Dvorapa added a comment.EditedApr 4 2020, 11:40 AM

I've shared link to this ticket with one of Fandom's contractors. Maybe their Staff member will clarify thing or two here.

That's awesome, thank you!

Any information about this move and the future would be useful.

Anomie added a subscriber: Anomie.Apr 8 2020, 5:17 PM