Change 303320 had a related patch set uploaded (by Gergő Tisza):
Revert "API: Remove deprecated response values from action=login"

https://gerrit.wikimedia.org/r/303320

Change 303320 merged by jenkins-bot:
Revert "API: Remove deprecated response values from action=login"

https://gerrit.wikimedia.org/r/303320

Mentioned in SAL [2016-08-05T23:39:43Z] <tgr@tin> Synchronized php-1.28.0-wmf.13/includes/api/ApiLogin.php: temporarily re-add dropped API feature to unbreak Pywikibot T142155 (duration: 00m 48s)

Reverted in 1.28.0-wmf.13 (meaning it will last until Tuesday). Is anyone planning to fix PWB and contact users about updating (or using OAuth)?

Eh. I don't think we should have reverted for now...we could have just fixed pywikibot pretty quickly by hardcoding the cookie prefix.

@Betacommand said something about the bot still working if you just delete the lines which deal with the prefix. Which seems plausible as it uses a cookie jar anyway. I'm not sure how to test that now though, there does not seem to be a wiki family for labs.

In any case, getting users to update should take a few days at least and it's not nice to break all bots for the weekend. (Well, the 16% of bots which are unlucky enough that their session would expire this week, but still.)

I'll make a step-by--step tutorial about using OAuth and we can send that and/or a call to update out with Tech News. (I suppose compat users will be screwed, or maybe that already happened in January?)

Using OAuth with Pywikibot is documented at https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth, maybe we need to publicize that more?

Hack to make Pywikibot work on Wikimedia wikis once cookieprefix is dropped:

diff --git a/pywikibot/data/api.py b/pywikibot/data/api.py
index 62ec818..3980c9a 100644
--- a/pywikibot/data/api.py
+++ b/pywikibot/data/api.py
@@ -3030,7 +3030,7 @@ class LoginManager(login.LoginManager):
             if u"login" not in login_result:
                 raise RuntimeError("API login response does not have 'login' key.")
             if login_result['login']['result'] == "Success":
-                prefix = login_result['login']['cookieprefix']
+                prefix = login_result['login'].get('cookieprefix', self.site.dbName())
                 cookies = []
                 for key in ('Token', 'UserID', 'UserName'):
                     cookies.append("%s%s=%s"

Haven't tested that, nor dropping that code entirely. I wouldn't be surprised if this code was entirely obsolete and not necessary.

In T142155#2529722, @Legoktm wrote:

Using OAuth with Pywikibot is documented at https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth, maybe we need to publicize that more?

Yes but it is not user-friendly (or rather the OAuth form is not user-friendly, and the doc sends people off to fill it).

Also apparently it's not in 2.0 and there is no more recent release :( Or do most people use master?

In T142155#2529761, @Tgr wrote:

Or do most people use master?

Pretty much, though ironically we've been trying to get people to use the proper releases so we can make breaking changes in master.

Hah, I found this gem:

def storecookiedata(self, data):
    """Ignore data; cookies are set by threadedhttp module."""
    http.cookie_jar.save()

So yes, that chunk of code isn't used and can be safely deleted. I'll put up a patch later tonight if no one beats me to it.

Tutorial: https://www.mediawiki.org/wiki/Manual:Pywikibot/OAuth/Wikimedia

See also T142282: Improve consumer registration experience for bot owners (owner-only consumers).

In T142155#2529638, @Tgr wrote:

Reverted in 1.28.0-wmf.13 (meaning it will last until Tuesday). Is anyone planning to fix PWB and contact users about updating (or using OAuth)?

This is affecting people at the #WikiConferenceIndia. Tuesday doesn't help them.
Yes, it looks like there is a fix coming.

But it is probably worth adding a logFeatureUsage to MediaWiki, as it takes a while for all bots to be updated to the latest code after we merge it, and push out a new release of 2.0.

In T142155#2530030, @jayvdb wrote:

In T142155#2529638, @Tgr wrote:

Reverted in 1.28.0-wmf.13 (meaning it will last until Tuesday). Is anyone planning to fix PWB and contact users about updating (or using OAuth)?

This is affecting people at the #WikiConferenceIndia. Tuesday doesn't help them.
Yes, it looks like there is a fix coming.

But it is probably worth adding a logFeatureUsage to MediaWiki, as it takes a while for all bots to be updated to the latest code after we merge it, and push out a new release of 2.0.

1.28.0-wmf.13 is the current version. The original patch was reverted until Tuesday (when 1.28.0-wmf.14 deployment begins). action=login should be working.

Ah, thanks for clarifying that.
Confirmed; login is working

Change 303342 had a related patch set uploaded (by Legoktm):
Stop reading 'cookieprefix' upon login

https://gerrit.wikimedia.org/r/303342

In T142155#2529638, @Tgr wrote:

Reverted in 1.28.0-wmf.13 (meaning it will last until Tuesday). Is anyone planning to fix PWB and contact users about updating (or using OAuth)?

Thank you Gergő. I can confirm too that login works again. Tuesday is way too early. Can you push it back at least 2 months?

Change 303342 merged by jenkins-bot:
Stop reading 'cookieprefix' upon login

https://gerrit.wikimedia.org/r/303342

Change 303367 had a related patch set uploaded (by John Vandenberg):
Stop reading 'cookieprefix' upon login

https://gerrit.wikimedia.org/r/303367

The gerrit reviewer bot seems to have stopped working. Maybe it's related to this task since it was only recently fixed and it uses the stable 2.0 branch.

https://mediawiki.org/wiki/Git/Reviewers

https://tools.wmflabs.org/gerrit-reviewer-bot/

I did a test here https://gerrit.wikimedia.org/r/#/c/303368/ and it should have added @hashar but it didn't.

https://github.com/valhallasw/gerrit-reviewer-bot/

Change 303367 merged by jenkins-bot:
Stop reading 'cookieprefix' upon login

https://gerrit.wikimedia.org/r/303367

Bot passwords don't work with Pywikibot, OAuth does not work very well (see T142269, T142303) and updating everyone to today's master cannot be expected to happen in a few days, so I'm reverting the API change in master. I'll leave it to @Anomie to decide whether two months is an acceptable delay (once he is back from vacation).

Change 303317 merged by jenkins-bot:
Revert "API: Remove deprecated response values from action=login"

https://gerrit.wikimedia.org/r/303317

Hello everybody,

my global interwiki bot "UT-interwiki bot" has global IP block exempt (GIPBE) because it runs on a Linux server whose IP is in a globally blocked range. In German Wiktionary the bot has also the local IP block exempt.

With the normal login via username and password everything works fine...

But if I switch to OAuth, then the bot can perform edits only in German Wiktionary (where it has the local IP block exempt, too).

In all other languages of Wiktionarys comes an error message: "WARNING: API error globalblocking-ipblocked-range: Your IP is in a range that has been globally blocked from editing".

======Post-processing [[de:Notbeleuchtung]]======
Updating links on page [[de:Notbeleuchtung]].
Changes to be made: Bot: Adding [[en:Notbeleuchtung]], [[hu:Notbeleuchtung]], [[zh:Notbeleuchtung]]
@@ -55 +55,5 @@
- {{Quellen}}
+ {{Quellen}}
+
+ [[en:Notbeleuchtung]]
+ [[hu:Notbeleuchtung]]
+ [[zh:Notbeleuchtung]]

Updating links on page [[hu:Notbeleuchtung]].
Changes to be made: Bot: Adding [[de:Notbeleuchtung]]
@@ -4,0 +5 @@
+ [[de:Notbeleuchtung]]

WARNING: API error globalblocking-ipblocked-range: Your IP is in a range that has been globally blocked from editing
Updating links on page [[zh:Notbeleuchtung]].
Changes to be made: Bot: Adding [[de:Notbeleuchtung]]
@@ -3,0 +4 @@
+ [[de:Notbeleuchtung]]

WARNING: API error globalblocking-ipblocked-range: Your IP is in a range that has been globally blocked from editing
Sleeping for 7.7 seconds, 2016-08-07 15:26:06

Is it possible that the GIPBE does not function properly with login via OAuth? Or may it be that all global rights (global bot flag) are affected?

Regards
Udo T.

In T142155#2531460, @Udo_T wrote:

But if I switch to OAuth, then the bot can perform edits only in German Wiktionary (where it has the local IP block exempt, too).

In all other languages of Wiktionarys comes an error message: "WARNING: API error globalblocking-ipblocked-range: Your IP is in a range that has been globally blocked from editing".

That's T142306: Adding globalblock-exempt to Grants in GlobalBlocking extension for OAuth use; it should be deployed as part of wmf.14, if I understand that task correctly.

In T142155#2531460, @Udo_T wrote:

Hello everybody,
[...]
Is it possible that the GIPBE does not function properly with login via OAuth? Or may it be that all global rights (global bot flag) are affected?

This is T142306, I've created 69d35960c11a which should fix this. Scheduled to rollout on wikis next week. Maybe it should be rolled now if many users are affected? Best regards.

In T142155#2530987, @Tgr wrote:

I'll leave it to @Anomie to decide whether two months is an acceptable delay (once he is back from vacation).

No, it's not acceptable. I already gave them seven months as part of the documented deprecation process for the API. But I suppose we may have to accept it anyway.

(Not back from vacation, just glancing through email)

In T142155#2530198, @Multichill wrote:

Tuesday is way too early. Can you push it back at least 2 months?

Two months have passed now (mostly due to me forgetting about this), so let's try re-reverting with the next train (Wed/Thu next week).

In T142155#2698611, @Tgr wrote:

In T142155#2530198, @Multichill wrote:

Tuesday is way too early. Can you push it back at least 2 months?

Two months have passed now (mostly due to me forgetting about this), so let's try re-reverting with the next train (Wed/Thu next week).

Excellent. Much appreciated. Let's see what breaks.....

This change broke KrinkleBot on Wikimedia Commons. File protection synchronisation was down for 13 hours. I last re-installed KrinkleBot 3 months ago in July. Using the then-latest version of Pywikibot core. It ran mostly uninterrupted – until yesterday. It started failing at 2016-10-12 19:17:10.

[..]
Sleeping for 9.5 seconds, 2016-10-12 11:01:52
Page [[Commons:Auto-protected files/wikipedia/fr]] saved

WARNING: API error maxlag: Waiting for 10.64.48.150: 6.2437400817871 seconds lagged
Traceback (most recent call last): [..]
CRITICAL: Closing network session.

Logging in to commons:commons as KrinkleBot@Autoprotect
Sleeping for 7.4 seconds, 2016-10-12 11:16:09
Page [[Commons:Auto-protected files/wikipedia/de]] saved
[..]
Sleeping for 9.3 seconds, 2016-10-12 19:17:10
Page [[Commons:Auto-protected files/wikipedia/bn]] saved
Logging in to commons:commons as KrinkleBot@Autoprotect

Traceback (most recent call last): [..]
    if loginMan.login(retry=True):
    cookiedata = self.getCookie()
    prefix = login_result['login']['cookieprefix']
KeyError: u'cookieprefix'
CRITICAL: Closing network session.

In attempt to remedy this, I updated the copy of pywikibot-core from Git and re-ran python setup.py. The next run failed with a different error:

    token = self.tokens['edit']
    assert self.site.user(), 'User must login in this site'
AssertionError: User must login in this site
CRITICAL: Closing network session.

I suspected this may be due to a breaking in bot passwords handling. The following configuration used to work:

("Username@Botname", "<bot password>")

import os
family = 'commons'
mylang = 'commons'
usernames['commons']['commons'] = u'Username@Botname'
sysopnames['commons']['commons'] = u'Username@Botname'
password_file = os.path.expanduser('~/.pywikibot/.pwd')

Per https://www.mediawiki.org/wiki/Manual:Pywikibot/BotPasswords I changed this to use plain "Username" and create a BotPassword() object in the password file.

("Username", BotPassword("Botname", "<bot password>"))

usernames['commons']['commons'] = u'Username'
sysopnames['commons']['commons'] = u'Username'

This fixed it.

Same problem with my bot this morning on wikipedia fr
Thanks to you guys for the fixes you already brought above.
My main problem is fixed now.

I am still having somes issues:

• connection with Username@Botname is logged as Username on Wikipedia (no 'Bot tag' applied on my commits).
• seems that the session login does not hold as it did before (I have to reenter my password for every pywikibot command) (I'll try the fix suggested by Krinkle tonight to see if it fixes this second issue). (update: 10/17/2016: it does fix this issue, providing the password_file= line in the user-config.php file).

cheers
Cernunnos

In T142155#2716184, @Gallicbot wrote:

• connection with Username@Botname is logged as Username on Wikipedia (no 'Bot tag' applied on my commits).

I'm not sure which one you mean:

• no bot flag on your edits? Make sure your application has the highvolume grant (and of course the user account needs to have the bot flag).
• no change tag on your edits? Yeah, bot passwords don't do that (but then neither do "traditional" bots).
• no change to your username (such as the @Botname part displayed)? Bot passwords don't do that either. (Feel free to submit feature requests though if you think it would be helpful!) They are just a more stable way to log in, nothing more.

Hello Tgr & thanks for your answer:

yes point 1: my bot account has the bot flag but not my user account, indeed.
as far as I know this is still the standard policy on Wikipedia fr..

What is the actual status of the pywikibot toolkit ? do you plan to fix the login commands ?
I don't know how to commit using a bot flag since the only way to connect is still the one described by Krinkle ?

thanks!!

@Gallicbot I'm a MediaWiki developer, so I can't really comment on the Pywikibot side, but I don't see what problem still needs to be fixed here. If frwiki requires you to have separate bot and personal accounts (which is probably the sane way for handling bot flags), just use Pywikibot as before, but replace the password to your bot account with a bot password to your bot account. We never intended to support a use case where you give the bot a bot password for your real account and then it somehow ends up editing with your bot account.

Work nothing: login, Oauth, Botpassword (case of Krinkle do: 'NameError: name 'BotPassword' is not defined'). :(
I don't understand, there the common login has broken at past week and it temporary bug? Or it forever, now only Oauth, which dont work?

@Vladis13 are you using an up-to-date version of Pywikibot? Old versions (which expected cookieprefix to be present in the login response) are not going to work anymore.

@Tgr yes, on https://www.mediawiki.org/wiki/Manual:Pywikibot/Installation again was not updated version number 2.0rc4. So I thought that my version Pwb is actual.

I make Oauth, but Pwb still ask password. I tried make Oauth from both accounts - user and bot, then write string in config, according manual, like:
authenticate['*.wikipedia.org'] = ('1920cc7b7.....','2be514c9....', 'd7cb99b0.....','6dd4a2206171......').

Note that in https://www.mediawiki.org/wiki/Manual:Pywikibot/login.py#Login_using_OAuth write Run python login.py -oauth, but that command not work.

Also, I tried BotPassword. I write in passwords.txt like:
("Accounname", BotPassword("Accounname", "sl2298i...."))
But it make the error:

Traceback (most recent call last):
  File "pwb.py", line 270, in <module>
    if not main():
  File "pwb.py", line 264, in main
    run_python_file(filename, [filename] + args, argvu, file_package)
  File "pwb.py", line 109, in run_python_file
    main_mod.__dict__)
  File ".\scripts\login.py", line 129, in <module>
    main()
  File ".\scripts\login.py", line 114, in main
    site.login(sysop)
  File "C:\pwb\pywikibot\site.py", line 1683, in login
    user=self._username[sysop])
  File "C:\pwb\pywikibot\tools\__init__.py", line 1105, in wrapper
    return obj(*__args, **__kw)
  File "C:\pwb\pywikibot\login.py", line 106, in __init__
    self.readPassword()
  File "C:\pwb\pywikibot\login.py", line 195, in readPassword
    entry = eval(line)
  File "<string>", line 1, in <module>
NameError: name 'BotPassword' is not defined
<class 'NameError'>
CRITICAL: Waiting for 1 network thread(s) to finish. Press ctrl-c to abort

I don't think the BotPasswords patch is part of the 2.0rc5 release (since it was released two days before that patch).

I'm actually don't think it is in the 2.0 branch at all (where login.py hasn't been changed since 2015).

Hi, I still have the error above with the latest version of pywikibot in PyPi.
https://pypi.python.org/pypi/pywikibot
When logging in (from wmflabs, with a bot account), I get the KeyError exception. Applying the hack above does not solve the problem, because I then get another exception:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "main.py", line 541, in test_run
    site.login()
  File "/data/project/oabot/www/python/venv/local/lib/python2.7/site-packages/pywikibot-2.0rc3-py2.7.egg/pywikibot/site.py", line 1684, in login
    if loginMan.login(retry=True):
  File "/data/project/oabot/www/python/venv/local/lib/python2.7/site-packages/pywikibot-2.0rc3-py2.7.egg/pywikibot/login.py", line 239, in login
    cookiedata = self.getCookie()
  File "/data/project/oabot/www/python/venv/local/lib/python2.7/site-packages/pywikibot-2.0rc3-py2.7.egg/pywikibot/data/api.py", line 2563, in getCookie
    login_result['login']['lg' + key.lower()]))
KeyError: u'lgtoken'

It seems to me that this is a critical issue! Please solve this as soon as possible!

If you need a temporary workaround you can install the developers branch of pywikiboy with pip install git+https://github.com/wikimedia/pywikibot-core.git the version you should have after that is 3.0-dev.

The reason why I call it a temporary fix is that that is in general and unstable build so things might break in the future.

Lokal_Profil: thanks, yes indeed that's what I did. I hope someone will be able to make a new release on PyPi soon.

In T142155#2783466, @Pintoch wrote:

Lokal_Profil: thanks, yes indeed that's what I did. I hope someone will be able to make a new release on PyPi soon.

At the very least a new one will be needed due to T150210 to ensure we don't add incorrect data to Wikidata. Ideally the login changes can be included in that one.

Hello,

When i login with compat Scripts i get Error:
Traceback (most recent call last):

File "login.py", line 480, in <module>

main()

File "login.py", line 475, in main

loginMan.login()

File "login.py", line 349, in login

cookiedata = self.getCookie(api)

File "login.py", line 263, in getCookie

self.site._token[index] = data['login']['lgtoken'] + "+\\"

KeyError: 'lgtoken'

is there any Solution?

@Jar unfortunately Pywikibot-compat is in the phase of becoming deprecated (see: T101214). There is not so much development in old compat branch (but You still welcome to report You issue there), but if I may suggest something - it would solve You a lot of issues, if You will migrate to Pywikibot.

@Magul is Pywikibot-core only use Python 3.x or there is some Scripts using 2.7?

@Jar Pywikibot should work with Python 2.6, 2.7, 3.3, 3.4 and 3.5. If You will find any issue with whatever version mentioned before, feel free to report an issue here.

@Multichill: I reopened it because the issue is still not solved in the latest version of the library on PyPI. Please just do a new release!

In T142155#2911377, @Pintoch wrote:

@Multichill: I reopened it because the issue is still not solved in the latest version of the library on PyPI. Please just do a new release!

Thank you for filing T154445: Outdated and misconfigured pywikibot PyPI package.