| revi | |
| Sep 23 2019, 10:52 AM |
| F31749530: image.png | |
| Apr 10 2020, 7:44 PM |
Here's the log. I am an admin on ko.wikinews (scroll it bit down), so I have no idea why pywikibot thinks I don't hold the ultimate authority? (Well, tbh, I should never see any of these errors on wikimedia wikis because of my hat.)
pywikibot-prod revi$ git show commit 3df33cb77efd037a561223f1599885ccce59f48b (HEAD -> master, origin/master, origin/HEAD) pywikibot-prod revi$ python3 --version Python 3.7.4
Originally posted on https://www.mediawiki.org/wiki/Topic:V7uz6t27614c6k9o
@revi: Can you please provide an excerpt of the code you are using (or if it is the vanilla delete.py, how you are calling it)?
It is vanilla delete.py within the scripts/ directory in gerrit.wikimedia.org/r/p/pywikibot/core.
I guess this this that listed above:
$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"
Got it. I can confirm that I can replicate it. My suspicion is that the -user parameter is causing it somehow. Investigating.
Change 538724 had a related patch set uploaded (by Huji; owner: Huji):
[pywikibot/core@master] Check user rights using a unicode string
I was wrong about the cause, but now I know the cause. Here is the line of code that causes it to happen: https://github.com/wikimedia/pywikibot/blob/master/pywikibot/page.py#L1922
The logic if 'delete' in self.site.userinfo['rights']: fails because self.site.userinfo['rights'] is a list of unicode strings (e.g. [ ..., u'delete', ...]) and as far as python 2.x is concerned, 'delete' !== u'delete'.
@revi: try checking out the patch that I submitted above, and please confirm that it works for you.
@Xqt: I don't think adding the u prefix is the right solution. I'm not sure how come userinfo[] contains unicode strings here (but when we use it in site.py we don't have this problem). What do you think is the right solution?
@Huji: I think it would be important see the userinfo content at this state. Using pprint is a good tool for showing dicts:
from pprint import pprint
pprint(self.site.userinfo)
Probably it isn’t filled.
@revi: can you please add these two lines in front of line 1922 inside page.py (the line Huji marked above)?
@revi: You aren't log in and working anonymously. Does it work if you are login first?
My siteinfo looks like this (for xqbot)
C:\pwb\GIT\core>pwb delete -page:user:xqt/Test -simulate
C:\pwb\GIT\core\pywikibot\config2.py:1138: _ConfigurationDeprecationWarning: "sy
sopnames" present in our user-config.py is no longer a supported configuration v
ariable. Please inform the maintainers if you depend on it.
'depend on it.'.format(_key), _ConfigurationDeprecationWarning)
Enter a reason for the deletion: No valid reason
>>> Benutzer:Xqt/Test <<<
WARNING: There are 12 pages that link to [[de:Benutzer:Xqt/Test]].
User: [2] 1 page
User talk: [3] 7 pages
Project: [4] 1 page
Project talk: [5] 3 pages
{'groups': ['bot', '*', 'user', 'autoconfirmed'],
'id': 627628,
'name': 'Xqbot',
'rights': ['editeditorprotected',
'noratelimit',
'bot',
'autoconfirmed',
'editsemiprotected',
'nominornewtalk',
'autopatrol',
'suppressredirect',
'apihighlimits',
'writeapi',
'autoreview',
'skipcaptcha',
'patrolmarks',
'createaccount',
'read',
'edit',
'createpage',
'createtalk',
'viewmywatchlist',
'editmywatchlist',
'viewmyprivateinfo',
'editmyprivateinfo',
'editmyoptions',
'urlshortener-create-url',
'centralauth-merge',
'abusefilter-view',
'abusefilter-log',
'vipsscaler-test',
'reupload-own',
'move-rootuserpages',
'move-categorypages',
'minoredit',
'editmyusercss',
'editmyuserjson',
'editmyuserjs',
'purge',
'sendemail',
'applychangetags',
'changetags',
'spamblacklistlog',
'mwoauthmanagemygrants',
'reupload',
'upload',
'move',
'collectionsaveasuserpage',
'collectionsaveascommunitypage',
'abusefilter-log-detail',
'transcode-reset']}
Can't delete [[de:Benutzer:Xqt/Test]]; do you want to mark it for deletion
instead? ([y]es, [N]o, [a]ll):Unfortunately the -user option doesn't work anymore:
C:\pwb\GIT\core>pwb delete -page:user:xqt/Test -simulate -user:xqt
C:\pwb\GIT\core\pywikibot\config2.py:1138: _ConfigurationDeprecationWarning: "sy
sopnames" present in our user-config.py is no longer a supported configuration v
ariable. Please inform the maintainers if you depend on it.
'depend on it.'.format(_key), _ConfigurationDeprecationWarning)
Enter a reason for the deletion: No valid deletion
>>> Benutzer:Xqt/Test <<<
WARNING: There are 12 pages that link to [[de:Benutzer:Xqt/Test]].
User: [2] 1 page
User talk: [3] 7 pages
Project: [4] 1 page
Project talk: [5] 3 pages
{'groups': ['bot', '*', 'user', 'autoconfirmed'],
'id': 627628,
'name': 'Xqbot',
'rights': ['editeditorprotected',
'noratelimit',
'bot',python3 dev/wikimedia/pywikibot/pywikibot-prod/pwb.py login -lang:ko -family:wikinews Logging in to wikinews:ko as -revi@Pywikibot Logged in on wikinews:ko as -revi.
Very strange: You are logged in successfully but you aren't obviously during running the script.
@revi can you share with us the relevant portions of your user-config.py (after redacting passwords and such) please? I want to know how you are using usernames[] to define your main bot account, any additional bot accounts, and the passwords for them, so I can replicate your setup as closely as possible.
I use one directory for -revi, and another directory for Revibot. Revibot doesn't hold any adminships so it cannot be used to replicate anything, so I didn't include it here.
I modified my config to use BotPaswords just like you, and then I forced my bot to log out (python3 pwb.py login -logout) and log back in. Then I ran the script; both with and without the -user argument, it would give me the "Can't delete" message, meaning that it forgot who it was logged in as.
This issue did not occur, however, if I was using a password-based configuration as opposed to one based on BotPasswords). And it happens with or without the -user argument. I am investigating it more, but just to be sure, can you also give it a try with a password-based configuration and confirm that things work that way?
@revi in addition to my request above, can you also checkout the 3rd version of that patch set and confirm that it works?
@Xqt I am surprised that when using BotPasswords, session information is lost. The patch above is not a solution; we need to figure out the root cause and fix that. Any thoughts?
I have not investigated in a systematic way but found some weird issues: I cannot use -user option for my account anymore but I can use the bot on sites even the name is not listed inside user-config.py. Two points to be digged deeper. And this listed problem too. I am on holidays currently and cannot help a lot. Btw I am not using botpasswords for my bot an I am using 2FA for my account. I guess we have to make a look into the password/cookie files behaviour.
I cannot use password login because I am required to use 2FA (mandated by WMF Security's policy for super advanced rights (like Stewards and/or Interface-Admins, which both applies to me)) and with it, I don't think password-based login would work.
I'll test the patch tomorrow: I don't have access to computer right now.
$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]" >>> 인도전문가의 인도사업비결 <<< WARNING: ./scripts/delete.py:217: DeprecationWarning: pywikibot.bot.MultipleSitesBot.site is deprecated for 4 years and 3 months; use the page's site property instead. if self.site.user() is None: WARNING: ./scripts/delete.py:218: DeprecationWarning: pywikibot.bot.MultipleSitesBot.site is deprecated for 4 years and 3 months; use the page's site property instead. self.site.login() Logging in to wikinews:ko as -revi@Pywikibot Do you want to delete [[ko:인도전문가의 인도사업비결]]? ([y]es, [N]o, [a]ll, [q]uit): ^C User quit DeletionRobot bot run... 0 pages read 0 pages written 0 pages skipped Execution time: 12 seconds Script terminated successfully.
Great. I will investigate more to see why BotPasswords doesn't retain the session info, but if it takes too long, we can merge the patch as is (it won't cause an extra login if session info is retained).
This could be one of the issues of token implementation as some token issues still remain unresolved
I'm having a similar issue with movepages.py. Each time I run the command, the bot will move one page and then say it isn't authorized to move.
@jhsoby from IRC: it works almost exactly 25 % of the time: https://hi.wikisource.org/wiki/%E0%A4%B5%E0%A4%BF%E0%A4%B6%E0%A5%87%E0%A4%B7:%E0%A4%B2%E0%A5%89%E0%A4%97/move
(the numbers in the moved pages there *should* be sequential)
I wrote a quick and ugly script that calls os.system with the commands to perform the moves one at a time. That's the script that works 25 % of the time. The rest of the time it gives the same error as the one I pasted above.
@Xqt: I think we should merge https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/538724/ for now to get the code into a working state at least. I know the Technical-Debt will remain, but I cannot afford a revamp of our login infrastructure for BotPasswords until at least December.
Change 538724 merged by jenkins-bot:
[pywikibot/core@master] Make sure the user is logged in before attempting a delete
Hi all, sorry to reopen this, but I think that problem I encountered is related.
I am using delete.py under PAWS. I applied the patch above, but I obtain the same result: not being able to "undelete" files in a project where I am sysop.
@PAWS:~$ pwb.py login -family:commons -lang:commons
Logged in on commons:commons as Ruthven.@PAWS:~$ pwb.py scripts/delete -undelete -file:ff -summary:"OTRS [[ticket:2019121410003163]]" -lang:commons -family:commons
File:Luigi Martignon 1969 - Jole.jpg <<<
0 pages read
0 pages written
Execution time: 0 seconds
Script terminated by exception:ERROR: NoUsername: No sysop is permitted with OAuth
Traceback (most recent call last):File "/srv/paws/pwb/pwb.py", line 250, in <module> if not main(): File "/srv/paws/pwb/pwb.py", line 243, in main run_python_file(filename, [filename] + args, argvu, file_package) File "/srv/paws/pwb/pwb.py", line 95, in run_python_file main_mod.__dict__) File "scripts/delete.py", line 307, in <module> main() File "scripts/delete.py", line 299, in main bot.run() File "/srv/paws/pwb/pywikibot/bot.py", line 1702, in run super(MultipleSitesBot, self).run() File "/srv/paws/pwb/pywikibot/bot.py", line 1505, in run self.treat(page) File "/srv/paws/pwb/pywikibot/bot.py", line 1732, in treat self.treat_page() File "scripts/delete.py", line 197, in treat_page self.current_page.undelete(self.summary) File "/srv/paws/pwb/pywikibot/tools/__init__.py", line 1738, in wrapper return obj(*__args, **__kw) File "/srv/paws/pwb/pywikibot/page.py", line 2030, in undelete self.site.undelete_page(self, reason, undelete_revs) File "/srv/paws/pwb/pywikibot/site.py", line 1317, in callee self.login(True) File "/srv/paws/pwb/pywikibot/site.py", line 2080, in login raise NoUsername('No sysop is permitted with OAuth')pywikibot.exceptions.NoUsername: No sysop is permitted with OAuth
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.NoUsername'>
Hm this is not the current stable release provided with paws. This is strange because paws should always use the last stable release. Is this a long term session you are using?
This does not seem to be the underlying issue here, but PAWS will not have admin permissions. T192237
In this sense the newly reopened task is invalid because the bot says "no sysop is permitted with OAuth" and as you said paws does not support sysops anyway.
Independently of that we removed sysop login with the last stable release which should be that released loaded with paws but with the given traceback I found out this is different.
Change 565624 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [IMPR] Replace @must_be with @need_right
Change 565624 merged by jenkins-bot:
[pywikibot/core@master] [IMPR] Replace @must_be with @need_right
It seems it is indeed still happening.
User list / Group: Administrators
However...
$ python3 pwb.py redirect broken -delete -lang:ko -family:wikiversity Logging in to wikiversity:ko as -revi@Pywikibot Retrieving broken redirect special page... Retrieving 6 pages from wikiversity:ko. >>> 사용자:Hosiryuhosi <<< Redirect target [[사용자:Rxy]] does not exist. Do you want to delete [[사용자:Hosiryuhosi]]? ([y]es, [N]o, [a]ll, [q]uit): y No sysop in user-config.py, put page to speedy deletion. @@ -0,0 +1 @@ + {{삭제}} Edit summary: 봇: 끊긴 넘겨주기 Do you want to accept these changes? ([y]es, [N]o, [a]ll, [q]uit): q User quit RedirectRobot bot run... 0 pages read 0 pages written Execution time: 4 seconds Script terminated successfully. $ git show commit d535e54980fecf5a3fdb1eb2f018659eb24c70d0 (HEAD -> stable, tag: stable, tag: 3.0.20190301) Author: xqt <info@gno.de> Date: Fri Mar 1 10:42:00 2019 +0100 [L10N] Update languages_by_size Change-Id: I95e02346c02a0d4db55954e80a6c4c2af3d99903
@revi You should update Pywikibot, it seems you have terribly outdated version (March 2019).
Yeah, I was in the wrong branch/tag, thanks to naive (and unverified) thinking that there would be origin/stable branch. Git went to 2019 edition. My bad.
$ git show commit 2cf67392f6b9855dd789bad2f89e2be2aa99456f (HEAD -> localstable, tag: 3.0.20200405) Merge: d14771f0 5afa1a85 Author: jenkins-bot <jenkins-bot@gerrit.wikimedia.org> Date: Sun Apr 5 16:57:37 2020 +0000 Merge "[doc] Update HISTORY.rst"
(it is just tags/stable, which is updated on every new release, but you have to fetch-update tags as well)
Version info:
$ git show commit 3a648ccd4678808ea151154cb158252dd77ea133 (HEAD -> master, origin/master, origin/HEAD) Author: xqt <info@gno.de> Date: Wed Apr 15 10:22:24 2020 +0200 [bugfix] Add missing write actions to write actions tuple - Add missing write actions from https://www.mediawiki.org/wiki/API:Changing_wiki_content - sort the write actions tuple Change-Id: Ibadd1456dc8b1525ea1f0465a837e7cfe4634bdc
Originally tried on...
$ git show commit 2cf67392f6b9855dd789bad2f89e2be2aa99456f (HEAD -> localstable, tag: 3.0.20200405) Merge: d14771f0 5afa1a85 Author: jenkins-bot <jenkins-bot@gerrit.wikimedia.org> Date: Sun Apr 5 16:57:37 2020 +0000 Merge "[doc] Update HISTORY.rst"
(For the record: I don't do PAWS for a sensitive rights such as delete.py — especially that I was a steward at that moment I preferred doing almost everything on my device — but yeah can't repro now)