Page MenuHomePhabricator

cannot use delete.py despite being admin on a wiki
Open, HighPublicBUG REPORT

Description

Here's the log. I am an admin on ko.wikinews (scroll it bit down), so I have no idea why pywikibot thinks I don't hold the ultimate authority? (Well, tbh, I should never see any of these errors on wikimedia wikis because of my hat.)

pywikibot-prod revi$ git show
commit 3df33cb77efd037a561223f1599885ccce59f48b (HEAD -> master, origin/master, origin/HEAD)
pywikibot-prod revi$ python3 --version
Python 3.7.4

Originally posted on https://www.mediawiki.org/wiki/Topic:V7uz6t27614c6k9o

Details

Related Gerrit Patches:

Event Timeline

revi created this task.Sep 23 2019, 10:52 AM
Restricted Application added subscribers: pywikibot-bugs-list, Aklapper. · View Herald TranscriptSep 23 2019, 10:52 AM
revi moved this task from Incoming to Radar on the User-revi board.Sep 23 2019, 10:52 AM
revi added a comment.Sep 23 2019, 10:57 AM

When I commented out 'sysopnames' from the user-config.py... It still fails.

1$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"
2
3
4
5>>> 인도전문가의 인도사업비결 <<<
6Can't delete [[ko:인도전문가의 인도사업비결]]; do you want to mark it for deletion
7instead? ([y]es, [N]o, [a]ll):
8
9>>> 동아에스티 개발 당뇨병치료제 인도에서 발매 <<<
10Can't delete [[ko:동아에스티 개발 당뇨병치료제 인도에서 발매]]; do you want to mark it for
11deletion instead? ([y]es, [N]o, [a]ll): ^C
12User quit DeletionRobot bot run...
13
141 pages read
150 pages written
160 pages skipped
17Execution time: 6 seconds
18Read operation time: 6 seconds
19Script terminated successfully.

Xqt triaged this task as High priority.Sep 23 2019, 11:07 AM
Xqt changed the subtype of this task from "Task" to "Bug Report".
Xqt added a subscriber: Huji.
Huji added a comment.Sep 23 2019, 11:16 AM

@revi: Can you please provide an excerpt of the code you are using (or if it is the vanilla delete.py, how you are calling it)?

revi added a comment.Sep 23 2019, 11:18 AM

It is vanilla delete.py within the scripts/ directory in gerrit.wikimedia.org/r/p/pywikibot/core.

Xqt added a subscriber: Xqt.Sep 23 2019, 11:18 AM

@revi: Can you please provide an excerpt of the code you are using (or if it is the vanilla delete.py, how you are calling it)?

I guess this this that listed above:
$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"

Huji claimed this task.Sep 24 2019, 12:25 AM

Got it. I can confirm that I can replicate it. My suspicion is that the -user parameter is causing it somehow. Investigating.

Change 538724 had a related patch set uploaded (by Huji; owner: Huji):
[pywikibot/core@master] Check user rights using a unicode string

https://gerrit.wikimedia.org/r/538724

Huji added a comment.EditedSep 24 2019, 12:44 AM

I was wrong about the cause, but now I know the cause. Here is the line of code that causes it to happen: https://github.com/wikimedia/pywikibot/blob/master/pywikibot/page.py#L1922

The logic if 'delete' in self.site.userinfo['rights']: fails because self.site.userinfo['rights'] is a list of unicode strings (e.g. [ ..., u'delete', ...]) and as far as python 2.x is concerned, 'delete' !== u'delete'.

@revi: try checking out the patch that I submitted above, and please confirm that it works for you.

@Xqt: I don't think adding the u prefix is the right solution. I'm not sure how come userinfo[] contains unicode strings here (but when we use it in site.py we don't have this problem). What do you think is the right solution?

revi added a comment.Sep 24 2019, 1:23 AM

Huji, see below.

1$ git pull "https://gerrit.wikimedia.org/r/pywikibot/core" refs/changes/24/538724/2
2remote: Counting objects: 449, done
3remote: Finding sources: 100% (12/12)
4remote: Total 12 (delta 9), reused 11 (delta 9)
5Unpacking objects: 100% (12/12), done.
6From https://gerrit.wikimedia.org/r/pywikibot/core
7 * branch refs/changes/24/538724/2 -> FETCH_HEAD
8Updating 3df33cb7..d2ef2f2e
9Fast-forward
10 pywikibot/page.py | 2 +-
11 pywikibot/site.py | 3 ++-
12 tests/site_tests.py | 28 ++++++++++++++--------------
13 3 files changed, 17 insertions(+), 16 deletions(-)
14$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"
15
16
17>>> 인도전문가의 인도사업비결 <<<
18Can't delete [[ko:인도전문가의 인도사업비결]]; do you want to mark it for deletion
19instead? ([y]es, [N]o, [a]ll): ^C
20User quit DeletionRobot bot run...
21
220 pages read
230 pages written
240 pages skipped
25Execution time: 9 seconds
26Script terminated successfully.
27$

Xqt added a comment.EditedSep 24 2019, 2:30 AM

@Huji: I think it would be important see the userinfo content at this state. Using pprint is a good tool for showing dicts:

from pprint import pprint
pprint(self.site.userinfo)

Probably it isn’t filled.

@revi: can you please add these two lines in front of line 1922 inside page.py (the line Huji marked above)?

Xqt added a comment.Sep 24 2019, 3:05 AM

Does it has to do something with T122705?

revi added a comment.Sep 24 2019, 3:18 AM

I don't think so,

1{redacted}-MacBook-Pro:pywikibot-prod {redacted}$ python3 pwb.py delete -v -debug -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"
2Python 3.7.4 (default, Sep 7 2019, 18:27:02)
3[Clang 10.0.1 (clang-1001.0.46.4)]
4handle_args() completed.
5Instantiated APISite object 'wikinews:ko'
6general siteinfo not loaded yet.
7general siteinfo not loaded yet.
8Load siteinfo properties 'namespaces', 'namespacealiases' along with 'general'
9Load siteinfo properties 'namespaces', 'namespacealiases' along with 'general'
10CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/2a76ff385cf98a4155129bdaa66eb952872107b3abb44f2727af13e2f3be8ea6) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'query'), ('continue', ''), ('format', 'json'), ('maxlag', '5'), ('meta', 'siteinfo|userinfo'), ('siprop', 'namespaces|namespacealiases|general'), ('uiprop', 'blockinfo|hasmsg')]
11CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/150466902097904a7865c31500f049a2b96020fc2213fc0aad0aaf75a956662e) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'paraminfo'), ('format', 'json'), ('maxlag', '5'), ('modules', 'main|paraminfo|query')]
12CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/53bdbc74cf43b9697ed5601d7c3e1e0730f2589c932d6f422faf443a41a5fcbb) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'paraminfo'), ('format', 'json'), ('maxlag', '5'), ('modules', 'query+info')]
13CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/2de2bf841b266132e4d458d715e1ed7af90ef8736b366452ca5946e9fc4b2a0d) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'query'), ('continue', ''), ('format', 'json'), ('maxlag', '5'), ('meta', 'siteinfo|userinfo'), ('siprop', 'extensions'), ('uiprop', 'blockinfo|hasmsg')]
14Checking multiplicity: pid = False
15Checking multiplicity: pid = False
16Found 1 wikinews:ko processes running, including this one.
17API request to wikinews:ko (uses get: True):
18Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
19URI: '/w/api.php?titles=%EC%9D%B8%EB%8F%84%EC%A0%84%EB%AC%B8%EA%B0%80%EC%9D%98+%EC%9D%B8%EB%8F%84%EC%82%AC%EC%97%85%EB%B9%84%EA%B2%B0&inprop=protection&prop=info&action=query&indexpageids=&continue=&meta=userinfo&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'
20Body: None
21API response received from wikinews:ko:
22{"batchcomplete":"","query":{"pageids":["4210"],"pages":{"4210":{"pageid":4210,"ns":0,"title":"\uc778\ub3c4\uc804\ubb38\uac00\uc758 \uc778\ub3c4\uc0ac\uc5c5\ube44\uacb0","contentmodel":"wikitext","pagelanguage":"ko","pagelanguagehtmlcode":"ko","pagelanguagedir":"ltr","touched":"2019-03-18T12:24:11Z","lastrevid":24194,"length":809,"new":"","protection":[],"restrictiontypes":["edit","move"]}},"userinfo":{"id":0,"name":"{redacted IP}","anon":""}}}
23PropertyGenerator received ['4210']; limit=None
24Working on '인도전문가의 인도사업비결'
25
26
27>>> 인도전문가의 인도사업비결 <<<
28CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/19277eba31e77d8e355f8a5c862b916ae750fae658fc1245556cccda5faca263) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'paraminfo'), ('format', 'json'), ('maxlag', '5'), ('modules', 'query+backlinks')]
29PageGenerator: Set query_limit to 500.
30PageGenerator: Set query_limit to 500.
31PageGenerator: query_limit: 500, api_limit: 500, limit: None, new_limit: 500, count: 0
32PageGenerator: gbllimit: ['500']
33CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/c8a8b73f5960988226dca44fe5b8385fd6cec75d603ad67fd86ae6bd06400d92) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'paraminfo'), ('format', 'json'), ('maxlag', '5'), ('modules', 'query+categoryinfo|query+imageinfo')]
34API request to wikinews:ko (uses get: False):
35Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
36URI: '/w/api.php'
37Body: 'gbltitle=%EC%9D%B8%EB%8F%84%EC%A0%84%EB%AC%B8%EA%B0%80%EC%9D%98+%EC%9D%B8%EB%8F%84%EC%82%AC%EC%97%85%EB%B9%84%EA%B2%B0&gblfilterredir=redirects&prop=info%7Cimageinfo%7Ccategoryinfo&inprop=protection&iiprop=timestamp%7Cuser%7Ccomment%7Curl%7Csize%7Csha1%7Cmetadata&iilimit=max&generator=backlinks&action=query&indexpageids=&continue=&gbllimit=500&meta=userinfo&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'
38API response received from wikinews:ko:
39{"batchcomplete":"","limits":{"imageinfo":500},"query":{"userinfo":{"id":0,"name":"{redacted IP}","anon":""}}}
40CachedRequest: cache hit (/Users/{redacted}/dev/wikimedia/pywikibot/pywikibot-prod/apicache-py3/53459dd4e8f0a69877c8dc5d061f4374637f697fea2b5d715ea2e2b6ec9a68dc) for API request: APISite("ko", "wikinews")LoginStatus(NOT_LOGGED_IN)[('action', 'paraminfo'), ('format', 'json'), ('maxlag', '5'), ('modules', 'query+embeddedin')]
41PageGenerator: Set query_limit to 500.
42PageGenerator: query_limit: 500, api_limit: 500, limit: None, new_limit: 500, count: 0
43PageGenerator: gbllimit: ['500']
44API request to wikinews:ko (uses get: False):
45Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
46URI: '/w/api.php'
47Body: 'gbltitle=%EC%9D%B8%EB%8F%84%EC%A0%84%EB%AC%B8%EA%B0%80%EC%9D%98+%EC%9D%B8%EB%8F%84%EC%82%AC%EC%97%85%EB%B9%84%EA%B2%B0&gblfilterredir=nonredirects&prop=info%7Cimageinfo%7Ccategoryinfo&inprop=protection&iiprop=timestamp%7Cuser%7Ccomment%7Curl%7Csize%7Csha1%7Cmetadata&iilimit=max&generator=backlinks&action=query&indexpageids=&continue=&gbllimit=500&meta=userinfo&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'
48API response received from wikinews:ko:
49{"batchcomplete":"","limits":{"imageinfo":500},"query":{"userinfo":{"id":0,"name":"{redacted IP}","anon":""}}}
50PageGenerator: query_limit: 500, api_limit: 500, limit: None, new_limit: 500, count: 0
51PageGenerator: geilimit: ['500']
52API request to wikinews:ko (uses get: False):
53Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
54URI: '/w/api.php'
55Body: 'geititle=%EC%9D%B8%EB%8F%84%EC%A0%84%EB%AC%B8%EA%B0%80%EC%9D%98+%EC%9D%B8%EB%8F%84%EC%82%AC%EC%97%85%EB%B9%84%EA%B2%B0&geifilterredir=nonredirects&prop=info%7Cimageinfo%7Ccategoryinfo&inprop=protection&iiprop=timestamp%7Cuser%7Ccomment%7Curl%7Csize%7Csha1%7Cmetadata&iilimit=max&generator=embeddedin&action=query&indexpageids=&continue=&geilimit=500&meta=userinfo&uiprop=blockinfo%7Chasmsg&maxlag=5&format=json'
56API response received from wikinews:ko:
57{"batchcomplete":"","limits":{"imageinfo":500},"query":{"userinfo":{"id":0,"name":"{redacted IP}","anon":""}}}
58API request to wikinews:ko (uses get: True):
59Headers: {'Content-Type': 'application/x-www-form-urlencoded'}
60URI: '/w/api.php?action=query&meta=userinfo&uiprop=blockinfo%7Cgroups%7Chasmsg%7Crights&rawcontinue=&maxlag=5&format=json'
61Body: None
62API response received from wikinews:ko:
63{"query":{"userinfo":{"id":0,"name":"{redacted IP}","anon":"","groups":["*"],"rights":["createaccount","read","edit","createpage","createtalk","writeapi","viewmywatchlist","editmywatchlist","viewmyprivateinfo","editmyprivateinfo","editmyoptions","urlshortener-create-url","centralauth-merge","abusefilter-view","abusefilter-log","vipsscaler-test"]}}}
64Can't delete [[ko:인도전문가의 인도사업비결]]; do you want to mark it for deletion
65instead? ([y]es, [N]o, [a]ll): ^C
66User quit DeletionRobot bot run...
67
680 pages read
690 pages written
700 pages skipped
71Execution time: 33 seconds
72Script terminated successfully.
73_flush() called
74Dropped throttle(s).
75Closing network session.
76Network session closed.
77{redacted}-MacBook-Pro:pywikibot-prod {redacted}$

1{redacted}-MacBook-Pro:pywikibot-prod {redacted}$ git pull "https://gerrit.wikimedia.org/r/pywikibot/core" refs/changes/24/538724/2
2From https://gerrit.wikimedia.org/r/pywikibot/core
3 * branch refs/changes/24/538724/2 -> FETCH_HEAD
4Updating 3df33cb7..d2ef2f2e
5Fast-forward
6 pywikibot/page.py | 2 +-
7 pywikibot/site.py | 3 ++-
8 tests/site_tests.py | 28 ++++++++++++++--------------
9 3 files changed, 17 insertions(+), 16 deletions(-)
10{redacted}-MacBook-Pro:pywikibot-prod {redacted}$ code pywikibot/page.py
11{redacted}-MacBook-Pro:pywikibot-prod {redacted}$ git diff
12diff --git a/pywikibot/page.py b/pywikibot/page.py
13index 0f625570..9a2fb49d 100644
14--- a/pywikibot/page.py
15+++ b/pywikibot/page.py
16@@ -1919,6 +1919,9 @@ class BasePage(UnicodeMixin, ComparableMixin):
17 reason = pywikibot.input('Please enter a reason for the deletion:')
18
19 # If user has 'delete' right, delete the page
20+ from pprint import pprint
21+ pprint(self.site.userinfo)
22+
23 if u'delete' in self.site.userinfo['rights']:
24 answer = 'y'
25 if prompt and not hasattr(self.site, '_noDeletePrompt'):
26{redacted}-MacBook-Pro:pywikibot-prod {redacted}$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"
27
28
29>>> 인도전문가의 인도사업비결 <<<
30{'anon': '',
31 'groups': ['*'],
32 'id': 0,
33 'name': '{redacted IP}',
34 'rights': ['createaccount',
35 'read',
36 'edit',
37 'createpage',
38 'createtalk',
39 'writeapi',
40 'viewmywatchlist',
41 'editmywatchlist',
42 'viewmyprivateinfo',
43 'editmyprivateinfo',
44 'editmyoptions',
45 'urlshortener-create-url',
46 'centralauth-merge',
47 'abusefilter-view',
48 'abusefilter-log',
49 'vipsscaler-test']}
50Can't delete [[ko:인도전문가의 인도사업비결]]; do you want to mark it for deletion
51instead? ([y]es, [N]o, [a]ll): ^C
52User quit DeletionRobot bot run...
53
540 pages read
550 pages written
560 pages skipped
57Execution time: 31 seconds
58Script terminated successfully.
59{redacted}-MacBook-Pro:pywikibot-prod {redacted}$

Xqt added a comment.EditedSep 24 2019, 8:48 AM

@revi: You aren't log in and working anonymously. Does it work if you are login first?

My siteinfo looks like this (for xqbot)

C:\pwb\GIT\core>pwb delete -page:user:xqt/Test -simulate
C:\pwb\GIT\core\pywikibot\config2.py:1138: _ConfigurationDeprecationWarning: "sy
sopnames" present in our user-config.py is no longer a supported configuration v
ariable. Please inform the maintainers if you depend on it.
  'depend on it.'.format(_key), _ConfigurationDeprecationWarning)
Enter a reason for the deletion: No valid reason


>>> Benutzer:Xqt/Test <<<
WARNING: There are 12 pages that link to [[de:Benutzer:Xqt/Test]].
    User:            [2]          1 page
    User talk:       [3]          7 pages
    Project:         [4]          1 page
    Project talk:    [5]          3 pages
{'groups': ['bot', '*', 'user', 'autoconfirmed'],
 'id': 627628,
 'name': 'Xqbot',
 'rights': ['editeditorprotected',
            'noratelimit',
            'bot',
            'autoconfirmed',
            'editsemiprotected',
            'nominornewtalk',
            'autopatrol',
            'suppressredirect',
            'apihighlimits',
            'writeapi',
            'autoreview',
            'skipcaptcha',
            'patrolmarks',
            'createaccount',
            'read',
            'edit',
            'createpage',
            'createtalk',
            'viewmywatchlist',
            'editmywatchlist',
            'viewmyprivateinfo',
            'editmyprivateinfo',
            'editmyoptions',
            'urlshortener-create-url',
            'centralauth-merge',
            'abusefilter-view',
            'abusefilter-log',
            'vipsscaler-test',
            'reupload-own',
            'move-rootuserpages',
            'move-categorypages',
            'minoredit',
            'editmyusercss',
            'editmyuserjson',
            'editmyuserjs',
            'purge',
            'sendemail',
            'applychangetags',
            'changetags',
            'spamblacklistlog',
            'mwoauthmanagemygrants',
            'reupload',
            'upload',
            'move',
            'collectionsaveasuserpage',
            'collectionsaveascommunitypage',
            'abusefilter-log-detail',
            'transcode-reset']}
Can't delete [[de:Benutzer:Xqt/Test]]; do you want to mark it for deletion
instead? ([y]es, [N]o, [a]ll):
revi added a comment.Sep 24 2019, 8:49 AM

Tried it and didn't work

Xqt added a comment.Sep 24 2019, 8:51 AM

Unfortunately the -user option doesn't work anymore:

C:\pwb\GIT\core>pwb delete -page:user:xqt/Test -simulate -user:xqt
C:\pwb\GIT\core\pywikibot\config2.py:1138: _ConfigurationDeprecationWarning: "sy
sopnames" present in our user-config.py is no longer a supported configuration v
ariable. Please inform the maintainers if you depend on it.
  'depend on it.'.format(_key), _ConfigurationDeprecationWarning)
Enter a reason for the deletion: No valid deletion


>>> Benutzer:Xqt/Test <<<
WARNING: There are 12 pages that link to [[de:Benutzer:Xqt/Test]].
    User:            [2]          1 page
    User talk:       [3]          7 pages
    Project:         [4]          1 page
    Project talk:    [5]          3 pages
{'groups': ['bot', '*', 'user', 'autoconfirmed'],
 'id': 627628,
 'name': 'Xqbot',
 'rights': ['editeditorprotected',
            'noratelimit',
            'bot',
Xqt added a comment.Sep 24 2019, 8:52 AM

@revi: Can you please tagg the result of the failing login please

revi added a comment.Sep 24 2019, 8:54 AM
python3 dev/wikimedia/pywikibot/pywikibot-prod/pwb.py login -lang:ko -family:wikinews
Logging in to wikinews:ko as -revi@Pywikibot
Logged in on wikinews:ko as -revi.
Xqt added a subscriber: Mpaa.Sep 24 2019, 9:44 AM

Very strange: You are logged in successfully but you aren't obviously during running the script.

Xqt added a subscriber: Framawiki.Sep 24 2019, 9:56 AM
Huji added a comment.Sep 24 2019, 12:10 PM

I have one more thought. Let me try that (may take me a day or two).

Huji added a comment.Sep 24 2019, 5:30 PM

@revi can you share with us the relevant portions of your user-config.py (after redacting passwords and such) please? I want to know how you are using usernames[] to define your main bot account, any additional bot accounts, and the passwords for them, so I can replicate your setup as closely as possible.

revi added a comment.Sep 25 2019, 8:45 AM

I use one directory for -revi, and another directory for Revibot. Revibot doesn't hold any adminships so it cannot be used to replicate anything, so I didn't include it here.

1# -*- coding: utf-8 -*-
2from __future__ import absolute_import, unicode_literals
3
4# This is an automatically generated file. You can find more configuration
5# parameters in 'config.py' file.
6
7# The family of sites to work on by default.
8#
9# ‘site.py’ imports ‘families/xxx_family.py’, so if you want to change
10# this variable, you need to use the name of one of the existing family files
11# in that folder or write your own, custom family file.
12#
13# For ‘site.py’ to be able to read your custom family file, you must
14# save it to ‘families/xxx_family.py’, where ‘xxx‘ is the codename of the
15# family that your custom ‘xxx_family.py’ family file defines.
16#
17# You can also save your custom family files to a different folder. As long
18# as you follow the ‘xxx_family.py’ naming convention, you can register your
19# custom folder in this configuration file with the following global function:
20#
21# register_families_folder(folder_path)
22#
23# Alternatively, you can register particular family files that do not need
24# to follow the ‘xxx_family.py’ naming convention using the following
25# global function:
26#
27# register_family_file(family_name, file_path)
28#
29# Where ‘family_name’ is the family code (the ‘xxx’ in standard family file
30# names) and ‘file_path’ is the absolute path to the target family file.
31#
32# If you use either of these functions to define the family to work on by
33# default (the ‘family’ variable below), you must place the function call
34# before the definition of the ‘family’ variable.
35family = 'wikipedia'
36
37# The language code of the site we're working on.
38mylang = 'ko'
39
40# The dictionary usernames should contain a username for each site where you
41# have a bot account. If you have a unique username for all languages of a
42# family , you can use '*'
43usernames['wikipedia']['*'] = '-revi'
44usernames['meta']['*'] = '-revi'
45usernames['commons']['*'] = '-revi'
46usernames['wikidata']['*'] = '-revi'
47usernames['wiktionary']['*'] = '-revi'
48usernames['wikibooks']['*'] = '-revi'
49usernames['wikinews']['*'] = '-revi'
50usernames['wikiquote']['*'] = '-revi'
51usernames['wikisource']['*'] = '-revi'
52usernames['wikiversity']['*'] = '-revi'
53usernames['wikivoyage']['*'] = '-revi'
54usernames['incubator']['*'] = '-revi'
55#sysopnames['wikipedia']['ko'] = '-revi'
56#sysopnames['wikinews']['ko'] = '-revi'
57#sysopnames['wikiversity']['ko'] = '-revi'
58#sysopnames['commons']['commons'] = '-revi'
59#sysopnames['wikidata']['wikidata'] = '-revi'
60#sysopnames['meta']['meta'] = '-revi'
61#sysopnames['wikimania']['wikimania'] = '-revi'
62password_file = "user-password.py"
63
64flickr = {
65 'api_key': '[REDACTED]',
66 'api_secret': '[REDACTED],
67 'review': False,
68 'reviewer': '-revi',
69}
.

revi added a comment.Sep 25 2019, 8:47 AM

Content of user-password.py: (u'-revi', BotPassword(u'Pywikibot', u'[REDACTED]'))

Huji added a comment.Sep 27 2019, 1:18 PM

I modified my config to use BotPaswords just like you, and then I forced my bot to log out (python3 pwb.py login -logout) and log back in. Then I ran the script; both with and without the -user argument, it would give me the "Can't delete" message, meaning that it forgot who it was logged in as.

This issue did not occur, however, if I was using a password-based configuration as opposed to one based on BotPasswords). And it happens with or without the -user argument. I am investigating it more, but just to be sure, can you also give it a try with a password-based configuration and confirm that things work that way?

Huji added a comment.Sep 27 2019, 1:33 PM

@revi in addition to my request above, can you also checkout the 3rd version of that patch set and confirm that it works?

@Xqt I am surprised that when using BotPasswords, session information is lost. The patch above is not a solution; we need to figure out the root cause and fix that. Any thoughts?

Xqt added a comment.Sep 28 2019, 8:54 AM

I have not investigated in a systematic way but found some weird issues: I cannot use -user option for my account anymore but I can use the bot on sites even the name is not listed inside user-config.py. Two points to be digged deeper. And this listed problem too. I am on holidays currently and cannot help a lot. Btw I am not using botpasswords for my bot an I am using 2FA for my account. I guess we have to make a look into the password/cookie files behaviour.

revi added a comment.Sep 28 2019, 3:27 PM

I cannot use password login because I am required to use 2FA (mandated by WMF Security's policy for super advanced rights (like Stewards and/or Interface-Admins, which both applies to me)) and with it, I don't think password-based login would work.

I'll test the patch tomorrow: I don't have access to computer right now.

revi added a comment.Sep 29 2019, 6:47 PM
$ python3 pwb.py delete -lang:ko -family:wikinews -user:"-revi" -file:../list.txt -summary:"[[Special:Permalink/24353#저작권 무단 침해 사용자 제재 요청|위키뉴스:관리자 요청]]"


>>> 인도전문가의 인도사업비결 <<<
WARNING: ./scripts/delete.py:217: DeprecationWarning: pywikibot.bot.MultipleSitesBot.site is deprecated for 4 years and 3 months; use the page's site property instead.
  if self.site.user() is None:

WARNING: ./scripts/delete.py:218: DeprecationWarning: pywikibot.bot.MultipleSitesBot.site is deprecated for 4 years and 3 months; use the page's site property instead.
  self.site.login()

Logging in to wikinews:ko as -revi@Pywikibot
Do you want to delete [[ko:인도전문가의 인도사업비결]]? ([y]es, [N]o, [a]ll, [q]uit): ^C
User quit DeletionRobot bot run...

0 pages read
0 pages written
0 pages skipped
Execution time: 12 seconds
Script terminated successfully.
Huji added a comment.Sep 30 2019, 1:44 PM

Great. I will investigate more to see why BotPasswords doesn't retain the session info, but if it takes too long, we can merge the patch as is (it won't cause an extra login if session info is retained).

Dvorapa added a subscriber: Dvorapa.Oct 8 2019, 9:01 PM

Very strange: You are logged in successfully but you aren't obviously during running the script.

This could be one of the issues of token implementation as some token issues still remain unresolved

jhsoby added a subscriber: jhsoby.Oct 8 2019, 9:03 PM

I'm having a similar issue with movepages.py. Each time I run the command, the bot will move one page and then say it isn't authorized to move.

1jhsoby@jhsoby:~/pywikibot$ python3 pwb.py movepages -family:wikisource -lang:hi -pairsfile:hiwikisource-move.txt -noredirect -pt:1
2Moving page [[पृष्ठ:Aaj Bhi Khare Hain Talaab (Hindi).pdf/113]] to [[पृष्ठ:Aaj Bhi Khare Hain Talaab (Hindi).pdf/११३]]
3Moving page [[पृष्ठ:Aaj Bhi Khare Hain Talaab (Hindi).pdf/114]] to [[पृष्ठ:Aaj Bhi Khare Hain Talaab (Hindi).pdf/११४]]
4WARNING: API error cantmove: You do not have permission to move pages.
5Traceback (most recent call last):
6 File "/home/jhsoby/pywikibot/pywikibot/site.py", line 5516, in movepage
7 result = req.submit()
8 File "/home/jhsoby/pywikibot/pywikibot/data/api.py", line 2080, in submit
9 raise APIError(**result['error'])
10pywikibot.data.api.APIError: cantmove: You do not have permission to move pages. [help:See https://hi.wikisource.org/w/api.php for API usage. Subscribe to the mediawiki-api-announce mailing list at &lt;https://lists.wikimedia.org/mailman/listinfo/mediawiki-api-announce&gt; for notice of API deprecations and breaking changes.]
11
12During handling of the above exception, another exception occurred:
13
14Traceback (most recent call last):
15 File "pwb.py", line 297, in <module>
16 if not main():
17 File "pwb.py", line 292, in main
18 run_python_file(filename, [filename] + args, argvu, file_package)
19 File "pwb.py", line 96, in run_python_file
20 main_mod.__dict__)
21 File "./scripts/movepages.py", line 271, in <module>
22 main()
23 File "./scripts/movepages.py", line 255, in main
24 bot.moveOne(page, pair[1])
25 File "./scripts/movepages.py", line 89, in moveOne
26 noredirect=self.getOption('noredirect'))
27 File "/home/jhsoby/pywikibot/pywikibot/tools/__init__.py", line 1744, in wrapper
28 return obj(*__args, **__kw)
29 File "/home/jhsoby/pywikibot/pywikibot/tools/__init__.py", line 1827, in wrapper
30 return obj(*new_args, **new_kwargs)
31 File "/home/jhsoby/pywikibot/pywikibot/page.py", line 1902, in move
32 noredirect=noredirect)
33 File "/home/jhsoby/pywikibot/pywikibot/site.py", line 1323, in callee
34 return fn(self, *args, **kwargs)
35 File "/home/jhsoby/pywikibot/pywikibot/site.py", line 5553, in movepage
36 raise Error(on_error % errdata)
37pywikibot.exceptions.Error: User JhsBot is not authorized to move pages on wikisource:hi wiki
38CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.Error'>

Dvorapa added a comment.EditedOct 8 2019, 9:33 PM

@jhsoby from IRC: it works almost exactly 25 % of the time: https://hi.wikisource.org/wiki/%E0%A4%B5%E0%A4%BF%E0%A4%B6%E0%A5%87%E0%A4%B7:%E0%A4%B2%E0%A5%89%E0%A4%97/move
(the numbers in the moved pages there *should* be sequential)

jhsoby added a comment.Oct 8 2019, 9:40 PM

@jhsoby from IRC: it works almost exactly 25 % of the time: https://hi.wikisource.org/wiki/%E0%A4%B5%E0%A4%BF%E0%A4%B6%E0%A5%87%E0%A4%B7:%E0%A4%B2%E0%A5%89%E0%A4%97/move
(the numbers in the moved pages there *should* be sequential)

I wrote a quick and ugly script that calls os.system with the commands to perform the moves one at a time. That's the script that works 25 % of the time. The rest of the time it gives the same error as the one I pasted above.

Huji added a comment.Nov 1 2019, 2:02 PM

@Xqt: I think we should merge https://gerrit.wikimedia.org/r/#/c/pywikibot/core/+/538724/ for now to get the code into a working state at least. I know the Technical-Debt will remain, but I cannot afford a revamp of our login infrastructure for BotPasswords until at least December.

Change 538724 merged by jenkins-bot:
[pywikibot/core@master] Make sure the user is logged in before attempting a delete

https://gerrit.wikimedia.org/r/538724

Is this solved too?

Xqt closed this task as Resolved.Mon, Jan 13, 1:58 PM
Ruthven reopened this task as Open.Fri, Jan 17, 1:24 PM
Ruthven added a subscriber: Ruthven.

Hi all, sorry to reopen this, but I think that problem I encountered is related.
I am using delete.py under PAWS. I applied the patch above, but I obtain the same result: not being able to "undelete" files in a project where I am sysop.

@PAWS:~$ pwb.py login -family:commons -lang:commons
Logged in on commons:commons as Ruthven.
@PAWS:~$ pwb.py scripts/delete -undelete -file:ff -summary:"OTRS [[ticket:2019121410003163]]" -lang:commons -family:commons
File:Luigi Martignon 1969 - Jole.jpg <<<
0 pages read
0 pages written
Execution time: 0 seconds
Script terminated by exception:
ERROR: NoUsername: No sysop is permitted with OAuth
Traceback (most recent call last):

File "/srv/paws/pwb/pwb.py", line 250, in <module>
  if not main():
File "/srv/paws/pwb/pwb.py", line 243, in main
  run_python_file(filename, [filename] + args, argvu, file_package)
File "/srv/paws/pwb/pwb.py", line 95, in run_python_file
  main_mod.__dict__)
File "scripts/delete.py", line 307, in <module>
  main()
File "scripts/delete.py", line 299, in main
  bot.run()
File "/srv/paws/pwb/pywikibot/bot.py", line 1702, in run
  super(MultipleSitesBot, self).run()
File "/srv/paws/pwb/pywikibot/bot.py", line 1505, in run
  self.treat(page)
File "/srv/paws/pwb/pywikibot/bot.py", line 1732, in treat
  self.treat_page()
File "scripts/delete.py", line 197, in treat_page
  self.current_page.undelete(self.summary)
File "/srv/paws/pwb/pywikibot/tools/__init__.py", line 1738, in wrapper
  return obj(*__args, **__kw)
File "/srv/paws/pwb/pywikibot/page.py", line 2030, in undelete
  self.site.undelete_page(self, reason, undelete_revs)
File "/srv/paws/pwb/pywikibot/site.py", line 1317, in callee
  self.login(True)
File "/srv/paws/pwb/pywikibot/site.py", line 2080, in login
  raise NoUsername('No sysop is permitted with OAuth')

pywikibot.exceptions.NoUsername: No sysop is permitted with OAuth
CRITICAL: Exiting due to uncaught exception <class 'pywikibot.exceptions.NoUsername'>

Xqt added a comment.Fri, Jan 17, 1:44 PM

Hm this is not the current stable release provided with paws. This is strange because paws should always use the last stable release. Is this a long term session you are using?

This does not seem to be the underlying issue here, but PAWS will not have admin permissions. T192237

Xqt added a comment.Fri, Jan 17, 1:57 PM

This does not seem to be the underlying issue here, but PAWS will not have admin permissions. T192237

In this sense the newly reopened task is invalid because the bot says "no sysop is permitted with OAuth" and as you said paws does not support sysops anyway.
Independently of that we removed sysop login with the last stable release which should be that released loaded with paws but with the given traceback I found out this is different.

Change 565624 had a related patch set uploaded (by Xqt; owner: Xqt):
[pywikibot/core@master] [IMPR] Replace @must_be with @need_right

https://gerrit.wikimedia.org/r/565624