Page MenuHomePhabricator

dualism between user and sysop needs to be overtaken
Open, HighPublic

Tokens
"Love" token, awarded by zhuyifei1999."Manufacturing Defect?" token, awarded by Dalba."Love" token, awarded by Ciencia_Al_Poder."Love" token, awarded by Ricordisamoa.
Assigned To
Authored By
Ricordisamoa, Aug 8 2014

Description

Some wikis, especially non-WMF ones, might have complementary user groups without clear distinction between users and administrators. So, one single user could have an account with "delete" permission, another with "editinterface" rights, etc., and use each of them for a different purpose.
Many bot-ops have multiple bot accounts for different tasks, too. Instead, Pywikibot only allows to configure one username per site.

The framework should also support CentralAuth (et similia) to automatically detect on which wikis a given account has bot/sysop/etc. rights.


Version: core-(2.0)
Severity: normal
See Also:
T67196: Cannot be logged in as user and sysop at the same time

Details

Reference
bz69283
Related Gerrit Patches:

Event Timeline

bzimport raised the priority of this task from to Needs Triage.Nov 22 2014, 3:30 AM
bzimport added a project: Pywikibot-General.
bzimport set Reference to bz69283.
bzimport added a subscriber: Unknown Object (????).

Having two accounts (user & sysop) breaks the 0,1,∞ design principle . https://en.wikipedia.org/wiki/Zero_one_infinity_rule

Either we have one user per APISite object, or any number of users per APISite object.

My preference is only one user per APISite, and the selection of user is done using a user-config variable or -user: command line argument. And if there are scripts which *need* to *write* using two different accounts, which is rare, they would instantiate two different site objects.

XZise added a subscriber: XZise.Feb 16 2015, 12:18 AM

But how should the script know what type of write they need? Maybe there should be a dict in the site which shows which user has which rights and then it selects from a list the first user which has that right.

@XZise , yes, that is a problem with "any number of users per APISite object" - it means APISite needs logic to determine which account to use when, and IMO that should be done in scripts with the bot operator in control, and not done inside the pywikibot library.

Change 209503 had a related patch set uploaded (by John Vandenberg):
Replace @must_be with @need_right

https://gerrit.wikimedia.org/r/209503

jayvdb added a comment.May 8 2015, 3:29 AM

After switching to need_right, I see the following tasks needed to completely remove the dualism:

  1. allow config.usernames to be a list , and move sysopnames values into config.usernames with a deprecation warning. the test class flag 'sysop' can still be supported as a static check whether the list of usernames has two values, however it would be better to replace the sysop flag with a list of rights needed for the test. however a list of rights cant be a static check, as the userinfo needs to be loaded to check
  2. allow site.login (or site.userlogin ?) to use a specific username (and maybe password- there is already a patch for this), which is then used, breaking the reliance on the config
  3. remove LoginStatus.AS_SYSOP
  4. change site.login to accept a list of rights needed, basically moving the process from need_right into login

They dont need to be done in that order, but that is how I would do it.

Some of this is playing on dangerous ground, as there are many bugs in the site login sequence.

jayvdb added a comment.May 8 2015, 3:51 AM

Tightly related, but orthogonal, is T67196, which is allows multiple accounts to be used concurrently. The code to solve that bug will be a lot more stable if it is done after step 1 (allow config.usernames to be a list of usernames per site) above.

jayvdb added a comment.May 8 2015, 9:35 AM

I mentioned this on the CS, but it will be a long standing problem so worth mentioning here.
Currently if there are two different usernames in for user and sysop, pywikibot drops down to the user account for non-sysop actions after a sysop action. It is bad behaviour on the wiki to split a sysop bots actions/edits between two accounts, which is why bot operators usually set both usernames to be the same username for a sysop bot which also performs normal edits. But we need to think about whether/how we support the prior behaviour, or fail in an orderly manner.

One approach is to set a flag when a user action has occurred, and error if the same bot attempts to perform a sysop action with a different username.

Ricordisamoa awarded a token.
This comment was removed by jayvdb.

Not to be seen in compat.

Ricordisamoa set Security to None.

By the way, I think Pywikibot should also support editing without logging in, if that's what the operator wants.

XZise added a comment.Jul 1 2015, 7:27 PM

There is a comment in @jayvdb's patch which talks about how to support that. Unfortunately it's got a bit stale because of the introduction of requests afaik.

There is a comment in @jayvdb's patch which talks about how to support that. Unfortunately it's got a bit stale because of the introduction of requests afaik.

Yea; I had a solution that was heavily tied to httplib2. I expect that multiple concurrent sessions will be easier with requests. Maybe existing packages solve the problem adequately.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 26 2015, 4:43 AM
Sn1per added a subscriber: Sn1per.Aug 26 2016, 8:37 PM
Xqt triaged this task as High priority.Apr 15 2018, 9:36 AM
Xqt moved this task from Backlog to Needs Review on the Pywikibot board.Feb 3 2019, 11:24 AM
Xqt assigned this task to jayvdb.Feb 7 2019, 4:23 PM
Dalba awarded a token.Aug 22 2019, 4:22 AM
Dalba added a subscriber: Dalba.
Huji added a subscriber: Huji.Aug 22 2019, 2:39 PM

I am considering to take this over after T229293 is fixed. Besides site.py is there any other place where we allow dualism?

config2.py needs to emit a warning that sysopnames don't work anymore (if we decide to just break it)

Change 532713 had a related patch set uploaded (by Huji; owner: Huji):
[pywikibot/core@master] Deprecate sysopnames

https://gerrit.wikimedia.org/r/532713

Huji added a comment.EditedAug 27 2019, 2:03 PM

config2.py needs to emit a warning that sysopnames don't work anymore (if we decide to just break it)

The patch I just submitted will mark sysopnames as deprecated.

The next step is to go through the code an eliminate all dependencies on sysopnames. I can find it in the following places:

  • pywikibot/config2.py
  • pywikibot/__init__.py
  • pywikibot/login.py
  • scripts/flickrripper.py
  • scripts/imagetransfer.py
  • scripts/login.py
  • scripts/version.py
  • scripts/watchlist.py
  • tests/aspects.py

Next, we should eliminate all places were the "sysop" status of an account is checked (because we should check the user's rights, not the group memberships).

At this point, all that would be left to be done is to devise a strategy for those who need to use different accounts on different wikis (or different accounts for different parts of the same script). The latter is such a rare use case, that I don't think we should prioritize working on it for now.

At this point, all that would be left to be done is to devise a strategy for those who need to use different accounts on different wikis (or different accounts for different parts of the same script). The latter is such a rare use case, that I don't think we should prioritize working on it for now.

user_a = Site(user='UserA')
user_b = Site(user='UserB')
Huji added a comment.Aug 27 2019, 4:45 PM

Correct. What I was trying to say was that we should not spend clarifying how to do that (updating documentation, etc) right away.

Huji removed jayvdb as the assignee of this task.Aug 27 2019, 8:10 PM
Xqt assigned this task to Huji.Aug 27 2019, 8:23 PM

The only thing needed is to support specifying two accounts, one bot, one sysop and allow to switch between them easily (by a parameter?)

BTW we should also update Stevard rights check (I think it was in site.py) to the new system

This comment was removed by Dvorapa.

The only thing needed is to support specifying two accounts, one bot, one sysop and allow to switch between them easily (by a parameter?)

This violates zero one infinity rule. If sysop is different, why would any other right be different? T71283#1040594

Besides, what is the need for the separation? And how frequent do they appear?

Change 533015 had a related patch set uploaded (by Huji; owner: Huji):
[pywikibot/core@master] Deprecate sysopnames

https://gerrit.wikimedia.org/r/533015

Change 532713 abandoned by Huji:
[WIP] Deprecate sysopnames

Reason:
Moving to 533015 for a minimal approach

https://gerrit.wikimedia.org/r/532713

Dvorapa added a subscriber: Urbanecm.EditedAug 28 2019, 4:52 PM

The only thing needed is to support specifying two accounts, one bot, one sysop and allow to switch between them easily (by a parameter?)

This violates zero one infinity rule. If sysop is different, why would any other right be different? T71283#1040594
Besides, what is the need for the separation? And how frequent do they appear?

This is quite common situation on Czech Wikipedia, one bot account and one sysop account for every sysop bot operator, edits by Pywikibot are made from each of them depending on whether sysop rights are needed or not. I suggest somethng like this (pseudocode, partially similar with what we've had before):

main_username[wikipedia][cs] = 'DvorapaBot'
other_usernames[wikipedia][cs] = ['Dvorapa', 'DvorapaTest']
$ python pwb.py category move -user:Dvorapa -family:wikibooks -lang:sk
$ python pwb.py category move -user:Dvorapa
( = -family:wikipedia -lang:cs)
$ python pwb.py category move
( = -user:DvorapaBot -family:wikipedia -lang:cs)

Another option would be something like this:

usernames[wikipedia][cs] = ['DvorapaBot', 'Dvorapa', 'DvorapaTest']
$ python pwb.py add_text
uses DvorapaBot as this does not need sysop rights
$ python pwb.py delete
tries DvorapaBot, but because of the lack of sysop rights for DvorapaBot it tries Dvorapa instead, successfully as this does require sysop rights

@Urbanecm What would be the best way to solve sysop/bot account dualism in Pywikibot? Currently important changes in this field are being merged to Pywikibot, we should think this through quickly and come up with some ideal solution

This violates zero one infinity rule. If sysop is different, why would any other right be different? T71283#1040594

What is ZOI rule? Never heard about it and Wiki article does not explain it at all. Not sure what do you want to say.

This is quite common situation on Czech Wikipedia, one bot account and one sysop account for every sysop bot operator, edits by Pywikibot are made from each of them depending on whether sysop rights are needed or not. I suggest somethng like this (pseudocode):

  1. Why is the separation necessary? It's not like you can't use BotPasswords...
  2. T71283#5442078:
user_a = Site(user='UserA')
user_b = Site(user='UserB')
pywikibot.Page(user_a, 'foobar').do_stuffs()
pywikibot.Page(user_b, 'foobar').do_stuffs()
usernames[wikipedia][cs] = ['DvorapaBot', 'Dvorapa', 'DvorapaTest']

Are you proposing to test each user for whether they have rights? This does not feels right to me. Errors should not fail silently by using a fallback, especially in the case when the 'actor' is being changed. If one account is blocked, the bot should stop, not continuing operating on an alt.

What is ZOI rule? Never heard about it and Wiki article does not explain it at all. Not sure what do you want to say.

How about these?

The point being, if we have dualism, why not tri-ism, quad-ism, quint-ism?

@Urbanecm What would be the best way to solve sysop/bot account dualism in Pywikibot? Currently important changes in this field are being merged to Pywikibot, we should think this through quickly and come up with some ideal solution

Hmm, that's a good question :). Let me introduce my workflow, which can give you some ideas. I have ~/.pywikibot folder, which looks like this:

tools.urbanecmbot@tools-sgebastion-07 ~
$ ll ~/.pywikibot/
total 120
drwxrws--- 2 tools.urbanecmbot tools.urbanecmbot 12288 Jan  1  2018 apicache
drwxr-sr-x 2 tools.urbanecmbot tools.urbanecmbot 12288 Aug  2 05:08 apicache-py2
drwxr-sr-x 2 tools.urbanecmbot tools.urbanecmbot 12288 Aug  8 11:56 apicache-py3
drwxrws--- 2 tools.urbanecmbot tools.urbanecmbot  4096 Mar 29  2016 logs
drwxr-sr-x 5 tools.urbanecmbot tools.urbanecmbot  4096 Apr  2 13:36 oznamovatel
-rw------- 1 tools.urbanecmbot tools.urbanecmbot   974 Jun 16  2016 pywikibot.lwp
drwxr-sr-x 5 tools.urbanecmbot tools.urbanecmbot  4096 Mar 29 17:10 sysop
-rw-r--r-- 1 tools.urbanecmbot tools.urbanecmbot    34 Aug 28 19:19 throttle.ctrl
-rw------- 1 tools.urbanecmbot tools.urbanecmbot 17328 Aug  6 20:11 user-config.bot.py
-rw------- 1 tools.urbanecmbot tools.urbanecmbot 17143 Jul 26 21:01 user-config.oznamovatel.py
lrwxrwxrwx 1 tools.urbanecmbot tools.urbanecmbot    18 Dec 16  2017 user-config.py -> user-config.bot.py
-rw------- 1 tools.urbanecmbot tools.urbanecmbot 17337 Jul 26 21:01 user-config.sysop.py
-rw------- 1 tools.urbanecmbot tools.urbanecmbot   101 Apr  2 13:32 user-password.oznamovatel.py
tools.urbanecmbot@tools-sgebastion-07 ~
$

As you can see, I have user-config file for every account I want to use with PWB, and so I have a subdirectory (oznamovatel and sysop in the ls output above), which is basically another .pywikibot folder dedicated for use with another accounts.

  • my personal account - user-config.sysop.py (sysop subdir)
  • UrbanecmBot - user-config.bot.py (no subdir, main acc)
  • Oznamovatel - user-bot.oznamovatel.py (oznamovatel subdir)

In addition to those assets, every non-main account has also a script in ~/bin, see an example:

tools.urbanecmbot@tools-sgebastion-07 ~
$ cat bin/oznamovatelbot
#!/bin/bash
PYWIKIBOT2_DIR=$HOME/.pywikibot/oznamovatel python3 "$@"
tools.urbanecmbot@tools-sgebastion-07 ~
$

Then, if I want to run a script saved under announce.py as User:Oznamovatel, I do oznamovatelbot announce.py. The same principle applies to my personal account, and in theory, I can add any account to this setup quickly and easily, I just need to copy the scripts and configs and edit a little.

Keep in mind my bots aren't object orientated, nnc.py is a real example.

To let me work how I work now (and have pywikibot do what I need it to, not me do what pywikibot needs me to let me use itself), I'd appreciate something like this.

user-config.py
accounts = {
    "oznamovatel": {
        "username": "Oznamovatel",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    },
    "urbanecmbot": {
        "username": "UrbanecmBot",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    }
}
usernames['wikipedia']['cs'] = [  'urbanecmbot', 'oznamovatelbot' ] # both urbanecmbot and oznamovatelbot used
usernames['wikipedia']['en'] = [  'urbanecmbot' ] # only urbanecmbot used

and then in shell:

tools.urbanecmbot@tools-sgebastion-07 ~
$ python3 nnc.py --user=urbanecmbot # PWB catches the user parameter internally, and runs the script as UrbanecmBot
tools.urbanecmbot@tools-sgebastion-07 ~
$ python3 nnc.py --user=oznamovatel # PWB catches the user parameter internally, and runs the script as Oznamovatel
tools.urbanecmbot@tools-sgebastion-07 ~
$

That would allow anyone to use any number of account in one PWB instance/config, and use one script with any of those as desired.

Less ideal (I'd probably alias it up to make it simple without much typing), but also good would be:

tools.urbanecmbot@tools-sgebastion-07 ~
$ PYWIKIBOT_USER=urbanecmbot python3 nnc.py # PWB catches the PYWIKIBOT_USER env variable internally, and runs the script as UrbanecmBot
tools.urbanecmbot@tools-sgebastion-07 ~
$ PYWIKIBOT_USER=oznamovatel python3 nnc.py # PWB catches the PYWIKIBOT_USER env variable internally, and runs the script as Oznamovatel
tools.urbanecmbot@tools-sgebastion-07 ~
$

Please note this is merely an end-user wish and I've little to none idea regarding the rest of PWB users' workflow,

This is quite common situation on Czech Wikipedia, one bot account and one sysop account for every sysop bot operator, edits by Pywikibot are made from each of them depending on whether sysop rights are needed or not. I suggest somethng like this (pseudocode):

  1. Why is the separation necessary? It's not like you can't use BotPasswords...

To put it simply, Czech Wikipedia is less developped than the English one speaking of policies, which leaves us with only a policy about how a human can become an admin, but none policy about how can a computer/bot become an admin. Bureacurats don't want to implement a precedent with granting sysop rights to alterego accounts, which is what bot account kinda-is. For instance, all my adminbot activity is distinguished only by summary prefixed with Bot, and runs from my own account.

BotPasswords can be used, but I use an owner-only OAuth token (probably allowed to do everything on behalf of the bot, and carefully selected actions on behalf of my personal account).

  1. T71283#5442078:
user_a = Site(user='UserA')
user_b = Site(user='UserB')
pywikibot.Page(user_a, 'foobar').do_stuffs()
pywikibot.Page(user_b, 'foobar').do_stuffs()
usernames[wikipedia][cs] = ['DvorapaBot', 'Dvorapa', 'DvorapaTest']

Are you proposing to test each user for whether they have rights? This does not feels right to me. Errors should not fail silently by using a fallback, especially in the case when the 'actor' is being changed. If one account is blocked, the bot should stop, not continuing operating on an alt.

Delete must be done with an user account that has delete, not ordinary bot - it is not going to succeed. I'd go with something slighly different, see above.

What is ZOI rule? Never heard about it and Wiki article does not explain it at all. Not sure what do you want to say.

How about these?

The point being, if we have dualism, why not tri-ism, quad-ism, quint-ism?

I believe he wants to propose to extend the idea of two accounts in pywikibot to any number of accounts, regardless their privilege. This is actually something that I consider important for my workflow, see above. {{support}}

I also have got and use several account for various things and I miss a way to simply switch between them. BTW Pywikibot tests miss the way to simply switch between several accounts (OAuth, BotPasswords, regular password) as well.

user-config.py
accounts = {
    "oznamovatel": {
        "username": "Oznamovatel",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    },
    "urbanecmbot": {
        "username": "UrbanecmBot",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    }
}
usernames['wikipedia']['cs'] = [  'urbanecmbot', 'oznamovatelbot' ] # both urbanecmbot and oznamovatelbot used
usernames['wikipedia']['en'] = [  'urbanecmbot' ] # only urbanecmbot used

I like the idea of supporting OAuth tokens for an infinite number of users per site. However, given that the user is provided by command line, and that the family/lang tuple is provided in the site constructor, what is the point of providing a user list per site? It is not like pywikibot will attempt to fallback through this list.

To put it simply, Czech Wikipedia is less developped than the English one speaking of policies, which leaves us with only a policy about how a human can become an admin, but none policy about how can a computer/bot become an admin. Bureacurats don't want to implement a precedent with granting sysop rights to alterego accounts, which is what bot account kinda-is. For instance, all my adminbot activity is distinguished only by summary prefixed with Bot, and runs from my own account.

This is risking your own personal account getting blocked for bot activity, in the case that it malfunctions.

BTW Pywikibot tests miss the way to simply switch between several accounts (OAuth, BotPasswords, regular password) as well.

The login manager is too stateful. You want PEP 554.

user-config.py
accounts = {
    "oznamovatel": {
        "username": "Oznamovatel",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    },
    "urbanecmbot": {
        "username": "UrbanecmBot",
        "oauth": [ consumer key, consumer secret, client key, client secret ]
    }
}
usernames['wikipedia']['cs'] = [  'urbanecmbot', 'oznamovatelbot' ] # both urbanecmbot and oznamovatelbot used
usernames['wikipedia']['en'] = [  'urbanecmbot' ] # only urbanecmbot used

I like the idea of supporting OAuth tokens for an infinite number of users per site. However, given that the user is provided by command line, and that the family/lang tuple is provided in the site constructor, what is the point of providing a user list per site? It is not like pywikibot will attempt to fallback through this list.

True that, it can be just one username per site, the preferred one? But listing that usernames can be actually useful as a safeguard, to not allow an username that's defined, but not allowed in usernames list, in case I don't want to use one username somewhere for some reason. Feel free to make it only default username, that's fine as well, just thinking loud :).

To put it simply, Czech Wikipedia is less developped than the English one speaking of policies, which leaves us with only a policy about how a human can become an admin, but none policy about how can a computer/bot become an admin. Bureacurats don't want to implement a precedent with granting sysop rights to alterego accounts, which is what bot account kinda-is. For instance, all my adminbot activity is distinguished only by summary prefixed with Bot, and runs from my own account.

This is risking your own personal account getting blocked for bot activity, in the case that it malfunctions.

I'm not saying that's the best way, I'm just saying that's the reason why it is separated :).

True that, it can be just one username per site, the preferred one? But listing that usernames can be actually useful as a safeguard, to not allow an username that's defined, but not allowed in usernames list, in case I don't want to use one username somewhere for some reason. Feel free to make it only default username, that's fine as well, just thinking loud :).

How about:

  • If it is not defined: No constraint, no defaults. One of my web tools use pywikibot this way and uses OAuth tokens from mwoauth to switch between users.
  • If it is a string: No constraint, default to string value. I use a few bots that switch to another non-default user by simply setting user= in site constructor
  • If it is a set of 0 strings: Unsatisfible constraint. Non-loginable, read-only.
  • If it is a set of 1 string: Constraint to default, default to string value.
  • If it is a set of multiple strings: Constraint to the set, no defaults.

I don't understand. Can you show me examples?

This is assuming the default family/lang is Commons. The same applies to any other site if the parameters passed to the Site constructor matches any case given to the usernames dict.

del usernames['commons']:

>>> pywikibot.Site().username()
[Error about username not set]
>>> pywikibot.Site(user='MyBot0').username()
'MyBot0'

usernames['commons']['commons'] = 'MyBot1':

>>> pywikibot.Site().username()
'MyBot1'
>>> pywikibot.Site(user='MyBot0').username()
'MyBot0'

usernames['commons']['commons'] = []:

>>> pywikibot.Site().username()
[Error about username not set]
>>> pywikibot.Site(user='MyBot0').username()
[Error about username constraint not satisfied]

usernames['commons']['commons'] = ['MyBot1']:

>>> pywikibot.Site().username()
'MyBot1'
>>> pywikibot.Site(user='MyBot1').username()
'MyBot1'
>>> pywikibot.Site(user='MyBot0').username()
[Error about username constraint not satisfied]

usernames['commons']['commons'] = ['MyBot1', 'MyBot2']:

>>> pywikibot.Site().username()
[Error about username not set]
>>> pywikibot.Site(user='MyBot1').username()
'MyBot1'
>>> pywikibot.Site(user='MyBot0').username()
[Error about username constraint not satisfied]

The list is not the focus. The container will be turned into a set object for de-duplication, and the order does not matter anyways.

Change 533015 merged by jenkins-bot:
[pywikibot/core@master] Deprecate sysopnames

https://gerrit.wikimedia.org/r/533015

Change 534927 had a related patch set uploaded (by Huji; owner: Huji):
[pywikibot/core@master] Deprecate the ability to login with a sysop account

https://gerrit.wikimedia.org/r/534927

Change 534927 merged by jenkins-bot:
[pywikibot/core@master] Deprecate the ability to login with a secondary sysop account

https://gerrit.wikimedia.org/r/534927