@drochford Would you happen to know how the users are being added manually in these cases? (The process for solving the issue in detail)

I don't @ARamirez_WMF but @jrbs or @sguebo_WMF would.

In T271884#6796608, @ARamirez_WMF wrote:

@drochford Would you happen to know how the users are being added manually in these cases? (The process for solving the issue in detail)

Right now, we use the override list, accessed through the "Voter eligibility" interface (example: https://vote.wikimedia.org/wiki/Special:SecurePoll/votereligibility/808)

Through there, you can access three lists, all of which are just text lists of usernames. Here's how the software describes them:

• The Eligibility list is a list of voters who are allowed to vote, if they also meet the basic requirements defined above. This list is often populated automatically.
• The Override list is a list of voters who are allowed to vote regardless of the basic requirements or the eligibility list.
• The Exclude list is a list of voters who are not allowed to vote, regardless of any other lists.

The override list is how we would add staff (or other users who should be eligible but are not automatically so).

The issue here isn't so much avoiding the server, it's being able to add people to that list based on e.g. a username filter so that staff can vote in Board elections. Otherwise we have to add them on a per-request basis which doesn't scale, or maintain a list of eligible staff accounts (which we probably have, though I don't know if it is readily available from ITS).

If one doesn't exist yet (and I think it doesn't), we could suggest that a user group be made for staff? It looks like the eligibility list is supposed to support "Include users in these groups, regardless of edits or other groups", and so that'd cover the whole thing without encoding any specific business logic. (It looks sort of broken because of hidden fields and cloner at the moment, so I'm kinda guessing about how it's supposed to work.)

tl;dr - this is probably the solution 👇 but there are a few caveats

In T271884#6853493, @DLynch wrote:

If one doesn't exist yet (and I think it doesn't), we could suggest that a user group be made for staff? It looks like the eligibility list is supposed to support "Include users in these groups, regardless of edits or other groups", and so that'd cover the whole thing without encoding any specific business logic. (It looks sort of broken because of hidden fields and cloner at the moment, so I'm kinda guessing about how it's supposed to work.)

Getting and maintaining a list of staff

I may be recalling incorrectly, but I believe @wikitrent brought that up once but I can't remember why it wasn't an option at the time. I know that the "staff" group is a misnomer (it's more like sysop iirc) but not sure what the work required would be for adding a new group + adding everyone to that group. @Tchanders I think you were the one who mentioned blockers?

If we can get the group, great! Either way, we have to get at list of WMF staff to prepopulate it. I don't know who that has to go through but this worked for me on sqlite with my 10 row table:

sqlite> SELECT user_name FROM user WHERE user_name LIKE "% (WMF)";
Test (WMF)

I'm not sure if there's a workflow for single time reads but we'd need to do something like that, add everyone, and then make adding new WMF staff to the group part of account set up. Since the foundation controls access to these accounts, I don't think there'd be any risk of past staff accounts having access when they shouldn't be, even if they don't get removed from the proposed wmfstaff usergroup.

If we can't, then if we can just do that one-time read right before the election and use that as an override list, I don't necessarily think that's the worst? It's once per election (and we run very few elections) and since part of the requirements is that you're registered before the election starts, if it's done right before the start then we'd know that we have everyone eligible.

Making eligibility lists work

tl;dr - They don't. There's a ticket open to investigate exactly how it's supposed to (T275470: Add option to allow specific user groups to vote to 'Basic options' list) but a few things I found while I was testing:

• cached sessions can make you incorrectly in/eligible since they check your eligibility when the session is created (so if the list changes, the session has no way of knowing that)
• if you don't check "Populate list automatically" but try to add a group to "Include users in these groups, regardless of edits or other groups", there will be no input to let you add a group
• I couldn't get my user to be considered eligible, even after I added the group and added the user to the group 🙃

Based on includes/Jobs/PopulateVoterListJob.php, it's supposed to be possible. Here's where it checks for the group override (list_include-groups) which it then is supposed to eventually be notated in securepoll_lists which itself then later gets used to figure out if a user is on some kind of pass/block list. I've yet to make any of this work for 'Eligibility list'.

I'm guessing here, but the caching and the use of jobs makes me think that this is all a bunch of vaguely intensive operations and it's unclear to me where it broke, but we should keep that in mind if/when we fix this up.

Alternatives

If we can't add a user group then some fallback solutions could include:

1. manually generate a list through a db read and use the list override. List overrides currently work so there'd be no additional engineering work on SecurePoll
2. manually scrape together a list (ie. this ticket is declined) and use the list override. There'd be no additional engineering work in general if there's also no db read.
3. write in some business logic, probably in Election::getQualifiedStatus() which we should not do because then if anyone else uses SecurePoll, then they now have WMF specific logic to contend with

I was curious how we currently deal with (WMF) usernames and we use the AntiSpoof extension to stop people from creating usernames containing WMF:

The user name "Test (WMF)" has been banned from creation. It matches the following blacklist entry: .*WMF.* <newaccountonly|antispoof>

I don't think doing something similar (but in reverse) is a particularly practical option here, especially since we already have a feature we can probably make do what we want it to (dependent on creating the user group). If we really wanted to, we could try to engineer some kind of regex criteria override feature and tack it onto the job (which we will have to fix in this case anyway so we should try for the user group first instead of doing this). If we wanted to do this we would:

1. Add the input somewhere in the Eligibility list fieldset
2. Presumably be concerned about sanitizing it somewhere (processConfig()?) before writing it in as a securepoll_properties row
3. Pull it as part of the job
4. Add it to the user list, similar to how the other lists are implemented

I may be recalling incorrectly, but I believe @wikitrent brought that up once but I can't remember why it wasn't an option at the time. I know that the "staff" group is a misnomer (it's more like sysop iirc) but not sure what the work required would be for adding a new group + adding everyone to that group.

There is already a group called staff but it currently grants the group member access to basically everything, including sensitive pages. It's currently only granted to a few people in the foundation, handed out by T&S on a case-by-case basis. In order to create a new group for all staff holders, we'll probably need to talk to someone in HR who manages user accounts. I am not very hopeful about a quick turnaround if we go that route (not that we are in a particular rush here).

Having this be too-WMF specific definitely complicates things a lot. Thanks for looking into AntiSpoof. I like the idea of allowing for a regex - but then again it's a fairly narrow use case and I can't think of other cases when it might be used.

I'm leaning towards checking in with People Ops to see if we can make a user-group happen and if so, we can look into fixing up the eligibility list.

In T271884#6856477, @Niharika wrote:

There is already a group called staff but it currently grants the group member access to basically everything, including sensitive pages. It's currently only granted to a few people in the foundation, handed out by T&S on a case-by-case basis. In order to create a new group for all staff holders, we'll probably need to talk to someone in HR who manages user accounts. I am not very hopeful about a quick turnaround if we go that route (not that we are in a particular rush here).

Yup, presently "staff" is essentially global superpowers given to a small number of staff members. I agree with Tran that this is a misnomer and that ideally "staff" would be applied to all staff accounts by ITS / People Ops, with a new superpowers userright to replace the existing "staff" that has a better name. I could swear there was an existing task for this but I can't find it; it's definitely a conversation we've had many times.

In T271884#6855343, @STran wrote:

If we can get the group, great! Either way, we have to get at list of WMF staff to prepopulate it. I don't know who that has to go through but this worked for me on sqlite with my 10 row table:

sqlite> SELECT user_name FROM user WHERE user_name LIKE "% (WMF)";
Test (WMF)

You probably already thought about this (sorry) but you could also run a check against the centralauth database and filter out locked accounts (i.e. where gu_locked = 1), which in theory should find every staff account and could be done before an election by someone with access. (Also just to note that some staff don't use brackets in their usernames, just to make things more complicated, e.g. User:GVarnum-WMF or User:Aaharoni-WMF.) In the long run the userright is the ideal solution though.

Making eligibility lists work
tl;dr - They don't.

Just to clarify, are you referring to the actual inputting of usernames into the list directly here, or the autogeneration functionality? I think adding names directly to the box should work, I tried it a week or so ago as part of the WMUA elections we're setting up this week.

You probably already thought about this (sorry) but you could also run a check against the centralauth database and filter out locked accounts (i.e. where gu_locked = 1), which in theory should find every staff account and could be done before an election by someone with access.

Do we lock (WMF) staff accounts when staff leave the foundation, such that this query+check would maybe get us "current staff"... or is that something which we can't work out from the database alone?

(I'm assuming that the rules don't allow former staff to vote regardless of the requirements, but admittedly haven't verified this.)

In T271884#6857656, @DLynch wrote:

Do we lock (WMF) staff accounts when staff leave the foundation, such that this query+check would maybe get us "current staff"... or is that something which we can't work out from the database alone?

Yes, it's part of the ITS off-boarding process, though it's not impossible some have been missed over the years.

(I'm assuming that the rules don't allow former staff to vote regardless of the requirements, but admittedly haven't verified this.)

It's complicated and might change but my understanding is that no, they cannot. 2017's requirements indicate that current staff were eligible but doesn't mention former staff, probably because they had their accounts locked even then :)

In T271884#6857715, @jrbs wrote:

Yes, it's part of the ITS off-boarding process, though it's not impossible some have been missed over the years.

My anecdotal impressions from the three months: It's pretty reliable nowadays; it can however take up to two months until an account of former staff gets locked.