Per https://www.mediawiki.org/wiki/Manual:Pywikibot/User-agent, users can set a custom UA easily by setting user_agent_format. Should we perhaps not be encouraging that vs blanket allow-listing the default UA? That then falls into line with our other UA policy for library defaults -- each bot should set a custom UA.

In T427491#11963857, @ssingh wrote:

Per https://www.mediawiki.org/wiki/Manual:Pywikibot/User-agent, users can set a custom UA easily by setting user_agent_format. Should we perhaps not be encouraging that vs blanket allow-listing the default UA? That then falls into line with our other UA policy for library defaults -- each bot should set a custom UA.

The default UA is customized automatically and contains contact information based on the user's login info. Why should the bot'S owner have to override the user-agent string if they are already supplying their user name? All we want to know is who is running the bot.

I'm not suggesting to whitelist pywikibot, I'm suggesting to support the format in which pywikibot supplies user information in the user-agent in accordance with our policy.

In T427491#11964208, @daniel wrote:

In T427491#11963857, @ssingh wrote:

Per https://www.mediawiki.org/wiki/Manual:Pywikibot/User-agent, users can set a custom UA easily by setting user_agent_format. Should we perhaps not be encouraging that vs blanket allow-listing the default UA? That then falls into line with our other UA policy for library defaults -- each bot should set a custom UA.

The default UA is customized automatically and contains contact information based on the user's login info. Why should the bot'S owner have to override the user-agent string if they are already supplying their user name? All we want to know is who is running the bot.

I'm not suggesting to whitelist pywikibot, I'm suggesting to support the format in which pywikibot supplies user information in the user-agent in accordance with our policy.

Yeah, that's fair I guess, I am a bit split so will defer to others here.

If pywikibot's main execution is scripts, then I'd say the username is sufficient. If it's used as a frequent import into other, bigger, tools, I would say it's not sufficient. Given my own limited use of pywikibot for maintenance on behalf of my user, I suspect it's mostly the former. I'd argue the username is acceptable as it gives us the differentiation we'd need (i.e. we can identify abuse when it comes up).

However, it does bring up the possibility of a malicious operator merely setting it to:

script_name (wikipedia:en; User:BCornwall-WMF) Pywikibot/10.2.0 (g19732) requests/2.32.3 Python/3.11.2.final.0

which brings another dimension to abuse: While spoofing isn't new, this allows operators to spoof users, not programs. It's another thing we'd have to sift through.

In T427491#11964475, @BCornwall wrote:

While spoofing isn't new, this allows operators to spoof users, not programs. It's another thing we'd have to sift through.

Since x-ua-contact is extracted from the user agent and people can set the user agent to whatever they want, I don't see how adding support for the way pywikibot represents the user changes anything. Anyone can already put (dkinzler@wikimedia.org) in their user-agent header.

Arguably, pywikibot makes this (slighly) less likely by using the user's login info to construct the user-agent string.

+1 from me on supporting the format pywikibot already emits for on-wiki usernames.

I think the right long-term answer to spoofability would be doing something like sending a valid JWT along with the automated traffic (or something else we can issue at the applayer and validate at the edge). I don't know what the status quo is re: OAuth flow support in Pywikibot, but it might be worth investing in that as well.

The pattern pywikibot uses for user-agents is nto only supported, but also unit-tested.

So I'm not sure what this task is about.

Care to provide examples?

Change #1295060 had a related patch set uploaded (by CDanis; author: CDanis):

[operations/puppet@production] cache: contact_info: test Pywikibot format

https://gerrit.wikimedia.org/r/1295060

Change #1295060 merged by CDanis:

[operations/puppet@production] cache: contact_info: test Pywikibot format

https://gerrit.wikimedia.org/r/1295060

Oh, I think I know what happened. There are log entries for category_redirect (commons:commons; User:RussBot) Pywikibot/10.2.0 (g19732) requests/2.32.3 Python/3.11.2.final.0 with no x-ua-contact set, but they are classified as known-network. I guess we don't extract the contact info for requests from known networks then?

In T427491#11965123, @daniel wrote:

Oh, I think I know what happened. There are log entries for category_redirect (commons:commons; User:RussBot) Pywikibot/10.2.0 (g19732) requests/2.32.3 Python/3.11.2.final.0 with no x-ua-contact set, but they are classified as known-network. I guess we don't extract the contact info for requests from known networks then?

Aha, yeah, this is indeed the case: https://codesearch.wmcloud.org/operations/?q=lua.set_contact_info

http-request lua.set_contact_info if !is_trusted_request !is_identified_bot_request

is_trusted_request is true for WMCS.

How sharp of a corner case is this for you?

In T427491#11967267, @CDanis wrote:

How sharp of a corner case is this for you?

It's not needed for the rate limiting to work, but it's useful when reading logs and investigating issues.

So, it would be Nice To Have (tm).