Page MenuHomePhabricator

Use a more efficient API in mark-locked.js user script and similar scripts/gadgets
Closed, ResolvedPublic

Description

There are some user scripts/gadgets that visually highlight signatures of locked or blocked users on discussion pages.

There is no efficient API that allows looking up the lock status for a batch of users, so when viewing a busy discussion page, these scripts fire off hundreds of cross-wiki API requests per page view (one per username).

https://en.wikipedia.org/wiki/User:GeneralNotability/mark-locked.js is a notable one, but there are some more of them. I'm not sure if the others are widely used.

We should introduce a suitable API (see subtasks), then identify widely used scripts/gadgets (help on this would be very welcome), and reach out to their authors (or propose patches).

Event Timeline

These scripts often end up with "centralauthtoken is invalid" warnings in our logs. I'm not sure why this happens. Maybe we just can't issue so many centralauthtokens so fast. Anyway, I added user-agent logging to these logs (https://gerrit.wikimedia.org/r/q/I0b78475875509e3a8c8ae3805117cdfef7c4dc71) to help identify more gadgets (at least the ones nice enough to set an user-agent – thanks for doing that).

User:GeneralNotability/mark-locked.js is is indeed commonly used (user-agent "Marked-locked-userscript"), and I found that https://ja.wikipedia.org/wiki/MediaWiki:Gadget-MarkBLocked-core.js is another common one.

There is at least one more widely used script sending queries with agufrom=...&agulimit=1&aguprop=lockinfo&aguto=..., but all I know is that it does not use mw.Api or mw.ForeignApi (only sending browser user-agent). Actually, I think this is wrong. I'm seeing mostly OPTIONS requests, which never have the Api-User-Agent header, so they could be coming from anything (including the other two scripts).

These scripts often end up with "centralauthtoken is invalid" warnings in our logs. I'm not sure why this happens.

I'm not sure why this happens either, and I actually had a workaround for this in MarkBLocked, which I recently removed after receiving a notice about the recent update in rate limits.
Anyway, I actually have a draft patch for T261752 which is about 60% complete in my local repo.

https://global-search.toolforge.org/?q=agufrom&namespaces=2%2C4%2C8&title=%28Gadgets-definition%7C.*%5C.%28js%7Ccss%7Cjson%29%29 SPIhelper maybe? I can throw a user agent at the multitude of Spihelper forks on enwiki if that would help

I've thrown in a useragent for the most used copy of SPIHelper in case it is required!

For those who are planning to update scripts/gadgets using list=globalusers with gusprop=locked, followed by cross-origin list=logevents requests to fetch lock details, T425972: POST by mw.ForeignApi is CORS-blocked when a Promise-Non-Write-API-Action header is provided may be of interest.

Thanks @Dragoniez.

I prepared a new version of mark-locked.js: https://en.wikipedia.org/wiki/Special:ComparePages?page1=User:GeneralNotability/mark-locked.js&page2=User:Matma_Rex/mark-locked.js

I'd appreciate if someone could give it a look, then copy it over the existing script (or let me know if I should do that myself). @Soda @GeneralNotability

Logstash query for "centralauthtoken is invalid" (which happens often on the list=globalallusers request), to check later: https://logstash.wikimedia.org/goto/88bcb77c7c6e777f71ceff5e9ef2e803

@daniel Do you have a convenient dashboard somewhere showing how often the list=globalallusers requests exceed rate limits, or something along these lines?

@matmarex @Soda @GeneralNotability

In mark-locked.js#L-92, there's an issue with how the locked property is evaluated. With gusprop=locked, the property is a boolean value and may be false. The current implementation probably mismarks unlocked users.

In mark-locked.js#L-112, the request may fail if lockLogIdsToUsers has a size of more than 50 (or 500 for those with apihighlimits).

Ah, never mind about the second point. I didn't realize the input users is already batched into groups of 50 or 500.

That said, mark-locked.js#L-158 needs to take into account both the API limit for the current wiki and that for the central wiki. Even if you have apihighlimits locally, you may not have it on the central wiki, in which case the list=logevents request may fail if the batch size is too large.

In mark-locked.js#L-92, there's an issue with how the locked property is evaluated. With gusprop=locked, the property is a boolean value and may be false. The current implementation probably mismarks unlocked users.

Ugh, never mind about the first point too. This is correct for formatversion=1. Sorry!

Thanks, fixed: https://en.wikipedia.org/w/index.php?title=User:GeneralNotability/mark-locked.js&diff=prev&oldid=1355638565

Maybe I should have changed it to use formatversion=2 at the same time, it's much more intuitive.

Logstash query for "centralauthtoken is invalid" (which happens often on the list=globalallusers request), to check later: https://logstash.wikimedia.org/goto/88bcb77c7c6e777f71ceff5e9ef2e803

Effect of these changes looks nice:

image.png (1,535×226 px, 29 KB)

Effect of these changes looks nice:

image.png (1,535×226 px, 29 KB)

Nice indeed!

matmarex claimed this task.

Thanks for your work on the API @Dragoniez, and thanks everyone for the help and reviews.