Thanks! I think dynamically changing the maxlag value is likely to still introduce some thresholds, whereas a continuous slowdown (by retrieving the lag and compute one's edit rate based on it) should in theory reach an equilibrium point.

Thanks for the analysis! Whether this is a breaking change or not is not my concern: Petscan and other mass-editing tools based on Widar should play by the book. I can provide a simple patch which ensures maxlag=5 is applied to all Widar edits: if someone wants to do a refined version which allows specific user-triggered edits to go through without a maxlag parameter, that is great. @Magnus, what is your take on this?

If clients are able to retrieve the current lag periodically (through some MediaWiki API call? which one?), then this should not require any server-side change. Clients can continue to use maxlag=5 but to also throttle themselves using the smoothed function proposed.

@Bugreporter have you got details of where this behaviour is currently implemented in PetScan? In particular, how do you request the current maxlag with the MediaWiki API?

@Theklan let's move your issue to a different ticket as your issue does not seem to be related: T240436

I've had a quick look at the code to see if I could submit a patch for this myself but it is not clear to me where the edits are done - I have looked in petscan_rs and wikibase_rs to no avail. Petscan edits might be done in the browser by sending them to some Widar-like interface?

@Bugreporter yes indeed! I was off by one hour there. Thanks for your help! Feel free to add more bots which match that period.

Matching pull request: https://github.com/arthurpsmith/author-disambiguator/pull/107

If you have not changed anything in user-config.py then you should be good to go, it might have been a false positive on my side. Sorry for the noise!

I am first getting in touch with people who seem to be running bots with maxlag greater than 5 or no maxlag parameter at all, to see if they would accept to follow @Addshore's advice never to use maxlag greater than 5 at all.

There has been one attempt I think but it did not go very far - and the ids do not seem to have been imported so far!

I think it is a bit harder to extract more than 1000 records (I didn't cap it on purpose to make it manageable for the workshop).

@Jheald I don't think anyone is working on this anymore: if you are still interested in the scraped dataset, it is here: http://pintoch.ulminfo.fr/adc2c9aaba/lakes-portal.tsv

OK! If you have ways to check what sort of maxlag values are used it would be great!

Actually, some tools seem to be doing something like that already, since edits are still going through despite max lag being above 5 for more than an hour now (Author Disambiguator does this, QuickStatements too probably, Edoderoobot too). So these tools use higher (more agressive) maxlag values than 5.

One problem with the current policy (requesting all automated editing processes to use maxlag=5) is that this creates a binary threshold: either the query service lag is under the threshold, in which case bots will edit at full speed, or the query service lag is above the threshold, in which case they should all stop editing entirely. This is likely to create an oscillating behaviour, where all bots start and stop periodically. This is probably not ideal neither for the infrastructure nor for the users.

We have an import here: https://tools.wmflabs.org/editgroups/b/OR/8cf42ae3c0/
Good job!

Moving to "Done" although this is not complete, but we did have a lot of new translations (not just in Dutch) during the event.

Data is getting into Wikidata!

If you have trouble reconciling, use this reconciliation service: http://ulminfo.fr:3894/en/api

For people who cannot install OpenRefine on their laptops, use: http://5.135.188.139:48379/

So this is what we get with an exponential back-off (1.5 factor), at the moment:

22:37:27.148 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 1000 milliseconds. (19338ms)
22:37:28.729 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 1500 milliseconds. (1581ms)
22:37:33.809 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 2250 milliseconds. (5080ms)
22:37:37.931 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 3375 milliseconds. (4122ms)
22:37:42.663 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 5062 milliseconds. (4732ms)
22:37:49.437 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 7593 milliseconds. (6774ms)
22:37:58.429 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 5.4916666666667 seconds lagged. -- pausing for 11389 milliseconds. (8992ms)
22:38:18.217 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 6 seconds lagged. -- pausing for 17083 milliseconds. (19788ms)
22:38:36.461 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 6 seconds lagged. -- pausing for 25624 milliseconds. (18244ms)
22:39:05.013 [..baseapi.WbEditingAction] [maxlag] Waiting for all: 6.4666666666667 seconds lagged. -- pausing for 38436 milliseconds. (28552ms)

So it looks like this means no OpenRefine edits at all with these new rules, in the current situation.

Thanks for the notification! I would be happy to release a new version of OpenRefine with a patch applied - I can do this in the coming days. The exponential back-off suggested by @Multichill makes sense intuitively - could WMDE confirm that this is the policy they recommend? Happy to adapt the policy as required.

@Ecritures shouldn't we rather assign tasks to the participants who decide to tackle them?

I should note that we do have a Dutch translation (which is incomplete) - you should be able to find it the language menu as "Nederlands", not "Dutch". It is the second option in the menu, just after "English".

Thanks! I am working on making OpenRefine easier to host, but it's a long term project indeed (exciting announcement about that soon).

Exciting developments! I am wondering if that comment_data could be (or already is) exposed in any public API? Exposing such diffs would also solve T106306 in the same go.

The simplest way to translate OpenRefine is to use Weblate:
https://hosted.weblate.org/projects/OpenRefine/
The UI strings are grouped into modules that can be translated independently. "Translations" is for the main features, "wikidata" is for the Wikidata-specific ones (from the Wikidata extension), "gdata" is for the Google extension, and so on.

Yes! The simplest way to translate OpenRefine is to use Weblate:
https://hosted.weblate.org/projects/OpenRefine/
The UI strings are grouped into modules that can be translated independently. "Translations" is for the main features, "wikidata" is for the Wikidata-specific ones (from the Wikidata extension), "gdata" is for the Google extension, and so on.

I thought for a moment that there was an issue with the fact that currently, filtering by tags only works for Special:RecentChanges (which only contain the most recent changes, not all of them). But Lucas pointed out that it is also supported by Special:Contributions, which is uncapped in time (I think?). EditGroups currently assumes that all edits in a given group are made by the same user, which I think makes sense as a constraint. So, in the current situation, if you had an edit group which corresponded to a single tag, it would be possible to retrieve all edits in it via Special:Contributions (assuming you know the user in the first place).

Great point! I did not think about that in this way. It sounds like a very sensible route to follow.

Sure, I hope you don't mind us having this discussion here anyway, since this is still at a very early stage. (unless you are already planning to work on this exact architecture?)

Here are a few issues with the current tool:

• If the tool goes down, this creates millions of dead links in edit summaries (which cannot be changed). Users who ran batches with the assumption that they could be undone if something goes wrong find themselves having to clean things up manually.
• If the recent changes listener dies for a long time, for instance for longer than the EventStream or recent changes cover, edits which were missed cannot be recovered easily (this could be solved by reading the corresponding edits from the public dumps, but it is not implemented at the moment).
• As a user, I can easily game EditGroups by imitating the edit summary of any tool, attributing edits to batches they are not actually part of (there are some protections against this, but they are not fully bullet-proof. For instance an edit will only be added to a batch if it was made under the account of the user who first created the batch)
• Bot/tool authors are reluctant to add grouping support for their bots since EditGroups is not officially part of the Wikidata infrastructure. The community is unlikely to systematically enforce edit grouping in requests for bot approvals as long as it relies on an external tool, which does not come with any sustainability guarantees, SLA, etc.
• Reverting relies on OAuth, and OAuth tokens can expire while a large edit group is being reverted.

With the introduction of Structured Data on Commons and the implementation of mass-editing tools there (AC/DC, QuickStatements), the need to generalize EditGroups to other wikis is gathering interest.

Here is a diff where one of the new fallback summaries failed to be translated to a human-readable form: https://www.wikidata.org/w/index.php?title=Property:P571&diff=1040450905&oldid=1038877052