Hi @Vajotwo, thanks for taking the time to report this!
Could you add a more structured description to this report (1) steps to reproduce, 2) actual outcome, 3) expected outcome), including links?

I guess I am supposed to go to https://vec.wikipedia.org/wiki/Speciale:%C3%99ltimiCanbiamenti?hidebots=1&hidecategorization=1&hideWikibase=1&limit=500&days=30&urlversion=2 and should not see what is listed on https://vec.wikipedia.org/wiki/Speciale:Contributi/Alvise_Morosin ? What makes you think that these edits listed on the last link were made with a flood flag?

Exactly, you understood everything. Those contributions should not appear in the latest changes because I personally assigned the flood flag to that user a few days ago. I am a sysop of this Wikipedia. But they continue to appear as if nothing had happened.

This might be problem of MobileFrontend (which may not respect the user group).

Do you have more or less idea how long it will take to fix this bug? It's not an urgent matter of course, it's just to get an idea. Thank you

This appears to be a problem with how the edits are saved into the database, since they don't have the bot flag on them. This is not a bug in the RecentChanges code; it's doing the right thing based on the data it sees, it's just that that data is wrong.

It's very possible that MobileFrontend is breaking this. All of the edits on vecwiki that should be marked as bot edits but aren't were mobile edits, and I can verify that the flood flag in general does work over on bnwiki (where ListeriaBot's edits are correctly flagged as bot edits).

I can replicate this on http://reading-web-staging.wmflabs.org/ - have added some notes, will continue to investigate.

As @Catrope says rc_bot is getting set as 0 for these edits. After debugging this I don't see any foul play in MobileFrontend, in fact this happens even if I disable the hooks that add the mobile-edit tag. I've realised I can replicate this with MobileFrontend disabled via the API.

The issue is mobile edits always use the API so this is why it gave the impression there is an issue with mobile.

@aaron and @daniel I see bot flag is set to 0 for api edits inside PageUpdater::doModify but I don't know enough about this code to trace this to the origin of the error.

Note, I see there is a bot flag on the edit api. If that's needs to be marked (ie. this can't be done auto-magically) please feel free to re-tag MobileFrontend and let me know how to update MobileFrontend's editor client to pass the correct value of bot to ensure this works correctly (I'm not sure if that's available in say mw.config). Does VisualEditor do anything special here?

The action API action=edit does indeed require the bot parameter to be supplied in order to mark the edit with rc_bot. That's how it has been since the module was added in rMWe0aae6a65895: API: Adding action=edit module.

For some reason EditPage works in the opposite manner, there if you have the bot userright you have to (somehow) set bot=0 in your form submission if you want to prevent the edit being marked with rc_bot. It looks like that's really old back-compat: the 'bot' user right used to force every edit to be marked with rc_bot, but that was changed for T14574 to allow bots to override with a form parameter. Of course, it has been recommended that bots use the API rather than screen-scraping for around a decade now.

It should be safe to always pass bot=1 to the API action=edit, it's silently ignored if you don't have the necessary userright. I have no idea what VE does.

Seems a bit counterintuitive to me. I'd expect without a bot field param the user would be used to check this.

Leaving open as I think it's unfair to expect every client to know about this and will lead to more bugs down the road (i imagine same issue is in any wikipedia apps)

Note that action API boolean parameters work like HTML checkboxes: any value, even 0 or the empty string, indicates "true" while the absence of the parameter indicates "false".

In T200089#4527178, @Jdlrobson wrote:

The issue is that unless the bot param is passed to the api, it is assumed the edit is not by a bot.

It's more that the bot parameter works much like the minor parameter: if supplied, the edit is intended to be flagged as "bot" or "minor".

Don't take the analogy too far though: yes, there is also a notminor parameter, which exists to override the minordefault user preference. There's currently no botdefault user preference or the like to override.

I suspect it would make sense for the bot parameter to default to true given it is ignored for non-bots or when absent the bot flag is calculated from the user.

This would be a breaking change for all existing client code. We'd need a pretty compelling reason to make such a break.

And as noted above, there's no way to specify "false" for an API boolean parameter, so such a parameter defaulting to true makes no sense. It would have to be a "notbot" parameter instead.

I talked to @Catrope yesterday and it seems the exact same problem occurs in VisualEditor.

However i have one question now - how is the user who is using the bot mode performing these edits? Is it common for flood users to make batch edits using VisualEditor or mobile web?

If they are using the api are they forgetting to set the flag? (Note hitting the api on the mobile domain will add the mobile edit tag). @Vajotwo can you find out if the user is using a script to make this changes?

In T200089#4530459, @Jdlrobson wrote:

@Vajotwo can you find out if the user is using a script to make this changes?

@Jdlrobson The user in question denies using scripts to make these changes. He says he does not know how to do it and simply makes a copy paste through the normal editor interface on Wikipedia to make massive changes.

Got it. Thanks @Vajotwo for the information. cc @Catrope @Anomie if users are using the flood/bot right in this way, it really does feel like this should default to true to avoid all the updates this requires to different clients. Could we limit this behaviour to when formatversion=2 is in the request?

In T200089#4535318, @Jdlrobson wrote:

cc @Catrope @Anomie if users are using the flood/bot right in this way, it really does feel like this should default to true to avoid all the updates this requires to different clients.

You're talking about changing the behavior to save updating four clients: MobileFrontend, the two mobile apps, and VE.

But changing the behavior would require a change to every other client, of which there are hundreds if not more.

It's a poor approximation, but filtering for a few obvious cases (e.g. Pywikibot) that identify themselves in the middle of the user-agent string and truncating the rest at the first space or slash leaves 132 unique values hitting action=edit on 2018-08-26. That's just one Sunday.

Could we limit this behaviour to when formatversion=2 is in the request?

The formatversion parameter affects ApiFormatJson or ApiFormatPhp, not ApiEditPage. We'd have to introduce a parameter specific to ApiEditPage to control this. But in that case you may as well just specify the existing 'bot' parameter in your query.

As far as MediaWiki-Action-API is concerned, this is working as intended and I'd mark it as declined or invalid. The bug as originally filed here has been refiled as T202638: Mobile editor should set bot field on all edits so bot edits are marked as bots.

Just from an API perspective (not index.php?action=edit), I do think the current default is backwards. Most clients with the bot flag want their edits to be marked as bot (the case that our defaults should optimize for), while it's a very slim majority of accounts that are flagged as bot don't want their edits to be marked as bot.

But changing it now after 10 years of people using it as it is would be a crazy amount of work and disruption for very minor benefit.

Look how much was involved with changing to the new-style continuation, which did bring major benefit to most clients who were implementing the old style wrong.

So basically you are telling me that it is not possible to solve this problem? Or better, that can not be solved without upsetting everything? Just to better understand the situation and what decisions will be taken. Thank you

IMO the proper solution would be for the mobile editor (and anything else that intends to replace EditPage by using API action=edit) to supply the bot parameter to action=edit. For mobile in particular that's what T202638 is about.

IMO the proper solution would be for the mobile editor (and anything else that intends to replace EditPage by using API action=edit) to supply the bot parameter to action=edit

So, manual edits with a bot account should be hidden from RC per default? Is that really a good idea?

Should the EditPage UI do the same, then? I don't see a reason why the web based editor should behave differently from the mobile editor.

In T200089#4541974, @daniel wrote:

So, manual edits with a bot account should be hidden from RC per default? Is that really a good idea?

Should the EditPage UI do the same, then?

The EditPage UI already does this. And it has been the behavior of the web UI edit form since rMWd82c14fb4fba: Initial revision (and probably earlier).

I don't know whether it's a good idea, but it's long-standing behavior that at least one person depends on (or this task wouldn't have been filed).

But changing it now after 10 years of people using it as it is would be a crazy amount of work and disruption for very minor benefit.

To quote yourself, you're talking about changing the behavior to save updating four clients: MobileFrontend, the two mobile apps, and VE.

But changing the behavior would require a change to every other client, of which there are hundreds if not more.

Sure, but it seems like this is broken for those clients too. If I'm an editor with the flood right, I'd expect my edits anywhere to be filed as bot edits, unless I state otherwise (e.g. via UI). If for instance I'm editing in Twinkle I'd expect Twinkle to set me as a bot (no idea if Twinkle does that). What is the downside of making a bot edit the default when Ive not specified? That's not clear to me.

What problems do you see with making this the default behaviour? Are there really any downsides if suddenly my edits get marked as a "bot" edit when I have the bot right? That seems useful behaviour to me and clients if they don't want this (which seems less of a problem that the status quo), can send the bot flag as false.

We'd have to introduce a parameter specific to ApiEditPage to control this. But in that case you may as well just specify the existing 'bot' parameter in your query.

I disagree. Versioning on the edit API which can be used in future for other breaking changes seems like a much more useful outcome than applying a bot edit that only fixes this change.

. The bug as originally filed here has been refiled as T202638: Mobile editor should set bot field on all edits so bot edits are marked as bots.

To be clear T202638 is stalled and I don't plan on introducing this code if I don't have to. It's technical debt that I'd like to avoid. The worst possible outcome here in my opinion is 5 identical patches for our existing clients (iOS, Android, Web, VisualEditor, VisualEditor mobile editor) and potentially other patches for other lesser known clients that we support working around this one particular problem, as this bug will come up in a different form somewhere down the road and I'd rather do this properly than throwing on unintuitive api parameters. Personally I'm hoping to decline T202638. Happy to create RFC if necessary.

@Vajotwo to be clear I want to get this fixed for you as it seems the behaviour is not matching expectations and that's bad and I'm sorry this is causing problems for your project. However it appears to a problem in other places (VisualEditor being one) so I am keen to fix it in all those other potential places to.

In T200089#4542577, @Jdlrobson wrote:

But changing it now after 10 years of people using it as it is would be a crazy amount of work and disruption for very minor benefit.

To quote yourself, you're talking about changing the behavior to save updating four clients: MobileFrontend, the two mobile apps, and VE.

I don't follow what you're trying to say here.

Sure, but it seems like this is broken for those clients too. If I'm an editor with the flood right, I'd expect my edits anywhere to be filed as bot edits, unless I state otherwise (e.g. via UI).

There's no flood right. The flood group applies the bot right with the expectation that the user is going to be editing via EditPage and so pick up EditPage's behavior.

If you want to introduce a flood right that applies the bot flag in all contexts, feel free to propose that somewhere.

What is the downside of making a bot edit the default when Ive not specified? That's not clear to me.

I already explained that: having to change every client that is depending on the existing behavior.

Are there really any downsides if suddenly my edits get marked as a "bot" edit when I have the bot right?

If there weren't downsides then T14574 wouldn't have ever existed. And the prior practice (at least on enwiki) of not giving some bots the 'bot' group to avoid the auto-flagging (in the days when bots screen-scraped EditPage's output) wouldn't have existed either.

I disagree. Versioning on the edit API which can be used in future for other breaking changes seems like a much more useful outcome than applying a bot edit that only fixes this change.

This discussion was already had on T41592.

The worst possible outcome here in my opinion is 5 identical patches for our existing clients (iOS, Android, Web, VisualEditor, VisualEditor mobile editor) and potentially other patches for other lesser known clients that we support

Versus patches to hundreds of existing clients that "we" (for any relevant value of "we") don't maintain (or even know much about) because they're written and maintained by various volunteer community members?

In T200089#4542718, @Anomie wrote:

In T200089#4542577, @Jdlrobson wrote:

But changing it now after 10 years of people using it as it is would be a crazy amount of work and disruption for very minor benefit.

To quote yourself, you're talking about changing the behavior to save updating four clients: MobileFrontend, the two mobile apps, and VE.

I don't follow what you're trying to say here.

I think Jon misinterpreted that to mean that those 4 are the only clients on the edit API, and so it should be easy to change the API. But that's not true: they're the only clients that power UIs for everyday editors. Most API clients are true bots, and this doesn't affect them in a particularly weird way, it's the user-facing "clients" that are affected. I agree with @Anomie here: it's not worth making a breaking change to this API to avoid updating 4 clients.

If there weren't downsides then T14574 wouldn't have ever existed

I'm still struggling to see how this is related. Bots can still use the bot flag. However not sending the bot flag would result in different behaviour.

it's not worth making a breaking change to this API to avoid updating 4 clients.

What I'm saying is that beyond Wikimedia there are more clients that need to be aware of this. If a 3rd party app becomes super popular with editors editors will complain about this same problem as if it's our fault, won't they (as they'll see it in RecentChanges)?

Versus patches to hundreds of existing clients that "we" (for any relevant value of "we") don't maintain (or even know much about) because they're written and maintained by various volunteer community members?

This seems purely hypothetical, to me and I feel like I'm missing something here.
Do we have any concrete examples of where someone may not be sending the bot flag and would care if suddenly those edits were tagged with "bot"? I'm struggling to imagine any situation where that would be a problem, so while the behaviour would change, I don't see how it can be called a "breaking change" since from a user pov it seems correct. It only seems like a breaking change in terms of the code used to do this, it now does this, but to be honest, if you're not being explicit in your API request that seems acceptable...

In T200089#4543381, @Jdlrobson wrote:

Do we have any concrete examples of where someone may not be sending the bot flag and would care if suddenly those edits were tagged with "bot"?

ClueBot NG on enwiki is one concrete example. People want to see those vandalism reverts, to know to check for additional vandalism.

AnomieBOT's fixes to orphaned references on enwiki are another. Sometimes the bot makes a mistake, it's better if the change isn't hidden.

I don't see how it can be called a "breaking change" since from a user pov it seems correct.

Then I don't know how you define "breaking change". I define it as "calling code that used to work now does not work." This change would make all the existing bot code not work correctly, in that edits that aren't supposed to be flagged as 'bot' will start being flagged.

It only seems like a breaking change in terms of the code used to do this, it now does this, but to be honest, if you're not being explicit in your API request that seems acceptable...

You seem to continue to misunderstand how the API works in this respect. I've already told you that this is a boolean option and API boolean options don't have any non-explicit state. I don't know what else to say to you.