It’s hard to tell. It could also be other bots/tools getting blocked at the account level with the anon-flag disabled causing the IP to be effectively be hard-blocked. I don’t know how the inner workings of MediaWiki work. All I know is that this error message comes from the API itself, and not IABot.

These error messages are not originating from the bot, it’s being passed back by the MW API itself when requesting the edit on behalf of the user. All users edit from the IP 185.15.56.1 which is Toolforge. This is either a glitch in the API, or there are admins hardblocking bots originating from it.

I'm hoping this is resolved.

It'll take some time before the backlog of 229 pending jobs are completed.

Yes it turns out the bot had robust error handling for prod errors. It didn’t have robust handling for reused OAuth nonces. For some reason it got a lot of OAuth nonce already used errors. This took repeated running the last few days to diagnose.

Unable to reproduce.

Closing this as resolved as the Wayback Machine has gotten better over the years.

Unable to reproduce. Tool claims you are not blocked.

If you are getting the first error, make sure you are not set to a "Mixed" status. You need to firmly define the state of the URLs. Alive and Dead resets every individual link while Whitelist and Blacklist sets it on all the selected domains as a whole and the bot will no longer assess if they are alive or not.

This does indeed look like a legitimate bug.

You did not whitelist it. No log entry for that URL at all aside from the bot scans it did of it. I have gone ahead and whitelisted it for you.

In T273003#6784499, @Dreamy_Jazz wrote:

In T273003#6782955, @Joe wrote:

In T273003#6780172, @Dreamy_Jazz wrote:

Just to note that cyberbot I has been blocked because of the blanking issues on enwiki. The bot has also been blocked on two other wikis for the blanking issues. It would be really great if this issue is fixed, as this has stopped a fair number of tasks.

The solution to this problem is pretty simple:

make the bot handle failures of the API (they will *always* happen, even if in ridiculously low numbers, as shown above) and stop blanking pages when receiving an error.

Even if we get to the bottom of which events caused the few errors we found evidence of, we can definitely guarantee there will be more errors in the future, and at times even a ton of errors, be it for a wrong deployment or some catastrophic hardware failure or something else.

I think the bot should only be unblocked once error management is properly taken into account, not before.

I don't disagree with you, and the block of the bot is because the bot was blanking (and not because of the server side issue). Primarily, the bot should be ready for strange situations like this. But secondarily, when the bot has to deal with the issues the root cause of the issue on the server side should be fixed with some level of speed.

Timestamps of these entries seem to suggest that when the bot encounters these failures almost all API requests being made are erroring out during the timeframe of these batch errors.

Got an influx of blank, 500s and 502 overnight. Your errors may be "few" overall, but they are happening in batches.

In T273003#6782382, @Urbanecm wrote:

@Cyberpower678 This doesn't sound to be a complete dump of a raw request. I'm pretty confident the response has at least one line (the one starting with HTTP/ that denotes the status etc.).

Normalization is by default on, but can be turned off if the community wants it off.

I've added additional logging data to the framework.

Got another surge of bad responses from production.

Joe, not all requests are 502s. They are insignificant compared to the amount of requests returning null responses. This is a very problematic occurrence as Cyberbot has begun to malfunction badly because of it. Is blanking pages repeatedly, my talk page is littered with malfunction reports. Granted some extra robustness could be implemented, this shouldn’t have to be something I would need to guard against at this level. Cyberbot has never had these problems in the past until I started getting reports around 11 days ago. I see my communication log is starting to grow to a large size recording these failures and has grown to 5 GB.

It won’t. It’s not designed to do that. It will only recognize the alias and use what’s there, or add the default preferred alias if not defined.

No changes have been made to the bot whatsoever.

The IP 172.16.2.21, I don't know what the UA is. The bot is pretty old.

Project Name: cyberbot
Type of quota increase requested: 8 CPUs and 16 GB RAM
Reason: IABot is expanding on to more WMF wikis, which requires more processing power. The current exec VM is reaching its limit.

Especially to Wikidata.

Thursday, as in yesterday? I’m not aware of anything that should have been running to create that massive level of requests.

https://ru.wikipedia.org/wiki/Модуль:Citation/CS1/Configuration

I don’t have access to your Citation module on ruwiki. A local sysop will need to make the changes.

IABot reads the configuration values of the CS1 module that drives the Cite Web template and related templates. dead-url and deadurl are valid parameters according to the configuration.

I need more info.

You simply lack the permissions to use the tool because you are a new user on Wikipedia. You gain permissions on this tool as your experience on Wikipedia increases. You need at least ten on wiki edits before the tool will allow you to use it.

The only wiki you’ve given the bot a flag is scruffywiki. I don’t see the flag on simpleelectronics.

Actually there is no bot flag on the bot.

@robkam I still need the bot flag in order to proceed.

Bot flag request: https://meta.wikimedia.org/wiki/Steward_requests/Bot_status#InternetArchiveBot@warwiki

Support for df was dropped because the CS1 module automatically formats dates based on the use dmy and similar templates which now the bot also follows. Lacking that, it will format dates based on the prevalent formatting used in the wiki text.

I went ahead and looked at your spam log and found this:

Sorry but this is not an issue I can fix. That message is produced by Wikipedia itself, so the tool doesn't know what URL is actually the problem. You will need to go to the local page that deals with the spam blacklist and have them look into what specific filter was triggered.

That is not a good reason to block the bot. Primarily since the link was marked as dead by someone else. This is not a bug.

Bot flag required

IABot will not be able to handle templates on the Incubator due to technical limitations. Not unless all wikis can universally use the same CS1 module and Cite templates, which shouldn't be much of an issue since the modules can easily be internationalized, upon moving to their own wikis.

Well here's a Catch-22. In order to access the wiki, I need to identify via OAuth, but I can't use OAuth becuase I am not identified. Clearly there's a Miraheze bug here. OAuth needs to be accessible despite a wiki being closed.

Approval request: https://war.wikipedia.org/wiki/Wikipedia:Polisiya_hit_bot#InternetArchiveBot

Approval request: https://vi.wikipedia.org/wiki/Wikipedia:Robot/Y%C3%AAu_c%E1%BA%A7u_c%E1%BA%A5p_quy%E1%BB%81n#C%E1%BA%A5p_quy%E1%BB%81n_cho_InternetArchiveBot

Approval request: https://ur.wikipedia.org/wiki/%D9%88%DB%8C%DA%A9%DB%8C%D9%BE%DB%8C%DA%88%DB%8C%D8%A7:%D8%AF%D8%B1%D8%AE%D9%88%D8%A7%D8%B3%D8%AA_%D8%A8%D8%B1%D8%A7%D8%A6%DB%92_%D8%A7%D8%AE%D8%AA%DB%8C%D8%A7%D8%B1%D8%A7%D8%AA_%D8%B1%D9%88%D8%A8%DB%81#InternetArchiveBot

Approval request: https://ta.wikipedia.org/wiki/%E0%AE%B5%E0%AE%BF%E0%AE%95%E0%AF%8D%E0%AE%95%E0%AE%BF%E0%AE%AA%E0%AF%8D%E0%AE%AA%E0%AF%80%E0%AE%9F%E0%AE%BF%E0%AE%AF%E0%AE%BE_%E0%AE%AA%E0%AF%87%E0%AE%9A%E0%AF%8D%E0%AE%9A%E0%AF%81:%E0%AE%A4%E0%AE%BE%E0%AE%A9%E0%AE%BF%E0%AE%AF%E0%AE%99%E0%AF%8D%E0%AE%95%E0%AE%BF%E0%AE%95%E0%AE%B3%E0%AF%8D#InternetArchiveBot

Approval request: https://simple.wikipedia.org/wiki/Wikipedia_talk:Bots#InternetArchiveBot

I need the bot flag to proceed.

Approval request: https://pl.wikipedia.org/wiki/Wikipedia:Boty/Zg%C5%82oszenia#InternetArchiveBot

Approval request: https://my.wikipedia.org/wiki/%E1%80%9D%E1%80%AE%E1%80%80%E1%80%AE%E1%80%95%E1%80%AE%E1%80%B8%E1%80%92%E1%80%AE%E1%80%B8%E1%80%9A%E1%80%AC%E1%80%B8:Bots/Requests_for_approval#InternetArchiveBot

Approval request: https://ka.wikipedia.org/wiki/%E1%83%95%E1%83%98%E1%83%99%E1%83%98%E1%83%9E%E1%83%94%E1%83%93%E1%83%98%E1%83%90:%E1%83%91%E1%83%9D%E1%83%A2%E1%83%94%E1%83%91%E1%83%98/%E1%83%A1%E1%83%A2%E1%83%90%E1%83%A2%E1%83%A3%E1%83%A1%E1%83%98%E1%83%A1_%E1%83%9B%E1%83%9D%E1%83%97%E1%83%AE%E1%83%9D%E1%83%95%E1%83%9C%E1%83%90#User:InternetArchiveBot