Same as T277328. I know you might have been told to open a new ticket each time, but I'm pretty sure it's the same issue, so it might be good to re-use the same ticket so we can see past discussions.

Restarting the bot has happened automatically many times over since this bug was filed, and I've restarted it manually too. Every time the number of pages in the buffer climbs to > 100 or more. I'm afraid I don't know what the issue is. Pinging @eranroz and also @JJMC89 in case they are interested in helping. I can revisit this next week but the usual trick of simply restarting is not cutting it this time.

After each restart, the bot finds a revision that it needs to upload to iThenticate. Then it gets stuck checking for results repeatedly (while the queue builds up).

I've checked and we are not out of credits, so that's not the issue. The iThenticate log is very, very slow (as I recall it's always been that way), but when it finally did load I see the first page of uploads are still in the "Processing" state. So it sounds like maybe this is not the same issue as T277328, but an issue on iThenticate's side. I'm not sure if the uploads listed are actually pending, and perhaps there's one that's stuck that I can delete, but I will try to paginate through and see if anything stands out. The general slowness of their site is a huge hurdle.

I have disabled the bot for the time being, as it's just going to keep uploading more documents and make it harder for me to paginate through.

The last upload that was fully processed was April 21, 2021 at 4:40 PM (they did not specify a timezone... I'm assuming UTC). I deleted the upload immediately following that one, thinking maybe that was the one that was "stuck". Here we are an hour or so later and it hasn't made any progress. My work day is about to end, so I'm afraid I'll have to get back to this Sunday or Monday. We may or may not need to contact iThenticate to find out what's going on. But, if things do catch up, you should start seeing new records show up in CopyPatrol (assuming at least some of these ~250 uploads are copyvios). I am leaving the bot turned off for all languages for the time being.

At the time of writing, the uploads are still stuck in the 'processing' state. I have contacted iThenticate support.

The support ticket with iThenticate has been escalated to a technical specialist, and I take it we're in some sort of queue now. If someone is not already looking into the issue, they should be soon. Sorry for the delay.

Signing my name onto this. We need CopyPatrol back, and we needed it back yesterday.

I've heard back from iThenticate that any new uploads should no longer become stuck, so I've re-enabled the cron jobs for all languages. If all is back to normal, you should start seeing new copyvios show up again. Initial spot check of the logs and the iThenticate upload list look good.

There are many hundreds of older uploads that are still stuck in the "processing" state. They said they intend to re-process those and will update us when that happens.

I'll remove the top notice from the tool and resolve this once I see new copyvios appear in the feed.

In T280899#7333570, @Majavah wrote:

In T280899#7333552, @Diannaa wrote:

Hello, the bot appears to have stopped about 10 hours ago and has failed to restart itself. Any help appreciated. Thank you!

Please open a new task for new issues instead of constantly re-opening this one.

This is the standard practice on Phabricator, but Community-Tech is actually content re-using the same task for CopyPatrol, specifically, since it's almost always the same issue, and almost always fixes itself – just as it did this time.

Frankly, there's no need to report downtime for the CopyPatrol feed at all, as we already get emails for this (T262767).