Page MenuHomePhabricator

Copypatrol is down
Closed, ResolvedPublic

Description

CopyPatrol is down for six hours, and has failed to re-start on its own. Any help appreciated. Thanks.

Event Timeline

It is likely the certificate expiration problem. I hope it will be solved asap since it hits a lot of our tools. (Probably the Ph task has been file, but I have no idea where to find it).

Yes, this one is probably relevant, thanks.

@eranroz: The logs have the bot prompting for a password.

I restored the session for the bot user so it should not promote for a password now

It didn't stick. Once it found a case to report, it is back to prompting for the password.

MusikAnimal claimed this task.
MusikAnimal moved this task from Backlog to Done on the CopyPatrol board.

Seems to be back now. For the record, I also had suspected we might see downtime from this bot due to the certificate problem, but ruled that out since that occurred on September 30 and the bot continued to function up until October 2. The bot runs on the grid (not Kubernetes), which apparently is unaffected.

The bot has stopped again, with no new reports for six hours. Thanks in advance for any assistance.

WARNING: API warning (paraminfo): The module "main" does not have a submodule "tokens".

This traces back to report_logger.py#L31. The code looks like it's doing things the right way, but the timing of the error makes me think it might be related to MediaWiki 1.37/Deprecation of legacy API token parameters.

When I start the enwiki job, I see that it's going through recent changes and querying iThenticate just fine. It's when it finds a copyvio and tries to add a tag to the revision that we hit the above warning.

Maybe pywikibot needs to be updated? Hopefully we can do this and still stay on Python 2. Pinging @eranroz @JJMC89 in case you have any ideas.

rPWBCc0cf17c196ca: [mw 1.37] Fix for removed action API token parameters is included in pywikibot 6.6.1; however, pywikibot hasn't supported python2 since 3.0.20200703.

We may need to patch in the new way of fetching tokens ourselves. I take this is the only interaction with the wiki that has a need for a token. If I was any good at Python/pywikibot I'd attempt this myself.

For now I have commented out the code that adds the revision tag, so hopefully the feed will start populating soon. The drawback is Special:NewPagesFeed will no longer indicate copyvios. I believe a good handful of users came to CopyPatrol from Special:NewPagesFeed, but surely having it recorded as a copyvio somewhere (in this case, the CopyPatrol web interface) is better than nowhere.

I've also gone ahead and filed T293688: Write new CopyPatrol backend to replace Plagiabot about porting the bot to Python 3, or rewriting it.

Feed is now back up. I have filed T293692: Fix Plagiabot to use new way of fetching API tokens about implementing a quick fix for the Page Curation integration.

CopyPatrol once again has been down for six hours, and has failed to re-start on its own. Any help appreciated. Thanks.

CopyPatrol once again has been down for six hours, and has failed to re-start on its own. Any help appreciated. Thanks.

Since this is a credits issue I've split it into a new task: T305318: Out of iThenticate credits.

Hi all. We've had no new reports at CopyPatrol for nearly six hours; the bot has apparently failed to re-start on its own. Thanks in advance for any help.

@eranroz It is prompting for your bot's password.

@eranroz could you put the bot credentials somewhere in a file somewhere and run chmod 400 on it? I think it would be good if more than one person is able to restart the bot.

Related, JJMC89 has graciously been working on a complete rewrite of the bot (T293688), so we will soon have new credentials and hopefully we'll be less prone in general to downtime.

I restored the bot credentials for eranbot tool

Hello, the bot is down again, with no new reports for about 10 hours. Thanks in advance for any assistance.

Resolved, functioning normally for 6-7 hours. Thanks.