Page MenuHomePhabricator

Migrate copypatrol & plagiabot to use tools.labsdb
Closed, ResolvedPublic

Description

Currently s51306__copyright_p database is on enwiki.labsdb. The database is used by:

  1. eranbot tool (running plagiabot) - writes suspected copyright violations to copyright_diffs
  2. copypatrol tool - read suspected copyright violations. (Possibly maintains wikiprojects table?)

AFAIK:

  • it will not be allowed to have user tables there by December. (see parent task)
  • There is no real use of being on labsdb.enwiki rather than any other DB host, except that it is useful for ad-hock analytics.

Preparations:

  • Copy s51306__copyright_p from enwiki.labsdb to tools.labsdb (data is up to Oct 28 07:00)
  • make sure both tools (eranbot and copypatrol) have access to this DB. (may require DBA, see T134392 for request on the current DB)

Migrate:

Event Timeline

eranroz created this task.Oct 28 2017, 6:53 AM
Restricted Application added a project: Community-Tech. · View Herald TranscriptOct 28 2017, 6:53 AM
Restricted Application added a subscriber: Cyberpower678. · View Herald Transcript
eranroz updated the task description. (Show Details)Oct 28 2017, 7:03 AM
eranroz updated the task description. (Show Details)Oct 28 2017, 7:13 AM

I replicated the DB and granted full access to both eranbot and copypatrol.

It is OK from my side (eranbot) to go on with migration.
@Niharika - Is it OK to migrate CopyPatrol? can you please confirm all the required changes are the .env? and maybe the README.md of CopyPatrol?

I can handle them both so both services will migrate almost the same time.
Thanks

Marostegui added a subscriber: Marostegui.
bd808 awarded a token.Oct 29 2017, 3:36 AM

@eranroz Can you tell me what changes might be required for the env? Here's what I have in it right now -

host=enwiki.labsdb
dbname=s51306__copyright_p
MusikAnimal added a subscriber: MusikAnimal.EditedNov 1 2017, 3:24 PM

I think we need to re-introduce separate credentials for the copyright database and the replicas. I've made a PR at https://github.com/wikimedia/CopyPatrol/pull/52

the database s51306__copyright_p was vanished from host enwiki.labsdb following the redirect (https://lists.wikimedia.org/pipermail/cloud-announce/2017-November/000007.html)
At least there is backup (synced to 28/10) in tools.labsdb - I'm changing the service to write there instead of enwiki.labsdb.
looks like env already use the new host

eranroz closed this task as Resolved.Nov 1 2017, 10:45 PM
eranroz claimed this task.
eranroz updated the task description. (Show Details)
Framawiki added a subscriber: Framawiki.EditedNov 21 2017, 7:55 PM

ehehe, my client script was broken due to this task :D

MusikAnimal moved this task from Backlog to Done on the CopyPatrol board.Feb 2 2018, 4:06 AM