Page MenuHomePhabricator

Run deleteAutopatrolLogs script for Wikidata (WMF)
Closed, ResolvedPublic

Description

After April 4th 2018, run the script from T189594: Create maintenance script to remove autopatrol actions from logging table..
This script should delete auto patrol actions in wikidata, a cronjob in operations/puppet (in modules/mediawiki/manifests/maintenance/wikidata.pp). Already tested the script on mediawikiwiki and worked just fine.

Event Timeline

Krinkle renamed this task from Remove old autopatrol entries from logging table to Run deleteAutopatrolLogs script for Wikidata (WMF).Mar 26 2018, 11:42 PM
Krinkle added a project: DBA.
Ladsgroup moved this task from Tasks to In Progress on the Wikidata-Ministry-Of-Magic board.
Ladsgroup updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2018-04-05T12:41:18Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=wikidatawiki --before 20180223210426 (T189596)

Change 424300 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] mediawiki: Start deleteAutoPatrolLogs from Wikidata logging table

https://gerrit.wikimedia.org/r/424300

Change 424300 merged by Marostegui:
[operations/puppet@production] mediawiki: Start deleteAutoPatrolLogs from Wikidata logging table

https://gerrit.wikimedia.org/r/424300

Also, once the script has finished going all the way to 23 February 2018, we should also do another run of the script for autopatrol entries from 23 February 2018 to 5 April 2018, as well as those old ones that still have 'log_action = patrol', with the information included in the log_params field instead.

This is also on my radar which I will get to it really soon :)

Change 428297 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] mediawiki: Delete pre-2016 autopatrol actions from logging table of wikidata

https://gerrit.wikimedia.org/r/428297

@Ladsgroup after the table rebuilding on dbstore2001, purging is going at a good pace:
https://grafana.wikimedia.org/dashboard/db/mysql?panelId=11&fullscreen&orgId=1&var-dc=codfw%20prometheus%2Fops&var-server=dbstore2001&var-port=13318&from=1524683251842&to=1524726451842

As soon as the counter gets close to 0, the row deletion can continue. At the current pace I predict it will be around 1 day of waitinf, if purge keeps happening at the same rate.

Change 428297 merged by Jcrespo:
[operations/puppet@production] mediawiki: Delete pre-2016 autopatrol actions from logging table of wikidata

https://gerrit.wikimedia.org/r/428297

Change 429407 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] mediawiki: remove --from-id from deleteAutoPatrolLog script run

https://gerrit.wikimedia.org/r/429407

Change 429407 merged by Jcrespo:
[operations/puppet@production] mediawiki: remove --from-id from deleteAutoPatrolLog script run

https://gerrit.wikimedia.org/r/429407

@Ladsgroup After all the old automatic patrols on Wikidata have been eliminated, we will need one more patch to get rid of all the remaining automatic patrols from 20180223210400 to 20180405131000. And once that is finished, congratulations! From then on, only manual patrols will be listed in Wikidata's patrol log and no more automatic patrols will ever be listed again.

Also on my radar. I will start deleting them around 8th of May to be sure all autoaptrolled recentchanges entries have rc_patrolled = 2 so it's okay to delete the logs.

Change 432350 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] mediawiki: Stop cronjob to delete autopatrol actions

https://gerrit.wikimedia.org/r/432350

Change 432350 merged by Jcrespo:
[operations/puppet@production] mediawiki: Stop cronjob to delete autopatrol actions

https://gerrit.wikimedia.org/r/432350

There are still two automatic patrols from 28 February 2018 as well as autopatrols from 27 March to 30 March 2016 left on Wikidata. We now need to get rid of both entries from the end of February 2018 as well as all the ones from the last week of March 2016.

Everything was still working, if you were patient, you could see they are gone by now.

Change 432570 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/puppet@production] mediawiki: remove stopped deleteAutoPatrol script for wikidata

https://gerrit.wikimedia.org/r/432570

Change 432570 merged by Jcrespo:
[operations/puppet@production] mediawiki: remove stopped deleteAutoPatrol script for wikidata

https://gerrit.wikimedia.org/r/432570

Ladsgroup moved this task from Blocked to Done on the Wikidata-Ministry-Of-Magic board.

This has been done \o/ 1.1B rows deleted including 99.1% of logging table of wikidatawiki