Page MenuHomePhabricator

Stop logging autopatrol actions
Closed, ResolvedPublic

Description

Problem statement

  • The majority of log entries for most wikis (especially RCPatrol wikis) are autopatrol actions. (Note that before April 2016, these were not distinct from explicit patrol actions.) There doesn't appear to be much value in logging these.
    • The data is almost identical to the revision row, except with some substituted localisation messages.
    • The only unique data in the logging row is the (boolean) fact that it was an autopatrolled revision.

Solution

  • Stop logging autopatrol actions (and possibly purge all the previous autopatrol actions).
  • Store the fact that it was autopatrolled in the rc_patrolled field instead (currently boolean tinyint, could add a third state for autopatrolled).

The proposed expansion of rc_patrolled is logically similar to the previous expansion we did of the patrol log type (from patrol/patrol, to also differentiating patrol/autopatrol).

Original task description by @kaldari

It seems that the majority of log entries for most wikis (especially RCPatrol wikis) are autopatrol actions. (Note that these were not distinct from patrol actions prior to April 2016.) I don't see any actual reason why we want to log autopatrol actions. The data is basically useless after 30 days and we have an rc_patrolled flag in recentchanges that is more useful.
Considering that some logging tables are approaching 1 billion records (see T12331#3876482) and that this is (soft-)blocking deployment of new, more useful, logging types, I think we should just stop logging autopatrol actions entirely (and possibly purge all the previous autopatrol actions while we're at it).
Is there any reason we need this data in the logging table?

See also

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 403115 restored by Ladsgroup:
Change default of $wgLogAutopatrol from true to false

Reason:
This is done now, let's move forward with this

https://gerrit.wikimedia.org/r/403115

Mentioned in SAL (#wikimedia-operations) [2018-04-04T08:19:33Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=mediawikiwiki --check-old --before 20160423210426 (T184485)

Mentioned in SAL (#wikimedia-operations) [2018-04-04T09:02:36Z] <Amir1> start of mwscript deleteAutoPatrolLogs.php --wiki=mediawikiwiki--before 20180223210426 --sleep 2 (T184485)

Mentioned in SAL (#wikimedia-operations) [2018-04-04T09:25:17Z] <Amir1> end of the deleteAutoPatrolLogs.php script on mediawikiwiki (T184485)

Change 424293 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Stop logging autopatrol actions in wikidatawiki

https://gerrit.wikimedia.org/r/424293

Change 424293 merged by jenkins-bot:
[operations/mediawiki-config@master] Stop logging autopatrol actions in wikidatawiki

https://gerrit.wikimedia.org/r/424293

Mentioned in SAL (#wikimedia-operations) [2018-04-05T13:09:54Z] <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:424293|Stop logging autopatrol actions in wikidatawiki (T184485)]] (duration: 01m 16s)

Change 403115 merged by jenkins-bot:
[mediawiki/core@master] Change default of $wgLogAutopatrol from true to false

https://gerrit.wikimedia.org/r/403115

Change 424362 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Disable logging autopatrol actins in commonswiki

https://gerrit.wikimedia.org/r/424362

Mentioned in SAL (#wikimedia-operations) [2018-04-05T17:33:26Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --before 20180223210426 (T184485)

Mentioned in SAL (#wikimedia-operations) [2018-04-05T17:56:45Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=commonswiki --before 20180223210426 --from-id 156008475 (T184485)

Change 424362 merged by jenkins-bot:
[operations/mediawiki-config@master] Disable logging autopatrol actins in commonswiki

https://gerrit.wikimedia.org/r/424362

Mentioned in SAL (#wikimedia-operations) [2018-04-05T18:08:30Z] <catrope@tin> Synchronized wmf-config/InitialiseSettings.php: Disable logging autopatrol actions on commonswiki (T184485) (duration: 01m 17s)

Mentioned in SAL (#wikimedia-operations) [2018-04-06T11:50:10Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=fawiki --before 20180223210426 --sleep 2 (T184485)

Mentioned in SAL (#wikimedia-operations) [2018-04-09T06:57:59Z] <Amir1> start of ladsgroup@terbium:~$ mwscript deleteAutoPatrolLogs.php --wiki=zhwiktionary --check-old --before 20180223210426 --sleep 2 (T184485)

Change 425719 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Stop logging autopatrol actions everywhere

https://gerrit.wikimedia.org/r/425719

The server kitties will be so happy!
Macro server_kitty:

Change 425719 merged by jenkins-bot:
[operations/mediawiki-config@master] Stop logging autopatrol actions everywhere

https://gerrit.wikimedia.org/r/425719

Mentioned in SAL (#wikimedia-operations) [2018-04-11T23:09:53Z] <dereckson@tin> Synchronized wmf-config/InitialiseSettings.php: Stop logging autopatrol actions everywhere (T184485) (duration: 01m 18s)

Zache added a subscriber: Zache.Apr 12 2018, 10:19 AM

Btw. By disabling the logging we lose the ability do stats/graphs how much of the edits were patrolled/autopatrolled/not patrolled if needed outside of the 30 days recentchanges table window.

Btw. By disabling the logging we lose the ability do stats/graphs how much of the edits were patrolled/autopatrolled/not patrolled if needed outside of the 30 days recentchanges table window.

Graph-wise it will be hard but stats-wise, you can just get list of autopatrolled users and consider their actions as patrolled. This is pretty straightforward thing to do

Change 425880 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@master] Drop code for the feature switch of logging autopatrol actions

https://gerrit.wikimedia.org/r/425880

Change 425880 merged by jenkins-bot:
[mediawiki/core@master] Remove the feature switch for disabling autopatrol logging

https://gerrit.wikimedia.org/r/425880

Mentioned in SAL (#wikimedia-operations) [2018-04-18T09:46:21Z] <Amir1> start of deleting auto patrol actions in small wikis (T184485)

@Ladsgroup The next patch you might want to upload is to request that the following lines (110 to 113) be completely removed from the InitialiseSettings.php file.

# T184485
'wgLogAutopatrol' => [
	'default' => false,
],

After removing the wgLogAutopatrol variable, the above lines are not needed any more and should be completely removed.

This is on my radar. Will get to it once the changes hit production.

EddieGP removed a subscriber: EddieGP.Apr 20 2018, 2:50 PM

Change 428624 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[operations/mediawiki-config@master] Clean up old config for logging autopatrol actions

https://gerrit.wikimedia.org/r/428624

Change 428624 merged by jenkins-bot:
[operations/mediawiki-config@master] Clean up old config for logging autopatrol actions

https://gerrit.wikimedia.org/r/428624

Mentioned in SAL (#wikimedia-operations) [2018-04-24T13:37:53Z] <ladsgroup@tin> Synchronized wmf-config/InitialiseSettings.php: [[gerrit:428624|Clean up old config for logging autopatrol actions (T184485)]] (duration: 00m 58s)

He7d3r added a subscriber: He7d3r.May 8 2018, 8:01 PM

Is this task related to the fact that no autopatrol actions are shown on ptwiki anymore? This list is empty (and takes forever to load):
https://pt.wikipedia.org/wiki/Special:Log/patrol?subtype=autopatrol

Yes, We have been removing autopatrol actions (at least anything after September 2011) as well.

I think this subtype should be deactivated to prevent this query to be run. It is of no use at all now.

Yes, I make a phabricator ticket and make it happen ASAP.

All automatic patrols on Wikipedia since 1 March 2012 (when Wikipedia started using MediaWiki 1.19) are now gone. Next, we should make the deleteAutopatrolLogs.php script also detect and get rid of pre-1.19 automatic patrols as well. Then, we could start running the script on wikis that existed prior to 2012 to completely get rid of all very old automatic patrols.

I have already deleted 1.1B rows from the database and deleting those rows that at the most will be about 10% of what we have accomplished by now will gain us next to none. I can't put too much time on it unfortunately. I will try to poke it when I have some free time but it's not high priority.

Change 433714 had a related patch set uploaded (by Ladsgroup; owner: Amir Sarabadani):
[mediawiki/core@master] Enable deleteAutoPatrolLogs to delete pre-2011 autopatrol logs as well

https://gerrit.wikimedia.org/r/433714

Change 433714 merged by jenkins-bot:
[mediawiki/core@master] Enable deleteAutoPatrolLogs to delete pre-2011 autopatrol logs as well

https://gerrit.wikimedia.org/r/433714

This comment was removed by Nirmos.
Ladsgroup closed this task as Resolved.May 28 2018, 9:24 AM
Ladsgroup claimed this task.
Ladsgroup moved this task from In Progress to Done on the Wikidata-Ministry-Of-Magic board.

It is officially done now 🎉🎉🎉🎉

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptMay 28 2018, 9:25 AM

wooot!

Am 28.05.2018 um 11:26 schrieb Ladsgroup:

Ladsgroup closed this task as "Resolved".
Ladsgroup claimed this task.
Ladsgroup added a comment.
Herald added a project: User-Ladsgroup.
It is officially done now 🎉🎉🎉🎉
*TASK DETAIL*
https://phabricator.wikimedia.org/T184485
*EMAIL PREFERENCES*
https://phabricator.wikimedia.org/settings/panel/emailpreferences/
*To: *Ladsgroup
*Cc: *Ciencia_Al_Poder, He7d3r, GeoffreyT2000, Lucas_Werkmeister_WMDE, Zache,
Stashbot, Liuxinyu970226, daniel, Nirmos, brion, Tgr, Nikerabbit, Ladsgroup,
Framawiki, ArielGlenn, MGChecker, MZMcBride, Nemo_bis, WMDE-leszek, Catrope,
gerritbot, MaxSem, Bawolff, Krinkle, jcrespo, Aklapper, kaldari, Versusxo,
Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Baloch007, Darkminds3113,
Bsandipan, Lordiis, lisong, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, Lewizho99,
Maathavan, Johan, SBisson, Luke081515, GWicke, jayvdb, fbstj, Lydia_Pintscher,
santhosh, TheDJ, Jdforrester-WMF, Rxy, Jay8g, Ltrlg, bd808, Legoktm

238482n375 set Security to Software security bug.Jun 15 2018, 8:06 AM
238482n375 changed the visibility from "Public (No Login Required)" to "Custom Policy".
This comment was removed by Vgutierrez.
Restricted Application added a project: Security. · View Herald TranscriptJun 15 2018, 10:15 AM
Vgutierrez changed the visibility from "Custom Policy" to "Public (No Login Required)".
Vgutierrez added a subscriber: Vgutierrez.
Vgutierrez removed a subscriber: Vgutierrez.
ArielGlenn moved this task from Watching now to Done on the User-ArielGlenn board.Aug 7 2018, 10:00 AM