Page MenuHomePhabricator

Create the CheckUserDataPruner service
Closed, ResolvedPublic2 Estimated Story Points

Description

A service is needed to be able to prune data from the CheckUser tables. This service will be created my moving code from other files into one service.

This is needed because the current pruning mechanisms have two different code paths (purgeOldData.php and the job) which presents code duplication. This means that in T353953: Don't use actor IDs for private CheckUser events when these actions are performed by IP addresses, where a new result table is added, we will only need to modify one service instead of having to modify two different files for the same changes.

Acceptance criteria
  • Create the CheckUserDataPruner service and use it in the job and maintenance script purging code paths

Related Objects

StatusSubtypeAssignedTask
Resolvedkostajh
DeclinedNone
ResolvedNiharika
ResolvedMadalina
OpenNone
Resolved Dreamy_Jazz
OpenNone
OpenNone
OpenNone
OpenNone
ResolvedSTran
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz
Resolved Dreamy_Jazz

Event Timeline

Change 929030 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Move Hooks::maybePruneIPData and ::pruneIPData to a new service

https://gerrit.wikimedia.org/r/929030

Change #929030 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] Move Hooks::maybePruneIPData and ::pruneIPData to a new service

Reason:

Will re-create this as it is merge conflicting a lot.

https://gerrit.wikimedia.org/r/929030

Change #1059424 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Create CheckUserDataPruner service

https://gerrit.wikimedia.org/r/1059424

Change #1059427 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Improve method names and documentation in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059427

Change #1059430 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] [WIP] Move pruning code from job to CheckUserDataPruner service.

https://gerrit.wikimedia.org/r/1059430

Change #1059432 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] [WIP] Move static function into method in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059432

Change #1064051 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064051

Change #1059432 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] [WIP] Move static function into method in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059432

Change #1059432 restored by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] [WIP] Move static function into method in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059432

Doing this is more complicated than first thought, so is not worth the time to implement (compared to the extra code).

Change #1059432 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] [WIP] Move static function into method in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059432

Change #1064051 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064051

Change #1059427 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] Improve method names and documentation in CheckUserDataPruner

https://gerrit.wikimedia.org/r/1059427

Change #1059424 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] Create CheckUserDataPruner service

https://gerrit.wikimedia.org/r/1059424

Change #1059430 abandoned by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] [WIP] Move pruning code from job to CheckUserDataPruner service

https://gerrit.wikimedia.org/r/1059430

Change #1064051 restored by Dreamy Jazz:

[mediawiki/extensions/CheckUser@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064051

Re-opening this with a more limited scope after further review.

Change #1064430 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Move PruneCheckUserDataJob::purgeDataFromLocalTable to a service

https://gerrit.wikimedia.org/r/1064430

Change #1064431 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Use CheckUserDataPruner in the purgeOldData.php maintenance script

https://gerrit.wikimedia.org/r/1064431

Change #1064432 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Replace slow PurgeOldDataTest with faster tests

https://gerrit.wikimedia.org/r/1064432

Change #1064051 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064051

Change #1064428 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Clean up code in PruneCheckUserDataJob

https://gerrit.wikimedia.org/r/1064428

Change #1064429 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] PruneCheckUserDataJob: Move static function into a private method

https://gerrit.wikimedia.org/r/1064429

Change #1064458 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064458

Change #1064051 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064051

Change #1064428 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Clean up code in PruneCheckUserDataJob

https://gerrit.wikimedia.org/r/1064428

Change #1064429 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] PruneCheckUserDataJob: Move static function into a private method

https://gerrit.wikimedia.org/r/1064429

Change #1064430 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Move PruneCheckUserDataJob::purgeDataFromLocalTable to a service

https://gerrit.wikimedia.org/r/1064430

Change #1064431 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Use CheckUserDataPruner in the purgeOldData.php maintenance script

https://gerrit.wikimedia.org/r/1064431

Change #1064432 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Replace slow PurgeOldDataTest with faster tests

https://gerrit.wikimedia.org/r/1064432

For this task, there should be no changes to how purging works for cu_changes, cu_log_event, and cu_private_event. So QA may wish to regression test for this ticket.

Change #1064458 merged by jenkins-bot:

[operations/mediawiki-config@master] Remove wgCheckUserPurgeOldClientHintsData

https://gerrit.wikimedia.org/r/1064458

Mentioned in SAL (#wikimedia-operations) [2024-09-02T12:58:45Z] <dreamyjazz@deploy1003> Started scap sync-world: Backport for [[gerrit:1064458|Remove wgCheckUserPurgeOldClientHintsData (T359560)]]

Mentioned in SAL (#wikimedia-operations) [2024-09-02T13:01:36Z] <dreamyjazz@deploy1003> dreamyjazz: Backport for [[gerrit:1064458|Remove wgCheckUserPurgeOldClientHintsData (T359560)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2024-09-02T13:07:35Z] <dreamyjazz@deploy1003> Finished scap sync-world: Backport for [[gerrit:1064458|Remove wgCheckUserPurgeOldClientHintsData (T359560)]] (duration: 08m 47s)

dom_walden subscribed.

I ran the purgeOldData script on my local to check that all the expired rows are being removed from the cu_* and cuci_* tables and none of the not expired rows.