Page MenuHomePhabricator
Create Task
Maniphest T340959

Update CheckUser prune job to remove client hint data
Closed, ResolvedPublic
Actions

Assigned To
Dreamy_Jazz
Authored By
Dreamy_Jazz
Jul 3 2023, 11:18 AM
Tags
Referenced Files
F37145736: T340959_CheckUser_ClientHintData_ModifyPrune3.png
Jul 21 2023, 6:12 PM
F37145729: T340959_CheckUser_ClientHintData_ModifyPrune2.png
Jul 21 2023, 6:12 PM
F37145727: T340959_CheckUser_ClientHintData_ModifyPrune1.png
Jul 21 2023, 6:12 PM
Subscribers
Aklapper
Dreamy_Jazz
GMikesell-WMF
kostajh

Description

Similar to T337943 the CheckUser prune job (checkuserPruneCheckUserDataJob) should also remove client hint data. The prune job is triggered on average 1/10 times a RecentChanges entry is saved and deletes expired entries in a similar way to maintenance/purgeOldData but doesn't prune the recentchanges table.

The job definition is at src/Jobs/PruneCheckUserDataJob.php.

Details

SubjectRepoBranchLines +/-
clienthints: Purge expired client hint datamediawiki/extensions/CheckUsermaster+924 -35
Disable purging of old client hint data by defaultoperations/mediawiki-configmaster+5 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

Dreamy_Jazz created this task.Jul 3 2023, 11:18 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 3 2023, 11:18 AM
Dreamy_Jazz updated the task description. (Show Details)Jul 3 2023, 11:20 AM
Dreamy_Jazz mentioned this in T257893: [EPIC] Support User-Agent Client Hints header in CheckUser.Jul 3 2023, 5:57 PM
Dreamy_Jazz mentioned this in T337943: Update maintenance/purgeOldData to remove client hint data.Jul 5 2023, 12:22 PM
gerritbot added a comment.Jul 6 2023, 3:09 PM

Change 935776 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] [WIP] clienthints: Purge expired client hint data

https://gerrit.wikimedia.org/r/935776

gerritbot added a project: Patch-For-Review.Jul 6 2023, 3:09 PM
Dreamy_Jazz claimed this task.Jul 6 2023, 3:09 PM
Dreamy_Jazz edited projects, added Anti-Harassment (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment.
Dreamy_Jazz moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to In Progress 💪 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.
gerritbot added a comment.Jul 6 2023, 3:14 PM

Change 936055 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Disable purging of old client hint data by default

https://gerrit.wikimedia.org/r/936055

Dreamy_Jazz triaged this task as Medium priority.Jul 6 2023, 4:01 PM
Dreamy_Jazz moved this task from In Progress 💪 to Code Review 🔍 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.
gerritbot added a comment.Jul 6 2023, 8:06 PM

Change 936055 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable purging of old client hint data by default

https://gerrit.wikimedia.org/r/936055

Stashbot added a comment.Jul 6 2023, 8:06 PM

Mentioned in SAL (#wikimedia-operations) [2023-07-06T20:06:31Z] <thcipriani@deploy1002> Started scap: Backport for [[gerrit:936055|Disable purging of old client hint data by default (T340959 T341076)]]

Stashbot mentioned this in T341076: Creation of database tables cu_useragent_clienthints and cu_useragent_clienthints_map.Jul 6 2023, 8:06 PM

Mentioned in SAL (#wikimedia-operations) [2023-07-06T20:07:54Z] <thcipriani@deploy1002> thcipriani and dreamyjazz: Backport for [[gerrit:936055|Disable purging of old client hint data by default (T340959 T341076)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet

Stashbot added a comment.Jul 6 2023, 8:16 PM

Mentioned in SAL (#wikimedia-operations) [2023-07-06T20:16:40Z] <thcipriani@deploy1002> Finished scap: Backport for [[gerrit:936055|Disable purging of old client hint data by default (T340959 T341076)]] (duration: 10m 08s)

Dreamy_Jazz moved this task from Code Review 🔍 to Blocked/Stalled 🚧 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 11 2023, 6:26 PM

While a response from legal is gained regarding deletion of data.

Dreamy_Jazz moved this task from Blocked/Stalled 🚧 to Code Review 🔍 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 11 2023, 10:21 PM
gerritbot added a comment.Jul 14 2023, 1:45 PM

Change 935776 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] clienthints: Purge expired client hint data

https://gerrit.wikimedia.org/r/935776

Dreamy_Jazz moved this task from Code Review 🔍 to QA/Testing 🐞 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 14 2023, 1:52 PM
Maintenance_bot removed a project: Patch-For-Review.Jul 14 2023, 2:11 PM
Dreamy_Jazz added a project: CheckUser.Jul 15 2023, 1:23 PM
Dreamy_Jazz added a comment.Jul 19 2023, 2:30 PM

Suggested QA steps:

  • Reduce $wgCUDMaxAge to a small enough value for the test
  • Empty cu_useragent_clienthints_map and cu_useragent_clienthints for the test
  • Make some test edits
  • Inspect cu_useragent_clienthints_map and check that rows exist
  • Modify Hooks::maybePruneIPData to temporarily comment out the if statement, such that when this method is called ::pruneIPData is always called.
  • Wait until these edits were made over $wgCUDMaxAge seconds old
  • Make a test edit and record the revision ID of this edit
  • Inspect cu_useragent_clienthints_map and check that the only rows have uachm_reference_id as the revision ID of the test edit.
GMikesell-WMF subscribed.Jul 21 2023, 6:12 PM

@Dreamy_Jazz I had no issues with the test steps above and the rows match with the revision id of the test edit as seen in the screenshots. I will move this to Done. Thanks for all your work and help!

T340959_CheckUser_ClientHintData_ModifyPrune1.png (479×870 px, 62 KB)

T340959_CheckUser_ClientHintData_ModifyPrune2.png (1×1 px, 197 KB)

T340959_CheckUser_ClientHintData_ModifyPrune3.png (493×1 px, 142 KB)

GMikesell-WMF moved this task from QA/Testing 🐞 to Done Q1 2023-2024 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 21 2023, 6:12 PM
dom_walden mentioned this in T258105: Implement storage for User-Agent Client Hints header data.Jul 26 2023, 12:52 PM
Tchanders closed this task as Resolved.Aug 14 2023, 11:02 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits