Page MenuHomePhabricator
Create Task
Maniphest T413868

Populate the cu_useragent table and agent_id columns on WMF wikis
Closed, ResolvedPublic
Actions

Description

Summary

This task is to run the maintenance script added in T361198: Create a maintenance script to populate the cu_useragent table and agent_id columns with values from the agent columns on WMF wikis

Background

Acceptance criteria

  • The maintenance script is run on all WMF wikis
    • s1
    • s2
    • s3
    • s4
    • s5
    • s6
    • s7
    • s8

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenFeatureNoneT315926 Provide a way to search all of the XFF addresses listed in the XFF string
 OpenFeatureNoneT326379 Normalise the XFF and user agent columns
 Resolved• TBolligerT120734 Epic ⚡️ Improve MediaWiki's blocking tools
 OpenNoneT146837 Add ability to search by user agent from CheckUser interface
 OpenNoneT249374 CU 2.0: Filter out exact UAs in the Compare tab
 OpenNoneT147894 Create index for agent_id columns in the CheckUser result tables
 OpenNoneT305930 Normalize cu_changes table
 OpenDreamy_JazzT361139 Normalise the user agent column in CheckUser result tables
 OpenDreamy_JazzT361208 Remove agent columns from CheckUser result tables
 ResolvedDreamy_JazzT361206 Stop writing old for user agent schema migration on WMF wikis
 ResolvedDreamy_JazzT361205 Stop writing old for user agent schema migration
 ResolvedDreamy_JazzT361199 Set user agent schema migration config to read new on WMF wikis
 ResolvedDreamy_JazzT413868 Populate the cu_useragent table and agent_id columns on WMF wikis
 ResolvedDreamy_JazzT361198 Create a maintenance script to populate the cu_useragent table and agent_id columns with values from the agent columns
 ResolvedDreamy_JazzT361196 Write to the cu_useragent table and agent_id columns on WMF wikis
 ResolvedDreamy_JazzT361197 Write to the cu_useragent table and agent_id columns on by default in extension.json
 ResolvedDreamy_JazzT361172 Purge rows from cu_useragent
 ResolvedDreamy_JazzT413873 Update remaining code that writes to the CheckUser result tables to write to the agent_id columns
 ResolvedDreamy_JazzT361174 Write to agent_id columns in the CheckUserInsert service when migration configuration allows it
 ResolvedDreamy_JazzT361173 Add schema migration config for cu_useragent table
 ResolvedDreamy_JazzT361140 Add user agent ID column to each CheckUser result table
 ResolvedDreamy_JazzT359312 Create cu_useragent table
 ResolvedMarosteguiT361631 Unable to create table cu_useragent on labtestwiki
 DeclinedNoneT328289 update labtestwiki user and password
 ResolvedPapaulT369308 Decommission clouddb2002-dev.codfw.wmnet
 ResolvedtaaviT378260 Retire labtestwiki
 ResolvedMarosteguiT361673 Filter cu_useragent on sanitarium
 OpenSnwachukwuT361210 Changes to the cuc_agent column in the cu_changes table

Event Timeline

Dreamy_Jazz created this task.Jan 6 2026, 11:20 AM
Dreamy_Jazz added subtasks: T361198: Create a maintenance script to populate the cu_useragent table and agent_id columns with values from the agent columns, T361196: Write to the cu_useragent table and agent_id columns on WMF wikis.
Dreamy_Jazz updated the task description. (Show Details)
Dreamy_Jazz moved this task from Inbox to Next sprint on the Product Safety and Integrity board.Jan 6 2026, 11:06 PM
mszwarc closed subtask T361198: Create a maintenance script to populate the cu_useragent table and agent_id columns with values from the agent columns as Resolved.Jan 7 2026, 2:32 PM
Dreamy_Jazz closed subtask T361196: Write to the cu_useragent table and agent_id columns on WMF wikis as Resolved.Jan 19 2026, 1:49 PM
Stashbot added a comment.Jan 19 2026, 1:52 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-19T13:52:52Z] <Dreamy_Jazz> Running populateUserAgentTable.php on group0 wikis for T413868

Dreamy_Jazz edited projects, added Product Safety and Integrity (Sprint Daffodil (Jan 19 - Feb 6)); removed Product Safety and Integrity.Jan 19 2026, 1:58 PM
Stashbot added a comment.Jan 19 2026, 3:24 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-19T15:23:59Z] <Dreamy_Jazz> Running populateUserAgentTable.php on group1 wikis for T413868

Stashbot added a comment.Jan 20 2026, 10:23 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:23:20Z] <Dreamy_Jazz> Running foreachwikiindblist group2.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 500 for T413868

Dreamy_Jazz claimed this task.Jan 20 2026, 10:23 AM
Dreamy_Jazz moved this task from Backlog to In progress on the Product Safety and Integrity (Sprint Daffodil (Jan 19 - Feb 6)) board.
Stashbot added a comment.Jan 20 2026, 10:41 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:41:23Z] <Dreamy_Jazz> Restarted group1 run with foreachwikiindblist group1.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 1000 --sleep 1 for T413868

Stashbot added a comment.Jan 20 2026, 10:42 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:42:06Z] <Dreamy_Jazz> Restarted group2 run with foreachwikiindblist group2.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 1000 --sleep 1 for T413868

Stashbot added a comment.Jan 20 2026, 10:43 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:43:07Z] <Dreamy_Jazz> Stopped all maintenance script runs for T413868

Stashbot added a comment.Jan 20 2026, 10:44 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:44:02Z] <Dreamy_Jazz> Running foreachwikiindblist large.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 1000 --sleep 1 for T413868

Stashbot added a comment.Jan 20 2026, 10:44 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T10:44:24Z] <Dreamy_Jazz> Running foreachwikiindblist small.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 1000 for T413868

Stashbot added a comment.Tue, Jan 20, 11:35 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-20T11:35:13Z] <Dreamy_Jazz> Running foreachwikiindblist medium.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 1000 -sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 11:58 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T11:58:29Z] <Dreamy_Jazz> Run of medium.dblist for T413868 has completed

Stashbot added a comment.Wed, Jan 21, 11:58 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T11:58:42Z] <Dreamy_Jazz> Stopped run of script for T413868 on large.dblist

Stashbot added a comment.Wed, Jan 21, 11:59 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T11:59:43Z] <Dreamy_Jazz> Running foreachwikiindblist s4.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 3000 --sleep 2 for T413868

Stashbot added a comment.Wed, Jan 21, 12:03 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T12:03:34Z] <Dreamy_Jazz> Running foreachwikiindblist s1.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 10000 --sleep 5 for T413868

Stashbot added a comment.Wed, Jan 21, 12:04 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T12:04:22Z] <Dreamy_Jazz> Running foreachwikiindblist s1.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 2 for T413868

Stashbot added a comment.Wed, Jan 21, 12:09 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T12:09:56Z] <Dreamy_Jazz> Running foreachwikiindblist s2.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 12:11 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T12:10:58Z] <Dreamy_Jazz> Running foreachwikiindblist s3.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 1:40 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:40:44Z] <Dreamy_Jazz> Running foreachwikiindblist s5.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 1:41 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:41:30Z] <Dreamy_Jazz> Run of script for T413868 has finished on s2

Stashbot added a comment.Wed, Jan 21, 1:42 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:42:04Z] <Dreamy_Jazz> Run of script for T413868 has finished on s3

Stashbot added a comment.Wed, Jan 21, 1:43 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:43:36Z] <Dreamy_Jazz> Running foreachwikiindblist s6.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 1:44 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:44:48Z] <Dreamy_Jazz> Running foreachwikiindblist s7.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 5000 --sleep 1 for T413868

Stashbot added a comment.Wed, Jan 21, 1:45 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T13:45:43Z] <Dreamy_Jazz> Running foreachwikiindblist s8.dblist extensions/CheckUser/maintenance/populateUserAgentTable.php --batch-size 3000 --sleep 2 for T413868

Stashbot added a comment.Wed, Jan 21, 2:37 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T14:36:58Z] <Dreamy_Jazz> Run of script for T413868 has finished on s5

Stashbot added a comment.Wed, Jan 21, 2:37 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T14:37:11Z] <Dreamy_Jazz> Run of script for T413868 has finished on s6

Stashbot added a comment.Wed, Jan 21, 2:37 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-21T14:37:40Z] <Dreamy_Jazz> Run of script for T413868 has finished on s7

Dreamy_Jazz updated the task description. (Show Details)Wed, Jan 21, 2:40 PM
Dreamy_Jazz updated the task description. (Show Details)Wed, Jan 21, 2:53 PM
Stashbot added a comment.Thu, Jan 22, 11:55 AM

Mentioned in SAL (#wikimedia-operations) [2026-01-22T11:55:32Z] <Dreamy_Jazz> Run of script for T413868 has finished on s8

Dreamy_Jazz updated the task description. (Show Details)Thu, Jan 22, 11:55 AM
Dreamy_Jazz updated the task description. (Show Details)
Dreamy_Jazz updated the task description. (Show Details)
Stashbot added a comment.Thu, Jan 22, 12:00 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-22T12:00:16Z] <Dreamy_Jazz> Run of script for T413868 has finished on s4

Stashbot added a comment.Thu, Jan 22, 12:00 PM

Mentioned in SAL (#wikimedia-operations) [2026-01-22T12:00:20Z] <Dreamy_Jazz> Run of script for T413868 has finished on s1

Dreamy_Jazz closed this task as Resolved.Thu, Jan 22, 12:55 PM
Dreamy_Jazz moved this task from In progress to Done on the Product Safety and Integrity (Sprint Daffodil (Jan 19 - Feb 6)) board.

Double checked with a run on all.dblist

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits