⚓ T330158 Enable write new for the event table migration

Subject	Repo	Branch	Lines +/-
Write new for event table migration on all wikis	operations/mediawiki-config	master	+1 -4
Write new on group1 except wikidatawiki for event table migration	operations/mediawiki-config	master	+2 -0
Write new on group0 for event table migration	operations/mediawiki-config	master	+1 -1
CheckUser event table migration: Write new on group0	operations/mediawiki-config	master	+1 -1
Enable write new on testwiki for CheckUser event tables migration	operations/mediawiki-config	master	+2 -1
Set event tables migration stage to write new and old, read old	mediawiki/extensions/CheckUser	master	+6 -2

Status	Subtype	Assigned	Task
Open		None	T340995 Display client hint data in Special:Investigate
			Restricted Task
			Restricted Task
			Restricted Task
Resolved	Security	Zabe	T311337 CVE-2022-39193: Edits with the performer suppressed still show the performer in results from the CheckUser extension
Resolved	Security	Dreamy_Jazz	T316414 CVE-2022-39193: Special:Investigate can expose supressed information in check results
Resolved	Security	Dreamy_Jazz	T318166 CVE-2022-39193: CheckUser API can expose the suppressed performer
Resolved	Security	Dreamy_Jazz	T316360 Oversighted action text is shown in Special:CheckUser when the checkuser does not have the right to see it
			Restricted Task
Open	BUG REPORT	None	T326866 Special:Investigate can expose suppressed information for log events
Open	BUG REPORT	None	T326867 CheckUser API can expose suppressed information for log events
Open	BUG REPORT	None	T326865 Special:CheckUser can expose suppressed information for log events
Open		Dreamy_Jazz	T253796 Make CheckUser record the log_id of logged actions
Open		None	T139810 RFC: Overhaul the CheckUser extension
Open		None	T132892 CheckUser UI revamp
Open		None	T69811 CheckUsers "Get users" design needs improvements
Open	Feature	Dreamy_Jazz	T24120 Enhance Blocking and tagging from checkuser interface: add dropdowns
Duplicate		None	T145340 Split CheckUser mass-block interface to separate special page
Open	Feature	None	T318458 Add dropdown options for the reason in Special:InvestigateBlock (like Special:Block has)
Open	Feature	None	T319517 Allow the OOUI selectandother field to have the required status indicate that either the dropdown or the input box could be used
Open	Feature	None	T324709 Allow the HTMLForm Select and other field to have a minimum length set
Resolved	Feature	Dreamy_Jazz	T316441 Enforce minimum length of the tag on the client side for the checkuser block form
Resolved	Feature	Dreamy_Jazz	T321427 Allow HTMLForm text based elements to specify a minimum length
Resolved	Feature	Dreamy_Jazz	T321439 Allow TextInputWidget to specify the "minlength" attribute
Open	Feature	None	T316444 The CheckUser block form should explain why users were not blocked or tagged
Open	Feature	None	T314700 Restore the bottom paging links on 'Get users'
Resolved	BUG REPORT	Dreamy_Jazz	T314217 Checkuserblock form is broken when paging links are shown on 'Get users'
Open	Feature	None	T313451 Separate out when the blocks are made for IPs and users in Special:CheckUser 'get users' mode
Open	Feature	None	T18306 Allow user to specify block settings from CU form
Resolved		Glaisher	T41213 Block interface after "Get users" check should allow talk page/send email blocking
Open		None	T27053 Replace user / talk pages feature of "Get users" creates a user/talk page if it doesnt exist
Open	Feature	None	T318459 Make a clearer visual distinction between the UA and IP list in 'get users'
Resolved		Glaisher	T49505 CheckUser results (userlinks) cannot be customized
Resolved		Niharika	T24119 Enhance Blocking and tagging from checkuser interface: add new page
Resolved		None	T14808 Blocking and tagging from checkuser interface
Resolved	Feature	Dreamy_Jazz	T16699 More versatile searching in CheckUser log
Declined		None	T15789 Restore the global CheckUser log
Resolved	Feature	Dreamy_Jazz	T309924 Add check links for both the target and performer in Special:CheckUserLog
Resolved	Feature	Dreamy_Jazz	T309925 Implement User:Amalthea/culoghelper.js into the CheckUser extension
Duplicate		None	T140345 Userlinks in CheckUser should be customizable
Open		None	T21964 Provide a link to the user on account creation
Duplicate		None	T144099 Make CheckUser unit-testable
Resolved		Dreamy_Jazz	T309815 Use OOUI in Special:CheckUser
Resolved		Dreamy_Jazz	T310019 The Special:CheckUser block form should use OOUI
Resolved		Dreamy_Jazz	T266586 Use OOUI in SpecialCheckUserLog
			Restricted Task
Open	Feature	None	T26231 Allow CheckUsers to filter checks for account creations only
Resolved		Dreamy_Jazz	T139809 Bad output of AbuseFilter blocks in CU get edits
Open	BUG REPORT	None	T41013 Incomplete i18n for log entries in CheckUser
Resolved	BUG REPORT	Dreamy_Jazz	T268156 Partial blocks from namespaces are displayed odd
Open		None	T311354 Make CheckUser results easier to parse by userscripts
Open	Feature	None	T311380 Make certain types of edits in 'Get edits' highlightable for ease of parsing the results
Declined	Feature	None	T54849 Checkuser option "get users" should point out when user password is changed
Open	Feature	None	T315488 Show failed login attempts in the results list for the user
			Restricted Task
Open		Dreamy_Jazz	T145265 Store check user data action text in structured format
In Progress		kostajh	T257893 [EPIC] Support User-Agent Client Hints header in CheckUser
Open		None	T341094 Create client hint mapping rows for CheckUser-worthy events
Open		None	T248926 Performance review of checkuser database queries [NOT READY]
Resolved		tstarling	T342613 CompareService::getTotalEditsFromIp queries exceeding TransactionProfiler limits
Open		None	T301992 Insert CheckUser row events during certain 2FA actions
Open		Tchanders	T326393 IP Address Reveal on Log page
Open	Feature	Dreamy_Jazz	T324907 Create separate tables for log events in CheckUser
Resolved		Ladsgroup	T326105 Create cu_log_event and cu_private_event on WMF wikis
Resolved		Milimetric	T327447 FYI: Other changes to the CheckUser tables
Open		None	T329189 Add read new support to Special:Investigate for event table migration
Open		None	T329200 Enable read new support in Special:Investigate's 'Timeline' tab
Open		None	T329201 Enable read new support in Special:Investigate's 'IPs & User agents' tab
Open		Dreamy_Jazz	T347102 Alias column names in Special:Investigate to not include the "cuc_" prefix
Open		None	T341827 Add read new support to the CheckUser API
Open		None	T341829 Enable read new for the event table migration
Resolved		Dreamy_Jazz	T341688 Create LogFormatter for cu_private_event log events
Resolved		Dreamy_Jazz	T330158 Enable write new for the event table migration
Resolved		Dreamy_Jazz	T328874 Create temporary column in cu_changes to assist in migration to the new tables
Resolved		Marostegui	T329203 Add new column cuc_only_for_read_old to cu_changes for migration purposes to wmf wikis
Resolved	PRODUCTION ERROR	Dreamy_Jazz	T342902 Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac
Resolved		Marostegui	T343174 Add missing column cuc_only_for_read_old to testcommonswiki
Open		Dreamy_Jazz	T328992 Add read new support to Special:CheckUser for event table migration
Resolved		Dreamy_Jazz	T328995 Enable read new support in Special:CheckUser's 'Get edits' mode
Resolved	BUG REPORT	Dreamy_Jazz	T347773 Log action text shown when reading new does not respect the deleted status of the log
Resolved		Dreamy_Jazz	T328997 Enable read new support in Special:CheckUser's 'Get users' mode
Open		Dreamy_Jazz	T328998 Enable read new support in Special:CheckUser's 'Get IP Addresses' mode
Resolved		Dreamy_Jazz	T328988 Create a SelectQueryBuilder that allows selecting from all three tables
Open		Dreamy_Jazz	T341830 Create maintenance script to delete entries only for read old
Resolved		Dreamy_Jazz	T341586 Allow write old and new for event table migration
Resolved		Dreamy_Jazz	T341934 Failing tests for CheckUser when event table migration config set to WRITE_BOTH and READ_NEW
Resolved		Dreamy_Jazz	T342371 Create a maintenance script used only for testing to insert a large volume of testing actions to CheckUser
Resolved	PRODUCTION ERROR	pmiazga	T343983 Error: Call to a member function getTimestamp() on null
Open		None	T346022 How does wgCheckUserLogAdditionalRights affect read new for event table migration
Resolved		Dreamy_Jazz	T346044 Remove CheckUserUnionQueryBuilder

Dreamy_Jazz updated the task description. (Show Details)Feb 21 2023, 2:07 PM

Dreamy_Jazz mentioned this in T324907: Create separate tables for log events in CheckUser.

Dreamy_Jazz added a subtask: T328874: Create temporary column in cu_changes to assist in migration to the new tables.

Dreamy_Jazz updated the task description. (Show Details)Feb 21 2023, 2:09 PM

Dreamy_Jazz updated the task description. (Show Details)

Dreamy_Jazz updated the task description. (Show Details)Apr 30 2023, 2:55 PM

Dreamy_Jazz added a parent task: T341829: Enable read new for the event table migration.Jul 13 2023, 8:03 PM

Dreamy_Jazz removed a parent task: T326393: IP Address Reveal on Log page.

Change 940470 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Set event tables migration stage to write new and old, read old

https://gerrit.wikimedia.org/r/940470

gerritbot added a project: Patch-For-Review.Jul 22 2023, 10:01 AM

Dreamy_Jazz edited projects, added Anti-Harassment (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment.Jul 22 2023, 10:02 AM

Dreamy_Jazz moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to Code Review 🔍 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.

Dreamy_Jazz moved this task from Code Review 🔍 to Blocked/Stalled 🚧 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 22 2023, 10:06 AM

Dreamy_Jazz updated the task description. (Show Details)Jul 22 2023, 12:29 PM

Dreamy_Jazz updated the task description. (Show Details)

Dreamy_Jazz updated the task description. (Show Details)Jul 22 2023, 4:31 PM

Change 940470 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Set event tables migration stage to write new and old, read old

https://gerrit.wikimedia.org/r/940470

ReleaseTaggerBot added a project: MW-1.41-notes (1.41.0-wmf.19; 2023-07-25).Jul 22 2023, 5:00 PM

Maintenance_bot removed a project: Patch-For-Review.Jul 22 2023, 5:12 PM

Dreamy_Jazz moved this task from Blocked/Stalled 🚧 to QA/Testing 🐞 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 22 2023, 5:43 PM

Dreamy_Jazz updated the task description. (Show Details)Jul 24 2023, 11:18 AM

Change 940927 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Enable write new for event table migration for CheckUser

https://gerrit.wikimedia.org/r/940927

gerritbot added a project: Patch-For-Review.Jul 24 2023, 11:28 AM

Dreamy_Jazz moved this task from QA/Testing 🐞 to Code Review 🔍 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 24 2023, 11:42 AM

Dreamy_Jazz updated the task description. (Show Details)Jul 24 2023, 11:56 AM

Change 941021 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] CheckUser event table migration: Write new on group0 and 1

https://gerrit.wikimedia.org/r/941021

Change 940927 merged by jenkins-bot:

[operations/mediawiki-config@master] Enable write new on testwiki for CheckUser event tables migration

https://gerrit.wikimedia.org/r/940927

Mentioned in SAL (#wikimedia-operations) [2023-07-25T13:40:11Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:940927|Enable write new on testwiki for CheckUser event tables migration (T330158)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-25T13:41:49Z] <urbanecm@deploy1002> urbanecm and dreamyjazz: Backport for [[gerrit:940927|Enable write new on testwiki for CheckUser event tables migration (T330158)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Mentioned in SAL (#wikimedia-operations) [2023-07-25T14:02:39Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:940927|Enable write new on testwiki for CheckUser event tables migration (T330158)]] (duration: 22m 27s)

Tchanders assigned this task to Dreamy_Jazz.Jul 26 2023, 8:41 AM

Change 941021 merged by jenkins-bot:

[operations/mediawiki-config@master] CheckUser event table migration: Write new on group0

https://gerrit.wikimedia.org/r/941021

Mentioned in SAL (#wikimedia-operations) [2023-07-26T20:08:02Z] <taavi@deploy1002> Started scap: Backport for [[gerrit:941954|clienthints: Start collecting client hints data on testwiki (T341110)]], [[gerrit:941021|CheckUser event table migration: Write new on group0 (T330158)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-26T20:09:41Z] <taavi@deploy1002> dreamyjazz and taavi: Backport for [[gerrit:941954|clienthints: Start collecting client hints data on testwiki (T341110)]], [[gerrit:941021|CheckUser event table migration: Write new on group0 (T330158)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD

Maintenance_bot removed a project: Patch-For-Review.Jul 26 2023, 8:10 PM

Dreamy_Jazz updated the task description. (Show Details)Jul 26 2023, 8:32 PM

Dreamy_Jazz updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2023-07-26T20:34:20Z] <taavi@deploy1002> Finished scap: Backport for [[gerrit:941954|clienthints: Start collecting client hints data on testwiki (T341110)]], [[gerrit:941021|CheckUser event table migration: Write new on group0 (T330158)]] (duration: 26m 17s)

Dreamy_Jazz added a subtask: T342902: Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac.Jul 27 2023, 3:44 PM

Dreamy_Jazz updated the task description. (Show Details)Jul 31 2023, 7:59 AM

Dreamy_Jazz moved this task from Code Review 🔍 to Blocked/Stalled 🚧 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Jul 31 2023, 11:41 PM

Dreamy_Jazz closed subtask T342902: Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac as Resolved.Aug 1 2023, 8:25 AM

Dreamy_Jazz moved this task from Blocked/Stalled 🚧 to Code Review 🔍 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Aug 1 2023, 9:29 AM

Change 944168 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Write new on group0 for event table migration

https://gerrit.wikimedia.org/r/944168

gerritbot added a project: Patch-For-Review.Aug 1 2023, 9:31 AM

Dreamy_Jazz updated the task description. (Show Details)Aug 1 2023, 9:43 AM

Dreamy_Jazz updated the task description. (Show Details)

Change 944168 merged by jenkins-bot:

[operations/mediawiki-config@master] Write new on group0 for event table migration

https://gerrit.wikimedia.org/r/944168

Mentioned in SAL (#wikimedia-operations) [2023-08-01T20:22:19Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:944168|Write new on group0 for event table migration (T330158)]]

Mentioned in SAL (#wikimedia-operations) [2023-08-01T20:23:51Z] <urbanecm@deploy1002> urbanecm and dreamyjazz: Backport for [[gerrit:944168|Write new on group0 for event table migration (T330158)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Maintenance_bot removed a project: Patch-For-Review.Aug 1 2023, 8:29 PM

Dreamy_Jazz updated the task description. (Show Details)Aug 1 2023, 8:43 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-01T20:44:05Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:944168|Write new on group0 for event table migration (T330158)]] (duration: 21m 46s)

Moving out of the sprint for now as progress will be made slowly.

Change 944350 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Write new on group1 except wikidatawiki for event table migration

https://gerrit.wikimedia.org/r/944350

gerritbot added a project: Patch-For-Review.Aug 1 2023, 11:05 PM

Dreamy_Jazz updated the task description. (Show Details)Aug 1 2023, 11:05 PM

Dreamy_Jazz updated the task description. (Show Details)

Change 944350 merged by jenkins-bot:

[operations/mediawiki-config@master] Write new on group1 except wikidatawiki for event table migration

https://gerrit.wikimedia.org/r/944350

Mentioned in SAL (#wikimedia-operations) [2023-08-03T20:07:43Z] <thcipriani@deploy1002> Started scap: Backport for [[gerrit:944350|Write new on group1 except wikidatawiki for event table migration (T330158)]]

Mentioned in SAL (#wikimedia-operations) [2023-08-03T20:09:12Z] <thcipriani@deploy1002> dreamyjazz and thcipriani: Backport for [[gerrit:944350|Write new on group1 except wikidatawiki for event table migration (T330158)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

Maintenance_bot removed a project: Patch-For-Review.Aug 3 2023, 8:10 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-03T20:23:38Z] <thcipriani@deploy1002> Finished scap: Backport for [[gerrit:944350|Write new on group1 except wikidatawiki for event table migration (T330158)]] (duration: 15m 54s)

Dreamy_Jazz updated the task description. (Show Details)Aug 7 2023, 10:57 AM

Change 946527 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Write new for event table migration on all wikis

https://gerrit.wikimedia.org/r/946527

gerritbot added a project: Patch-For-Review.Aug 7 2023, 11:00 AM

Dreamy_Jazz updated the task description. (Show Details)Aug 7 2023, 11:00 AM

Dreamy_Jazz added a parent task: T326393: IP Address Reveal on Log page.Aug 7 2023, 11:04 AM

Change 946527 merged by jenkins-bot:

[operations/mediawiki-config@master] Write new for event table migration on all wikis

https://gerrit.wikimedia.org/r/946527

Mentioned in SAL (#wikimedia-operations) [2023-08-07T13:05:24Z] <urbanecm@deploy1002> Started scap: Backport for [[gerrit:945939|Update knwiktionary logos (T343662)]], [[gerrit:946527|Write new for event table migration on all wikis (T330158)]], [[gerrit:946540|zhwiki: Grant "suppressredirect"to autoreviewer (T343711)]]

Mentioned in SAL (#wikimedia-operations) [2023-08-07T13:06:49Z] <urbanecm@deploy1002> anzx and dreamyjazz and stang and urbanecm: Backport for [[gerrit:945939|Update knwiktionary logos (T343662)]], [[gerrit:946527|Write new for event table migration on all wikis (T330158)]], [[gerrit:946540|zhwiki: Grant "suppressredirect"to autoreviewer (T343711)]] synced to the testservers mwdebug1002.eqiad.wmnet, mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, and mw-d

Maintenance_bot removed a project: Patch-For-Review.Aug 7 2023, 1:11 PM

Mentioned in SAL (#wikimedia-operations) [2023-08-07T13:19:18Z] <urbanecm@deploy1002> Finished scap: Backport for [[gerrit:945939|Update knwiktionary logos (T343662)]], [[gerrit:946527|Write new for event table migration on all wikis (T330158)]], [[gerrit:946540|zhwiki: Grant "suppressredirect"to autoreviewer (T343711)]] (duration: 13m 54s)

Dreamy_Jazz updated the task description. (Show Details)Aug 7 2023, 1:20 PM

Dreamy_Jazz edited projects, added Anti-Harassment (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment.

Dreamy_Jazz moved this task from Code Review 🔍 to QA/Testing 🐞 on the Anti-Harassment (AHaT Sprint 32 - Baseball Cap) board.Aug 7 2023, 1:22 PM

Moved to QA/Testing column to review the changes that have been made by changing the default value of CheckUserEventTablesMigrationStage.

Suggested QA steps:

Pull the latest code on the main branch
Ensure that the default of wgCheckUserEventTablesMigrationStage is being used
Set wgCheckUserLogLogins to true (this is not the default value)
Login to an account
Make a edit
Move a page
Check that rows exist in the tables cu_log_event and cu_private_event

Checking the values in the rows has been done in QA for previous tickets. These steps check that "write new" is enabled by default and that writing new does not cause any problems.

Testing locally, I see the default behaviour is to write to cu_log_event, cu_private_event and cu_changes.

I believe I tested all the different actions from Hooks.php and the maintenance scripts moveLogEntriesFromCuChanges.php, populateCheckUserTable.php and purgeOldData.php.

I checked that, where applicable, the rows written to cu_changes matched those written to cu_log_event and cu_private_event.

Also tested:

IPs from XFF headers correctly recorded
cupe_private column set correctly (matches cuc_private)
On local wikibase and wikilambda installs, in case that made a difference (it did not appear to)