Create a maintenance script used only for testing to insert a large volume of testing actions to CheckUser
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Dreamy_Jazz
	Jul 20 2023, 5:10 PM

Description

One issue with CheckUser is that issues crop up when the extension encounters a DB with a large number of edits and actions. This cannot adequately done in CI tests as the setup time would add too much time to set-up each test. However, having a instance of CheckUser that has a DB filled with varying actions means that in code review and QA the extension can be tested against an environment that isn't usually present on a local testing wiki.

Acceptance criteria

Create a maintenance script used only for testing that adequately fills the DB with events in CheckUser

QA Results - Local

AC	Status	Details
1	✅	https://phabricator.wikimedia.org/T342371#9128336

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Fix use of 'suppress' log type in simulated data maintenance script	mediawiki/extensions/CheckUser	master	+2 -1
	Create a maintenance script to populate tables with test data	mediawiki/extensions/CheckUser	master	+1 K -1

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved	Feature	Dreamy_Jazz	T311375 Instead of not showing any results on too many results show the results to the truncation point with a pager that allows generation of more results
Resolved	Feature	Dreamy_Jazz	T313446 Make non-CUs linking users and IPs when CUs use block forms in CheckUser harder
Open		None	T364943 InvestigateBlock does not allow many custom options
			Restricted Task
			Restricted Task
Resolved	Security	Dreamy_Jazz	T268147 CVE-2024-40611: Special:CheckUser shows deleted edits to non-admins
Resolved	Security	Zabe	T311337 CVE-2022-39193: Edits with the performer suppressed still show the performer in results from the CheckUser extension
Resolved	Security	Dreamy_Jazz	T316414 CVE-2022-39193: Special:Investigate can expose supressed information in check results
Resolved	Security	Dreamy_Jazz	T318166 CVE-2022-39193: CheckUser API can expose the suppressed performer
Resolved	Security	Dreamy_Jazz	T316360 Oversighted action text is shown in Special:CheckUser when the checkuser does not have the right to see it
			Restricted Task
Resolved	BUG REPORT	Dreamy_Jazz	T326866 CVE-2024-40596: Special:Investigate can expose suppressed information for log events
Resolved	BUG REPORT	Dreamy_Jazz	T326867 CVE-2024-40598: CheckUser API can expose suppressed information for log events
Resolved	BUG REPORT	Dreamy_Jazz	T326865 CVE-2024-40597: Special:CheckUser can expose suppressed information for log events
Resolved		Dreamy_Jazz	T253796 Make CheckUser record the log_id of logged actions
Resolved		None	T139810 RFC: Overhaul the CheckUser extension
Resolved		Dreamy_Jazz	T132892 CheckUser UI revamp
Resolved		Dreamy_Jazz	T69811 CheckUsers "Get users" design needs improvements
Resolved	Feature	Dreamy_Jazz	T24120 Enhance Blocking and tagging from checkuser interface: add dropdowns
Duplicate		None	T145340 Split CheckUser mass-block interface to separate special page
Resolved	Feature	Dreamy_Jazz	T316441 Enforce minimum length of the tag on the client side for the checkuser block form
Resolved	Feature	Dreamy_Jazz	T321427 Allow HTMLForm text based elements to specify a minimum length
Resolved	Feature	Dreamy_Jazz	T321439 Allow TextInputWidget to specify the "minlength" attribute
Resolved	Feature	Dreamy_Jazz	T316444 The CheckUser block form should explain why users were not blocked or tagged
Resolved	Feature	Dreamy_Jazz	T314700 Restore the bottom paging links on 'Get users'
Resolved	BUG REPORT	Dreamy_Jazz	T314217 Checkuserblock form is broken when paging links are shown on 'Get users'
Declined	Feature	None	T313451 Separate out when the blocks are made for IPs and users in Special:CheckUser 'get users' mode
Resolved	Feature	Dreamy_Jazz	T18306 Allow user to specify block settings from CU form
Resolved		Glaisher	T41213 Block interface after "Get users" check should allow talk page/send email blocking
Resolved	Feature	Dreamy_Jazz	T329493 Replace Special:CheckUser's 'get users' block form with a usage of Special:InvestigateBlock
Resolved		Dreamy_Jazz	T339914 Remove CheckUserEnableSpecialInvestigate config
Resolved	Feature	Dreamy_Jazz	T318458 Add dropdown options for the reason in Special:InvestigateBlock (like Special:Block has)
Resolved		Dreamy_Jazz	T362569 Test SpecialInvestigateBlock
Resolved		Dreamy_Jazz	T366265 Limit the number of blocks that can be performed in a single use of Special:InvestigateBlock to wgCheckUserMaxBlocks
Declined		None	T27053 Replace user / talk pages feature of "Get users" creates a user/talk page if it doesnt exist
Resolved	Feature	Dreamy_Jazz	T318459 Make a clearer visual distinction between the UA and IP list in 'get users'
Resolved		Glaisher	T49505 CheckUser results (userlinks) cannot be customized
Resolved		Niharika	T24119 Enhance Blocking and tagging from checkuser interface: add new page
Resolved		None	T14808 Blocking and tagging from checkuser interface
Resolved	Feature	Dreamy_Jazz	T16699 More versatile searching in CheckUser log
Declined		None	T15789 Restore the global CheckUser log
Resolved	Feature	Dreamy_Jazz	T309924 Add check links for both the target and performer in Special:CheckUserLog
Resolved	Feature	Dreamy_Jazz	T309925 Implement User:Amalthea/culoghelper.js into the CheckUser extension
Duplicate		None	T140345 Userlinks in CheckUser should be customizable
Resolved		Dreamy_Jazz	T21964 Provide a link to the user on account creation
Duplicate		None	T144099 Make CheckUser unit-testable
Resolved		Dreamy_Jazz	T309815 Use OOUI in Special:CheckUser
Resolved		Dreamy_Jazz	T310019 The Special:CheckUser block form should use OOUI
Resolved		Dreamy_Jazz	T266586 Use OOUI in SpecialCheckUserLog
			Restricted Task
Open	Feature	None	T26231 Allow CheckUsers to filter checks for account creations only
Resolved		Dreamy_Jazz	T139809 Bad output of AbuseFilter blocks in CU get edits
Resolved	BUG REPORT	Dreamy_Jazz	T41013 Incomplete i18n for log entries in CheckUser
Resolved	BUG REPORT	Dreamy_Jazz	T268156 Partial blocks from namespaces are displayed odd
Open		None	T311354 Make CheckUser results easier to parse by userscripts
Open	Feature	None	T311380 Make certain types of edits in 'Get edits' highlightable for ease of parsing the results
Declined	Feature	None	T54849 Checkuser option "get users" should point out when user password is changed
Open	Feature	None	T315488 Show failed login attempts in the results list for the user
			Restricted Task
Resolved		Dreamy_Jazz	T145265 Store check user data action text in structured format
Resolved		kostajh	T257893 [EPIC] Support User-Agent Client Hints header in CheckUser
Resolved		kostajh	T341094 Create client hint mapping rows for CheckUser-worthy events
Resolved		None	T248926 Performance review of checkuser database queries [NOT READY]
Resolved		tstarling	T342613 CompareService::getTotalEditsFromIp queries exceeding TransactionProfiler limits
Resolved		Tchanders	T326393 IP Address Reveal on Log page
Declined	BUG REPORT	None	T28843 CheckUser results do not appear cleanly for AbuseFilter modifications
Open		None	T366719 Update AccountInfo InfoLookup::getUserInfo to read data from cu_log_event and cu_private_event
Open		None	T366739 Update ArticleFeedbackv5 rebuildCheckUser.php maintenance script for CheckUser extension database changes
Resolved		Dreamy_Jazz	T371099 No longer use removed cuc_actiontext column in analytics/refinery
Resolved	Feature	Dreamy_Jazz	T324907 Create separate tables for log events in CheckUser
Resolved		Dreamy_Jazz	T342371 Create a maintenance script used only for testing to insert a large volume of testing actions to CheckUser

Event Timeline

Dreamy_Jazz created this task.Jul 20 2023, 5:10 PM

Restricted Application removed a project: Patch-For-Review. · View Herald TranscriptJul 20 2023, 5:10 PM

Dreamy_Jazz renamed this task from Create a test helper to insert a large volume of testing edits to CheckUser for testing to Create a maintenance script used only for testing to insert a large volume of testing actions to CheckUser.Jul 20 2023, 5:11 PM

Dreamy_Jazz moved this task from Untriaged to CheckUser on the Anti-Harassment-Team board.Jul 22 2023, 5:46 PM

Dreamy_Jazz edited projects, added Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment-Team.Aug 1 2023, 8:47 PM

Dreamy_Jazz edited projects, added Anti-Harassment-Team; removed Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap).

Dreamy_Jazz mentioned this in T328995: Enable read new support in Special:CheckUser's 'Get edits' mode.Aug 9 2023, 7:24 PM

Dreamy_Jazz edited projects, added Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap); removed Anti-Harassment-Team.

Dreamy_Jazz moved this task from Ready 🎬 (ONLY IF YOU HAVE NO MORE CODE TO REVIEW) to In Progress 💪 on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.Aug 10 2023, 5:51 PM

Change 947900 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] [WIP] Create a maintenance script to populate tables with test data

https://gerrit.wikimedia.org/r/947900

gerritbot added a project: Patch-For-Review.Aug 10 2023, 5:52 PM

I think this is ready for code review. Happy to discuss and help testing this.

Change 947900 merged by jenkins-bot:

[mediawiki/extensions/CheckUser@master] Create a maintenance script to populate tables with test data

https://gerrit.wikimedia.org/r/947900

kostajh moved this task from Code Review 🔍 to QA/Testing 🐞 on the Anti-Harassment-Team (AHaT Sprint 32 - Baseball Cap) board.Aug 23 2023, 7:53 AM

The steps to QA test this are maybe not as clear-cut as other tasks, but I would suggest the following steps. The maintenance script will create users and make edits, so don't do this on a wiki where you don't want this to happen:

Enable wgCheckUserDeveloperMode by setting it to true
Enable temporary account creation
Run the maintenance script added by this task. You can use docker compose exec mediawiki php maintenance/run.php CheckUser:populateCheckUserTablesWithSimulatedData.php to run the script
Ensure the script runs without errors (it should output usernames, temporary account names and then IPs)
Load Special:CheckUser and/or Special:Investigate
Run a check on 127.0.0.1/24
Verify that actions exist from multiple users, temporary accounts and IPs within the range 127.0.0.1/24.

Further QA testing could be performed, but as this maintenance script is designed for use to set-up a local testing wiki doing in-depth QA testing may not be necessary.

@Dreamy_Jazz After running the script, I see in Special:Log a few entries:

<user> performed unknown action "delete/suppress" on <page>

These log entries still appear in Special:Investigate and Special:CheckUser, so perhaps it does not matter.

In T342371#9136872, @dom_walden wrote:
@Dreamy_Jazz After running the script, I see in Special:Log a few entries:
<user> performed unknown action "delete/suppress" on <page>
These log entries still appear in Special:Investigate and Special:CheckUser, so perhaps it does not matter.

I had noticed this after the merge. I will fix the script. I can also write something to remove these incorrect logs if wanted.

Change 954325 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[mediawiki/extensions/CheckUser@master] Fix use of 'suppress' log type in simulated data maintenance script

https://gerrit.wikimedia.org/r/954325

@Dreamy_Jazz No errors when the script ran and verified that actions exist from multiple users, temporary accounts and IPs are within the range 127.0.0.1/24. I will move this to Done. Also, I will update this task with the task for <user> performed unknown action "delete/suppress" on <page> when you have created it. Thanks for all your work!

Update: https://phabricator.wikimedia.org/T346105

Status: ✅ PASS
Environment: Local 1.41.0-alpha (2f2802e) 20:03, 11 September 2023. CheckUser: 2.5 (7402d99) 13:46, 11 September 2023
OS: macOS Ventura
Browser: Chrome 116
Device: MBP
Emulated Device:: N/A
Test Links:
Special:Investigate
Special:CheckUser