Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Krinkle
	Jul 27 2023, 3:26 PM

Description

Error

service.version: 1.41.0-wmf.19
trace.id: 9765454d-a7c9-485b-bfdc-ca387b69c929
Find trace.id in Logstash

labels.normalized_message

[{reqId}] {exception_url}   Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'
Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAudit
Query: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac

error.stack_trace

from /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1300)
#0 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1284): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
#1 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1258): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(743): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#3 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1564): Wikimedia\Rdbms\Database->query(Wikimedia\Rdbms\Query, string)
#4 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->insert(string, array, string)
#5 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/DBConnRef.php(397): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6 /srv/mediawiki/php-1.41.0-wmf.19/extensions/CheckUser/src/Hooks.php(442): Wikimedia\Rdbms\DBConnRef->insert(string, array, string)
#7 /srv/mediawiki/php-1.41.0-wmf.19/extensions/CheckUser/src/Hooks.php(710): MediaWiki\CheckUser\Hooks::insertIntoCuChangesTable(array, string, MediaWiki\User\UserIdentityValue)
#8 /srv/mediawiki/php-1.41.0-wmf.19/includes/HookContainer/HookContainer.php(160): MediaWiki\CheckUser\Hooks->onAuthManagerLoginAuthenticateAudit(MediaWiki\Auth\AuthenticationResponse, User, string, array)
#9 /srv/mediawiki/php-1.41.0-wmf.19/includes/HookContainer/HookRunner.php(890): MediaWiki\HookContainer\HookContainer->run(string, array)
#10 /srv/mediawiki/php-1.41.0-wmf.19/includes/auth/AuthManager.php(556): MediaWiki\HookContainer\HookRunner->onAuthManagerLoginAuthenticateAudit(MediaWiki\Auth\AuthenticationResponse, NULL, string, array)
#11 /srv/mediawiki/php-1.41.0-wmf.19/includes/auth/AuthManager.php(462): MediaWiki\Auth\AuthManager->continueAuthentication(array)
#12 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(374): MediaWiki\Auth\AuthManager->beginAuthentication(array, string)
#13 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(506): AuthManagerSpecialPage->performAuthenticationStep(string, array)
#14 /srv/mediawiki/php-1.41.0-wmf.19/includes/htmlform/HTMLForm.php(748): AuthManagerSpecialPage->handleFormSubmit(array, CodexHTMLForm)
#15 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(437): HTMLForm->trySubmit()
#16 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/LoginSignupSpecialPage.php(317): AuthManagerSpecialPage->trySubmit()
#17 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/SpecialPage.php(701): LoginSignupSpecialPage->execute(NULL)
#18 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/SpecialPageFactory.php(1564): SpecialPage->run(NULL)
#19 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(344): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#20 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(948): MediaWiki->performRequest()
#21 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(597): MediaWiki->main()
#22 /srv/mediawiki/php-1.41.0-wmf.19/index.php(50): MediaWiki->run()
#23 /srv/mediawiki/php-1.41.0-wmf.19/index.php(46): wfIndexMain()
#24 /srv/mediawiki/w/index.php(3): require(string)
#25 {main}

Impact

Notes

Looks like it relates to T329203: Add new column cuc_only_for_read_old to cu_changes for migration purposes to wmf wikis. Maybe it didn't get provisioned on all db hosts? (\cc @Dreamy_Jazz)

Details

Request URL: https://test-commons.wikimedia.org/w/index.php?returnto=*&returntoquery=*&title=*

	Subject	Repo	Branch	Lines +/-
	Revert "CheckUser event table migration: Write new on group0"	operations/mediawiki-config	master	+1 -1

Customize query in gerrit

Related Objects
Search...

Status	Subtype	Assigned	Task
Open		None	T340995 Display client hint data in Special:Investigate
			Restricted Task
			Restricted Task
			Restricted Task
Resolved	Security	Zabe	T311337 CVE-2022-39193: Edits with the performer suppressed still show the performer in results from the CheckUser extension
Resolved	Security	Dreamy_Jazz	T316414 CVE-2022-39193: Special:Investigate can expose supressed information in check results
Resolved	Security	Dreamy_Jazz	T318166 CVE-2022-39193: CheckUser API can expose the suppressed performer
Resolved	Security	Dreamy_Jazz	T316360 Oversighted action text is shown in Special:CheckUser when the checkuser does not have the right to see it
			Restricted Task
Open	BUG REPORT	None	T326866 Special:Investigate can expose suppressed information for log events
Open	BUG REPORT	None	T326867 CheckUser API can expose suppressed information for log events
Open	BUG REPORT	None	T326865 Special:CheckUser can expose suppressed information for log events
Open		Dreamy_Jazz	T253796 Make CheckUser record the log_id of logged actions
Open		None	T139810 RFC: Overhaul the CheckUser extension
Open		None	T132892 CheckUser UI revamp
Open		None	T69811 CheckUsers "Get users" design needs improvements
Open	Feature	Dreamy_Jazz	T24120 Enhance Blocking and tagging from checkuser interface: add dropdowns
Duplicate		None	T145340 Split CheckUser mass-block interface to separate special page
Open	Feature	None	T318458 Add dropdown options for the reason in Special:InvestigateBlock (like Special:Block has)
Open	Feature	None	T319517 Allow the OOUI selectandother field to have the required status indicate that either the dropdown or the input box could be used
Open	Feature	None	T324709 Allow the HTMLForm Select and other field to have a minimum length set
Resolved	Feature	Dreamy_Jazz	T316441 Enforce minimum length of the tag on the client side for the checkuser block form
Resolved	Feature	Dreamy_Jazz	T321427 Allow HTMLForm text based elements to specify a minimum length
Resolved	Feature	Dreamy_Jazz	T321439 Allow TextInputWidget to specify the "minlength" attribute
Open	Feature	None	T316444 The CheckUser block form should explain why users were not blocked or tagged
Open	Feature	None	T314700 Restore the bottom paging links on 'Get users'
Resolved	BUG REPORT	Dreamy_Jazz	T314217 Checkuserblock form is broken when paging links are shown on 'Get users'
Open	Feature	None	T313451 Separate out when the blocks are made for IPs and users in Special:CheckUser 'get users' mode
Open	Feature	None	T18306 Allow user to specify block settings from CU form
Resolved		Glaisher	T41213 Block interface after "Get users" check should allow talk page/send email blocking
Open		None	T27053 Replace user / talk pages feature of "Get users" creates a user/talk page if it doesnt exist
Open	Feature	None	T318459 Make a clearer visual distinction between the UA and IP list in 'get users'
Resolved		Glaisher	T49505 CheckUser results (userlinks) cannot be customized
Resolved		Niharika	T24119 Enhance Blocking and tagging from checkuser interface: add new page
Resolved		None	T14808 Blocking and tagging from checkuser interface
Resolved	Feature	Dreamy_Jazz	T16699 More versatile searching in CheckUser log
Declined		None	T15789 Restore the global CheckUser log
Resolved	Feature	Dreamy_Jazz	T309924 Add check links for both the target and performer in Special:CheckUserLog
Resolved	Feature	Dreamy_Jazz	T309925 Implement User:Amalthea/culoghelper.js into the CheckUser extension
Duplicate		None	T140345 Userlinks in CheckUser should be customizable
Open		None	T21964 Provide a link to the user on account creation
Duplicate		None	T144099 Make CheckUser unit-testable
Resolved		Dreamy_Jazz	T309815 Use OOUI in Special:CheckUser
Resolved		Dreamy_Jazz	T310019 The Special:CheckUser block form should use OOUI
Resolved		Dreamy_Jazz	T266586 Use OOUI in SpecialCheckUserLog
			Restricted Task
Open	Feature	None	T26231 Allow CheckUsers to filter checks for account creations only
Resolved		Dreamy_Jazz	T139809 Bad output of AbuseFilter blocks in CU get edits
Open	BUG REPORT	None	T41013 Incomplete i18n for log entries in CheckUser
Resolved	BUG REPORT	Dreamy_Jazz	T268156 Partial blocks from namespaces are displayed odd
Open		None	T311354 Make CheckUser results easier to parse by userscripts
Open	Feature	None	T311380 Make certain types of edits in 'Get edits' highlightable for ease of parsing the results
Declined	Feature	None	T54849 Checkuser option "get users" should point out when user password is changed
Open	Feature	None	T315488 Show failed login attempts in the results list for the user
			Restricted Task
Open		Dreamy_Jazz	T145265 Store check user data action text in structured format
In Progress		kostajh	T257893 [EPIC] Support User-Agent Client Hints header in CheckUser
Open		None	T341094 Create client hint mapping rows for CheckUser-worthy events
Open		None	T248926 Performance review of checkuser database queries [NOT READY]
Open		None	T342613 CompareService::getTotalEditsFromIp queries exceeding TransactionProfiler limits
Open		None	T301992 Insert CheckUser row events during certain 2FA actions
Open		Tchanders	T326393 IP Address Reveal on Log page
Open	Feature	Dreamy_Jazz	T324907 Create separate tables for log events in CheckUser
Resolved		Ladsgroup	T326105 Create cu_log_event and cu_private_event on WMF wikis
Resolved		Milimetric	T327447 FYI: Other changes to the CheckUser tables
Open		None	T329189 Add read new support to Special:Investigate for event table migration
Open		None	T329200 Enable read new support in Special:Investigate's 'Timeline' tab
Open		None	T329201 Enable read new support in Special:Investigate's 'IPs & User agents' tab
Open		Dreamy_Jazz	T347102 Alias column names in Special:Investigate to not include the "cuc_" prefix
Open		None	T341827 Add read new support to the CheckUser API
Open		None	T341829 Enable read new for the event table migration
Resolved		Dreamy_Jazz	T341688 Create LogFormatter for cu_private_event log events
Resolved		Dreamy_Jazz	T330158 Enable write new for the event table migration
Resolved		Dreamy_Jazz	T328874 Create temporary column in cu_changes to assist in migration to the new tables
Resolved		Marostegui	T329203 Add new column cuc_only_for_read_old to cu_changes for migration purposes to wmf wikis
Resolved	PRODUCTION ERROR	Dreamy_Jazz	T342902 Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac
Resolved		Marostegui	T343174 Add missing column cuc_only_for_read_old to testcommonswiki
Open		Dreamy_Jazz	T328992 Add read new support to Special:CheckUser for event table migration
Resolved		Dreamy_Jazz	T328995 Enable read new support in Special:CheckUser's 'Get edits' mode
Resolved	BUG REPORT	Dreamy_Jazz	T347773 Log action text shown when reading new does not respect the deleted status of the log
Resolved		Dreamy_Jazz	T328997 Enable read new support in Special:CheckUser's 'Get users' mode
Open		Dreamy_Jazz	T328998 Enable read new support in Special:CheckUser's 'Get IP Addresses' mode
Resolved		Dreamy_Jazz	T328988 Create a SelectQueryBuilder that allows selecting from all three tables
Open		Dreamy_Jazz	T341830 Create maintenance script to delete entries only for read old
Resolved		Dreamy_Jazz	T341586 Allow write old and new for event table migration
Resolved		Dreamy_Jazz	T341934 Failing tests for CheckUser when event table migration config set to WRITE_BOTH and READ_NEW
Resolved		Dreamy_Jazz	T342371 Create a maintenance script used only for testing to insert a large volume of testing actions to CheckUser
Resolved	PRODUCTION ERROR	pmiazga	T343983 Error: Call to a member function getTimestamp() on null
Open		None	T346022 How does wgCheckUserLogAdditionalRights affect read new for event table migration
Resolved		Dreamy_Jazz	T346044 Remove CheckUserUnionQueryBuilder

Event Timeline

Krinkle created this task.Jul 27 2023, 3:26 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 27 2023, 3:26 PM

Krinkle added a project: Anti-Harassment.Jul 27 2023, 3:26 PM

Krinkle moved this task from Untriaged to Jul 2023 on the Wikimedia-production-error board.

Krinkle updated the task description. (Show Details)

Krinkle added a subscriber: Dreamy_Jazz.

Hmm. I was under the impression that all wikis had cuc_only_for_read_old on their DBs.

Best plan is to probably revert https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/941021 and see what other DBs are missing that column.

Change 942466 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Revert "CheckUser event table migration: Write new on group0"

https://gerrit.wikimedia.org/r/942466

gerritbot added a project: Patch-For-Review.Jul 27 2023, 3:33 PM

Change 942466 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert "CheckUser event table migration: Write new on group0"

https://gerrit.wikimedia.org/r/942466

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:37:23Z] <zabe@deploy1002> Started scap: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:38:50Z] <zabe@deploy1002> zabe and dreamyjazz: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

RhinosF1 added a subscriber: RhinosF1.Jul 27 2023, 3:42 PM

P49765 (An Untitled Masterwork)

1	wikiadmin2023@10.64.48.232(testcommonswiki)> desc cu_changes;
2	+----------------+---------------------+------+-----+----------------+----------------+
3	\| Field \| Type \| Null \| Key \| Default \| Extra \|
4	+----------------+---------------------+------+-----+----------------+----------------+
5	\| cuc_id \| int(10) unsigned \| NO \| PRI \| NULL \| auto_increment \|
6	\| cuc_namespace \| int(11) \| NO \| \| 0 \| \|
7	\| cuc_title \| varbinary(255) \| NO \| \| \| \|
8	\| cuc_user \| int(11) \| NO \| MUL \| 0 \| \|
9	\| cuc_user_text \| varbinary(255) \| NO \| \| \| \|
10	\| cuc_actor \| bigint(20) unsigned \| NO \| MUL \| NULL \| \|
11	\| cuc_actiontext \| varbinary(255) \| NO \| \| \| \|
12	\| cuc_comment_id \| bigint(20) unsigned \| NO \| \| NULL \| \|
13	\| cuc_minor \| tinyint(1) \| NO \| \| 0 \| \|
14	\| cuc_page_id \| int(10) unsigned \| NO \| \| 0 \| \|
15	\| cuc_this_oldid \| int(10) unsigned \| NO \| \| 0 \| \|
16	\| cuc_last_oldid \| int(10) unsigned \| NO \| \| 0 \| \|
17	\| cuc_type \| tinyint(3) unsigned \| NO \| \| 0 \| \|
18	\| cuc_timestamp \| binary(14) \| NO \| MUL \| \| \|
19	\| cuc_ip \| varbinary(255) \| YES \| \| \| \|
20	\| cuc_ip_hex \| varbinary(255) \| YES \| MUL \| NULL \| \|
21	\| cuc_xff \| varbinary(255) \| YES \| \| \| \|
22	\| cuc_xff_hex \| varbinary(255) \| YES \| MUL \| NULL \| \|
23	\| cuc_agent \| varbinary(255) \| YES \| \| NULL \| \|
24	\| cuc_private \| mediumblob \| YES \| \| NULL \| \|
25	+----------------+---------------------+------+-----+----------------+----------------+
26	20 rows in set (0.002 sec)
27
28	wikiadmin2023@10.64.48.232(testcommonswiki)>

shows the current structure for cu_changes on testcommonswiki and suggests that the column does not exist. This should mean that after fixing the schema drift, group0 can go back to write new.

Dreamy_Jazz added a parent task: T330158: Enable write new for the event table migration.Jul 27 2023, 3:44 PM

Dreamy_Jazz added a subtask: T321063: Fix CheckUser database schema drifts in production.

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:45:06Z] <zabe@deploy1002> Finished scap: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]] (duration: 07m 43s)