Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Assigned To

Authored By

	Krinkle
	Jul 27 2023, 3:26 PM

Description

Error

service.version: 1.41.0-wmf.19
trace.id: 9765454d-a7c9-485b-bfdc-ca387b69c929
Find trace.id in Logstash

labels.normalized_message

[{reqId}] {exception_url}   Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'
Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAudit
Query: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac

error.stack_trace

from /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1300)
#0 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1284): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
#1 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1258): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#2 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(743): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#3 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/Database.php(1564): Wikimedia\Rdbms\Database->query(Wikimedia\Rdbms\Query, string)
#4 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/DBConnRef.php(119): Wikimedia\Rdbms\Database->insert(string, array, string)
#5 /srv/mediawiki/php-1.41.0-wmf.19/includes/libs/rdbms/database/DBConnRef.php(397): Wikimedia\Rdbms\DBConnRef->__call(string, array)
#6 /srv/mediawiki/php-1.41.0-wmf.19/extensions/CheckUser/src/Hooks.php(442): Wikimedia\Rdbms\DBConnRef->insert(string, array, string)
#7 /srv/mediawiki/php-1.41.0-wmf.19/extensions/CheckUser/src/Hooks.php(710): MediaWiki\CheckUser\Hooks::insertIntoCuChangesTable(array, string, MediaWiki\User\UserIdentityValue)
#8 /srv/mediawiki/php-1.41.0-wmf.19/includes/HookContainer/HookContainer.php(160): MediaWiki\CheckUser\Hooks->onAuthManagerLoginAuthenticateAudit(MediaWiki\Auth\AuthenticationResponse, User, string, array)
#9 /srv/mediawiki/php-1.41.0-wmf.19/includes/HookContainer/HookRunner.php(890): MediaWiki\HookContainer\HookContainer->run(string, array)
#10 /srv/mediawiki/php-1.41.0-wmf.19/includes/auth/AuthManager.php(556): MediaWiki\HookContainer\HookRunner->onAuthManagerLoginAuthenticateAudit(MediaWiki\Auth\AuthenticationResponse, NULL, string, array)
#11 /srv/mediawiki/php-1.41.0-wmf.19/includes/auth/AuthManager.php(462): MediaWiki\Auth\AuthManager->continueAuthentication(array)
#12 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(374): MediaWiki\Auth\AuthManager->beginAuthentication(array, string)
#13 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(506): AuthManagerSpecialPage->performAuthenticationStep(string, array)
#14 /srv/mediawiki/php-1.41.0-wmf.19/includes/htmlform/HTMLForm.php(748): AuthManagerSpecialPage->handleFormSubmit(array, CodexHTMLForm)
#15 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/AuthManagerSpecialPage.php(437): HTMLForm->trySubmit()
#16 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/LoginSignupSpecialPage.php(317): AuthManagerSpecialPage->trySubmit()
#17 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/SpecialPage.php(701): LoginSignupSpecialPage->execute(NULL)
#18 /srv/mediawiki/php-1.41.0-wmf.19/includes/specialpage/SpecialPageFactory.php(1564): SpecialPage->run(NULL)
#19 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(344): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#20 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(948): MediaWiki->performRequest()
#21 /srv/mediawiki/php-1.41.0-wmf.19/includes/MediaWiki.php(597): MediaWiki->main()
#22 /srv/mediawiki/php-1.41.0-wmf.19/index.php(50): MediaWiki->run()
#23 /srv/mediawiki/php-1.41.0-wmf.19/index.php(46): wfIndexMain()
#24 /srv/mediawiki/w/index.php(3): require(string)
#25 {main}

Impact

Notes

Looks like it relates to T329203: Add new column cuc_only_for_read_old to cu_changes for migration purposes to wmf wikis. Maybe it didn't get provisioned on all db hosts? (\cc @Dreamy_Jazz)

Details

Request URL: https://test-commons.wikimedia.org/w/index.php?returnto=*&returntoquery=*&title=*

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Revert "CheckUser event table migration: Write new on group0"	operations/mediawiki-config	master	+1 -1

Customize query in gerrit

Related Objects
Search...

View Standalone Graph

This task is connected to more than 200 other tasks. Only direct parents and subtasks are shown here. Use View Standalone Graph to show more of the graph.

Status	Subtype	Assigned	Task
			· · ·
Resolved		• Dreamy_Jazz	T330158 Enable write new for the event table migration
Resolved	PRODUCTION ERROR	• Dreamy_Jazz	T342902 Wikimedia\Rdbms\DBQueryError: Error 1054: Unknown column 'cuc_only_for_read_old' in 'field list'Function: MediaWiki\CheckUser\Hooks::onAuthManagerLoginAuthenticateAuditQuery: INSERT INTO `cu_changes` (cuc_page_id,cuc_namespac
Resolved		• Marostegui	T343174 Add missing column cuc_only_for_read_old to testcommonswiki
			· · ·

Event Timeline

Krinkle created this task.Jul 27 2023, 3:26 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 27 2023, 3:26 PM

Krinkle added a project: Anti-Harassment-Team.Jul 27 2023, 3:26 PM

Krinkle moved this task from Untriaged to Jul 2023 on the Wikimedia-production-error board.

Krinkle updated the task description. (Show Details)

Krinkle added a subscriber: • Dreamy_Jazz.

Hmm. I was under the impression that all wikis had cuc_only_for_read_old on their DBs.

Best plan is to probably revert https://gerrit.wikimedia.org/r/c/operations/mediawiki-config/+/941021 and see what other DBs are missing that column.

Change 942466 had a related patch set uploaded (by Dreamy Jazz; author: Dreamy Jazz):

[operations/mediawiki-config@master] Revert "CheckUser event table migration: Write new on group0"

https://gerrit.wikimedia.org/r/942466

gerritbot added a project: Patch-For-Review.Jul 27 2023, 3:33 PM

Change 942466 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert "CheckUser event table migration: Write new on group0"

https://gerrit.wikimedia.org/r/942466

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:37:23Z] <zabe@deploy1002> Started scap: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]]

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:38:50Z] <zabe@deploy1002> zabe and dreamyjazz: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]] synced to the testservers mwdebug2001.codfw.wmnet, mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, and mw-debug kubernetes deployment (accessible via k8s-experimental XWD option)

RhinosF1 subscribed.Jul 27 2023, 3:42 PM

P49765 (An Untitled Masterwork)

1	wikiadmin2023@10.64.48.232(testcommonswiki)> desc cu_changes;
2	+----------------+---------------------+------+-----+----------------+----------------+
3	\| Field \| Type \| Null \| Key \| Default \| Extra \|
4	+----------------+---------------------+------+-----+----------------+----------------+
5	\| cuc_id \| int(10) unsigned \| NO \| PRI \| NULL \| auto_increment \|
6	\| cuc_namespace \| int(11) \| NO \| \| 0 \| \|
7	\| cuc_title \| varbinary(255) \| NO \| \| \| \|
8	\| cuc_user \| int(11) \| NO \| MUL \| 0 \| \|
9	\| cuc_user_text \| varbinary(255) \| NO \| \| \| \|
10	\| cuc_actor \| bigint(20) unsigned \| NO \| MUL \| NULL \| \|
11	\| cuc_actiontext \| varbinary(255) \| NO \| \| \| \|
12	\| cuc_comment_id \| bigint(20) unsigned \| NO \| \| NULL \| \|
13	\| cuc_minor \| tinyint(1) \| NO \| \| 0 \| \|
14	\| cuc_page_id \| int(10) unsigned \| NO \| \| 0 \| \|
15	\| cuc_this_oldid \| int(10) unsigned \| NO \| \| 0 \| \|
16	\| cuc_last_oldid \| int(10) unsigned \| NO \| \| 0 \| \|
17	\| cuc_type \| tinyint(3) unsigned \| NO \| \| 0 \| \|
18	\| cuc_timestamp \| binary(14) \| NO \| MUL \| \| \|
19	\| cuc_ip \| varbinary(255) \| YES \| \| \| \|
20	\| cuc_ip_hex \| varbinary(255) \| YES \| MUL \| NULL \| \|
21	\| cuc_xff \| varbinary(255) \| YES \| \| \| \|
22	\| cuc_xff_hex \| varbinary(255) \| YES \| MUL \| NULL \| \|
23	\| cuc_agent \| varbinary(255) \| YES \| \| NULL \| \|
24	\| cuc_private \| mediumblob \| YES \| \| NULL \| \|
25	+----------------+---------------------+------+-----+----------------+----------------+
26	20 rows in set (0.002 sec)
27
28	wikiadmin2023@10.64.48.232(testcommonswiki)>

shows the current structure for cu_changes on testcommonswiki and suggests that the column does not exist. This should mean that after fixing the schema drift, group0 can go back to write new.

• Dreamy_Jazz added a parent task: T330158: Enable write new for the event table migration.Jul 27 2023, 3:44 PM

• Dreamy_Jazz added a subtask: T321063: Fix CheckUser database schema drifts in production.

Mentioned in SAL (#wikimedia-operations) [2023-07-27T15:45:06Z] <zabe@deploy1002> Finished scap: Backport for [[gerrit:942466|Revert "CheckUser event table migration: Write new on group0" (T342902)]] (duration: 07m 43s)