Special:UserRights not allowing cross-wiki user rights change
Closed, ResolvedPublicPRODUCTION ERROR
Actions

Description

Error

[[ URL | Logtash single document ]]

message

[192b4fdf-fcb4-4739-b305-286ca8a11999] 2022-09-29 08:19:58: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"
[7f94c7de-6388-492c-9ae9-7d3276d6a731] 2022-09-29 08:21:59: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"
[ee5b9c5f-2bb5-439c-884b-7d7bf96ff774] 2022-09-29 08:29:25: Fatal exception of type "Wikimedia\Rdbms\DBQueryError"

detailled message

Error 1049: Unknown database 'metawiki'
Function: Wikimedia\Rdbms\DatabaseMysqlBase::doSelectDomain
Query: USE `metawiki`

trace

from /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/database/Database.php(1618)
#0 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/database/Database.php(1602): Wikimedia\Rdbms\Database->getQueryException(string, integer, string, string)
#1 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/database/Database.php(1576): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#2 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/database/DatabaseMysqlBase.php(214): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string)
#3 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/database/Database.php(1869): Wikimedia\Rdbms\DatabaseMysqlBase->doSelectDomain(Wikimedia\Rdbms\DatabaseDomain)
#4 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/loadbalancer/LoadBalancer.php(1138): Wikimedia\Rdbms\Database->selectDomain(Wikimedia\Rdbms\DatabaseDomain)
#5 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/loadbalancer/LoadBalancer.php(970): Wikimedia\Rdbms\LoadBalancer->reuseOrOpenConnectionForNewRef(integer, Wikimedia\Rdbms\DatabaseDomain, integer)
#6 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2095): Wikimedia\Rdbms\LoadBalancer->getServerConnection(integer, string, integer)
#7 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/objectcache/wancache/WANObjectCache.php(1689): Wikimedia\Rdbms\LoadBalancer->Wikimedia\Rdbms\{closure}(boolean, integer, array, NULL, array)
#8 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/objectcache/wancache/WANObjectCache.php(1522): WANObjectCache->fetchOrRegenerate(string, integer, Closure, array, array)
#9 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2113): WANObjectCache->getWithSetCallback(string, integer, Closure, array)
#10 /srv/mediawiki/php-1.40.0-wmf.3/includes/libs/rdbms/loadbalancer/LoadBalancer.php(2016): Wikimedia\Rdbms\LoadBalancer->isPrimaryRunningReadOnly(Wikimedia\Rdbms\DatabaseDomain)
#11 /srv/mediawiki/php-1.40.0-wmf.3/includes/ReadOnlyMode.php(45): Wikimedia\Rdbms\LoadBalancer->getReadOnlyReason()
#12 /srv/mediawiki/php-1.40.0-wmf.3/includes/ReadOnlyMode.php(29): ReadOnlyMode->getReason()
#13 /srv/mediawiki/php-1.40.0-wmf.3/includes/user/UserGroupManager.php(760): ReadOnlyMode->isReadOnly()
#14 /srv/mediawiki/php-1.40.0-wmf.3/includes/user/UserRightsProxy.php(243): MediaWiki\User\UserGroupManager->addUserToGroup(UserRightsProxy, string, NULL, boolean)
#15 /srv/mediawiki/php-1.40.0-wmf.3/includes/specials/SpecialUserrights.php(433): UserRightsProxy->addGroup(string, NULL)
#16 /srv/mediawiki/php-1.40.0-wmf.3/includes/specials/SpecialUserrights.php(367): UserrightsPage->doSaveUserGroups(UserRightsProxy, array, array, string, array, array)
#17 /srv/mediawiki/php-1.40.0-wmf.3/includes/specials/SpecialUserrights.php(238): UserrightsPage->saveUserGroups(string, string, UserRightsProxy)
#18 /srv/mediawiki/php-1.40.0-wmf.3/includes/specialpage/SpecialPage.php(701): UserrightsPage->execute(NULL)
#19 /srv/mediawiki/php-1.40.0-wmf.3/includes/specialpage/SpecialPageFactory.php(1428): SpecialPage->run(NULL)
#20 /srv/mediawiki/php-1.40.0-wmf.3/includes/MediaWiki.php(316): MediaWiki\SpecialPage\SpecialPageFactory->executePath(string, RequestContext)
#21 /srv/mediawiki/php-1.40.0-wmf.3/includes/MediaWiki.php(904): MediaWiki->performRequest()
#22 /srv/mediawiki/php-1.40.0-wmf.3/includes/MediaWiki.php(562): MediaWiki->main()
#23 /srv/mediawiki/php-1.40.0-wmf.3/index.php(50): MediaWiki->run()
#24 /srv/mediawiki/php-1.40.0-wmf.3/index.php(46): wfIndexMain()
#25 /srv/mediawiki/w/index.php(3): require(string)
#26 {main}

Impact

Cannot add CheckUser and remove bureaucrat. I can edit confirmed status just fine.

Notes

Details

Request URL: https://meta.wikimedia.org/wiki/Special:UserRights

Subject	Repo	Branch	Lines +/-
rdbms: remove DB domain parameter from various lag/read-only methods	mediawiki/core	master	+39 -59
Revert "rdbms: improve LoadBalancer connection pool reuse"	mediawiki/core	master	+278 -198
Revert "rdbms: improve LoadBalancer connection pool reuse"	mediawiki/core	wmf/1.40.0-wmf.3	+278 -198
Revert "group1 wikis to 1.40.0-wmf.3"	operations/mediawiki-config	master	+531 -531

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved	Release	brennen	T314192 1.40.0-wmf.3 deployment blockers
		Resolved	PRODUCTION ERROR	Ladsgroup	T318904 Special:UserRights not allowing cross-wiki user rights change

Event Timeline

revi created this task.Sep 29 2022, 8:30 AM

Restricted Application added a project: User-revi. · View Herald TranscriptSep 29 2022, 8:30 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

revi updated the task description. (Show Details)Sep 29 2022, 8:31 AM

revi moved this task from Incoming to Radar on the User-revi board.

I just tried to add CU to myself at testwiki, and I can confirm that I get the same error.

Yeah, [8701d83f-51e1-4140-967e-42ade58b0294] 2022-09-29 08:38:44: Fatal exception of type "Wikimedia\Rdbms\DBQueryError" for trying to add oversight on testwiki. But since I can modify my rights for confirmed at metawiki, it seems it is about modifying cross-wiki user rights.

MdsShakil subscribed.Sep 29 2022, 8:51 AM

Johannnes89 subscribed.Sep 29 2022, 8:55 AM

hashar updated the task description. (Show Details)Sep 29 2022, 8:56 AM

jnuche triaged this task as Unbreak Now! priority.Sep 29 2022, 8:59 AM

Change 836728 had a related patch set uploaded (by Jaime Nuche; author: Jaime Nuche):

[operations/mediawiki-config@master] Revert "group1 wikis to 1.40.0-wmf.3"

https://gerrit.wikimedia.org/r/836728

Change 836728 merged by jenkins-bot:

[operations/mediawiki-config@master] Revert "group1 wikis to 1.40.0-wmf.3"

https://gerrit.wikimedia.org/r/836728

Very likely caused by the refactor @aaron did https://gerrit.wikimedia.org/r/c/mediawiki/core/+/519305 (Tim is on vacation) so ccing @Krinkle

Restricted Application added a project: Performance-Team. · View Herald TranscriptSep 29 2022, 9:19 AM

Urbanecm subscribed.Sep 29 2022, 9:27 AM

Maintenance_bot removed a project: Patch-For-Review.Sep 29 2022, 9:30 AM

_joe_> Giuseppe Lavagetto this is breaking important on-wiki workflows

The problem was considered important enough to warrant a rollback of group1 wikis to version 1.40.0-wmf.2

Change 836712 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/core@master] Revert "rdbms: improve LoadBalancer connection pool reuse"

https://gerrit.wikimedia.org/r/836712

Change 836713 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/core@wmf/1.40.0-wmf.3] Revert "rdbms: improve LoadBalancer connection pool reuse"

https://gerrit.wikimedia.org/r/836713

Change 836713 merged by jenkins-bot:

[mediawiki/core@wmf/1.40.0-wmf.3] Revert "rdbms: improve LoadBalancer connection pool reuse"

https://gerrit.wikimedia.org/r/836713

Mentioned in SAL (#wikimedia-operations) [2022-09-29T12:34:55Z] <ladsgroup@deploy1002> Started scap: Backport for [[gerrit:836713|Revert "rdbms: improve LoadBalancer connection pool reuse" (T318904)]]

Mentioned in SAL (#wikimedia-operations) [2022-09-29T12:35:27Z] <ladsgroup@deploy1002> ladsgroup and ladsgroup: Backport for [[gerrit:836713|Revert "rdbms: improve LoadBalancer connection pool reuse" (T318904)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2022-09-29T12:44:00Z] <ladsgroup@deploy1002> Finished scap: Backport for [[gerrit:836713|Revert "rdbms: improve LoadBalancer connection pool reuse" (T318904)]] (duration: 09m 05s)

With revert it's now fixed.

MdsShakil unsubscribed.Sep 29 2022, 12:53 PM

ReleaseTaggerBot added a project: MW-1.40-notes (1.40.0-wmf.3; 2022-09-26).Sep 29 2022, 1:00 PM

Change 836712 merged by jenkins-bot:

[mediawiki/core@master] Revert "rdbms: improve LoadBalancer connection pool reuse"

https://gerrit.wikimedia.org/r/836712

Maintenance_bot removed a project: Patch-For-Review.Sep 29 2022, 1:30 PM

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.4; 2022-10-03); removed MW-1.40-notes (1.40.0-wmf.3; 2022-09-26).Sep 29 2022, 2:00 PM

hashar mentioned this in T226595: Refactor LoadBalancer connection pooling to be more efficient.Sep 29 2022, 3:32 PM

Zabe subscribed.Sep 29 2022, 3:49 PM

tstarling subscribed.Oct 3 2022, 3:59 PM

Change 838234 had a related patch set uploaded (by Aaron Schulz; author: Aaron Schulz):

[mediawiki/core@master] rdms: remove DB domain parameter from various lag/read-only methods

https://gerrit.wikimedia.org/r/838234

gerritbot added a project: Patch-For-Review.Oct 4 2022, 5:53 PM

Change 838234 merged by jenkins-bot:

[mediawiki/core@master] rdbms: remove DB domain parameter from various lag/read-only methods

https://gerrit.wikimedia.org/r/838234

Maintenance_bot removed a project: Patch-For-Review.Oct 18 2022, 11:30 PM

ReleaseTaggerBot edited projects, added MW-1.40-notes (1.40.0-wmf.7; 2022-10-24); removed MW-1.40-notes (1.40.0-wmf.4; 2022-10-03).Oct 19 2022, 12:00 AM

Krinkle moved this task from Untriaged to Sep 2022 on the Wikimedia-production-error board.Nov 17 2022, 4:21 PM

Special:UserRights not allowing cross-wiki user rights changeClosed, ResolvedPublicPRODUCTION ERRORActions