Page MenuHomePhabricator

WikibaseLexeme LexemeSpecialWhatLinksHereTest tests fail on Math CI
Closed, ResolvedPublic

Description

see https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/452897/ error message:

00:18:52.022 Fatal error: Uncaught exception 'Wikimedia\Rdbms\DBQueryError' with message 'A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
00:18:52.022 Query: SELECT  ug_user,ug_group,ug_expiry  FROM `LexemeSpecialWhatLinksHereTest-table-user_groups`    WHERE ug_user = '1'  
00:18:52.023 Function: UserGroupMembership::getMembershipsForUser
00:18:52.023 Error: 1146 Table 'wikidb.LexemeSpecialWhatLinksHereTest-table-user_groups' doesn't exist (/tmp/quibble-mysql-6ld6oukh/socket)
00:18:52.024 ' in /workspace/src/includes/libs/rdbms/database/Database.php:1446
00:18:52.024 Stack trace:
00:18:52.024 #0 /workspace/src/includes/libs/rdbms/database/Database.php(1416): Wikimedia\Rdbms\Database->makeQueryException()
00:18:52.024 #1 /workspace/src/includes/libs/rdbms/database/Database.php(1186): Wikimedia\Rdbms\Database->reportQueryError()
00:18:52.025 #2 /workspace/src/includes/libs/rdbms/database/Database.php(1643): Wikimedia\Rdbms\Database->query()
00:18:52.025 #3 /workspace/src/includes/user/UserGroupMembership.php(317): Wikimedia\Rdbms\Database->select()
00:18:52.025 #4 /workspace/src/includes/user/User.php(1611): UserGroupMembership::getMembershipsForUser()
00:18:52.026 #5 /workspace/src/includes/user/User.php(3570): User->loadGroups()
00:18:52.026 #6 /workspace/src/includes/user/User.php(3597): User->getGroups()
00:18:52.026 #7 /workspace/src/includes/user/User.php(3529): User->getEffectiveGroups()
00:18:52.027 #8 /workspace/src/includes/user/User.php(3840): User->getRights()
00:18:52.027 #9 /workspace/src/includes/Title.php(2265): User->isAllowed()
00:18:52.027 #10 /workspace/src/includes/Title.php(2825): Title->checkQuickPermissions()
00:18:52.027 #11 /workspace/src/includes/Title.php(2161): Title->getUserPermissionsErrorsInternal()
00:18:52.028 #12 /workspace/src/includes/page/WikiPage.php(1846): Title->userCan()
00:18:52.028 #13 /workspace/src/tests/phpunit/MediaWikiTestCase.php(1249): WikiPage->doEditContent()
00:18:52.028 #14 /workspace/src/tests/phpunit/MediaWikiTestCase.php(1803): MediaWikiTestCase->addCoreDBData()
00:18:52.028 #15 /workspace/src/tests/phpunit/MediaWikiTestCase.php(478): MediaWikiTestCase->resetDB()
00:18:52.029 #16 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(722): MediaWikiTestCase->run()
00:18:52.029 #17 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(722): PHPUnit_Framework_TestSuite->run()
00:18:52.029 #18 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(722): PHPUnit_Framework_TestSuite->run()
00:18:52.030 #19 /workspace/src/vendor/phpunit/phpunit/src/Framework/TestSuite.php(722): PHPUnit_Framework_TestSuite->run()
00:18:52.030 #20 /workspace/src/vendor/phpunit/phpunit/src/TextUI/TestRunner.php(440): PHPUnit_Framework_TestSuite->run()
00:18:52.031 #21 /workspace/src/vendor/phpunit/phpunit/src/TextUI/Command.php(149): PHPUnit_TextUI_TestRunner->doRun()
00:18:52.031 #22 /workspace/src/vendor/phpunit/phpunit/src/TextUI/Command.php(100): PHPUnit_TextUI_Command->run()
00:18:52.031 #23 /workspace/src/tests/phpunit/phpunit.php(135): PHPUnit_TextUI_Command::main()
00:18:52.032 #24 /workspace/src/maintenance/doMaintenance.php(94): PHPUnitMaintClass->execute()
00:18:52.032 #25 /workspace/src/tests/phpunit/phpunit.php(174): include()
00:18:52.032 #26 {main}

see https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/11345/console

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 19 2018, 10:01 AM
Addshore renamed this task from Math/Wikibase CI tests fail to Wikibase tests fail on Math CI.Aug 19 2018, 10:15 AM
Addshore moved this task from incoming to other stuff on the Lexicographical data board.
Addshore moved this task from incoming to ready to go on the Wikidata board.
Addshore added a subscriber: Addshore.
Addshore added a subscriber: Pablo-WMDE.
Addshore renamed this task from Wikibase tests fail on Math CI to WikibaseLexeme LexemeSpecialWhatLinksHereTest tests fail on Math CI.Aug 19 2018, 10:17 AM
Addshore updated the task description. (Show Details)Aug 20 2018, 8:23 AM
Pablo-WMDE added a comment.EditedAug 20 2018, 9:00 AM

Trying to reproduce in local quibble and see if changing NonTempTableTestCase to see if running its custom tear down on tearDown instead of tearDownAfterClass will help.

Reproducible with local quibble.

Exception also happens when running tearDownAfterClass as tearDown in NonTempTableTestCase

I can produce a similar result (2 errorous tests inside LexemeSpecialWhatLinksHereTest, then DBQueryError) if I prepend the prefix used inside NonTempTableTestCase::setupNonTempTableDbAndUser() by "unittest_" (= MediaWikiTestCase::DB_PREFIX). I got the idea when I saw that MediaWikiTestCase has a method isNotUnittest() which does naive string processing on table names. -> probably simply due to the sheer length of the table name

When dumping the existing tables right before the error happens the list of existing non-temp test tables (prefixed "LexemeSpecialWhatLinksHereTest-") ends at "LexemeSpecialWhatLinksHereTest-table-transcode", all following are missing. At the same time the temp tables do have "unittest_mathlatexml" at this exact spot.

MathLaTeXMLDatabaseTest::setup() is very involved when it comes to database connections.

Pablo-WMDE added a comment.EditedAug 20 2018, 9:52 AM

by-catch when running (quibble) set-up with the math extension => T202266

Creating tables for enabled extensions
PHP Warning:  array_map(): Argument #2 should be an array in /workspace/src/extensions/Math/src/MathRenderer.php on line 671
PHP Warning:  in_array() expects parameter 2 to be array, null given in /workspace/src/extensions/Math/src/MathHooks.php on line 322
PHP Warning:  array_map(): Argument #2 should be an array in /workspace/src/extensions/Math/src/MathRenderer.php on line 671
PHP Warning:  in_array() expects parameter 2 to be array, null given in /workspace/src/extensions/Math/src/MathHooks.php on line 334

by-catch when running (quibble) set-up with the math extension => T202266

Creating tables for enabled extensions
PHP Warning:  array_map(): Argument #2 should be an array in /workspace/src/extensions/Math/src/MathRenderer.php on line 671
PHP Warning:  in_array() expects parameter 2 to be array, null given in /workspace/src/extensions/Math/src/MathHooks.php on line 322
PHP Warning:  array_map(): Argument #2 should be an array in /workspace/src/extensions/Math/src/MathRenderer.php on line 671
PHP Warning:  in_array() expects parameter 2 to be array, null given in /workspace/src/extensions/Math/src/MathHooks.php on line 334

This looks like getValidModes is called before the ExtensionRegistration stuff sets the configs / globals, hence $wgMathValidModes is not an array

Change 454064 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/Math@master] phpunit: relish MediaWikiTestCase db setup

https://gerrit.wikimedia.org/r/454064

Pablo-WMDE added a comment.EditedAug 20 2018, 5:24 PM

This looks like getValidModes is called before the ExtensionRegistration stuff sets the configs / globals, hence $wgMathValidModes is not an array

It is works for me locally when running through docker-compose exec --user root "web" bash //var/www/mediawiki/.docker/installdbs default

array(3) {
  [0] =>
  string(3) "png"
  [1] =>
  string(6) "source"
  [2] =>
  string(6) "mathml"
}

However my local quibble spits out these warnings as well.

installdbs in the mediawiki-docker-dev setup does install.php and then update.php

quibble does install.php with the --with-extensions parameter.

I remember there being some issues with some extensions and --with-extensions for install.php before, so I imagine this is something similar.

Pablo-WMDE changed the task status from Open to Stalled.
Pablo-WMDE removed Pablo-WMDE as the assignee of this task.

Change 455482 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/Math@master] Avoid globals in MathRenderer::getValidModes

https://gerrit.wikimedia.org/r/455482

I am not sure what to do. Creating all tables is certainly a workaround. However, that would be different from the production setup and create tables that might never be used. I suspected that it could be related to the use of the globals to access the config. However, that did not help either.

This is likely caused by T202266. Should it turn out this remains a problem after T202266 is fixed, please resurrect/report again.

Pablo-WMDE closed this task as Invalid.Aug 29 2018, 2:24 PM
Physikerwelt reopened this task as Open.Sep 3 2018, 8:32 PM

Adding fallback configuration does not solve this problem. We need to find a way to fix / workaround this issue asap.

@WMDE-leszek can you run both database-modifying tests from math and wikibase in one pass with your local setup? My suspicion is that the test database is not cleaned up properly.
Look at the warning:

20:05:43 Warning: Destructor threw an object exception: exception 'Wikimedia\Rdbms\DBConnectionError' with message 'Cannot access the database: Unknown error (/tmp/quibble-mysql-jao3ypn6/socket)' in /workspace/src/includes/libs/rdbms/loadbalancer/LoadBalancer.php:1180

Will run tests in a sec as requested. Will report results shortly.

Isn't the quoted warning though something which is printed in basically all PHPunit runs in extensions since couple of weeks at least?

Just run tests (--testsuite extensions --group Database --exclude-group Broken,ParserFuzz,Stub) with Wikibase, WikibaseLexeme and Math enabled, and both in case of master, and https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/455482/10 there is no test failures, nor errors.

We're looking into https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/455482 having fixed the DB setup problem still fails.

zeljkofilipin added a subscriber: zeljkofilipin.

It would be great if someone could merge https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/455482/ so that the failing tests do not block other developments. Thank you in advance.

Change 457886 had a related patch set uploaded (by Jakob; owner: Jakob):
[mediawiki/extensions/Math@master] Ensure database tests run as part of --group Database

https://gerrit.wikimedia.org/r/457886

Pablo-WMDE closed this task as Resolved.Sep 5 2018, 8:44 AM
Pablo-WMDE reopened this task as Open.
Pablo-WMDE assigned this task to Jakob_WMDE.

Change 458163 had a related patch set uploaded (by Jakob; owner: Jakob):
[mediawiki/extensions/WikibaseLexeme@master] Replace SpecialWhatLinksHereTest with equivalent LexemeApiQueryBacklinksTest

https://gerrit.wikimedia.org/r/458163

Change 458458 had a related patch set uploaded (by Pablo Grass (WMDE); owner: Pablo Grass (WMDE)):
[mediawiki/extensions/Wikibase@master] EntityIdFormatterFactory: prepare for param change

https://gerrit.wikimedia.org/r/458458

Change 458163 merged by jenkins-bot:
[mediawiki/extensions/WikibaseLexeme@master] Replace SpecialWhatLinksHereTest with equivalent LexemeApiQueryBacklinksTest

https://gerrit.wikimedia.org/r/458163

Jakob_WMDE closed this task as Resolved.Sep 6 2018, 10:32 AM

We replaced the flaky test with something less hacky. Math CI is no longer blocked as far as I can tell.
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Math/+/457886 <- green!

Change 455482 abandoned by Physikerwelt:
Fix broken test in quibble

Reason:
root cause for this issue was fixed

https://gerrit.wikimedia.org/r/455482

Change 457886 merged by jenkins-bot:
[mediawiki/extensions/Math@master] Ensure database tests run as part of --group Database

https://gerrit.wikimedia.org/r/457886

Change 458458 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] EntityIdFormatterFactory: expect Language input

https://gerrit.wikimedia.org/r/458458

Change 454064 abandoned by Pablo Grass (WMDE):
phpunit: relish MediaWikiTestCase db setup

Reason:
Mirrored (and then discontinued) in I2fb89ebeded45f97fa91aaf773ac55ee77314e86

https://gerrit.wikimedia.org/r/454064