Started from this weekend, suddenly Wikibase started to run phpunit tests of Scribunto, basically doubling the tests run time (went from 12 minutes on hhvm quible wmf to 20 minutes)
Description
Description
Related Objects
Related Objects
- Mentioned In
- T361118: [REPO][CLIENT][SW] Reduce CI test runtime for Wikibase and related extensions
T225730: Reduce runtime of MW shared gate Jenkins jobs to 5 min
T232626: CI mediawiki/core run times have increased since July 26th, 2019
T231198: Wikibase and WikibaseLexeme tests now take more than 1 hour
T125050: [Task] Add Scribunto to extension-gate in CI - Mentioned Here
- T227855: Add MobileFrontend to Wikibase CI dependencies
rCICF8cc0216b3732: add Wikibase and Wikidata optional dependencies
rCICF70df1d2f922c: Include MobileFrontend and MinervaNeue in Wikibase dependencies
Event Timeline
Comment Actions
Wikibase has depended on Scribunto right from the start in 8cc0216b3 in December 2015.
The last direct change to Wikibase dependencies was 70df1d2f, merged on Friday 12 July. That added MobileFrontend and MinervaNeue (per T227855) which shouldn't have changed anything.
There aren't any obviously suspect changes to Scribunto itself in the past fortnight. Possibly some of the changes to unit test running in core has made them run when they were previously wrongly skipped, or make it more visible that they're running?
Comment Actions
It wasn't running before, that's for sure. Now they basically own the "Slowest tests" part which was other things last week. Plus the fact that its run time jumped from 12 minutes to 20 :/
Comment Actions
Note: as of today, this was 'temporarily' disabled for slowness: https://gerrit.wikimedia.org/r/c/integration/config/+/532379/
Comment Actions
Jenkins has been running Scribunto tests in Wikibase patches since 2015.
This is not correct, I have been monitoring jenkins slow tests (things like this):
You should really fix these slow tests (>50ms)... 12:57:49 1. 3218ms to run WikibaseQuality\ConstraintReport\Tests\Maintenance\ImportConstraintEntitiesTest:testImportEntityFromJson 12:57:49 2. 1593ms to run SpecialPageFatalTest:testSpecialPageDoesNotFatal with data set "Version" 12:57:49 3. 1397ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #15 12:57:49 4. 1357ms to run Wikibase\MediaInfo\Tests\Integration\MultiLingualCaptionsTest:testEditCaptions 12:57:49 5. 1332ms to run Wikibase\MediaInfo\Tests\Integration\StatementsTest:testEditStatements 12:57:49 6. 1073ms to run Wikibase\Tests\RemoveTermsInLanguageTest:testTermsOfLanguageAreRemoved 12:57:49 7. 1070ms to run Scribunto_LuaStandaloneInterpreterTest:testGetStatus 12:57:49 8. 1041ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #64 12:57:49 9. 1037ms to run Scribunto_LuaStandaloneInterpreterTest:testTimeLimit 12:57:49 10. 1012ms to run Wikibase\Lexeme\Tests\DataModel\Services\Diff\LexemeDifferPatcherTest:testProperty_PatchingLexemeWithGeneratedDiffAlwaysRestoresItToTheTargetState 12:57:49 11. 995ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testFormatStatements_usage with data set #0 12:57:49 12. 975ms to run CirrusSearch\UserTestingTest:testSamplesFairlyWithDefaultCallback 12:57:49 13. 945ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #24 12:57:49 14. 921ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryInProcessEntityCacheTest:testLua with data set #0 12:57:49 15. 898ms to run Wikibase\Lib\Tests\LanguageFallbackChainTest:testExtractPreferredValue with data set #1 12:57:49 16. 887ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #78 12:57:49 17. 882ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #57 12:57:49 18. 881ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #108 12:57:49 19. 870ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #71 12:57:49 20. 862ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #25 12:57:49 21. 856ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #4 12:57:49 22. 855ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #59 12:57:49 23. 851ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #49 12:57:49 24. 846ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #8 12:57:49 25. 846ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #33 12:57:49 26. 841ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #16 12:57:49 27. 840ms to run Wikibase\Lib\Tests\LanguageWithConversionTest:testFactoryCodeException with data set #9 12:57:49 28. 838ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testFormatPropertyValues_noPropertyId 12:57:49 29. 838ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryNoArbitraryAccessTest:testValidCovers 12:57:49 30. 830ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryNoLinkedEntityTest:testValidCovers 12:57:49 31. 829ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromUserAndLanguageCode with data set #19 12:57:49 32. 827ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #0 12:57:49 33. 814ms to run Wikibase\Search\Elastic\Tests\ShowSearchHitHandlerTest:testShowSearchHit with data set "all languages" 12:57:49 34. 810ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #41 12:57:49 35. 802ms to run Wikibase\Lib\Tests\LanguageFallbackChainTest:testExtractPreferredValueOrAny with data set #1 12:57:49 36. 797ms to run Wikibase\Repo\Tests\Specials\SpecialListDatatypesTest:testExecute 12:57:49 37. 797ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #72 12:57:49 38. 782ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #43 12:57:49 39. 776ms to run Wikibase\Repo\Tests\View\RepoSpecialPageLinkerTest:testGetLink with data set #1 12:57:49 40. 774ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #30 12:57:49 41. 767ms to run Wikibase\Client\Tests\Hooks\ParserOutputUpdateHookHandlersTest:testDoContentAlterParserOutput with data set "noexternallanglinks=*" 12:57:49 42. 763ms to run Wikibase\Client\Tests\WikibaseClientRepositorySettingsTest:testGivenOldForeignRepositorySettings_oldSettingsAreUsedForForeignRepository 12:57:49 43. 753ms to run Scribunto_LuaLanguageLibraryTest:testLua with data set #42 12:57:49 44. 743ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #16 12:57:49 45. 741ms to run Wikibase\Lib\Tests\MockRepositoryTest:testSaveRedirect 12:57:49 46. 738ms to run Wikibase\Client\Tests\Hooks\InfoActionHookHandlerTest:testHandle with data set #4 12:57:49 47. 728ms to run Wikibase\Client\Tests\Hooks\OtherProjectsSidebarGeneratorTest:testBuildProjectLinkSidebar_hook with data set "Invalid hook #1, original data is being used" 12:57:49 48. 718ms to run Wikibase\Repo\Tests\Hooks\ShowSearchHitHandlerTest:testShowSearchHitPlain with data set "no fallback" 12:57:49 49. 718ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testConstructor 12:57:49 50. 717ms to run Wikibase\Client\Tests\Hooks\OtherProjectsSidebarGeneratorTest:testBuildProjectLinkSidebar_hook with data set "Invalid hook #4, original data is being used"
Scribunto tests got added to this list, and doubled the run time in the weekend of July 22nd. I might be missing something but we weren't running Scribunto tests until July 22nd.