Page MenuHomePhabricator

Jenkins runs Scribunto tests in Wikibase patches
Open, Needs TriagePublic

Description

Started from this weekend, suddenly Wikibase started to run phpunit tests of Scribunto, basically doubling the tests run time (went from 12 minutes on hhvm quible wmf to 20 minutes)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 23 2019, 10:51 AM

Wikibase has depended on Scribunto right from the start in 8cc0216b3 in December 2015.

The last direct change to Wikibase dependencies was 70df1d2f, merged on Friday 12 July. That added MobileFrontend and MinervaNeue (per T227855) which shouldn't have changed anything.

There aren't any obviously suspect changes to Scribunto itself in the past fortnight. Possibly some of the changes to unit test running in core has made them run when they were previously wrongly skipped, or make it more visible that they're running?

Wikibase has depended on Scribunto right from the start in 8cc0216b3 in December 2015.
The last direct change to Wikibase dependencies was 70df1d2f, merged on Friday 12 July. That added MobileFrontend and MinervaNeue (per T227855) which shouldn't have changed anything.
or make it more visible that they're running?

It wasn't running before, that's for sure. Now they basically own the "Slowest tests" part which was other things last week. Plus the fact that its run time jumped from 12 minutes to 20 :/

Jdforrester-WMF renamed this task from Jenkins runs Scribunto tests in Wikibase patches to Jenkins has been running Scribunto tests in Wikibase patches since 2015.Aug 26 2019, 2:38 PM

Note: as of today, this was 'temporarily' disabled for slowness: https://gerrit.wikimedia.org/r/c/integration/config/+/532379/

Jenkins has been running Scribunto tests in Wikibase patches since 2015.

This is not correct, I have been monitoring jenkins slow tests (things like this):

 You should really fix these slow tests (>50ms)...
12:57:49  1. 3218ms to run WikibaseQuality\ConstraintReport\Tests\Maintenance\ImportConstraintEntitiesTest:testImportEntityFromJson
12:57:49  2. 1593ms to run SpecialPageFatalTest:testSpecialPageDoesNotFatal with data set "Version"
12:57:49  3. 1397ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #15
12:57:49  4. 1357ms to run Wikibase\MediaInfo\Tests\Integration\MultiLingualCaptionsTest:testEditCaptions
12:57:49  5. 1332ms to run Wikibase\MediaInfo\Tests\Integration\StatementsTest:testEditStatements
12:57:49  6. 1073ms to run Wikibase\Tests\RemoveTermsInLanguageTest:testTermsOfLanguageAreRemoved
12:57:49  7. 1070ms to run Scribunto_LuaStandaloneInterpreterTest:testGetStatus
12:57:49  8. 1041ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #64
12:57:49  9. 1037ms to run Scribunto_LuaStandaloneInterpreterTest:testTimeLimit
12:57:49  10. 1012ms to run Wikibase\Lexeme\Tests\DataModel\Services\Diff\LexemeDifferPatcherTest:testProperty_PatchingLexemeWithGeneratedDiffAlwaysRestoresItToTheTargetState
12:57:49  11. 995ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testFormatStatements_usage with data set #0
12:57:49  12. 975ms to run CirrusSearch\UserTestingTest:testSamplesFairlyWithDefaultCallback
12:57:49  13. 945ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #24
12:57:49  14. 921ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryInProcessEntityCacheTest:testLua with data set #0
12:57:49  15. 898ms to run Wikibase\Lib\Tests\LanguageFallbackChainTest:testExtractPreferredValue with data set #1
12:57:49  16. 887ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #78
12:57:49  17. 882ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #57
12:57:49  18. 881ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #108
12:57:49  19. 870ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #71
12:57:49  20. 862ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #25
12:57:49  21. 856ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromLanguage with data set #4
12:57:49  22. 855ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #59
12:57:49  23. 851ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #49
12:57:49  24. 846ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #8
12:57:49  25. 846ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #33
12:57:49  26. 841ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #16
12:57:49  27. 840ms to run Wikibase\Lib\Tests\LanguageWithConversionTest:testFactoryCodeException with data set #9
12:57:49  28. 838ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testFormatPropertyValues_noPropertyId
12:57:49  29. 838ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryNoArbitraryAccessTest:testValidCovers
12:57:49  30. 830ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseLibraryNoLinkedEntityTest:testValidCovers
12:57:49  31. 829ms to run Wikibase\Lib\Tests\LanguageFallbackChainFactoryTest:testNewFromUserAndLanguageCode with data set #19
12:57:49  32. 827ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #0
12:57:49  33. 814ms to run Wikibase\Search\Elastic\Tests\ShowSearchHitHandlerTest:testShowSearchHit with data set "all languages"
12:57:49  34. 810ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testLua with data set #41
12:57:49  35. 802ms to run Wikibase\Lib\Tests\LanguageFallbackChainTest:testExtractPreferredValueOrAny with data set #1
12:57:49  36. 797ms to run Wikibase\Repo\Tests\Specials\SpecialListDatatypesTest:testExecute
12:57:49  37. 797ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #72
12:57:49  38. 782ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #43
12:57:49  39. 776ms to run Wikibase\Repo\Tests\View\RepoSpecialPageLinkerTest:testGetLink with data set #1
12:57:49  40. 774ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #30
12:57:49  41. 767ms to run Wikibase\Client\Tests\Hooks\ParserOutputUpdateHookHandlersTest:testDoContentAlterParserOutput with data set "noexternallanglinks=*"
12:57:49  42. 763ms to run Wikibase\Client\Tests\WikibaseClientRepositorySettingsTest:testGivenOldForeignRepositorySettings_oldSettingsAreUsedForForeignRepository
12:57:49  43. 753ms to run Scribunto_LuaLanguageLibraryTest:testLua with data set #42
12:57:49  44. 743ms to run Wikibase\Lib\Tests\Formatters\MwTimeIsoFormatterTest:testFormat with data set #16
12:57:49  45. 741ms to run Wikibase\Lib\Tests\MockRepositoryTest:testSaveRedirect
12:57:49  46. 738ms to run Wikibase\Client\Tests\Hooks\InfoActionHookHandlerTest:testHandle with data set #4
12:57:49  47. 728ms to run Wikibase\Client\Tests\Hooks\OtherProjectsSidebarGeneratorTest:testBuildProjectLinkSidebar_hook with data set "Invalid hook #1, original data is being used"
12:57:49  48. 718ms to run Wikibase\Repo\Tests\Hooks\ShowSearchHitHandlerTest:testShowSearchHitPlain with data set "no fallback"
12:57:49  49. 718ms to run Wikibase\Client\Tests\DataAccess\Scribunto\Scribunto_LuaWikibaseEntityLibraryTest:testConstructor
12:57:49  50. 717ms to run Wikibase\Client\Tests\Hooks\OtherProjectsSidebarGeneratorTest:testBuildProjectLinkSidebar_hook with data set "Invalid hook #4, original data is being used"

Scribunto tests got added to this list, and doubled the run time in the weekend of July 22nd. I might be missing something but we weren't running Scribunto tests until July 22nd.

Ladsgroup renamed this task from Jenkins has been running Scribunto tests in Wikibase patches since 2015 to Jenkins runs Scribunto tests in Wikibase patches.Sep 11 2019, 6:00 PM