Page MenuHomePhabricator

WikibaseMediaInfo test MultiLingualCaptionsTest fails
Closed, ResolvedPublic

Description

When running CI for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseLexemeCirrusSearch/+/507847, this is produced:

01:07:59 Undefined index: imageinfo
01:07:59 
01:07:59 /workspace/src/extensions/WikibaseMediaInfo/tests/phpunit/integration/MultiLingualCaptionsTest.php:133
01:07:59 /workspace/src/extensions/WikibaseMediaInfo/tests/phpunit/integration/MultiLingualCaptionsTest.php:138
01:07:59 /workspace/src/tests/phpunit/MediaWikiTestCase.php:427
01:07:59 /workspace/src/maintenance/doMaintenance.php:96

the log has this:

01:07:59 [wfDebug] [debug] ApiUpload::performStash Stashing temporary file failed: UploadStashFileException Could not store file "/tmp/MW_PHPUnit_Wikibase\MediaInfo\Tests\Integration\MultiLingualCap8mTFoR" at "mwstore://local-backend/local-temp/6/60/20190503010521!MW_PHPUnit_Wikibase\MediaInfo\Tests\Integration\MultiLingualCap8mTFoR.png". {"private":false}

Details

Related Gerrit Patches:

Event Timeline

Smalyshev created this task.May 3 2019, 3:01 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 3 2019, 3:01 AM
Smalyshev triaged this task as High priority.May 3 2019, 3:02 AM
Smalyshev updated the task description. (Show Details)

@Jdforrester-WMF any idea why this might be happening? Permissions in /tmp?

Possibly a permissions issue, yes. It's happening in the gate job but not the test job, which is confusing; possibly the order of operations is different and it's effectively a race condition with another test?

Krinkle added a subscriber: Krinkle.May 8 2019, 3:42 PM

@Smalyshev Looks like one or more WikibaseLexemeCirrusSearch commits have since been merged without overriding Verified. Is this resolved? Or was it not a consistent failure?

Looks like it's not happening anymore, and it happened only in old branch so maybe since we're moving on from that branch it may be irrelevant now.

Marking as resolved on the shared failure board. Keeping open for Multimedia to triage/close as needed.

Oh, here is the correct task which is still open.

I left a comment at T222436#5181994

dcausse added a subscriber: dcausse.

it's happening again on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CirrusSearch/+/516637 :

07:16:01 Undefined index: imageinfo
07:16:01 
07:16:01 /workspace/src/extensions/WikibaseMediaInfo/tests/phpunit/integration/WBMIApiTestCase.php:118
07:16:01 /workspace/src/extensions/WikibaseMediaInfo/tests/phpunit/integration/MultiLingualCaptionsTest.php:28
07:16:01 /workspace/src/tests/phpunit/MediaWikiTestCase.php:427

but surprisingly only for wmf-quibble-vendor-mysql-php71-docker.

Change 518168 had a related patch set uploaded (by Smalyshev; owner: Smalyshev):
[mediawiki/extensions/WikibaseMediaInfo@master] Add more checks for failing test

https://gerrit.wikimedia.org/r/518168

Change 518168 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Add more checks for failing test

https://gerrit.wikimedia.org/r/518168

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM

Just got this error again on a backport for 1.34.0-wmf.25 (Ida2ba52fd0) – output:

16:17:51 Bad response: {"upload":{"result":"Warning","warnings":{"badfilename":"Angelological.png"},"filekey":"16xx0vnigzsg.knbhvx.71.png","sessionkey":"16xx0vnigzsg.knbhvx.71.png"}}
16:17:51 Failed asserting that an array has the key 'imageinfo'.

I’m not sure why Angelological.png is supposed to be a bad file name (though it seems to be missing the File: namespace?); I don’t see any failed stash in the console output.

@Lucas_Werkmeister_WMDE I think the issue might be that it is sent as lowercase instead of ucfirst.

Grepping core for badfilename, I get to UploadBase::checkWarnings where it calls checkBadFileName() and if this method returns non-null, it's return value is sent out as { "badfilename": <return>}. This means Angelological.png` is not the problematic input, but rather the would-be-valid normalised output.

From what I can see, it takes a title-ish value (not full page name / prefixed title), so the omission of namespace prefix is okay.

I couldn't find where this test case came from, but then found it doesn't use a fixture, but rather RandomImageGenerator which picks random words from includes/api/words.txt, where angelological appears and is, as all other words, lowercase.

I'm not sure why this would fail only sometimes, as it seems like it should fail always. I think using ucfirst would fix it?

But it looks like RandomImageGenerator::getRandomFilenames() always generates file names from a pair of words, e. g. foo_bar.jpg.

And then there’s this in the debug log a few lines earlier:

16:17:51 [exec] [debug] MediaWiki\Shell\Command::execute: /bin/bash '/workspace/src/includes/shell/limit.sh' ''\''/usr/bin/convert'\'' '\''-size'\'' '\''712x476'\'' '\''xc:rgb(130, 202, 41)'\'' '\''-draw'\'' '\''fill rgb(23, 208, 186)  polygon -172,303 -53,620 -291,620 -172,303'\'' '\''-draw'\'' '\''fill rgb(17, 98, 6)  polygon 501,337 520,485 482,485 501,337'\'' '\''-draw'\'' '\''fill rgb(238, 0, 114)  polygon 432,377 502,516 362,516 432,377'\'' '\''-draw'\'' '\''fill rgb(11, 252, 99)  polygon 377,259 391,302 363,302 377,259'\'' '\''-draw'\'' '\''fill rgb(94, 69, 46)  polygon 216,125 242,163 190,163 216,125'\'' '\''-draw'\'' '\''fill rgb(97, 230, 40)  polygon 769,385 864,509 674,509 769,385'\'' '\''/tmp/MW_PHPUnit_Wikibase_MediaInfo_Tests_Integration_MultiLingualCapI3paGn/_angelological.png'\''' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=180; MW_CGROUP='\'''\''; MW_MEM_LIMIT=307200; MW_FILE_SIZE_LIMIT=102400; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes' []
16:17:51 [wfDebug] [debug] WebRequestUpload::getName: _angelological.png normalized to '_angelological.png' {"private":false}

That’s a very suspicious _angelological.png… has RandomImageGenerator somehow chosen the empty string as the first word?

Yeah, I think there’s some off-by-one errors in the dictionary line selection:

Psy Shell v0.9.9 (PHP 7.2.19-0ubuntu0.19.04.2 — cli) by Justin Hileman
>>> require_once 'tests/phpunit/includes/api/RandomImageGenerator.php'
=> 1
>>> $rig = new RandomImageGenerator( [ 'dictionaryFile' => 'data:text/plain;charset=utf-8,line1%0D%0Aline2' ] )
=> RandomImageGenerator {#1951}
>>> sudo for ( $i = 0; $i < 16; $i++ ) { echo $rig->getRandomFilenames( 1, 'jpg', null )[0] . PHP_EOL; }
/var/www/html/wiki1/line2_line1.jpg
/var/www/html/wiki1/line1_.jpg
/var/www/html/wiki1/_line2.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/line1_line2.jpg
/var/www/html/wiki1/line1_.jpg
/var/www/html/wiki1/line2_line1.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/line1_.jpg
/var/www/html/wiki1/_line2.jpg
/var/www/html/wiki1/line1_.jpg
/var/www/html/wiki1/_line1.jpg
/var/www/html/wiki1/line1_.jpg

And in a sufficiently large dictionary, this would only become apparent very rarely – even more so since a file name ending in _ is not a problem.

Change 542499 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[mediawiki/core@master] Fix line selection in RandomImageGenerator

https://gerrit.wikimedia.org/r/542499

Change 542537 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] tests: Replace RandomImageGenerator dictionary logic with rand+time

https://gerrit.wikimedia.org/r/542537

Change 545584 had a related patch set uploaded (by Anomie; owner: Anomie):
[mediawiki/core@master] Fix line selection in RandomImageGenerator

https://gerrit.wikimedia.org/r/545584

Change 545584 merged by jenkins-bot:
[mediawiki/core@master] Fix line selection in RandomImageGenerator

https://gerrit.wikimedia.org/r/545584

Change 542499 abandoned by Lucas Werkmeister (WMDE):
Fix line selection in RandomImageGenerator

Reason:
in favor of If136928bda (already merged)

https://gerrit.wikimedia.org/r/542499

Lucas_Werkmeister_WMDE closed this task as Resolved.Oct 23 2019, 3:51 PM
Lucas_Werkmeister_WMDE claimed this task.

Okay, let’s close this task and hope the above change fixes the problem – if anyone notices it again, please reopen. (I feel like if we don’t close it now, we’ll just forget to do it and it’ll stay open for ages.)

Change 542537 merged by jenkins-bot:
[mediawiki/core@master] tests: Replace RandomImageGenerator dictionary logic with rand+time

https://gerrit.wikimedia.org/r/542537