Page MenuHomePhabricator

[Bug] the changed job configuration extension-unittests -> extension-unittests-generic for Wikidata.git makes it not run all tests and fail
Closed, ResolvedPublic

Description

The changed job configuration extension-unittests -> extension-unittests-generic for Wikidata.git makes it not run all tests and fail. Compare after https://integration.wikimedia.org/ci/job/mwext-testextension-zend/75/consoleFull and before https://integration.wikimedia.org/ci/job/mwext-Wikidata-testextension-zend/175/consoleFull .

The change in question is: 44a943a115f328a541c8ad94349bd8670eca0815

However the diff of the jobs does not explain this for me:

$ diff -u output-before/mwext-Wikidata-testextension-zend output-after/mwext-testextension-zend 
--- output-before/mwext-Wikidata-testextension-zend     2015-04-13 16:38:13.321862204 +0200
+++ output-after/mwext-testextension-zend       2015-04-13 16:32:33.537852125 +0200
@@ -16,13 +16,6 @@
     <artifactNumToKeep>-1</artifactNumToKeep>
   </logRotator>
   <properties>
-    <hudson.plugins.throttleconcurrents.ThrottleJobProperty>
-      <maxConcurrentPerNode>1</maxConcurrentPerNode>
-      <maxConcurrentTotal>0</maxConcurrentTotal>
-      <throttleEnabled>true</throttleEnabled>
-      <throttleOption>project</throttleOption>
-      <configVersion>1</configVersion>
-    </hudson.plugins.throttleconcurrents.ThrottleJobProperty>
     <hudson.model.ParametersDefinitionProperty>
       <parameterDefinitions>
         <hudson.model.StringParameterDefinition>
@@ -129,16 +122,6 @@
 </command>
     </hudson.tasks.Shell>
     <hudson.tasks.Shell>
-      <command># Build dependencies list
-rm -f deps.txt
-touch deps.txt
-echo -e &quot;mediawiki/core\nmediawiki/vendor\nmediawiki/extensions/Wikidata&quot; &gt;&gt; deps.txt
-if [ &quot;&quot; != &quot;&quot; ]; then
-    echo &quot;$(IFS=,; for dep in `echo `; do echo mediawiki/extensions/$dep; done;)&quot; &gt;&gt; deps.txt
-fi
-</command>
-    </hudson.tasks.Shell>
-    <hudson.tasks.Shell>
       <command>zuul-cloner --version
 zuul-cloner \
     --color \
@@ -146,21 +129,14 @@
     --map /srv/deployment/integration/slave-scripts/etc/zuul-clonemap.yaml \
     --workspace src \
     https://gerrit.wikimedia.org/r/p \
-    $(cat deps.txt) 
-</command>
-    </hudson.tasks.Shell>
-    <hudson.tasks.Shell>
-      <command>if grep -q mediawiki/extensions/VisualEditor deps.txt; then
-   cd src/extensions/VisualEditor
-   git submodule update --init
-fi
+    mediawiki/core mediawiki/vendor $ZUUL_PROJECT 
 </command>
     </hudson.tasks.Shell>
     <hudson.tasks.Shell>
       <command>/srv/deployment/integration/slave-scripts/bin/mw-install-mysql.sh</command>
     </hudson.tasks.Shell>
     <hudson.tasks.Shell>
-      <command>cp deps.txt src/extensions_load.txt</command>
+      <command>echo $ZUUL_PROJECT &gt; src/extensions_load.txt</command>
     </hudson.tasks.Shell>
     <hudson.tasks.Shell>
       <command>/srv/deployment/integration/slave-scripts/bin/mw-apply-settings.sh</command>

Event Timeline

JanZerebecki raised the priority of this task from to High.
JanZerebecki updated the task description. (Show Details)
JanZerebecki added subscribers: JanZerebecki, Legoktm.

Change 203858 had a related patch set uploaded (by Legoktm):
Use non-generic job for Wikidata extension

https://gerrit.wikimedia.org/r/203858

Change 203858 merged by jenkins-bot:
Use non-generic job for Wikidata extension

https://gerrit.wikimedia.org/r/203858

Legoktm removed a subscriber: gerritbot.

Reverted for now, the generic jobs are experimental so we can debug what is going wrong.

To test extensions we load them from their PHP entry point such as foo/foo.php.

Wikidata is a bit specific since it has several components and originally had no entry point (iirc). So there a hack to detect which Jenkins job is running and use different settings:

So it looks at the JOB_NAME being 'mwext-Wikidata-testextension':

if ( PHP_SAPI === 'cli' && strpos( getenv( 'JOB_NAME' ), 'mwext-Wikidata-testextension' ) !== false ) {
    // in future, run as non-experimental
    if ( !defined( 'WB_EXPERIMENTAL_FEATURES' ) || !WB_EXPERIMENTAL_FEATURES ) {
        define( 'WB_EXPERIMENTAL_FEATURES', true );
    }

    $wmgUseWikibaseRepo = true;
    $wmgUseWikibaseClient = true;
}

The two $wmg settings loads different components:

if ( !empty( $wmgUseWikibaseRepo ) ) {
    include_once "$wgWikidataBaseDir/extensions/Wikibase/repo/Wikibase.php";
    include_once "$wgWikidataBaseDir/extensions/Wikidata.org/WikidataOrg.php";
    include_once "$wgWikidataBaseDir/extensions/PropertySuggester/PropertySuggester.php";
}

if ( !empty( $wmgUseWikibaseClient ) ) {
    include_once "$wgWikidataBaseDir/extensions/Wikibase/client/WikibaseClient.php";
}

So by changing the job name, those includes are not realized..

Note: Wikibase has a similar hack though based on $wgWikimediaJenkinsCI.

JanZerebecki lowered the priority of this task from High to Medium.Apr 16 2015, 4:06 PM
JanZerebecki moved this task from incoming to ready to go on the Wikidata board.

in integration/jenkins.git mediawiki/conf.d/10_set_wgWikimediaJenkinsCI.php we have:

$wgWikimediaJenkinsCI = true;

So you can:

if ( isset( $wgWikimediaJenkinsCI ) && $wgWikimediaJenkinsCI == true ) {
    echo "I hate Jenkins\n"; exit(0);
}

We should convert it to use something like `if ( isset( $wgWikimediaJenkinsCI ) && $wgWikimediaJenkinsCI == true )` which is set in integration/jenkins.git mediawiki/conf.d/10_set_wgWikimediaJenkinsCI.php .

Jonas renamed this task from the changed job configuration extension-unittests -> extension-unittests-generic for Wikidata.git makes it not run all tests and fail to [Bug] the changed job configuration extension-unittests -> extension-unittests-generic for Wikidata.git makes it not run all tests and fail.Aug 15 2015, 12:50 PM

Change 231779 had a related patch set uploaded (by Jdlrobson):
Revert "WikidataPageBanner should run Wikibase tests"

https://gerrit.wikimedia.org/r/231779

Change 250990 had a related patch set uploaded (by JanZerebecki):
ext:Wikidata repo: switch to generic and add hhvm

https://gerrit.wikimedia.org/r/250990

Change 250990 merged by jenkins-bot:
ext:Wikidata repo: switch to generic and add hhvm

https://gerrit.wikimedia.org/r/250990