Submodule not being updated in Jenkins jobs
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Krinkle
	Jun 2 2015, 2:47 PM

Description

The mwext-VisualEditor-* jobs are no longer taking the submodule into account when starting a build.

It checks out the commit of the root repository, but the submodules are stale from whatever commit/branch was last used on that slave/workspace before the bug started happening.

While all VE jobs are affected, this is most significantly affecting mwext-VisualEditor-npm because that job asserts the git checkout is clean.

https://integration.wikimedia.org/ci/job/mwext-VisualEditor-npm/

I temporarily fixed it by updating the live job config. It looks like the "Disable submodule" option no longer exists, and is now inside a "Advanced submodule options" menu. Since this sub-section did not exist in the XML, this part of the plugin was not triggered at all.

Looking at the diff you can see this is caused by the Git plugin having been upgraded and it has a breaking change.

Before:

<scm class="hudson.plugins.git.GitSCM" plugin="git@1.5.0">
  ..
  <disableSubmodules>false</disableSubmodules>
 ..
</scm>

After

<scm class="hudson.plugins.git.GitSCM" plugin="git@2.3.5">
  ..
  <extensions>
  <hudson.plugins.git.extensions.impl.SubmoduleOption>
    <disableSubmodules>false</disableSubmodules>
    ..
  ..
 ..
</scm>

jjb will need updating, and re-generating all relevant jobs.

Details

Subject	Repo	Branch	Lines +/-
Force JJB to use new git submodule syntax (2)	integration/config	master	+1 -0
Force JJB to use new git submodule syntax	integration/config	master	+1 -0
Switch WikidataQuality to the generic npm job	integration/config	master	+1 -6
Switch Wikibase to the generic npm job	integration/config	master	+2 -6
Update Git plugin configuration	integration/config	master	+38 -19

Customize query in gerrit

Related Objects

Mentioned In: rCICFc94e5e32af64: Force JJB to use new git submodule syntax (2)
rCICF95bf3f5af877: Force JJB to use new git submodule syntax
rCICFb1e51b0929c5: Switch WikidataQuality to the generic npm job
rCICFa0f3bd91a576: Switch Wikibase to the generic npm job
rCICF2113cb1538ca: Update Git plugin configuration
T100655: Upgrade Jenkins git related plugins

Event Timeline

Krinkle created this task.Jun 2 2015, 2:47 PM

Krinkle raised the priority of this task from to Unbreak Now!.

Krinkle updated the task description. (Show Details)

Krinkle added projects: Regression, VisualEditor, Contributors-Team, Continuous-Integration-Infrastructure.

Krinkle subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 2 2015, 2:47 PM

hashar moved this task from Untriaged to In-progress on the Continuous-Integration-Infrastructure board.Jun 2 2015, 2:50 PM

hashar mentioned this in T100655: Upgrade Jenkins git related plugins.Jun 2 2015, 3:09 PM

The JJB version on integration/jenkins-job-builder is at the commit just before Update git-submodule configuration to new extension style.

It changes JJB syntax https://review.openstack.org/#/c/148057/6/tests/scm/fixtures/git-submodule01.yaml but style recognize the current one.

Gotta update our JJB copy and update our yaml config files.

I have updated integration/jenkins-job-builder to e3199d9

Change 215335 had a related patch set uploaded (by Hashar):
Update Git plugin configuration

https://gerrit.wikimedia.org/r/215335

gerritbot added a project: Patch-For-Review.Jun 2 2015, 3:30 PM

Further updated JJB from e3199d9 to c7231fe

Now gotta merge in https://gerrit.wikimedia.org/r/#/c/215335/ and refresh all the jobs :-/

Jdforrester-WMF subscribed.Jun 2 2015, 5:17 PM

Finished too late, had some meetings in the evening. So will process updating jobs carefully tomorrow morning.

Change 215335 merged by jenkins-bot:
Update Git plugin configuration

https://gerrit.wikimedia.org/r/215335

hashar mentioned this in rCICF2113cb1538ca: Update Git plugin configuration.Jun 3 2015, 9:04 AM

I have refreshed all the jobs. Should be fine now.

I sent a mail to the qa list to have people to update their JJB installation from our fork.

hashar closed this task as Resolved.Jun 3 2015, 10:04 AM

hashar moved this task from In-progress to Done on the Continuous-Integration-Infrastructure board.

Hm.. that didn't fix it. A couple days ago I updated the VE job config on Jenkins to re-enable the submodule handling (using the newer git plugin version).

It looks like your deployment didn't use the newer git config. Instead, it did the opposite and deployed the old code again. This diff removed the hudson.plugins.git.extensions.impl.SubmoduleOption extension and adds the old <disableSubmodules>false</disableSubmodules> back.
https://integration.wikimedia.org/ci/job/mwext-VisualEditor-npm/jobConfigHistory/showDiffFiles?timestamp1=2015-06-01_22-24-01&timestamp2=2015-06-03_12-14-50

https://integration.wikimedia.org/ci/job/mwext-VisualEditor-npm/

Screen Shot 2015-06-04 at 01.38.04.png (1×1 px, 307 KB)

I've reverted the job config update to mwext-VisualEditor-npm on Jenkins for now.

Krinkle moved this task from Done to In-progress on the Continuous-Integration-Infrastructure board.Jun 4 2015, 12:40 AM

The mwext-VisualEditor-npm job is generated using:

template: '{name}-{ext-name}-npm'
defaults: use-remoteonly-zuul
scm macro: git-remoteonly-zuul

And that macro does not proceed submodules.

The template has a comment stating it is only used by VE, but it is also used by Wikibase extension, though that last one has no submodule and could use the regular npm job.

So I am going to:

switch Wikibase to use the npm job
make the mwext-Wikibase-npm a job instead of a job-template

Change 215875 had a related patch set uploaded (by Hashar):
Switch Wikibase to the generic npm job

https://gerrit.wikimedia.org/r/215875

Change 215876 had a related patch set uploaded (by Hashar):
Switch WikidataQuality to the generic npm job

https://gerrit.wikimedia.org/r/215876

Change 215875 merged by jenkins-bot:
Switch Wikibase to the generic npm job

https://gerrit.wikimedia.org/r/215875

hashar mentioned this in rCICFa0f3bd91a576: Switch Wikibase to the generic npm job.Jun 4 2015, 11:02 AM

Change 215876 merged by jenkins-bot:
Switch WikidataQuality to the generic npm job

https://gerrit.wikimedia.org/r/215876

hashar mentioned this in rCICFb1e51b0929c5: Switch WikidataQuality to the generic npm job.Jun 4 2015, 11:06 AM

Change 215888 had a related patch set uploaded (by Hashar):
Force JJB to use new git submodule syntax

https://gerrit.wikimedia.org/r/215888

Change 215893 had a related patch set uploaded (by Hashar):
Force JJB to use new git submodule syntax (2)

https://gerrit.wikimedia.org/r/215893

Change 215888 merged by jenkins-bot:
Force JJB to use new git submodule syntax

https://gerrit.wikimedia.org/r/215888

hashar mentioned this in rCICF95bf3f5af877: Force JJB to use new git submodule syntax.Jun 4 2015, 12:22 PM

Change 215893 merged by jenkins-bot:
Force JJB to use new git submodule syntax (2)

https://gerrit.wikimedia.org/r/215893

hashar mentioned this in rCICFc94e5e32af64: Force JJB to use new git submodule syntax (2).Jun 4 2015, 12:25 PM

All blame goes to JJB :-D We have to set a submodule parameter to trigger the proper config :/

Jdforrester-WMF removed a project: Patch-For-Review.Jun 10 2015, 6:09 PM

Jdforrester-WMF moved this task from To Triage to FY 18-19 Q3/Q4 on the VisualEditor board.Jun 17 2015, 11:33 PM

hashar moved this task from In-progress to Done on the Continuous-Integration-Infrastructure board.Jun 23 2015, 9:08 AM

Jdforrester-WMF added a project: VisualEditor 2014/15 Q4 blockers.Aug 5 2015, 1:22 AM

	F174179: Screen Shot 2015-06-04 at 01.38.04.png
	Jun 4 2015, 12:40 AM

Submodule not being updated in Jenkins jobsClosed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

Submodule not being updated in Jenkins jobs
Closed, ResolvedPublic
Actions