Page MenuHomePhabricator

Create docker based CI job to build the wikidata-query-gui
Closed, ResolvedPublic8 Estimated Story Points

Description

Desired Behavior

As a WMDE developer
Once a GUI change has been merged on Gerrit
I can go to Jenkins to invoke a gui build job
That will submit a new build change to the deployment repository
So that I don't have to do manually on my local machine.

Acceptance Criteria

  • There is a Jenkins job defined that will build the query gui on demand and push the build to the qui-deploy repository for review and actual deployment.

Notes

Per the original description, the existing wikimedia-portals-build job does something similar.

Original Description

The old wikidata-query-gui-build CI job was disabled to avoid blocking migration of some CI things.
This decision was made as the job was broken anyway.

A new job should be created building the GUI based on the CI docker systems.

See also T213806 for wikimedia-portals-build which share common logic.

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Right now the decision needs to happen between T210286 and T192006.
And the question there is are we going to deploy the query service UI on the k8s cluster, which could be done as part of T235639.
If we do go the k8s route, we don't need to migrate the thing in this task we can just stop using it.

Once T241291 is done I guess we will just be able to close this one

So the decision by the QS team was not to do this as part of their work, so I guess this swings back to WMDE.

Right now the build job is broken anyway, so if this is the last blocked please done block on this, instead just disable the job.

Moving forward we will recreate the job using docker, or change the build process or something else etc.

Anything I can do to help with this? It's a blocker for us in Release Engineering (both for T211784 and for T239981 and thus T236576).

(As you said, I can just delete the job, but I would really like to help the team get this working again.)

Right now the goal would still be to not really use / have this job and instead complete T192006 creating a docker image and deploying the UI on k8s.

Change 575381 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: [wikidata-query-gui-build] Drop for now, awaiting replacement with Docker

https://gerrit.wikimedia.org/r/575381

Change 575381 merged by jenkins-bot:
[integration/config@master] jjb: [wikidata-query-gui-build] Drop for now, awaiting replacement with Docker

https://gerrit.wikimedia.org/r/575381

Addshore renamed this task from Migrate wikidata-query-gui-build to Docker containers to Create docker based CI job to build the wikidata-query-gui.Apr 28 2020, 2:59 PM
Addshore lowered the priority of this task from Medium to Low.
Addshore updated the task description. (Show Details)

This was prioritized in a prio session, but was determined to perhaps be better as a trailblaze.
This ticket needs more content either way, but for now I have put it in the backlog.

I can go to Jenkins to invoke a gui build job
There is a Jenkins job defined that will build the query gui on demand and push the build to the qui-deploy repository for review and actual deployment.

Is it a requirement that the job is triggered manually? As far as I recall, the old job used to run automatically after every merge to the GUI repo, which seems more convenient to me.

darthmon_wmde set the point value for this task to 8.Feb 3 2021, 1:29 PM

Change 664848 had a related patch set uploaded (by Lucas Werkmeister (WMDE); owner: Lucas Werkmeister (WMDE)):
[integration/config@master] WIP: Reinstate wikidata-query-gui-build job

https://gerrit.wikimedia.org/r/664848

Change 664848 merged by jenkins-bot:
[integration/config@master] Reinstate wikidata-query-gui-build job

https://gerrit.wikimedia.org/r/664848

Change 668038 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] Remove zuul trigger for wikidata-query-gui-build

https://gerrit.wikimedia.org/r/668038

So the triggers - zuul needs to be removed otherwise the zuul parameters get added again, overwriting the defaults set by the job.
This now runs, you can use one of the build buttons in the jenkins UI.

https://integration.wikimedia.org/ci/job/wikidata-query-gui-build/

image.png (432×383 px, 30 KB)

Seems to fail right now:

11:13:41 Running "qunit:all" (qunit) task
11:13:41 >> There was an error with headless chrome
11:13:41 Fatal error: Failed to launch chrome!
11:13:41 /src/node_modules/puppeteer/.local-chromium/linux-686378/chrome-linux/chrome: error while loading shared libraries: libX11.so.6: cannot open shared object file: No such file or directory

Could relate to some other tickets currently on camp!

Change 668038 merged by jenkins-bot:
[integration/config@master] Remove zuul trigger for wikidata-query-gui-build

https://gerrit.wikimedia.org/r/668038

Change 668069 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] wikidata-query-gui-build run webdriver

https://gerrit.wikimedia.org/r/668069

Now failing with:

12:26:08 
12:26:08 * Please tell me who you are.
12:26:08 
12:26:08 Run
12:26:08 
12:26:08   git config --global user.email "you@example.com"
12:26:08   git config --global user.name "Your Name"
12:26:08 
12:26:08 to set your account's default identity.
12:26:08 Omit --global to set the identity only in this repository.
12:26:08 
12:26:08 fatal: unable to auto-detect email address (got 'nobody@b11404240ad1.(none)')
12:26:08  Use --force to continue.
12:26:08 
12:26:08 Aborted due to warnings.
12:26:08 npm ERR! code ELIFECYCLE

https://integration.wikimedia.org/ci/job/wikidata-query-gui-build/5/console

Change 668069 merged by jenkins-bot:
[integration/config@master] wikidata-query-gui-build run webdriver

https://gerrit.wikimedia.org/r/668069

The main difference between our current approach and that used by wikimedia-portals-build is that we are trying to push to gerrit within our npm / grunt command, thus within a container.
wikimedia-portals-build instead only builds in the container, pushing on the jenkins host itself.

This might end up being easier to do so that we can easily use the ssh agent etc (as sssh details are provided by ssh-agent-credentials)

Change 672436 had a related patch set uploaded (by Addshore; owner: Addshore):
[wikidata/query/gui@master] Add a build_for_deploy grunt task that doesn't commit or push

https://gerrit.wikimedia.org/r/672436

Change 672436 merged by jenkins-bot:
[wikidata/query/gui@master] Add a build_for_deploy grunt task that doesn't commit or push

https://gerrit.wikimedia.org/r/672436

Change 672440 had a related patch set uploaded (by Addshore; owner: Addshore):
[wikidata/query/gui@master] Add npm script for build_for_deploy

https://gerrit.wikimedia.org/r/672440

Change 672440 merged by jenkins-bot:
[wikidata/query/gui@master] Add npm script for grunt

https://gerrit.wikimedia.org/r/672440

Change 672459 had a related patch set uploaded (by WDQSGuiBuilder; owner: WDQSGuiBuilder):
[wikidata/query/gui-deploy@production] Merging from f99989440cc067ae37d8970e0dafea996a32b35e

https://gerrit.wikimedia.org/r/672459

Change 672464 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] Alter wikidata-query-gui-build job to successfully push

https://gerrit.wikimedia.org/r/672464

The job in it's current state makes a build and pushes it with the commit message of the last commit merged into the gui repo.
This might make sense if we make these builds be postmerge, but perhaps this doesn't make sense for triggered builds?

Change 672464 merged by jenkins-bot:
[integration/config@master] Alter wikidata-query-gui-build job to successfully push

https://gerrit.wikimedia.org/r/672464

/me sees this as one more argument for the builds being postmerge…

Change 672467 had a related patch set uploaded (by WDQSGuiBuilder; owner: WDQSGuiBuilder):
[wikidata/query/gui-deploy@production] Merging from f99989440cc067ae37d8970e0dafea996a32b35e

https://gerrit.wikimedia.org/r/672467

Change 672459 abandoned by Addshore:
[wikidata/query/gui-deploy@production] Merging from f99989440cc067ae37d8970e0dafea996a32b35e

Reason:
In favour of https://gerrit.wikimedia.org/r/c/wikidata/query/gui-deploy/ /672467 which keeps the custom config

https://gerrit.wikimedia.org/r/672459

I accidently pushed the last change to the job directly to master (missing gerrit).
You can find it here.... https://github.com/wikimedia/integration-config/commit/8523412cdb1546a2ad021c9047a172b817cd0fb1
(I already told releng I did this)

/me sees this as one more argument for the builds being postmerge…

I partly agree, though in the case that the previous build commit was not yet merged, the commit message still will not make any sense. So I would be tempted to change the commit message even if we do post merge builds
The portals build that we are copying doesn't include things like the previous commit msg in the build commit message.

Change 672471 had a related patch set uploaded (by Addshore; owner: Addshore):
[integration/config@master] wikidata-query-gui-build: don't clean up custom config

https://gerrit.wikimedia.org/r/672471

Change 672471 abandoned by Addshore:
[integration/config@master] wikidata-query-gui-build: don't clean up custom config

Reason:
I pushed this directly to master by accident.
PS1 is a representation of what I pushed.
Added to gerrit so that it is linked to phab etc.

https://gerrit.wikimedia.org/r/672471

Change 672467 merged by Addshore:
[wikidata/query/gui-deploy@production] Merging from f99989440cc067ae37d8970e0dafea996a32b35e

https://gerrit.wikimedia.org/r/672467

The integration-config patches are already merged as this is already deployed into the the ci system.
The easiest way to review this would now be to look at https://github.com/wikimedia/integration-config/blob/master/jjb/wikidata.yaml#L71-L140

This successfully ran in CI here: https://integration.wikimedia.org/ci/job/wikidata-query-gui-build/21/
The CI job created this new build job for the deployment repo: https://gerrit.wikimedia.org/r/c/wikidata/query/gui-deploy/+/672467

LGTM

          #
  #      ##
  #     # #
#####     #
  #       #
  #       #
        #####

Random note: merged builds are logged in the wdqs-gui-build mailing list, originally set up at T189810 (we use its email address as the Git email, and Gerrit sends an email to it when the change is merged). Currently I’m the only list admin; let me know if anyone else is interested ^^