Page MenuHomePhabricator

Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects
Open, NormalPublic

Description

[Please create sub-tasks for relevant jobs/repos.]

Things still using node6 as of 2019-06-12:

RepoStateTask?
analytics/mediawiki-storage Experimental, doesn't passNo task yet
analytics/wikistats2 Experimental, doesn't passNo task yet
data-values/value-view Experimental, doesn't passNo task yet
oojs/ui Needs custom imageNo task yet
mediawiki/services/mobileapps & …/deploy Running in parallel, waiting for production switchT218733
mediawiki/services/recommendation-api & …/deploy Running in parallel, waiting for production switchT225107
3d2png & …/deploy Not yet running in parallel, doesn't passT225678
mediawiki/services/chromium-render & …/deploy Not yet running in parallel, doesn't passT217114
mediawiki/services/parsoid & …/deploy ???No task yet
maps/kartotherian & …/deploy and maps/tilerator & …/deploy To migrateT213077
wikidata/query/gui & …-deploy To migrateT210286

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 489120 merged by jenkins-bot:
[integration/config@master] Switch labs/tools/wikidata-slicer to node10

https://gerrit.wikimedia.org/r/489120

Change 489128 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Switch npm-audit job to node10

https://gerrit.wikimedia.org/r/489128

Change 489128 merged by jenkins-bot:
[integration/config@master] Switch npm-audit job to node10

https://gerrit.wikimedia.org/r/489128

Mentioned in SAL (#wikimedia-releng) [2019-02-08T15:59:36Z] <addshore> this reload also included "Switch npm-audit job to node10"? T211784, which did touch the zuul file

Change 486188 merged by jenkins-bot:
[mediawiki/core@master] build: Use headless Chrome and Firefox for QUnit testing

https://gerrit.wikimedia.org/r/486188

Change 491379 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Remove redundant 'install /cache' from node10-test image

https://gerrit.wikimedia.org/r/491379

Change 491380 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] jjb: Update node10-test* images in jobs to version 0.3.3

https://gerrit.wikimedia.org/r/491380

Change 491379 merged by jenkins-bot:
[integration/config@master] Remove redundant 'install /cache' from node10-test image

https://gerrit.wikimedia.org/r/491379

Mentioned in SAL (#wikimedia-releng) [2019-02-18T22:32:08Z] <Krinkle> Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/491379 / T211784

Change 491380 merged by jenkins-bot:
[integration/config@master] jjb: Update node10-test* images in jobs to version 0.3.3

https://gerrit.wikimedia.org/r/491380

Change 492425 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Try node10 on pywikibot/i18n

https://gerrit.wikimedia.org/r/492425

Change 492425 merged by jenkins-bot:
[integration/config@master] Try node10 on pywikibot/i18n

https://gerrit.wikimedia.org/r/492425

Change 492427 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Add generic job for node10/npm-test with browsers

https://gerrit.wikimedia.org/r/492427

Change 492427 merged by jenkins-bot:
[integration/config@master] Add generic job for node10/npm-test with browsers

https://gerrit.wikimedia.org/r/492427

Change 482764 abandoned by Krinkle:
Add node10-docker job variants for mwgate

https://gerrit.wikimedia.org/r/482764

Change 496882 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Update mwext-EventLogging postmerge from jsduck to generic node10

https://gerrit.wikimedia.org/r/496882

Change 496882 merged by jenkins-bot:
[integration/config@master] Update mwext-EventLogging postmerge from jsduck to generic node10

https://gerrit.wikimedia.org/r/496882

Krinkle added a comment.EditedMar 20 2019, 3:47 AM

Last quarter I've put the basics in place for Node 10 and npm 6. This was mainly for Fresnel, but I took the time to make it generic for other standalone repositories as well, and for MediaWiki repos. Quibble is now fully on Debian Stretch (the HHVM flavour was on Jessie still, which didn't match prod; this is now fixed).

I'm signing this back to Release-Engineering to work on further, including the porting of Quibble and Node.js service-deploy jobs to Node 10.

When we started we were a bit ahead of schedule (in terms of production only just beginning to use Node 10). By now, however, we're right schedule (or slightly behind, as more production services are using Node 10 now, maybe some that don't use it in CI yet, not sure).

Also:

Node 6 will be EOL by end of April 2019 (nodejs/Release  ·  Announcement).

Change 501428 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[integration/config@master] Switch various jobs from node6/npm3 to node10/npm6

https://gerrit.wikimedia.org/r/501428

Change 501428 merged by jenkins-bot:
[integration/config@master] Switch various jobs from node6/npm3 to node10/npm6

https://gerrit.wikimedia.org/r/501428

awight added a subscriber: awight.

Change 514012 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] Temporarily don't run selenium tests with quibble

https://gerrit.wikimedia.org/r/514012

Change 487896 abandoned by Hashar:
Update Quibble containers to Node 10 with npm 6

Reason:
It is way too outdated

https://gerrit.wikimedia.org/r/487896

Mentioned in SAL (#wikimedia-releng) [2019-06-03T14:38:18Z] <James_F> hashar and I are temporarily disabling running selenium tests in CI. See T211784 T222406 for more details.

Change 514012 abandoned by Jforrester:
Temporarily don't run selenium tests with quibble

Reason:
Done in core instead.

https://gerrit.wikimedia.org/r/514012

Change 514089 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] Replace mwgate-npm-node-6-docker with mwgate-node10-docker

https://gerrit.wikimedia.org/r/514089

Change 514089 merged by jenkins-bot:
[integration/config@master] Replace mwgate-npm-node-6-docker with mwgate-node10-docker

https://gerrit.wikimedia.org/r/514089

Things still using node6 as of 2019-06-12:

  • Services – T210704
    • 3d2png & …/deploy – ToCreate
    • mediawiki/services/chromium-render & …/deploy – T217114
    • mediawiki/services/mobileapps & …/deploy – T218733
    • mediawiki/services/parsoid & …/deploy – ToCreate
    • mediawiki/services/recommendation-api & …/deploy – T225107
    • mediawiki/services/trending-edits & …/deploy – ToCreate
    • maps/kartotherian & …/deploy and maps/tilerator & …/deploy – T213077
    • wikidata/query/gui & …-deploy – T210286
  • Analytics things
    • analytics/aqs
    • analytics/dashiki
    • analytics/mediawiki-storage
    • analytics/wikistats2
  • Randoms
    • data-values/value-view
    • design/landing-page
    • labs/tools/graphql
    • mapdata
    • oojs/ui
    • operations/dumps/dcat
    • react.i18n
    • wikidata/build-resources
    • wikimedia-cz/tracker
    • wikimedia/fundraising/dash
    • wikimedia/portals & …/deploy
    • wikimedia/wikimania-scholarships

Most probably all of those repositories can just be upgraded from Node 6 to 10. The conservative way I have been doing for such upgrades is to add a job in the experimental pipeline, trigger the job for all repositories, assert they pass, then switch.

Then I guess we can just f****ing do it. They are probably just running grunt-jsonlint / banana checker.

oojs/ui might be a special case.

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

wikidata/build-resources might have some kind of magic build script which would be affected.

Beside those, the rest is probably just fine.

Most probably all of those repositories can just be upgraded from Node 6 to 10. The conservative way I have been doing for such upgrades is to add a job in the experimental pipeline, trigger the job for all repositories, assert they pass, then switch.

Then I guess we can just f****ing do it. They are probably just running grunt-jsonlint / banana checker.

Yeah, probably.

oojs/ui might be a special case.

It definitely is – it needs an image with npm, composer, and jsduck right now. :-(

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

Yeah – didn't want to break that workflow. @Jdrewniak, are you the expert on this?

wikidata/build-resources might have some kind of magic build script which would be affected.

I imagine we can just change it and make it @Addshore's problem if it breaks. :-)

Beside those, the rest is probably just fine.

OK, experimental patch incoming.

Change 516674 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/aqs] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516674

Change 516675 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/dashiki] Experiment with generic-node10-docker; no current CI

https://gerrit.wikimedia.org/r/516675

Change 516676 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/{mediawiki-storage,wikistats2}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516676

For jsduck, if a repository is still using it, just break it. It has been obsolete for a few years now and the last commit is from 2013 or 6 years ago. It is time to break! If we want to play it nice, we can have E_YET_ANOTHER_TASK another task that announce Jsduck being dropped and just get rid of it from all repos: https://codesearch.wmflabs.org/search/?q=jsduck&i=nope&files=&repos=

:-]

Change 516674 merged by jenkins-bot:
[integration/config@master] layout: [analytics/aqs] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516674

Change 516675 merged by jenkins-bot:
[integration/config@master] layout: [analytics/dashiki] Experiment with generic-node10-docker; no current CI

https://gerrit.wikimedia.org/r/516675

Change 516676 merged by jenkins-bot:
[integration/config@master] layout: [analytics/{mediawiki-storage,wikistats2}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516676

Change 516682 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [data-values/value-view] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516682

Change 516683 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [design/landing-page] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516683

Change 516684 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [labs/tools/graphql] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516684

Change 516685 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [mapdata] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516685

Change 516686 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [operations/dumps/dcat] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516686

Change 516687 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [react.i18n] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516687

Change 516688 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikidata/build-resources] Archive, unused

https://gerrit.wikimedia.org/r/516688

Change 516689 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia-cz/tracker] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516689

Change 516690 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/fundraising/dash] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516690

Change 516691 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/portals{,/deploy}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516691

Change 516692 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/wikimania-scholarships] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516692

Change 516682 merged by jenkins-bot:
[integration/config@master] layout: [data-values/value-view] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516682

Change 516683 merged by jenkins-bot:
[integration/config@master] layout: [design/landing-page] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516683

Change 516684 merged by jenkins-bot:
[integration/config@master] layout: [labs/tools/graphql] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516684

Change 516685 merged by jenkins-bot:
[integration/config@master] layout: [mapdata] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516685

Change 516686 merged by jenkins-bot:
[integration/config@master] layout: [operations/dumps/dcat] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516686

Change 516687 merged by jenkins-bot:
[integration/config@master] layout: [react.i18n] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516687

Change 516688 merged by jenkins-bot:
[integration/config@master] layout: [wikidata/build-resources] Archive, unused

https://gerrit.wikimedia.org/r/516688

Change 516689 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia-cz/tracker] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516689

Change 516690 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/fundraising/dash] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516690

Change 516691 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/portals{,/deploy}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516691

Change 516692 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/wikimania-scholarships] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516692

Change 516707 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [3d2png, mediawiki/services/{chromium-render,recommendation-api,trending-edits}] Add service-pipeline-test

https://gerrit.wikimedia.org/r/516707

Change 516707 merged by jenkins-bot:
[integration/config@master] layout: [3d2png, mediawiki/services/{chromium-render,recommendation-api,trending-edits}] Add service-pipeline-test

https://gerrit.wikimedia.org/r/516707

Change 516716 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: Disable service-pipeline-test from 3d2png, chromium-render, and trending-edits for now

https://gerrit.wikimedia.org/r/516716

Change 516716 merged by jenkins-bot:
[integration/config@master] layout: Disable service-pipeline-test from 3d2png, chromium-render, and trending-edits for now

https://gerrit.wikimedia.org/r/516716

OK, that's enough for today.