Page MenuHomePhabricator

Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects
Open, Stalled, NormalPublic

Description

[Please create sub-tasks for relevant jobs/repos.]

Things still using node6 as of 2019-07-18:

RepoStateTask?
mediawiki/services/mobileapps & …/deploy Running in parallel, waiting for production switchT218733
mediawiki/services/recommendation-api & …/deploy Running in parallel, waiting for production switchT225107
3d2png & …/deploy Not yet running in parallel, doesn't passT225678
mediawiki/services/chromium-render & …/deploy Not yet running in parallel, doesn't passT217114
mediawiki/services/parsoid & …/deploy ???No task yet
maps/kartotherian/deploy and maps/tilerator/deploy Waiting for production switch, then deleteNo task yet
wikidata/query/gui & …-deploy To migrateT210286
wikimedia/portalsTo migrateT213806
data-values/value-view Not yet running in parallel, doesn't passT229276

Related Objects

StatusAssignedTask
StalledJdforrester-WMF
InvalidJdforrester-WMF
ResolvedMoritzMuehlenhoff
ResolvedKrinkle
ResolvedKrinkle
Resolvedhashar
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF
DeclinedJdforrester-WMF
DuplicateNone
ResolvedMilimetric
ResolvedMilimetric
OpenNone
OpenNone
OpenNone
OpenNone
OpenNone
OpenMSantos
OpenNone
StalledNone
ResolvedAddshore
DeclinedNone
DuplicateNone
OpenNone
OpenNone
ResolvedJdforrester-WMF
ResolvedJdforrester-WMF

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Things still using node6 as of 2019-06-12:

  • Services – T210704
    • 3d2png & …/deploy – ToCreate
    • mediawiki/services/chromium-render & …/deploy – T217114
    • mediawiki/services/mobileapps & …/deploy – T218733
    • mediawiki/services/parsoid & …/deploy – ToCreate
    • mediawiki/services/recommendation-api & …/deploy – T225107
    • mediawiki/services/trending-edits & …/deploy – ToCreate
    • maps/kartotherian & …/deploy and maps/tilerator & …/deploy – T213077
    • wikidata/query/gui & …-deploy – T210286
  • Analytics things
    • analytics/aqs
    • analytics/dashiki
    • analytics/mediawiki-storage
    • analytics/wikistats2
  • Randoms
    • data-values/value-view
    • design/landing-page
    • labs/tools/graphql
    • mapdata
    • oojs/ui
    • operations/dumps/dcat
    • react.i18n
    • wikidata/build-resources
    • wikimedia-cz/tracker
    • wikimedia/fundraising/dash
    • wikimedia/portals & …/deploy
    • wikimedia/wikimania-scholarships

Most probably all of those repositories can just be upgraded from Node 6 to 10. The conservative way I have been doing for such upgrades is to add a job in the experimental pipeline, trigger the job for all repositories, assert they pass, then switch.

Then I guess we can just f****ing do it. They are probably just running grunt-jsonlint / banana checker.

oojs/ui might be a special case.

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

wikidata/build-resources might have some kind of magic build script which would be affected.

Beside those, the rest is probably just fine.

Most probably all of those repositories can just be upgraded from Node 6 to 10. The conservative way I have been doing for such upgrades is to add a job in the experimental pipeline, trigger the job for all repositories, assert they pass, then switch.
Then I guess we can just f****ing do it. They are probably just running grunt-jsonlint / banana checker.

Yeah, probably.

oojs/ui might be a special case.

It definitely is – it needs an image with npm, composer, and jsduck right now. :-(

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

Yeah – didn't want to break that workflow. @Jdrewniak, are you the expert on this?

wikidata/build-resources might have some kind of magic build script which would be affected.

I imagine we can just change it and make it @Addshore's problem if it breaks. :-)

Beside those, the rest is probably just fine.

OK, experimental patch incoming.

Change 516674 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/aqs] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516674

Change 516675 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/dashiki] Experiment with generic-node10-docker; no current CI

https://gerrit.wikimedia.org/r/516675

Change 516676 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics/{mediawiki-storage,wikistats2}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516676

For jsduck, if a repository is still using it, just break it. It has been obsolete for a few years now and the last commit is from 2013 or 6 years ago. It is time to break! If we want to play it nice, we can have E_YET_ANOTHER_TASK another task that announce Jsduck being dropped and just get rid of it from all repos: https://codesearch.wmflabs.org/search/?q=jsduck&i=nope&files=&repos=

:-]

Change 516674 merged by jenkins-bot:
[integration/config@master] layout: [analytics/aqs] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516674

Change 516675 merged by jenkins-bot:
[integration/config@master] layout: [analytics/dashiki] Experiment with generic-node10-docker; no current CI

https://gerrit.wikimedia.org/r/516675

Change 516676 merged by jenkins-bot:
[integration/config@master] layout: [analytics/{mediawiki-storage,wikistats2}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516676

Change 516682 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [data-values/value-view] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516682

Change 516683 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [design/landing-page] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516683

Change 516684 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [labs/tools/graphql] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516684

Change 516685 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [mapdata] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516685

Change 516686 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [operations/dumps/dcat] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516686

Change 516687 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [react.i18n] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516687

Change 516688 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikidata/build-resources] Archive, unused

https://gerrit.wikimedia.org/r/516688

Change 516689 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia-cz/tracker] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516689

Change 516690 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/fundraising/dash] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516690

Change 516691 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/portals{,/deploy}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516691

Change 516692 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [wikimedia/wikimania-scholarships] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516692

Change 516682 merged by jenkins-bot:
[integration/config@master] layout: [data-values/value-view] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516682

Change 516683 merged by jenkins-bot:
[integration/config@master] layout: [design/landing-page] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516683

Change 516684 merged by jenkins-bot:
[integration/config@master] layout: [labs/tools/graphql] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516684

Change 516685 merged by jenkins-bot:
[integration/config@master] layout: [mapdata] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516685

Change 516686 merged by jenkins-bot:
[integration/config@master] layout: [operations/dumps/dcat] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516686

Change 516687 merged by jenkins-bot:
[integration/config@master] layout: [react.i18n] Migrate to node10-docker, already passes

https://gerrit.wikimedia.org/r/516687

Change 516688 merged by jenkins-bot:
[integration/config@master] layout: [wikidata/build-resources] Archive, unused

https://gerrit.wikimedia.org/r/516688

Change 516689 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia-cz/tracker] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516689

Change 516690 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/fundraising/dash] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516690

Change 516691 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/portals{,/deploy}] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516691

Change 516692 merged by jenkins-bot:
[integration/config@master] layout: [wikimedia/wikimania-scholarships] Experiment with generic-node10-browser-docker

https://gerrit.wikimedia.org/r/516692

Change 516707 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [3d2png, mediawiki/services/{chromium-render,recommendation-api,trending-edits}] Add service-pipeline-test

https://gerrit.wikimedia.org/r/516707

Change 516707 merged by jenkins-bot:
[integration/config@master] layout: [3d2png, mediawiki/services/{chromium-render,recommendation-api,trending-edits}] Add service-pipeline-test

https://gerrit.wikimedia.org/r/516707

Change 516716 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: Disable service-pipeline-test from 3d2png, chromium-render, and trending-edits for now

https://gerrit.wikimedia.org/r/516716

Change 516716 merged by jenkins-bot:
[integration/config@master] layout: Disable service-pipeline-test from 3d2png, chromium-render, and trending-edits for now

https://gerrit.wikimedia.org/r/516716

OK, that's enough for today.

Jdforrester-WMF changed the task status from Open to Stalled.Jul 18 2019, 4:44 PM
Jdforrester-WMF updated the task description. (Show Details)

Blocked on action from other teams.

Change 524303 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Drop analytics-mediawiki-storage-npm-browser-node-6-docker, unused

https://gerrit.wikimedia.org/r/524303

Change 524303 merged by jenkins-bot:
[integration/config@master] jjb: Drop analytics-mediawiki-storage-npm-browser-node-6-docker, unused

https://gerrit.wikimedia.org/r/524303

FWIW:

  • Now that kartotherian is monorepo and is fully hosted in gerrit, some clean-up could be done:
    • maps/kartotherian and maps/tilerator repos have been useless for a while (at least more than 1 year) and can be archived
    • the package and deploy repos for both kartotherian and tilerator are still needed for deployment, but might be gone when these services move to k8s
  • Maps already upgraded to node 10 production wise

FWIW:

  • Now that kartotherian is monorepo and is fully hosted in gerrit, some clean-up could be done:
    • maps/kartotherian and maps/tilerator repos have been useless for a while (at least more than 1 year) and can be archived
    • the package and deploy repos for both kartotherian and tilerator are still needed for deployment, but might be gone when these services move to k8s
  • Maps already upgraded to node 10 production wise

Thanks, have updated accordingly.

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

Yeah – didn't want to break that workflow. @Jdrewniak, are you the expert on this?

I just tested the build-step with Node 10 locally and it works just fine, so this repo can be updated.

wikimedia/portals uses some build step to generate assets which are then deployed as a submodule of operations/mediawiki-config.git.

Yeah – didn't want to break that workflow. @Jdrewniak, are you the expert on this?

I just tested the build-step with Node 10 locally and it works just fine, so this repo can be updated.

Thanks for the confirmation, Jan – we've moved it.

awight removed a subscriber: awight.Jul 19 2019, 7:14 AM
MSantos removed a subscriber: MSantos.Jul 25 2019, 3:10 PM

Change 525581 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] layout: [analytics-wikistats2] Drop node 6, switch to node10-browser-docker

https://gerrit.wikimedia.org/r/525581

Change 525582 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[integration/config@master] jjb: Drop analytics-wikistats2-npm-browser-node-6-docker, unused

https://gerrit.wikimedia.org/r/525582

Change 525581 merged by jenkins-bot:
[integration/config@master] layout: [analytics-wikistats2] Drop node 6, switch to node10-browser-docker

https://gerrit.wikimedia.org/r/525581

Mentioned in SAL (#wikimedia-releng) [2019-07-25T16:41:08Z] <James_F> Zuul: [analytics-wikistats2] Drop node 6, switch to node10-browser-docker T211784

Change 525582 merged by jenkins-bot:
[integration/config@master] jjb: Drop analytics-wikistats2-npm-browser-node-6-docker, unused

https://gerrit.wikimedia.org/r/525582