Page MenuHomePhabricator
Create Task
Maniphest T284345

Upgrade all CI jobs for WMF-deployed projects from Node 10 to Node 12
Closed, ResolvedPublic
Actions

Description

T211784: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects | T267890: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Jdforrester-WMF created this task.Jun 4 2021, 6:55 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 4 2021, 6:55 PM
Jdforrester-WMF mentioned this in T267890: Upgrade all CI jobs for WMF-deployed projects from Node 12 to Node 14 LTS.Jun 4 2021, 6:56 PM
Jdforrester-WMF added a subtask: T284343: Create WMF CI image for Node.js 12.
Jdforrester-WMF closed subtask T284343: Create WMF CI image for Node.js 12 as Resolved.Jun 4 2021, 7:35 PM
gerritbot added a comment.Jun 4 2021, 7:50 PM

Change 698244 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Provide node12 versions of existing node10 jobs

https://gerrit.wikimedia.org/r/698244

gerritbot added a project: Patch-For-Review.Jun 4 2021, 7:50 PM

Change 698245 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: [VisualEditor/VisualEditor] Switch node10 jobs to node12

https://gerrit.wikimedia.org/r/698245

gerritbot added a comment.Jun 4 2021, 7:51 PM

Change 698246 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: Migrate all node10 jobs to node12 in bulk

https://gerrit.wikimedia.org/r/698246

gerritbot added a comment.Jun 4 2021, 7:51 PM

Change 698247 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Drop unused node10 jobs

https://gerrit.wikimedia.org/r/698247

gerritbot added a comment.Jun 4 2021, 7:51 PM

Change 698248 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] jjb: Switch special jobs from node10 to node12 images

https://gerrit.wikimedia.org/r/698248

gerritbot added a comment.Jun 4 2021, 7:57 PM

Change 698244 merged by jenkins-bot:

[integration/config@master] jjb: Provide node12 versions of existing node10 jobs

https://gerrit.wikimedia.org/r/698244

gerritbot added a comment.Jun 4 2021, 7:57 PM

Change 698245 merged by jenkins-bot:

[integration/config@master] Zuul: [VisualEditor/VisualEditor] Switch node10 jobs to node12

https://gerrit.wikimedia.org/r/698245

Stashbot added a comment.Jun 4 2021, 8:11 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-04T20:11:44Z] <James_F> Zuul: [VisualEditor/VisualEditor] Switch node10 jobs to node12 T284345

gerritbot added a comment.Jun 4 2021, 8:12 PM

Change 698246 merged by jenkins-bot:

[integration/config@master] Zuul: Migrate all node10 jobs to node12 in bulk

https://gerrit.wikimedia.org/r/698246

Stashbot added a comment.Jun 4 2021, 8:13 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-04T20:13:55Z] <James_F> Zuul: Switch almost all node10 jobs to node12 T284345

gerritbot added a comment.Jun 4 2021, 8:19 PM

Change 698252 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/MobileFrontend@master] .nvmrc: Update to node 12.21.0 as CI has upgraded

https://gerrit.wikimedia.org/r/698252

gerritbot added a comment.Jun 4 2021, 9:03 PM

Change 698247 merged by jenkins-bot:

[integration/config@master] jjb: Drop unused node10 jobs

https://gerrit.wikimedia.org/r/698247

gerritbot added a comment.Jun 4 2021, 9:18 PM

Change 698255 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/Popups@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698255

gerritbot added a comment.Jun 4 2021, 9:20 PM

Change 698248 merged by jenkins-bot:

[integration/config@master] jjb: Switch special jobs from node10 to node12 images

https://gerrit.wikimedia.org/r/698248

gerritbot added a comment.Jun 4 2021, 9:36 PM

Change 698257 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [WIP] dockerfiles: Provide node12-*php*-composer images

https://gerrit.wikimedia.org/r/698257

gerritbot added a comment.Jun 4 2021, 9:36 PM

Change 698258 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] [WIP] dockerfiles: Provide a node12-based Kartotherian image

https://gerrit.wikimedia.org/r/698258

Jdforrester-WMF added a comment.Jun 4 2021, 9:37 PM

OK, this is mostly done except for two sets of jobs:

  • Jobs using the special Kartotherian image.
  • Jobs using the special PHP + Node images.

Both of those need some further work.

gerritbot added a comment.Jun 4 2021, 9:42 PM

Change 698259 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/skins/MinervaNeue@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698259

gerritbot added a comment.Jun 4 2021, 9:45 PM

Change 698260 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/skins/Vector@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698260

gerritbot added a comment.Jun 4 2021, 9:49 PM

Change 698261 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/Wikibase@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698261

gerritbot added a comment.Jun 4 2021, 9:49 PM

Change 698262 had a related patch set uploaded (by Jforrester; author: Jforrester):

[wikibase/vuejs-components@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698262

Jdforrester-WMF added a comment.Jun 4 2021, 9:59 PM

Announced on wikitech-l in case it breaks anything: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/ZOMZ376CCVD7HNTL2XYA5ASCABUS4DG5/

gerritbot added a comment.Jun 5 2021, 2:17 AM

Change 698252 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] .nvmrc: Update to node 12.21.0 as CI has upgraded

https://gerrit.wikimedia.org/r/698252

gerritbot added a comment.Jun 5 2021, 2:20 AM

Change 698260 merged by jenkins-bot:

[mediawiki/skins/Vector@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698260

gerritbot added a comment.Jun 5 2021, 2:20 AM

Change 698259 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698259

gerritbot added a comment.Jun 5 2021, 2:20 AM

Change 698255 merged by jenkins-bot:

[mediawiki/extensions/Popups@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698255

ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.9; 2021-06-07).Jun 5 2021, 3:00 AM
hashar subscribed.Jun 5 2021, 10:05 AM
In T284345#7135717, @Jdforrester-WMF wrote:

Announced on wikitech-l in case it breaks anything: https://lists.wikimedia.org/hyperkitty/list/wikitech-l@lists.wikimedia.org/thread/ZOMZ376CCVD7HNTL2XYA5ASCABUS4DG5/

Great! :)

• Mholloway awarded a token.Jun 5 2021, 6:47 PM
hashar moved this task from INBOX to Doing on the Release-Engineering-Team board.Jun 7 2021, 8:46 AM
hashar edited projects, added Release-Engineering-Team (Doing); removed Release-Engineering-Team.
WMDE-Fisch subscribed.Jun 7 2021, 9:44 AM

While playing with this locally I realized that, npm v7 uses a new lockfile version and whenever I run npm to install the packages it also updates the v1 lockfile to v2 adding a lot of diff overhead. - So I wonder if it might be useful ( when ever we are sure to stick with this version ) to run Libraryupgrader and mass adjust the repos.

zeljkofilipin awarded a token.Jun 7 2021, 10:30 AM
zeljkofilipin subscribed.
zeljkofilipin added a subscriber: Sahilgrewalhere.Jun 7 2021, 6:23 PM

Selenium jobs are not updated to node v12?

Example patch: 678287.
Example job: 55939.

...
INFO:quibble.commands:node --version: v10.24.0
INFO:quibble.commands:npm --version: 6.14.5
...
hashar added a comment.Jun 7 2021, 6:40 PM
In T284345#7139795, @zeljkofilipin wrote:

Selenium jobs are not updated to node v12?
...
INFO:quibble.commands:node --version: v10.24.0

Obviously they haven't been migrated yet.

This task has been opened late last Friday and is open. You can check the progress by looking at changes that have been attached to the task (press the Show related patches button below the task description). Or check https://gerrit.wikimedia.org/r/q/bug:T284345 :]

Jdforrester-WMF added a comment.Jun 7 2021, 6:55 PM
In T284345#7139795, @zeljkofilipin wrote:

Selenium jobs are not updated to node v12?

Example patch: 678287.
Example job: 55939.

...
INFO:quibble.commands:node --version: v10.24.0
INFO:quibble.commands:npm --version: 6.14.5
...

Yeah, this task and announcement is about the JS jobs, not the quibble jobs.

Migrating the latter will be difficult. Quibble jobs currently run on buster, to be as close as possible to a production environment. SRE have vetoed backporting node 12+ to buster properly, so we'd have to either have to over-ride in CI and load a different version of node from what SRE provide, or switch the quibble selenium jobs over to bullseye. I slightly prefer the latter, but it'll be a chore.

gerritbot added a comment.Jun 7 2021, 7:57 PM

Change 698258 abandoned by Jforrester:

[integration/config@master] [WIP] dockerfiles: Provide a node12-based Kartotherian image

Reason:

Dropping instead, in I621dc416be773285c315a143f830ce8d6fb2a85c.

https://gerrit.wikimedia.org/r/698258

matmarex mentioned this in T284492: CI failures in "git-status" task related to submodules.Jun 7 2021, 7:59 PM
zeljkofilipin added a project: User-zeljkofilipin.Jun 8 2021, 9:17 AM
zeljkofilipin moved this task from Backlog 🪒 to Watching 📺 on the User-zeljkofilipin board.
zeljkofilipin added a comment.Jun 8 2021, 9:19 AM
In T284345#7139881, @Jdforrester-WMF wrote:

Yeah, this task and announcement is about the JS jobs, not the quibble jobs.

Thanks! I misunderstood then. I thought most of the jobs are moved to node v12.

Mvolz subscribed.Jun 8 2021, 9:30 AM
In T284345#7137909, @WMDE-Fisch wrote:

While playing with this locally I realized that, npm v7 uses a new lockfile version and whenever I run npm to install the packages it also updates the v1 lockfile to v2 adding a lot of diff overhead. - So I wonder if it might be useful ( when ever we are sure to stick with this version ) to run Libraryupgrader and mass adjust the repos.

We've had some issues with npm 7, namely that sometimes it randomly replaces an https link to a repo with SSH. This breaks libraryupgrader, so for the time being we're asking people to downgrade to 6, as the bug is still open: https://github.com/npm/cli/issues/2610

(Example where my npm 7 package-lock file broke things!) https://gerrit.wikimedia.org/r/c/mediawiki/services/citoid/+/674644/2#message-167c788635c5b18ac245a155c3bbba5e3a5a5a44

Ottomata subscribed.Jun 11 2021, 3:01 PM

Hi, I think this broke CI for packages that use node-rdkafka, e.g. node-rdkakfa-factory, eventgate, etc.

Upgrading node-rdkafka may be error prone and take a lot of manual work to ensure that things don't break (we have to upgrade librdkafka too). Is there a reason we have to upgrade to node12 in CI, when the production services aren't using node 12?

https://integration.wikimedia.org/ci/job/generic-node12-docker/39/console

The eventgate, etc. CI still works, because they use service-pipeline, which has control over the docker images and versions that are used for both CI and prod. Should I switch these node-rdkafka-* library packages to using service-pipeline for CI too?

Ottomata added a subscriber: • razzi.Jun 11 2021, 3:02 PM

cc @razzi ^

Ottomata mentioned this in T283671: Replace Content::getNativeData() calls with TextContent::getText() in EventLogging.Jun 11 2021, 3:04 PM
Jdforrester-WMF added a comment.Jun 11 2021, 3:08 PM
In T284345#7151622, @Ottomata wrote:

Hi, I think this broke CI for packages that use node-rdkafka, e.g. node-rdkakfa-factory, eventgate, etc.

Sorry about that. :-(

Upgrading node-rdkafka may be error prone and take a lot of manual work to ensure that things don't break (we have to upgrade librdkafka too). Is there a reason we have to upgrade to node12 in CI, when the production services aren't using node 12?

https://integration.wikimedia.org/ci/job/generic-node12-docker/39/console

Node10 left security support at the end of April.

The eventgate, etc. CI still works, because they use service-pipeline, which has control over the docker images and versions that are used for both CI and prod. Should I switch these node-rdkafka-* library packages to using service-pipeline for CI too?

Yes, as those are intended for running in production then using the pipeline would be a better fit. The legacy CI jobs are meant for development testing code, not production, really.

Happy to help migrate if needed.

gerritbot added a comment.Jun 11 2021, 3:12 PM

Change 699431 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Restore generic-node10-docker

https://gerrit.wikimedia.org/r/699431

gerritbot added a comment.Jun 11 2021, 3:14 PM

Change 699431 merged by jenkins-bot:

[integration/config@master] Restore generic-node10-docker

https://gerrit.wikimedia.org/r/699431

gerritbot added a comment.Jun 11 2021, 3:18 PM

Change 699435 had a related patch set uploaded (by Ottomata; author: Ottomata):

[integration/config@master] Use service-pipeline-test for node-rdkafka-factory

https://gerrit.wikimedia.org/r/699435

gerritbot added a comment.Jun 11 2021, 3:24 PM

Change 699435 merged by jenkins-bot:

[integration/config@master] Zuul: [node-rdkafka-factory] Switch to service-pipeline-test

https://gerrit.wikimedia.org/r/699435

Stashbot added a comment.Jun 11 2021, 3:25 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-11T15:25:30Z] <James_F> Zuul: [node-rdkafka-factory] Switch to service-pipeline-test T284345

gerritbot added a comment.Jun 11 2021, 3:33 PM

Change 699437 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: [node-rdkafka-statsd] Switch to service-pipeline-test

https://gerrit.wikimedia.org/r/699437

gerritbot added a comment.Jun 11 2021, 3:44 PM

Change 699443 had a related patch set uploaded (by Ottomata; author: Ottomata):

[node-rdkafka-statsd@master] Add .pipeline for CI tests

https://gerrit.wikimedia.org/r/699443

gerritbot added a comment.Jun 11 2021, 3:48 PM

Change 699437 merged by jenkins-bot:

[integration/config@master] Zuul: [node-rdkafka-statsd] Switch to service-pipeline-test

https://gerrit.wikimedia.org/r/699437

Stashbot added a comment.Jun 11 2021, 3:50 PM

Mentioned in SAL (#wikimedia-releng) [2021-06-11T15:50:31Z] <James_F> Zuul: [node-rdkafka-statsd] Switch to service-pipeline-test T284345

gerritbot added a comment.Jun 11 2021, 5:13 PM

Change 699443 merged by Ottomata:

[node-rdkafka-statsd@master] Add .pipeline for CI tests

https://gerrit.wikimedia.org/r/699443

hashar mentioned this in T285239: wikimedia/portals/deploy webdriver.io has to be updated in order to support NodeJS 12.Jun 21 2021, 3:40 PM
hashar added a subtask: T285239: wikimedia/portals/deploy webdriver.io has to be updated in order to support NodeJS 12.
gerritbot added a comment.Jun 21 2021, 3:46 PM

Change 700654 had a related patch set uploaded (by Hashar; author: Hashar):

[integration/config@master] Restore generic-node10-browser-docker

https://gerrit.wikimedia.org/r/700654

gerritbot added a comment.Jun 21 2021, 3:51 PM

Change 700654 merged by jenkins-bot:

[integration/config@master] Restore generic-node10-browser-docker

https://gerrit.wikimedia.org/r/700654

hashar closed subtask T285239: wikimedia/portals/deploy webdriver.io has to be updated in order to support NodeJS 12 as Resolved.Jun 29 2021, 1:45 PM
Michael subscribed.Jul 15 2021, 10:12 AM
Ladsgroup awarded a token.Jul 15 2021, 10:17 AM
Ladsgroup subscribed.
Catrope added a parent task: T284253: Upgrade WVUI node.js version.Jul 28 2021, 11:19 PM
Jdforrester-WMF closed this task as Resolved.Jul 29 2021, 8:54 AM
Jdforrester-WMF claimed this task.
Jdforrester-WMF removed a parent task: T284253: Upgrade WVUI node.js version.

This was completed in June (except for repos using the pipeline, which rely on SRE's images; see T284346).

gerritbot added a comment.Sep 9 2021, 7:40 PM

Change 698257 abandoned by Jforrester:

[integration/config@master] [WIP] dockerfiles: Provide node12-*php*-composer images

Reason:

Done in 242dc1456a5f08a10d149a7e21cb5179f3208930.

https://gerrit.wikimedia.org/r/698257

Michael mentioned this in T295036: Migrate all repos to lockfileVersion 2 to avoid "The package-lock.json file was created with an old version of npm".Nov 4 2021, 2:55 PM
gerritbot added a comment.Feb 25 2022, 10:56 AM

Change 698261 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Update to Node 12.21.0 as CI has migrated

https://gerrit.wikimedia.org/r/698261

ReleaseTaggerBot added a project: MW-1.38-notes (1.38.0-wmf.24; 2022-02-28).Feb 25 2022, 11:00 AM
• dduvall mentioned this in T307507: Fully deprecate service-pipeline-test and service-pipeline-test-and-publish jobs.May 3 2022, 8:53 PM
gerritbot added a comment.Aug 3 2022, 1:59 PM

Change 698262 abandoned by Jforrester:

[wikibase/vuejs-components@master] .nvmrc: Update to Node 12.21.0 as CI has migrated

Reason:

https://gerrit.wikimedia.org/r/698262

zeljkofilipin unsubscribed.Aug 23 2022, 10:49 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL