Page MenuHomePhabricator

Migrate node-based services in production to node18
Open, Needs TriagePublic

Description

T308371: Migrate node-based services in production to node16 | {No Node20 Task Yet}

Note that you may wish to complete the migration to node16 first, rather than make the migration in one go.

Imperfect search:

  • Abstract Wikipedia team
    • services/function-evaluator
    • services/function-orchestrator
  • Content Transformation
    • mediawiki/services/chromium-render
    • mediawiki/services/geoshapes [never deployed - stalled work]
    • mediawiki/services/kartotherian [not in k8s yet]
    • mediawiki/services/mobileapps
    • mediawiki/services/push-notifications
    • mediawiki/services/wikifeeds T358017: Migrate wikifeeds from Node16 to Node18
  • Data Engineering
    • mediawiki/services/eventstreams
    • wikimedia/eventgate
    • mediawiki/services/similar-users [only the 'htmldocs' build]
    • node-rdkafka-statsd [not a deploy target?]
  • Editing
    • mediawiki/services/citoid
    • mediawiki/services/zotero
  • Language Engineering
    • mediawiki/services/cxserver
  • MediaWiki Engineering
    • mediawiki/services/example-node-api [being decommissioned?]
    • mediawiki/services/image-suggestion-api [being decommissioned?]
    • mediawiki/services/restbase [being decommissioned]
    • mediawiki/services/service-scaffold-node [being decommissioned?]
  • Wikidata
    • wikibase/termbox
  • ???
    • mediawiki/services/mathoid
    • mediawiki/services/change-propagation
    • mediawiki/services/recommendation-api

Details

SubjectRepoBranchLines +/-
mediawiki/services/zoteromaster+5 -5
operations/deployment-chartsmaster+1 -1
wikimedia/portalsmaster+1 K -331
operations/deployment-chartsmaster+2 -2
mediawiki/services/mobileappsmaster+5 K -5 K
operations/deployment-chartsmaster+1 -1
mediawiki/services/wikifeedsmaster+347 -5 K
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
wikibase/termboxmaster+7 -5
operations/deployment-chartsmaster+2 -7
operations/deployment-chartsmaster+6 -0
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+1 -1
mediawiki/services/recommendation-apimaster+13 -10
operations/deployment-chartsmaster+1 -1
operations/deployment-chartsmaster+9 -9
operations/deployment-chartsmaster+1 -1
mediawiki/services/cxservermaster+1 K -1 K
Show related patches Customize query in gerrit
ReferenceSource BranchDest BranchAuthorTitle
repos/abstract-wiki/wikifunctions/function-orchestrator!75T349118mainjforresterDraft: build: Migrate image from node16 to node18
Customize query in GitLab

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@Jdforrester-WMF here's a few questions that I have:

Recommendation API ownership
The former Product-Infrastructure-Team-Backlog-Deprecated did some patches on`mediawiki/services/recommendation-api` but didn't own it, therefore CTT also doesn't. It used to be maintained by Research AFAIK (please correct me if I'm wrong). Not sure how to proceed here.

Planned for deprecation
A few repos are expected to be removed/deprecated by API Platform, like:

  • mediawiki/services/example-node-api
  • mediawiki/services/image-suggestion-api
  • mediawiki/services/service-scaffold-node

Restbase
restbase is still on bare-metal and was recently bumped to bullseye, it seems that it's on version 12 (I don't have access to restbase machines so I checked our default bullseye node version):

mbsantos@mwlog1002:~$ apt list -a nodejs
Listing... Done
nodejs/oldstable,oldstable-security 12.22.12~dfsg-1~deb11u4 amd64

mbsantos@mwlog1002:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

A question would be, should it be bumped to a different OS version in order to get node18 available?

Data Engineering ownerhship

eventgate is missing from your list, but has WIP to target node 18 https://phabricator.wikimedia.org/T347477 (cc / @Ottomata @elukey).

Ownership of changeprop is currently unclear, same for similar users afaik. They used to fall under the Platform Engineering (at some point), we are trying to figure out what that would map to nowadays (cc / @Ahoelzl @WDoranWMF ).

Also, mediawiki/services/similar-users is a Python service.

@Jdforrester-WMF here's a few questions that I have:

Recommendation API ownership
The former Product-Infrastructure-Team-Backlog-Deprecated did some patches on`mediawiki/services/recommendation-api` but didn't own it, therefore CTT also doesn't. It used to be maintained by Research AFAIK (please correct me if I'm wrong). Not sure how to proceed here.

Ack. Have moved to the ??? section for now.

Planned for deprecation
A few repos are expected to be removed/deprecated by API Platform, like:

  • mediawiki/services/example-node-api
  • mediawiki/services/image-suggestion-api
  • mediawiki/services/service-scaffold-node

Understood; marked as such. This isn't a centrally-driven piece of work, and it's up for the responsible teams to decide their risk/change appetite; if the services get undeployed in the next while, we can just strike them off.

Restbase
restbase is still on bare-metal and was recently bumped to bullseye, it seems that it's on version 12 (I don't have access to restbase machines so I checked our default bullseye node version):

mbsantos@mwlog1002:~$ apt list -a nodejs
Listing... Done
nodejs/oldstable,oldstable-security 12.22.12~dfsg-1~deb11u4 amd64

mbsantos@mwlog1002:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye

A question would be, should it be bumped to a different OS version in order to get node18 available?

I don't think it feels reasonable for the team to do lots of work there if it's being pushed out of production soon.

Data Engineering ownerhship

eventgate is missing from your list, but has WIP to target node 18 https://phabricator.wikimedia.org/T347477 (cc / @Ottomata @elukey).

Ownership of changeprop is currently unclear, same for similar users afaik. They used to fall under the Platform Engineering (at some point), we are trying to figure out what that would map to nowadays (cc / @Ahoelzl @WDoranWMF ).

eventstreams is listed (and listed as resolved); have marked T347477 as fixing this task (and its three predecessors!).

I didn't realise that eventgate was run out of GitHub, which is why it doesn't come up in code search. Added.

Also, mediawiki/services/similar-users is a Python service.

Yes, but the htmldocs build uses node, and specifically node 10(!): https://gerrit.wikimedia.org/g/mediawiki/services/similar-users/+/19acabe7138d9f4baf73d0b2a8e6ff132595b460/.pipeline/blubber.yaml#37

Data Engineering ownerhship

I didn't realise that eventgate was run out of GitHub, which is why it doesn't come up in code search. Added.

Ack. There's two parts to that service: the eventgate "generic" code that lives in github, and the service implementation atop said code at https://gerrit.wikimedia.org/g/eventgate-wikimedia.

Both are being updated.

Change 854500 merged by Santhosh:

[mediawiki/services/cxserver@master] Migrate to nodejs18

https://gerrit.wikimedia.org/r/854500

Change 972323 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2023-11-07-081511-production

https://gerrit.wikimedia.org/r/972323

Change 972323 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2023-11-07-081511-production

https://gerrit.wikimedia.org/r/972323

Change 975740 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] cxserver: Force 127.0.0.1 instead of localhost

https://gerrit.wikimedia.org/r/975740

Change 976369 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2023-11-20-052250-production

https://gerrit.wikimedia.org/r/976369

Change 975740 merged by jenkins-bot:

[operations/deployment-charts@master] cxserver: Force 127.0.0.1 instead of localhost

https://gerrit.wikimedia.org/r/975740

Change 976369 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2023-11-20-052250-production

https://gerrit.wikimedia.org/r/976369

Change 977751 had a related patch set uploaded (by Elukey; author: Elukey):

[mediawiki/services/recommendation-api@master] Upgrade to nodejs 18 and Bookworm

https://gerrit.wikimedia.org/r/977751

I filed https://gerrit.wikimedia.org/r/c/mediawiki/services/recommendation-api/+/977751 for the recommendation-api service, if anybody has time to review it I will take care of the build/deploy :)

Change 977751 merged by jenkins-bot:

[mediawiki/services/recommendation-api@master] Upgrade to nodejs 18 and Bookworm

https://gerrit.wikimedia.org/r/977751

Change 980391 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: upgrade recommendation-api's Docker image

https://gerrit.wikimedia.org/r/980391

Change 980391 merged by Elukey:

[operations/deployment-charts@master] services: upgrade recommendation-api's Docker image

https://gerrit.wikimedia.org/r/980391

Change 980407 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: use 127.0.0.1 instead of localhost for rec-api's mw host

https://gerrit.wikimedia.org/r/980407

Change 980407 merged by Elukey:

[operations/deployment-charts@master] services: use 127.0.0.1 instead of localhost for rec-api's mw host

https://gerrit.wikimedia.org/r/980407

Change 981956 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] service: update recommendation-api's docker image

https://gerrit.wikimedia.org/r/981956

Change 981956 merged by Elukey:

[operations/deployment-charts@master] service: update recommendation-api's docker image

https://gerrit.wikimedia.org/r/981956

Change 983404 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: deploy the new rec-api-ng Docker image in staging

https://gerrit.wikimedia.org/r/983404

Change 983404 merged by Elukey:

[operations/deployment-charts@master] services: deploy the new rec-api-ng Docker image in staging

https://gerrit.wikimedia.org/r/983404

Change 983694 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] services: update Docker image and settings for Recommendation API

https://gerrit.wikimedia.org/r/983694

Change 983694 merged by Elukey:

[operations/deployment-charts@master] services: update Docker image and settings for Recommendation API

https://gerrit.wikimedia.org/r/983694

Change 989895 had a related patch set uploaded (by Jforrester; author: Jforrester):

[wikibase/termbox@master] [WIP] Migrate from node16 to node18

https://gerrit.wikimedia.org/r/989895

Change 989895 merged by jenkins-bot:

[wikibase/termbox@master] Migrate from node16 to node18

https://gerrit.wikimedia.org/r/989895

Change 992387 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] termbox(test): update to 2024-01-22-163619-production

https://gerrit.wikimedia.org/r/992387

Change 992387 merged by jenkins-bot:

[operations/deployment-charts@master] termbox(test): update to 2024-01-22-163619-production

https://gerrit.wikimedia.org/r/992387

Change 992446 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] termbox: update to 2024-01-22-163619-production

https://gerrit.wikimedia.org/r/992446

Change 992446 merged by jenkins-bot:

[operations/deployment-charts@master] termbox: update to 2024-01-22-163619-production

https://gerrit.wikimedia.org/r/992446

Change 992452 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] Revert "termbox: update to 2024-01-22-163619-production"

https://gerrit.wikimedia.org/r/992452

Change 992452 merged by jenkins-bot:

[operations/deployment-charts@master] Revert "termbox: update to 2024-01-22-163619-production"

https://gerrit.wikimedia.org/r/992452

Change 999024 had a related patch set uploaded (by Sbailey; author: Sbailey):

[mediawiki/services/wikifeeds@master] wikifeeds: upgrade to node18 from node16

https://gerrit.wikimedia.org/r/999024

Change 1000296 had a related patch set uploaded (by Func; author: Func):

[wikimedia/portals@master] build: Update eslint-config-wikimedia to 0.26.0 for node18

https://gerrit.wikimedia.org/r/1000296

Change 1002592 had a related patch set uploaded (by Sbailey; author: Sbailey):

[mediawiki/services/wikifeeds@master] wikifeeds: upgrade to node18 from node16

https://gerrit.wikimedia.org/r/1002592

Change 999024 abandoned by Sbailey:

[mediawiki/services/wikifeeds@master] wikifeeds: upgrade to node18 from node16

Reason:

replaced due to git problem

https://gerrit.wikimedia.org/r/999024

Change 1003400 had a related patch set uploaded (by Lucas Werkmeister (WMDE); author: Lucas Werkmeister (WMDE)):

[operations/deployment-charts@master] Reapply "termbox: update to 2024-01-22-163619-production"

https://gerrit.wikimedia.org/r/1003400

Change 1003400 merged by jenkins-bot:

[operations/deployment-charts@master] Reapply "termbox: update to 2024-01-22-163619-production"

https://gerrit.wikimedia.org/r/1003400

Change 1003880 had a related patch set uploaded (by Sbailey; author: Sbailey):

[mediawiki/services/mobileapps@master] WIP mobileapps: upgrade to node18 from node12

https://gerrit.wikimedia.org/r/1003880

Change 1007353 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-02-12-160222 to 2024-02-26-150300

https://gerrit.wikimedia.org/r/1007353

Change 1007353 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-02-12-160222 to 2024-02-26-150300

https://gerrit.wikimedia.org/r/1007353

Change 1000296 merged by jenkins-bot:

[wikimedia/portals@master] build: Update eslint-config-wikimedia to 0.26.0 for node18

https://gerrit.wikimedia.org/r/1000296

Change 1011269 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/zotero@master] Update Zotero to node 18

https://gerrit.wikimedia.org/r/1011269

Change #1011269 merged by jenkins-bot:

[mediawiki/services/zotero@master] Update Zotero to node 18

https://gerrit.wikimedia.org/r/1011269

Change #1016728 had a related patch set uploaded (by Mvolz; author: Mvolz):

[operations/deployment-charts@master] Update zotero to node18

https://gerrit.wikimedia.org/r/1016728

Change #1016728 merged by jenkins-bot:

[operations/deployment-charts@master] Update zotero to node18

https://gerrit.wikimedia.org/r/1016728