Page MenuHomePhabricator
Create Task
Maniphest T361602

Remove usage of preq in all services and replace with node fetch; requires node >=18.x.x
Open, Needs TriagePublic
Actions

Description

  • mediawiki/services/cxserver : T350773
  • mediawiki/services/citoid done, but need to remove undici wmf fork once node 26 is released
  • html-metadata: released as 3.0.0
  • service-runner: T309772, done but should replace with service-utils instead.
  • mediawiki/services/example-node-api removal in progress
  • mediawiki/services/geoshapes
  • mediawiki/services/recommendation-api sunsetting in progress
  • mediawiki/services/chromium-render
  • mediawiki/services/kartotherian
  • mediawiki/services/mobileapps
  • mediawiki/services/push-notifications
  • mediawiki/services/servicelib-node sunsetting in progress
  • mediawiki/services/wikifeeds
  • mediawiki/services/change-propagation
  • mediawiki/services/restbase - sunsetting in progress

https://libraryupgrader2.wmcloud.org/library/npm/preq?branch=main

Details

Related Changes in Gerrit:
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

Mvolz created this task.Apr 2 2024, 2:52 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 2 2024, 2:52 PM
Mvolz added subtasks: T293853: Service-runner depends on preq, a wrapper of request, which is deprecated, T350773: Remove preq and use node fetch.Apr 2 2024, 2:55 PM
Mvolz moved this task from Needs removal to Migrate / Replace on the Technical-Debt board.Dec 12 2024, 1:30 PM
Mvolz added a subtask: T373280: Preq parses websites in html which claim to be in json as json, which causes a syntax error in citoid.
gerritbot added a comment.Feb 25 2025, 1:26 PM

Change #1122574 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq from spec tests

https://gerrit.wikimedia.org/r/1122574

gerritbot added a project: Patch-For-Review.Feb 25 2025, 1:26 PM
Mvolz updated the task description. (Show Details)Feb 25 2025, 1:29 PM
gerritbot added a comment.Feb 25 2025, 9:09 PM

Change #1122574 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq from spec tests

https://gerrit.wikimedia.org/r/1122574

Maintenance_bot removed a project: Patch-For-Review.Feb 25 2025, 9:30 PM
gerritbot added a comment.Feb 26 2025, 1:07 PM

Change #1122934 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq from app tests

https://gerrit.wikimedia.org/r/1122934

gerritbot added a project: Patch-For-Review.Feb 26 2025, 1:07 PM
gerritbot added a comment.Feb 26 2025, 1:15 PM

Change #1122937 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] [WIP] Remove preq from app tests

https://gerrit.wikimedia.org/r/1122937

gerritbot added a comment.Feb 26 2025, 1:17 PM

Change #1122934 abandoned by Mvolz:

[mediawiki/services/citoid@master] Remove preq from app tests

Reason:

duplicate

https://gerrit.wikimedia.org/r/1122934

gerritbot added a comment.Feb 26 2025, 3:33 PM

Change #1122937 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq from app tests

https://gerrit.wikimedia.org/r/1122937

gerritbot added a comment.Feb 26 2025, 3:45 PM

Change #1122978 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq from error tests

https://gerrit.wikimedia.org/r/1122978

gerritbot added a comment.Feb 26 2025, 5:06 PM

Change #1122978 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq from error tests

https://gerrit.wikimedia.org/r/1122978

Maintenance_bot removed a project: Patch-For-Review.Feb 26 2025, 5:30 PM
gerritbot added a comment.Mar 6 2025, 11:55 AM

Change #1125127 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] build: Update dependencies

https://gerrit.wikimedia.org/r/1125127

gerritbot added a project: Patch-For-Review.Mar 6 2025, 11:55 AM
Mvolz updated the task description. (Show Details)Mar 6 2025, 12:42 PM
Mvolz added projects: Citoid, service-runner.
gerritbot added a comment.Mar 7 2025, 1:58 PM

Change #1125127 merged by jenkins-bot:

[mediawiki/services/citoid@master] build: Update dependencies

https://gerrit.wikimedia.org/r/1125127

Jdforrester-WMF subscribed.Mar 7 2025, 2:11 PM

Is this just for Editing services, or do we want to do the others? https://libraryupgrader2.wmcloud.org/library/npm/preq?branch=main

Maintenance_bot removed a project: Patch-For-Review.Mar 7 2025, 2:30 PM
Mvolz updated the task description. (Show Details)Mar 8 2025, 2:35 PM
Mvolz updated the task description. (Show Details)Mar 8 2025, 2:37 PM
Mvolz updated the task description. (Show Details)
Mvolz added a comment.Mar 8 2025, 2:39 PM
In T361602#10613261, @Jdforrester-WMF wrote:

Is this just for Editing services, or do we want to do the others? https://libraryupgrader2.wmcloud.org/library/npm/preq?branch=main

It should be done everywhere, as its a wrapper of the request library, which is itself deprecated, see: https://www.npmjs.com/package/request

Jdforrester-WMF updated the task description. (Show Details)Mar 10 2025, 1:31 PM
Mvolz added a subscriber: tchin.Mar 10 2025, 2:38 PM

Is mediawiki/services/servicelib-node being used anywhere? It looks like an abandoned attempt to convert service-template-node into a library. @tchin maybe knows if we should we empty this repo / decomm it (as with example-node-api)?

Mvolz edited projects, added: Maps (Kartotherian), Push-Notification-Service, Wikifeeds, ChangeProp, RESTBase, Mobile; removed: service-runner.Mar 10 2025, 2:40 PM
Jdforrester-WMF added a comment.EditedMar 10 2025, 3:10 PM
In T361602#10619075, @Mvolz wrote:

Is mediawiki/services/servicelib-node being used anywhere? It looks like an abandoned attempt to convert service-template-node into a library. @tchin maybe knows if we should we empty this repo / decomm it (as with example-node-api)?

It was being created as part of T289207; I filed T365512 for service-scaffold-node but missed servicelib-node, which @tchin noted in T365512#9880928. Will file a task. [Edit] T388424

Jdforrester-WMF updated the task description. (Show Details)Mar 11 2025, 3:27 PM
Mvolz moved this task from Backlog to Next & Doing on the Citoid board.May 15 2025, 10:43 AM
Mvolz added a project: Essential-Work.Jun 5 2025, 11:42 AM
gerritbot added a comment.Jun 5 2025, 11:43 AM

Change #1153997 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Replace preq with fetch in test server

https://gerrit.wikimedia.org/r/1153997

gerritbot added a project: Patch-For-Review.Jun 5 2025, 11:43 AM
gerritbot added a comment.Jun 8 2025, 7:01 PM

Change #1153997 merged by jenkins-bot:

[mediawiki/services/citoid@master] Replace preq with fetch in test server

https://gerrit.wikimedia.org/r/1153997

Maintenance_bot removed a project: Patch-For-Review.Jun 8 2025, 7:31 PM
gerritbot added a comment.Jun 12 2025, 10:10 AM

Change #1156292 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq from ZoteroService

https://gerrit.wikimedia.org/r/1156292

gerritbot added a project: Patch-For-Review.Jun 12 2025, 10:10 AM
gerritbot added a comment.Jun 16 2025, 2:18 PM

Change #1156292 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq from ZoteroService

https://gerrit.wikimedia.org/r/1156292

Maintenance_bot removed a project: Patch-For-Review.Jun 16 2025, 2:31 PM
Jdlrobson-WMF removed a project: Mobile.Oct 9 2025, 8:32 PM
gerritbot added a comment.Nov 6 2025, 11:02 AM

Change #1182150 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Redo "Remove preq from ZoteroService"

https://gerrit.wikimedia.org/r/1182150

gerritbot added a project: Patch-For-Review.Nov 6 2025, 11:02 AM
gerritbot added a comment.Nov 6 2025, 3:59 PM

Change #1182150 merged by jenkins-bot:

[mediawiki/services/citoid@master] Redo "Remove preq from ZoteroService"

https://gerrit.wikimedia.org/r/1182150

Maintenance_bot removed a project: Patch-For-Review.Nov 6 2025, 4:32 PM
gerritbot added a comment.Nov 18 2025, 11:38 AM

Change #1153119 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] [WIP] Remove preq usage

https://gerrit.wikimedia.org/r/1153119

gerritbot added a project: Patch-For-Review.Nov 18 2025, 11:38 AM
Mvolz merged a task: T373280: Preq parses websites in html which claim to be in json as json, which causes a syntax error in citoid.Nov 18 2025, 11:50 AM
Mvolz added subscribers: Lwgph, diegodlh.
Mvolz updated the task description. (Show Details)Nov 18 2025, 11:53 AM
Mvolz updated the task description. (Show Details)Jan 17 2026, 2:40 PM
Mvolz closed subtask T293853: Service-runner depends on preq, a wrapper of request, which is deprecated as Resolved.Jan 17 2026, 3:09 PM
Mvolz updated the task description. (Show Details)
Lwgph unsubscribed.Jan 19 2026, 1:20 PM
gerritbot added a comment.Jan 22 2026, 11:35 AM

Change #1230285 had a related patch set uploaded (by Mvolz; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq dependency

https://gerrit.wikimedia.org/r/1230285

gerritbot added a comment.Jan 22 2026, 11:36 AM

Change #1153119 abandoned by Mvolz:

[mediawiki/services/citoid@master] Remove preq and request usage

https://gerrit.wikimedia.org/r/1153119

gerritbot added a comment.Jan 27 2026, 7:18 PM

Change #1230285 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq dependency

https://gerrit.wikimedia.org/r/1230285

Maintenance_bot removed a project: Patch-For-Review.Jan 27 2026, 7:31 PM
Mvolz created subtask T416471: update nodejs22-slim image to 22.21.0 to support proxy env variables for outbound requests.Feb 4 2026, 1:11 PM
Mvolz closed subtask T416471: update nodejs22-slim image to 22.21.0 to support proxy env variables for outbound requests as Declined.Mar 10 2026, 11:22 AM
gerritbot added a comment.Apr 4 2026, 10:37 AM

Change #1243079 had a related patch set uploaded (by Jforrester; author: Mvolz):

[mediawiki/services/citoid@master] Remove preq dependency again

https://gerrit.wikimedia.org/r/1243079

gerritbot added a project: Patch-For-Review.Apr 4 2026, 10:37 AM
gerritbot added a comment.Apr 4 2026, 10:42 AM

Change #1243079 merged by jenkins-bot:

[mediawiki/services/citoid@master] Remove preq dependency again

https://gerrit.wikimedia.org/r/1243079

Maintenance_bot removed a project: Patch-For-Review.Apr 4 2026, 11:30 AM
Mvolz updated the task description. (Show Details)Mon, Jun 8, 11:20 AM
Mvolz updated the task description. (Show Details)
Mvolz added a comment.EditedMon, Jun 8, 11:28 AM

This is now done in citoid, however note that for outbound requests, fetch in node versions <= 24 does not use the correct protocol for http when contacting it through a proxy.

This means if you try to make outbound requests using the http protocol, our squid proxy url-downloader used in production rejects the requests. This is not caught by CI because external requests in CI does not pass through url-downloader: T427499

We are therefore temporarily use a WMF fork of undici here: https://github.com/wikimedia/undici-proxy in production

Undici plans to support this natively in node from node 26. They are not planning to backport it. So this is "done" in citoid but there is further work to be done once we have node 26 in production to switch from this fork back to node 26's native undici. See thread: https://github.com/nodejs/undici/pull/5116

Mvolz updated the task description. (Show Details)Mon, Jun 8, 3:18 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits