Page MenuHomePhabricator
Create Task
Maniphest T350773

Remove preq and use node fetch
Closed, ResolvedPublic
Actions

Description

Currently cxserver uses https://github.com/wikimedia/preq library for http(s) requests. However, preq is an unmaintained library. It is a wrapper over other deprecated or unmaintained libraries. preq also brings two high severity security issues as per npm audit checks(T309772: npm audit reports several security issues with Service runner).

cxserver recently upgraded to Node 18. Node 18 has native fetch. So we can start replacing preq with fetch. Since cxserver depends on service-runner, we cannot fully remove the usage of preq. service-runner uses bluebird and preq(T293853: Service-runner depends on preq, a wrapper of request, which is deprecated )

Details

SubjectRepoBranchLines +/-
Update cxserver to 2024-03-18-053939-productionoperations/deployment-chartsmaster+1 -1
Update tests to use node fetch, remove preq dependencymediawiki/services/cxservermaster+154 -762
Update cxserver to 2024-03-14-053505-productionoperations/deployment-chartsmaster+3 -3
MW API requests: Remove preq usage, use native fetchmediawiki/services/cxservermaster+141 -75
Update cxserver to 2024-03-12-113634-productionoperations/deployment-chartsmaster+1 -1
Set the proxy to dispatcher option of fetch APImediawiki/services/cxservermaster+16 -28
Update cxserver to 2024-03-11-120258-productionoperations/deployment-chartsmaster+1 -1
Google: Use native fetch API instead of preqmediawiki/services/cxservermaster+38 -26
Update cxserver to 2024-03-11-035839-productionoperations/deployment-chartsmaster+1 -1
Elia: Migrate from preq to native fetchmediawiki/services/cxservermaster+64 -31
Update cxserver to 2024-03-05-082211-productionoperations/deployment-chartsmaster+1 -1
Yandex: Migrate from preq to native fetchmediawiki/services/cxservermaster+57 -27
Update cxserver to 2024-03-04-113412-productionoperations/deployment-chartsmaster+1 -1
LingoCloud: Migrate from preq to native fetchmediawiki/services/cxservermaster+102 -52
Update cxserver to 2024-03-04-023843-productionoperations/deployment-chartsmaster+1 -1
Apertium: Migrate from preq to native fetchmediawiki/services/cxservermaster+44 -28
Update cxserver to 2023-12-04-055024-productionoperations/deployment-chartsmaster+1 -1
MinT: use node native fetch instead of preqmediawiki/services/cxservermaster+35 -28
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

• santhosh created this task.Nov 8 2023, 10:49 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 8 2023, 10:49 AM
• santhosh added projects: Technical-Debt, Language-Team (Language-2023-October-December).Nov 8 2023, 10:50 AM
gerritbot added a comment.Nov 8 2023, 10:50 AM

Change 972727 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] MinT: use node native fetch instead of preq

https://gerrit.wikimedia.org/r/972727

gerritbot added a project: Patch-For-Review.Nov 8 2023, 10:50 AM
• santhosh claimed this task.Nov 8 2023, 10:50 AM
• santhosh changed the task status from Open to In Progress.Nov 8 2023, 10:58 AM
• santhosh triaged this task as Medium priority.
• santhosh moved this task from Quarter Backlog to In Progress on the Language-Team (Language-2023-October-December) board.
Nikerabbit added a project: Unplanned-Sprint-Work.Nov 8 2023, 2:49 PM
gerritbot added a comment.Nov 28 2023, 8:19 AM

Change 972727 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MinT: use node native fetch instead of preq

https://gerrit.wikimedia.org/r/972727

Maintenance_bot removed a project: Patch-For-Review.Nov 28 2023, 8:30 AM
gerritbot added a comment.Dec 4 2023, 5:54 AM

Change 979487 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2023-12-04-055024-production

https://gerrit.wikimedia.org/r/979487

gerritbot added a project: Patch-For-Review.Dec 4 2023, 5:54 AM
gerritbot added a comment.Dec 4 2023, 5:57 AM

Change 979487 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2023-12-04-055024-production

https://gerrit.wikimedia.org/r/979487

Stashbot mentioned this in T270060: Package apertium-fra-frp (French-Arpitan).Dec 4 2023, 6:08 AM

Mentioned in SAL (#wikimedia-operations) [2023-12-04T06:08:03Z] <kart_> Updated cxserver to 2023-12-04-055024-production (T270060, T350773, T352620)

Stashbot mentioned this in T352620: Failure to start new translations (item.dispose is not a function).Dec 4 2023, 6:08 AM
Maintenance_bot removed a project: Patch-For-Review.Dec 4 2023, 6:10 AM
KartikMistry moved this task from In Progress to Needs QA on the Language-Team (Language-2023-October-December) board.Dec 4 2023, 6:45 AM
Pginer-WMF edited projects, added Language-Team (Language-2024-January-March); removed Language-Team (Language-2023-October-December).Jan 8 2024, 10:34 AM
Pginer-WMF moved this task from Quarter Backlog to Needs QA on the Language-Team (Language-2024-January-March) board.
ngkountas subscribed.Jan 8 2024, 7:48 PM

I do not believe this task is completed and ready for review. The usage of the "preq" library has only been replaced for Mint MT client, but the library is still in use for a lot of cases in cxserver codebase. Moving this back to priority backlog.

ngkountas moved this task from Needs QA to Priority: Translation on the Language-Team (Language-2024-January-March) board.Jan 8 2024, 7:49 PM
kostajh subscribed.Jan 9 2024, 12:30 PM
• santhosh mentioned this in T357950: Remove servicerunner dependency for cxserver.Feb 21 2024, 7:02 AM
gerritbot added a comment.Feb 22 2024, 5:56 AM

Change 1005645 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] LingoCloud: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005645

gerritbot added a project: Patch-For-Review.Feb 22 2024, 5:56 AM
gerritbot added a comment.Feb 22 2024, 6:08 AM

Change 1005646 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Yandex: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005646

gerritbot added a comment.Feb 22 2024, 6:35 AM

Change 1005648 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Elia: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005648

gerritbot added a comment.Feb 22 2024, 6:47 AM

Change 1005651 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Apertium: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005651

gerritbot added a comment.Feb 22 2024, 10:19 AM

Change 1005717 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Google: Use native fetch API instead of preq

https://gerrit.wikimedia.org/r/1005717

Nikerabbit moved this task from Priority: Translation to In Review on the Language-Team (Language-2024-January-March) board.Feb 22 2024, 10:43 AM
gerritbot added a comment.Feb 22 2024, 1:28 PM

Change 1005757 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Remove preq usage, use native fetch

https://gerrit.wikimedia.org/r/1005757

gerritbot added a comment.Mar 4 2024, 2:38 AM

Change 1005651 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Apertium: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005651

gerritbot added a comment.Mar 4 2024, 2:59 AM

Change 1008119 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-04-023843-production

https://gerrit.wikimedia.org/r/1008119

gerritbot added a comment.Mar 4 2024, 6:59 AM

Change 1008119 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-04-023843-production

https://gerrit.wikimedia.org/r/1008119

Stashbot added a comment.Mar 4 2024, 7:08 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-04T07:08:41Z] <kart_> Updated cxserver to 2024-03-04-023843-production (T350773)

gerritbot added a comment.Mar 4 2024, 11:33 AM

Change 1005645 merged by jenkins-bot:

[mediawiki/services/cxserver@master] LingoCloud: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005645

gerritbot added a comment.Mar 4 2024, 11:42 AM

Change 1008420 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-04-113412-production

https://gerrit.wikimedia.org/r/1008420

gerritbot added a comment.Mar 5 2024, 4:49 AM

Change 1008420 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-04-113412-production

https://gerrit.wikimedia.org/r/1008420

Stashbot added a comment.Mar 5 2024, 5:15 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-05T05:15:46Z] <kart_> Updated cxserver to 2024-03-04-113412-production (T350773)

gerritbot added a comment.Mar 5 2024, 5:45 AM

Change 1005646 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Yandex: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005646

gerritbot added a comment.Mar 5 2024, 11:39 AM

Change 1008836 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-05-082211-production

https://gerrit.wikimedia.org/r/1008836

Mvolz subscribed.Mar 5 2024, 8:25 PM
gerritbot added a comment.Mar 7 2024, 4:40 AM

Change 1008836 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-05-082211-production

https://gerrit.wikimedia.org/r/1008836

Stashbot mentioned this in T353259: Set MinT as the default service in Content Translation for a set of languages based on their machine translation usage.Mar 7 2024, 4:55 AM
Stashbot mentioned this in T353136: Set MinT as the default service in Content Translation for languages supported by IndicTrans2.

Mentioned in SAL (#wikimedia-operations) [2024-03-07T04:55:17Z] <kart_> Updated cxserver to 2024-03-05-082211-production (T353136, T353259, T350773)

gerritbot added a comment.Mar 7 2024, 6:37 AM

Change 1005648 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Elia: Migrate from preq to native fetch

https://gerrit.wikimedia.org/r/1005648

gerritbot added a comment.Mar 11 2024, 4:04 AM

Change 1009941 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-11-035839-production

https://gerrit.wikimedia.org/r/1009941

gerritbot added a comment.Mar 11 2024, 4:32 AM

Change 1009941 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-11-035839-production

https://gerrit.wikimedia.org/r/1009941

Stashbot added a comment.Mar 11 2024, 4:52 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-11T04:52:28Z] <kart_> Updated cxserver to 2024-03-11-035839-production (T350773)

gerritbot added a comment.Mar 11 2024, 12:02 PM

Change 1005717 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Google: Use native fetch API instead of preq

https://gerrit.wikimedia.org/r/1005717

gerritbot added a comment.Mar 11 2024, 12:07 PM

Change 1010169 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-11-120258-production

https://gerrit.wikimedia.org/r/1010169

gerritbot added a comment.Mar 12 2024, 4:54 AM

Change 1010169 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-11-120258-production

https://gerrit.wikimedia.org/r/1010169

Stashbot added a comment.Mar 12 2024, 5:11 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-12T05:11:34Z] <kart_> Updated cxserver to 2024-03-11-120258-production (T350773)

gerritbot added a comment.Mar 12 2024, 6:49 AM

Change 1010449 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Set the proxy to dispatcher option of fetch API

https://gerrit.wikimedia.org/r/1010449

gerritbot added a comment.Mar 12 2024, 10:54 AM

Change 1010498 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/cxserver@master] Update tests to use node fetch, remove preq dependency

https://gerrit.wikimedia.org/r/1010498

gerritbot added a comment.Mar 12 2024, 11:35 AM

Change 1010449 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Set the proxy to dispatcher option of fetch API

https://gerrit.wikimedia.org/r/1010449

gerritbot added a comment.Mar 12 2024, 11:46 AM

Change 1010508 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-12-113634-production

https://gerrit.wikimedia.org/r/1010508

gerritbot added a comment.Mar 13 2024, 4:34 AM

Change 1010508 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-12-113634-production

https://gerrit.wikimedia.org/r/1010508

Stashbot mentioned this in T359525: MinT: Translation with MinT/Apertium are failing: fetch failed.Mar 13 2024, 4:50 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-13T04:50:27Z] <kart_> Updated cxserver to 2024-03-12-113634-production (T350773, T359525)

gerritbot added a comment.Mar 14 2024, 5:34 AM

Change 1005757 merged by jenkins-bot:

[mediawiki/services/cxserver@master] MW API requests: Remove preq usage, use native fetch

https://gerrit.wikimedia.org/r/1005757

gerritbot added a comment.Mar 14 2024, 5:46 AM

Change 1011033 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-14-053505-production

https://gerrit.wikimedia.org/r/1011033

gerritbot added a comment.Mar 14 2024, 5:50 AM

Change 1011033 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-14-053505-production

https://gerrit.wikimedia.org/r/1011033

Stashbot added a comment.Mar 14 2024, 7:15 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-14T07:15:43Z] <kart_> Updated cxserver to 2024-03-14-065833-production (T350773)

gerritbot added a comment.Mar 14 2024, 11:32 AM

Change 1010498 merged by jenkins-bot:

[mediawiki/services/cxserver@master] Update tests to use node fetch, remove preq dependency

https://gerrit.wikimedia.org/r/1010498

Maintenance_bot removed a project: Patch-For-Review.Mar 14 2024, 12:31 PM
Nikerabbit moved this task from In Review to Done on the Language-Team (Language-2024-January-March) board.Mar 14 2024, 6:37 PM
gerritbot added a comment.Mar 18 2024, 5:54 AM

Change 1012130 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update cxserver to 2024-03-18-053939-production

https://gerrit.wikimedia.org/r/1012130

gerritbot added a project: Patch-For-Review.Mar 18 2024, 5:54 AM
gerritbot added a comment.Mar 18 2024, 6:05 AM

Change 1012130 merged by jenkins-bot:

[operations/deployment-charts@master] Update cxserver to 2024-03-18-053939-production

https://gerrit.wikimedia.org/r/1012130

Stashbot added a comment.Mar 18 2024, 6:25 AM

Mentioned in SAL (#wikimedia-operations) [2024-03-18T06:25:20Z] <kart_> Updated cxserver to 2024-03-18-053939-production (T350773)

Maintenance_bot removed a project: Patch-For-Review.Mar 18 2024, 6:30 AM
Pginer-WMF closed this task as Resolved.Mar 27 2024, 12:18 PM
Mvolz mentioned this in T361602: Remove usage of preq in all services and replace with node fetch; requires node >=18.x.x.Tue, Apr 2, 2:52 PM
Mvolz added a parent task: T361602: Remove usage of preq in all services and replace with node fetch; requires node >=18.x.x.Tue, Apr 2, 2:55 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL