Page MenuHomePhabricator
Create Task
Maniphest T329443

Cloud services enhancement proposal: Toolforge API gateway
Closed, ResolvedPublic
Actions

Description

Proposal Title:Toolforge API gateway

Brief description: Implement a new single API gateway that will route traffic to the various Toolforge APIs (for example to the Jobs API and the Build service API).

Why: This will provide an unified entry point for the API clients and simplify the set up of new services.

Risks: If implemented poorly, the new service could end up as an unnecessary complication without any value add, or in the worst case decrease service reliability or add security risks.

Design documentation: https://wikitech.wikimedia.org/wiki/Wikimedia_Cloud_Services_team/EnhancementProposals/Toolforge_API_gateway

Details

SubjectRepoBranchLines +/-
tests: add support for new api-gateway setupcloud/toolforge/jobs-framework-apimain+3 -3
tests: add support for new api-gateway setupcloud/toolforge/jobs-framework-climaster+2 -2
deployment: Drop direct accesscloud/toolforge/jobs-framework-apimain+3 -177
P:toolforge::k8s::haproxy: drop standalone jobs ingressoperations/puppetproduction+2 -24
P:toolforge: use api gateway for jobs clioperations/puppetproduction+1 -1
deployment: Add extra Nginx server for the API gatewaycloud/toolforge/jobs-framework-apimain+67 -10
P:toolforge::k8s::haproxy: add api gateway load balanceroperations/puppetproduction+28 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

taavi created this task.Feb 12 2023, 11:12 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 12 2023, 11:12 AM
dcaro awarded a token.Feb 12 2023, 1:20 PM
dcaro subscribed.
taavi added a subtask: T329453: Deploy cert-manager to Toolforge.Feb 12 2023, 4:47 PM
aborrero subscribed.Feb 13 2023, 11:01 AM

I think this idea is excellent.

taavi closed subtask T329453: Deploy cert-manager to Toolforge as Resolved.Feb 15 2023, 10:15 AM
Slst2020 subscribed.Feb 15 2023, 4:22 PM
Andrew subscribed.Feb 15 2023, 5:01 PM

This sounds like a good idea. I briefly thought that you might use a new Keystone install for this but it sounds like what you want is a shared gateway rather than just catalog/discovery?

taavi added a comment.Feb 15 2023, 8:57 PM
In T329443#8619300, @Andrew wrote:

This sounds like a good idea. I briefly thought that you might use a new Keystone install for this but it sounds like what you want is a shared gateway rather than just catalog/discovery?

Yes, correct, at this point I just want to add an additional HTTP proxy for these requests.

fnegri subscribed.Feb 21 2023, 4:17 PM
taavi claimed this task.Feb 22 2023, 9:39 AM
taavi moved this task from Inbox to Implementation on the Cloud Services Proposals board.
gerritbot added a comment.Feb 22 2023, 9:55 AM

Change 891239 had a related patch set uploaded (by Majavah; author: Majavah):

[cloud/toolforge/jobs-framework-api@main] deployment: Add extra Nginx server for the API gateway

https://gerrit.wikimedia.org/r/891239

gerritbot added a project: Patch-For-Review.Feb 22 2023, 9:55 AM

Change 891240 had a related patch set uploaded (by Majavah; author: Majavah):

[cloud/toolforge/jobs-framework-api@main] deployment: Drop direct access

https://gerrit.wikimedia.org/r/891240

Slst2020 awarded a token.Feb 22 2023, 10:50 AM
gerritbot added a comment.Feb 23 2023, 4:29 PM

taavi merged https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/1

Initial implementation

gerritbot added a comment.Feb 24 2023, 1:23 PM

Change 891827 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:toolforge::k8s::haproxy: add api gateway load balancer

https://gerrit.wikimedia.org/r/891827

gerritbot added a comment.Feb 27 2023, 9:39 AM

Change 892370 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:toolforge: use api gateway for jobs cli

https://gerrit.wikimedia.org/r/892370

gerritbot added a comment.Feb 27 2023, 1:00 PM

Change 891827 merged by David Caro:

[operations/puppet@production] P:toolforge::k8s::haproxy: add api gateway load balancer

https://gerrit.wikimedia.org/r/891827

gerritbot added a comment.Feb 28 2023, 5:01 PM

Change 891239 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-api@main] deployment: Add extra Nginx server for the API gateway

https://gerrit.wikimedia.org/r/891239

gerritbot added a comment.Mar 1 2023, 11:27 AM

Change 892370 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] P:toolforge: use api gateway for jobs cli

https://gerrit.wikimedia.org/r/892370

gerritbot added a comment.Mar 1 2023, 1:05 PM

Change 893474 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] P:toolforge::k8s::haproxy: drop standalone jobs ingress

https://gerrit.wikimedia.org/r/893474

gerritbot added a comment.Mar 1 2023, 1:11 PM

Change 893474 merged by Arturo Borrero Gonzalez:

[operations/puppet@production] P:toolforge::k8s::haproxy: drop standalone jobs ingress

https://gerrit.wikimedia.org/r/893474

gerritbot added a comment.Mar 1 2023, 1:14 PM

Change 891240 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-api@main] deployment: Drop direct access

https://gerrit.wikimedia.org/r/891240

taavi closed this task as Resolved.Mar 1 2023, 9:13 PM

I wrote some documentation here: https://wikitech.wikimedia.org/wiki/Portal:Toolforge/Admin/APIs. This is done.

gerritbot added a comment.Mar 2 2023, 11:47 AM

aborrero opened https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/22

k8s: deploy api-gateway component

gerritbot added a comment.Mar 2 2023, 12:26 PM

aborrero opened https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/3

api-gateway: assume local deployment has jobs-api loaded

gerritbot added a comment.Mar 2 2023, 12:30 PM

Change 893707 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[cloud/toolforge/jobs-framework-api@main] tests: add support for new api-gateway setup

https://gerrit.wikimedia.org/r/893707

gerritbot added a comment.Mar 2 2023, 12:39 PM

Change 893709 had a related patch set uploaded (by Arturo Borrero Gonzalez; author: Arturo Borrero Gonzalez):

[cloud/toolforge/jobs-framework-cli@master] tests: add support for new api-gateway setup

https://gerrit.wikimedia.org/r/893709

gerritbot added a comment.Mar 6 2023, 10:54 AM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/lima-kilo/-/merge_requests/22

k8s: deploy api-gateway component

gerritbot added a comment.Mar 6 2023, 10:54 AM

aborrero merged https://gitlab.wikimedia.org/repos/cloud/toolforge/api-gateway/-/merge_requests/3

api-gateway: assume local deployment has jobs-api loaded

gerritbot added a comment.Mar 6 2023, 11:08 AM

Change 893709 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-cli@master] tests: add support for new api-gateway setup

https://gerrit.wikimedia.org/r/893709

gerritbot added a comment.Mar 6 2023, 11:09 AM

Change 893707 merged by jenkins-bot:

[cloud/toolforge/jobs-framework-api@main] tests: add support for new api-gateway setup

https://gerrit.wikimedia.org/r/893707

aborrero mentioned this in rCTKF10fc50ca460d: tests: add support for new api-gateway setup.Mar 6 2023, 11:16 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits