Page MenuHomePhabricator

[jobs-api,builds-api,envvars-api] consolidate api paths
Closed, ResolvedPublic

Description

We want to have the same path structure for all our sub-apis, this means (on the sub-api level):

  • Using singular for the resources for our resources (ex. /v1/build or /v1/job/<id>/logs) (because we have 2 apis using it, happy to use plural instead if anyone has a strong preference, if so the checkmarks should be swapped): (update: we will go with plural)
    • jobs-api (quota -> quotas)
    • builds-api
    • envvars-api
  • Prefix the urls with the version /<version>/* for example /v1/build
    • jobs-api (only the /healthz endpoint doesn't have a version prefix)
    • builds-api
    • envars-api
  • Remove /api from the paths
    • jobs-api
    • builds-api
    • envvars-api

Note that these changes should be done in a backwards-compatible way, as in:

  • change the API server to support both paths
  • change the client to use the new path
  • change the API server and remove the old path support

Final paths structure examples:

  • For resources: '/v1/build/logs'
  • For actions/misc: /v1/clean, /v1/healthz /v1/quota

Note that this is related to T356974: [builds-api,jobs-api,envvars-api,api-gateway] Figure out and document how to do non-backwards compatible changes, that somewhat consolidates the expected responses from the apis.
It's also related to T363808: [builds-api, builds-cli] Prefix all endpoints with `/tool/<toolname>`, T363346: [jobs-api, jobs-cli] Prefix all endpoints with `/tool/<toolname>` and T363809: [envvars-api, envvars-cli] Prefix all endpoints with `/tool/<toolname>`

Details

Related Changes in Gerrit:
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
functional-tests/direct-api: update direct api endpointsrepos/cloud/toolforge/toolforge-deploy!448sstefanovaslavina/update-direct-api-endpointsmain
builds-api: bump to 0.0.166-20240723115306-1e901808repos/cloud/toolforge/toolforge-deploy!444ghostbump_builds-apimain
envvars-api: bump to 0.0.58-20240723115225-033f7657repos/cloud/toolforge/toolforge-deploy!443ghostbump_envvars-apimain
jobs-api: bump to 0.0.323-20240723115142-863de5d7repos/cloud/toolforge/toolforge-deploy!442ghostbump_jobs-apimain
api: remove deprecated endpointsrepos/cloud/toolforge/builds-api!106sstefanovaslavina/remove-deprecated-endpointsmain
d/changelog: bump to 0.0.17repos/cloud/toolforge/builds-cli!82sstefanovaslavina/bump_to_0.0.17main
builds-api: bump to 0.0.165-20240718140844-131a3480repos/cloud/toolforge/toolforge-deploy!432ghostbump_builds-apimain
api: remove deprecated endpointsrepos/cloud/toolforge/envvars-api!42sstefanovaslavina/remove-deprecated-endpointsmain
api endpoints: use plural pathsrepos/cloud/toolforge/builds-cli!81sstefanovaslavina/use-plural-endpointsmain
api: rename api resources to pluralrepos/cloud/toolforge/builds-api!105sstefanovaslavina/endpoints-to-pluralmain
envvars-api: bump to 0.0.56-20240718074100-89430f97repos/cloud/toolforge/toolforge-deploy!431ghostbump_envvars-apimain
api: drop deprecated endpointsrepos/cloud/toolforge/jobs-api!108sstefanovaslavina/drop-deprecated-endpointsmain
d/changelog: bump to 16.0.15repos/cloud/toolforge/jobs-cli!55sstefanovaslavina/bump_to_16.0.15main
d/changelog: bump to 16.0.15repos/cloud/toolforge/jobs-cli!54sstefanovaslavina/bump_to_16.0.5main
jobs-api: bump to 0.0.321-20240717143951-2c8a3296repos/cloud/toolforge/toolforge-deploy!430ghostbump_jobs-apimain
cli: use /quotas endpointrepos/cloud/toolforge/jobs-cli!53sstefanovaslavina/use-plural-quotamain
api: make quota pluralrepos/cloud/toolforge/jobs-api!107sstefanovaslavina/make-quota-pluralmain
jobs-api: bump to 0.0.315-20240715135730-fb3bd3e7repos/cloud/toolforge/toolforge-deploy!417ghostbump_jobs-apimain
api endpoints: use plural pathsrepos/cloud/toolforge/envvars-cli!51sstefanovaslavina/use-plural-pathsmain
api: rename api resources to pluralrepos/cloud/toolforge/envvars-api!40sstefanovaslavina/resources-to-pluralmain
Show related patches Customize query in GitLab

Event Timeline

dcaro triaged this task as High priority.
dcaro updated the task description. (Show Details)
Slst2020 changed the task status from Open to In Progress.Jun 12 2024, 12:50 PM
Slst2020 moved this task from Next Up to In Progress on the Toolforge (Toolforge iteration 11) board.

Flask blueprints default to introducing a trailing slash at the "root" URL for each blueprint in jobs-api. Is this something we want to remove too? E.g

10.244.0.1 - - [12/Jun/2024:12:55:00 +0000] "GET /envvars/v1/tool/tf-test/envvar HTTP/1.1" 200 63 "-" "envvars client toolforge_weld/1.4.0 python-requests/2.28.1"
10.244.0.1 - - [12/Jun/2024:12:55:05 +0000] "GET /jobs/api/v1/tool/tf-test/jobs/ HTTP/1.1" 200 520 "-" "tool-tf-test@lima-kilo toolforge_weld/1.4.0 python-requests/2.28.1"
10.244.0.1 - - [12/Jun/2024:12:55:37 +0000] "GET /builds/v1/tool/tf-test/build HTTP/1.1" 200 28 "-" "toolforge_builds_cli toolforge_weld/1.4.0 python-requests/2.28.1"

Flask blueprints default to introducing a trailing slash at the "root" URL for each blueprint in jobs-api. Is this something we want to remove too? E.g

10.244.0.1 - - [12/Jun/2024:12:55:00 +0000] "GET /envvars/v1/tool/tf-test/envvar HTTP/1.1" 200 63 "-" "envvars client toolforge_weld/1.4.0 python-requests/2.28.1"
10.244.0.1 - - [12/Jun/2024:12:55:05 +0000] "GET /jobs/api/v1/tool/tf-test/jobs/ HTTP/1.1" 200 520 "-" "tool-tf-test@lima-kilo toolforge_weld/1.4.0 python-requests/2.28.1"
10.244.0.1 - - [12/Jun/2024:12:55:37 +0000] "GET /builds/v1/tool/tf-test/build HTTP/1.1" 200 28 "-" "toolforge_builds_cli toolforge_weld/1.4.0 python-requests/2.28.1"

I would say that we should support both, if it's too complicated, maybe we can just move jobs-api to fastapi where should be easier I think

Shouldn't be too complicated hopefully, I think there's an option when creating the Blueprint.

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/417

jobs-api: bump to 0.0.315-20240715135730-fb3bd3e7

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/430

jobs-api: bump to 0.0.321-20240717143951-2c8a3296

Change #1055146 had a related patch set uploaded (by Slavina Stefanova; author: Slavina Stefanova):

[operations/puppet@production] envvars backend: update endpoints

https://gerrit.wikimedia.org/r/1055146

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/431

envvars-api: bump to 0.0.56-20240718074100-89430f97

Change #1055146 merged by David Caro:

[operations/puppet@production] envvars backend: update endpoints

https://gerrit.wikimedia.org/r/1055146

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/432

builds-api: bump to 0.0.165-20240718140844-131a3480

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/442

jobs-api: bump to 0.0.323-20240723115142-863de5d7

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/443

envvars-api: bump to 0.0.58-20240723115225-033f7657

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/444

builds-api: bump to 0.0.166-20240723115306-1e901808

Slst2020 updated the task description. (Show Details)
Slst2020 moved this task from In Review to Done on the Toolforge (Toolforge iteration 13) board.

Change #1056494 had a related patch set uploaded (by Slavina Stefanova; author: Slavina Stefanova):

[operations/puppet@production] replica-cnf-api: fix envvars endpoint

https://gerrit.wikimedia.org/r/1056494

Change #1056494 abandoned by Slavina Stefanova:

[operations/puppet@production] replica-cnf-api: fix envvars endpoint

Reason:

dcaro got to it first :)

https://gerrit.wikimedia.org/r/1056494