Original doc: https://docs.google.com/document/d/1y7cIX3oiqOH8hEuPhSEuqWx-yElxTq9Ga8qJRvqAJjY/edit
This is an epic task to keep track of all the subtasks needed for a first iteration of the Toolforge Component API.
After this iteration, the user should be able to:
- Define build configuration and trigger the build with one or more triggers (TBD) in a yaml file and persist it in toolforge
- Define any kind of job (webservice included) in that yaml file that will be created/restarted when depolyed (toolforge deploy)
- Redeploy automatically components (jobs) when their images are built by a trigger
- Specify the dependencies between the jobs to deploy in a specific order
- Specify ports to expose to other components defined in the yaml (internal services)
This will be open only as 'preview' for users.
Potential example:
$ cat mytool.yaml config-version: 0.1 metadata: name: Toolhunt description: A web application for editing Toolhub records in a fun and easy way. version: v1.0.0 components: toolhunt-ui: description: a vue.js frontend type: webservice build: repository: https://github.com/wikimedia/toolhunt-ui triggers: - webhook wait-for: - toolhunt-api toolhunt-api: description: a python-flask api type: continuous build: repository: https://github.com/wikimedia/toolhunt run: port: 5000 toolhunt-flower: description: dashboard for monitoring and managing celery type: continuous build: reuse-from: toolhunt-api run: port: 5555 command: flower wait-for: - toolhunt-api $ toolforge deploy mytool.yaml # or toolforge component deploy mytool.yaml if we don't have a single cli yet Starting deploy for toolhunt v1.0.0... Building images: [✅] toolhunt-ui [✅] toolhunt-api Deploying components: [✅] toolhunt-api [✅] toolhunt-ui (waiting for toolhunt-api) [✅] toolhunt-flower (waiting for toolhunt-api) All components deployed $ toolforge status # very speculative, might be toolforge components status if we don't have a single cli yet Current components: # similar to `toolforge jobs list` with deployment info --------------------------------------------------------------------------- name status build-id deploy time ports type --------------------------------------------------------------------------- toolhunt-ui RUNNING build-id-12345 12:15:00 - webservice toolhunt-api RUNNING build-id-12344 12:12:00 5000 continuous toolhunt-flower ERROR build-id-12344 12:12:00 5555 continuous Last triggered builds: # similar to `toolforge build list` with deployment info 2024-04-04 12:11:00 UTC OK build-id-12345 toolhunt-ui triggerd by webhook some more information 2024-04-04 12:00:00 UTC OK build-id-12344 toolhunt-api triggerd manually some more information Last deployments: toolhunt-ui 2024-04-04 12:15:00 UT build-id-12345 Current 2024-03-02 12:00:00 UT build-id-12342 OK toolhunt-api 2024-04-04 12:15:00 UT build-id-12344 Current 2024-03-02 12:00:00 UT build-id-12222 FAILED toolhunt-flower (image: toolhunt-api) 2024-04-04 12:15:00 UT build-id-12344 Current 2024-03-02 12:00:00 UT build-id-12322 OK
The details on how that configuration file looks like and the command line interface are to be defined.
Direct subtasks:
- T362070: [components-api] Get a minimal version of the config with build-only data
- T362069: [components-api] Get a skeleton of API webservice and implement `/tool/<toolname>/deploy` with build-only features
- T362066: [components-api] Develop the webhook mechanism to trigger a deployment
- T362075: [components-api] add one-off, scheduled and continuous jobs support to the yaml + api
- T348755: [jobs-api,webservice] Run webservices via the jobs framework
- T362077: [components-api] Add webservice support (to refine)
- T362072: [components-api] Add support for non-public services
- T362076: [components-api] Add support for pre-build images (to refine)
- T362071: [components-api] Extend the list of build triggers (unrefined)