Page MenuHomePhabricator

[jobs-api] Split the API, business, and k8s models
Open, HighPublic

Description

We have three sets of data structures, the ones used in the API requests and responses, the internal objects, and the k8s objects we create from those.

This task is to split those three sets, creating a clear distinction between them and allowing us to modularize the application.

  • API layer:

This contains the logic to validate API requests, and how to create business model objects from them

  • Business layer:

This contains our internal objects (ScheduledJob, OneOffJob, ContinuousJob, ...) and has logic related only to how to manage those.

  • Execution layer:

This will have the logic on how to instantiate the business layer objects into k8s, probably using an abstract class to be able to easily change the infa layer in the future:

from abc import ABC, abstractmethod

class Engine(ABC):
   @abstractmethod
   def run_continuous_job(self, continuous_job: ContinuousJob) -> RunResult:
      pass

   @abstractmethod
   def run_scheduled_job(self, scheduled_job: ScheduledJob) -> RunResult:
      pass
...


class K8sEngine(Engine):
    def run_continuous_job(self, continuous_job: ContinuousJob) -> RunResult:
        ... do stuff on k8s to get the job running

Details

TitleReferenceAuthorSource BranchDest Branch
jobs-api: bump to 0.0.291-20240415170518-fde04b2crepos/cloud/toolforge/toolforge-deploy!259project_1317_bot_df3177307bed93c3f34e421e26c86e38bump_jobs-apimain
Customize query in GitLab

Event Timeline

dcaro created this task.
dcaro triaged this task as High priority.Mar 11 2024, 12:52 PM
dcaro moved this task from Backlog to Ready to be worked on on the Toolforge board.

project_1317_bot_df3177307bed93c3f34e421e26c86e38 opened https://gitlab.wikimedia.org/repos/cloud/toolforge/toolforge-deploy/-/merge_requests/259

jobs-api: bump to 0.0.291-20240415170518-fde04b2c