Some things that we can simplify before adding the webservice complexity:
- Remove flask-restful (we don't use rest), and use bare flask, this will make sure we can decouple API models from business models, will be specially useful once we start generating the code from the openapi definition
- Split the 3 model sets:
- Api requests (ex. data sent when starting a job, or requesting a job)
- Internal logic (ex. ScheduledJob, ContinuousJob, OneOffJob should be classes by themselves as they are quite different and have different data attached to them)
- K8s mapping, we can abstract this in something we might want to call 'engine', so we might have a better time swapping/modifying it in the future, for example:
from abc import ABC, abstractmethod
class Engine(ABC):
@abstractmethod
def run_continuous_job(self, continuous_job: ContinuousJob) -> RunResult:
pass
@abstractmethod
def run_scheduled_job(self, scheduled_job: ScheduledJob) -> RunResult:
pass
...
class K8sEngine(Engine):
def run_continuous_job(self, continuous_job: ContinuousJob) -> RunResult:
... do stuff on k8s to get the job runningThat would help keep the k8s side, the business logic side, and the API side models separated.
- Use a database to store our internal models (this could be etcd with custom resources, read-only for users as we want to only modify them through the API), that would allow us to untangle k8s objects from business model objects (ex. having to parse the k8s template.spec.command, or having to add some versioning information to the k8s objects), this would be a step after the Engine above, and could be something like:
from abc import ABC, abstractmethod
class Store(ABC):
@abstractmethod
def get_continuous_job(self, job_id: str) -> ContinuousJob:
pass
@abstractmethod
def get_continuous_jobs(self, tool_name: str) -> list[ContinuousJob]:
pass
...
class K8sStore(Engine):
def get_continuous_job(self, job_id: str) -> ContinuousJob:
... retrieve all the jobs from k8s, could be started as the same logic we currently have, listing all the deployments in the user's namespace, etc., and then be replaced by reading the custom resourcesNote: working on the subtasks in parallel is easily going to end up in conflicts, so make sure to sync with whomever is already working on them.