Page MenuHomePhabricator

[Discuss] Changes to ores codebase for migrating to kubernets
Open, NormalPublic

Description

This is a task about to discuss and spec out how changes to ores should be made to make it work with the blubber/helm/kubernetes. In order to achieve this, we need to archive ores deploy repo. That means:

  • All of model files need to be either pulled when building the images (That would make the images gigantic) or make service pulls them down when it's starting (make the configs accept URLs as model path)
  • Configs need to be overhauled into one big yaml file that's going to be checked out into helm charts
  • Model files wouldn't work without the codes (features and feature lists). This means the repos need to be turned to libraries and they would be installed building the image using a file like requirements-production.txt
  • Assets ores use, they also need to pulled down when starting the service.
  • Scap configs need to turn to helm configs
  • wheels repo need to be archived and completely ditched
  • helm chart for ores need to have redis subchart for tests, meaning redis should also have a chart in releases.wikimedia.org/charts
  • Speaking of tests, it's better to wrap all of python tests into one tox.ini file so entrypoint for the "test" stage of docker images can be easily defined
  • The blubber files need to be added to ores repo but it's blocked on T210267: Execution of the deployment pipeline should be configurable via .pipeline/config.yaml

(More?)

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 30 2019, 4:02 PM
Harej added a comment.Apr 2 2019, 9:11 PM

@Ladsgroup I feel like this should be a meeting?

Harej triaged this task as High priority.Apr 2 2019, 9:16 PM
Harej moved this task from Untriaged to Ready to go on the Scoring-platform-team board.

@Ladsgroup I feel like this should be a meeting?

yup

Just some notes, I agree on premise with most of the above.

Configs need to be overhauled into one big yaml file that's going to be checked out into helm charts

That's not necessary. We can ship more than 1 config files via helm charts. We can ship config directories as well.

Assets ores use, they also need to pulled down when starting the service.

I am not so sure about this one. Very often assets are tightly coupled with the deployed code version and usually relatively small in size and fine to ship with the application.

Scap configs need to turn to helm configs

Which parts and in which sense ? The 2 systems are very different in both features and shortcomings, it probably makes more sense to just create the helm chart.

helm chart for ores need to have redis subchart for tests, meaning redis should also have a chart in releases.wikimedia.org/charts

Indeed. I 'll take care of that

Configs need to be overhauled into one big yaml file that's going to be checked out into helm charts

That's not necessary. We can ship more than 1 config files via helm charts. We can ship config directories as well.

Nice, thank you for letting me know!

Assets ores use, they also need to pulled down when starting the service.

I am not so sure about this one. Very often assets are tightly coupled with the deployed code version and usually relatively small in size and fine to ship with the application.

We have an asset for ores that is 1.3 GB (word2vec dictionary)

Scap configs need to turn to helm configs

Which parts and in which sense ? The 2 systems are very different in both features and shortcomings, it probably makes more sense to just create the helm chart.

Good question, what I meant was scap configs (things in the scap folder) and puppet configs all will be tossed meaning we should not assume anything there. Things like "the scap config says 3 parallel deploys at the same time will happen"

Configs need to be overhauled into one big yaml file that's going to be checked out into helm charts

That's not necessary. We can ship more than 1 config files via helm charts. We can ship config directories as well.

Nice, thank you for letting me know!

Assets ores use, they also need to pulled down when starting the service.

I am not so sure about this one. Very often assets are tightly coupled with the deployed code version and usually relatively small in size and fine to ship with the application.

We have an asset for ores that is 1.3 GB (word2vec dictionary)

Ah, good point. We 'll probably need to figure that out indeed.

Scap configs need to turn to helm configs

Which parts and in which sense ? The 2 systems are very different in both features and shortcomings, it probably makes more sense to just create the helm chart.

Good question, what I meant was scap configs (things in the scap folder) and puppet configs all will be tossed meaning we should not assume anything there. Things like "the scap config says 3 parallel deploys at the same time will happen"

Ah ok understood. Probably should be reworded to "Investigate which behaviors of scap configs (and how) should be kept while migrating to helm charts"

Met with @Ladsgroup and @akosiaris to understand some of this better. Here are our notes: https://etherpad.wikimedia.org/p/ores-to-k8s

Harej removed a subscriber: Harej.Jul 4 2019, 9:27 AM
Halfak added a subscriber: ACraze.Aug 16 2019, 9:21 PM
Halfak lowered the priority of this task from High to Normal.Wed, Sep 11, 9:17 PM