Page MenuHomePhabricator

Investigate memory usage of ORES in kubernetes
Open, NormalPublic

Description

ORES is a pretty memory hungry service, each celery node loads up lots of huge pickle files into memory and the only way it's handleable is because of COW. If we move to containers we might end up having lots of issues. How to address this? We might end up giving dedicated service for different wikis.

Does COW work with kubernetes? Can we load multiple ORES containers and expect that they will share memory?

If not, what strategy for splitting up ORES models/assets might work for us in kubernetes?

This task is done when the above questions have answers.