Our Celery workers are forked after pre-initializing all models, so that the bulk of the application memory is shared between all sibling processes. However, child processes still use around ~310MB each once they "warm up". We want to know whether this is normal, unavoidable runtime stuff or if there's more pre-initialization we could do which would cause more of that memory to be shared.
Description
Description
Related Objects
Related Objects
Event Timeline
Comment Actions
We are moving to Lift Wing: https://wikitech.wikimedia.org/wiki/Machine_Learning/LiftWing
I am closing old tasks related to ORES since it is being deprecated, please re-open if you feel that any work could be done on Lift Wing.