Why do we run ORES with celery?
Celery allows us to separate IO requests from CPU parallelization. This is most useful for batch processing. We are able to request data needed for scoring in batch and then split the data into celery "tasks" for the CPU intensive work. 20-25% of our scores are delivered via batch requests and a batch request is generally ~3x faster than requesting single scores. Our batch request use-case to increase in frequency over time.
If we could allow people to send 3x as many single-score requests, then the tradeoff would cancel out on our end. I don't think we could triple our workers if we dropped celery but we could potentially increase them by 20% or so with the memory we'd make available.
For what it's worth, we can run already run ORES in "single-threaded" mode (each UWSGI worker does both IO and CPU one task at a time) so this should be easy to test.
Why might we want to run ORES without celery?
What costs and benefits exist for running ORES without celery
This is what we mean to flesh out in this task.
This task is done when we have laid out a proposal for and discussed the considerations of deploying ORES without celery.