Description
Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)
- memory consumption must be comparable to current evaluator
- currently, the Rust evaluator container consumes ~8GiB of memory, while the Node one consumes ~1
- performance must not suffer under load
- currently, the Rust evaluator can handle about 15 concurrent requests before threads start to block; Node has no such limitation
- blocking calls must be limited
- this means, among other things, not reading the WASM files when creating executors but reading them once into memory
I believe the major culprit here is wasi_common, which is deprecated and makes some blocking calls. We should switch to wasmtime_wasi::p2. This library has much better async support. A good way to test is to set tokio::main(flavor="current_thread") in the top-level evaluator service. This will tell us very quickly if any calls are blocking. Once we can run in "current_thread" mode, we can switch back to the multi-threaded runtime and fine-tune the amount of worker threads to optimize performance.
Completion checklist
- Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist