Page MenuHomePhabricator

Benchmark WASM Evaluators with Compile Step at Different Stages
Open, LowPublic



We have recently moved the compile-wasm-binary command to run during the serve or test steps, rather than during the initial build steps, for WASM-based evaluators. There is a small chance that, due to restrictions on write permissions, these compile-wasm-binary commands are silently failing.

A straightforward (if tedious) way to determine whether this is the case is via differential benchmarking: run the WASM benchmarks over the code as-is, and then run them over a version of the code where we are more certain that compilation is completing successfully.

Alternatively, it would suffice to provide some other proof that these compilation steps are completing successfully.

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • prove that WASM compilation step is completing successfully

Completion checklist