Will we see the wait time separately from the actual execution time? And will the wait times be excluded from the comparisons for determining the fastest implementation (and consequential ordering of the implementation list)?

apine updated https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/254

Add Evaluator class to encapsulate information about ExecutorPools and manage metadata.

We won't see them yet, no. Conceptually, there are three different interesting time spans in the evaluator:

queueTime: how long we wait for an executor subprocess to be available;
startUpTime: how long we wait for an executor process to be READY (wasmedge startup cost, which we still incur under load);
executionTime: how long we spend actually inside the executor.

The current change makes this possible, but we're not there yet. All we have is evaluationDuration, which (as of this change) is basically queueTime + startupTime (if relevant) + executionTime.

We do have other means of measuring this more granular time spans, but we don't expose them in the UI. It would be useful to exclude wait times when determining the fastest implementation.

Yes, we should exclude wait times (at least in principle). If we arrange for them to appear as separate items in the metadata map, it will be easy to adapt the implementation-ordering code to take that into consideration. However, the success of this strategy will depend on whether these wait times differ based on language (Python vs. JavaScript). If one language consistently gets better wait times than the other, then it might make sense to continue to base the ordering on evaluationDuration.

ecarg merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/254

Add Evaluator class to encapsulate information about ExecutorPools and manage metadata.

Change #1075555 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-08-20-132618 to 2024-09-24-221243

https://gerrit.wikimedia.org/r/1075555

Change #1075555 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-08-20-132618 to 2024-09-24-221243

https://gerrit.wikimedia.org/r/1075555

This is now live.