If we create a working evaluator service in Rust, we can measure its performance and stability characteristics and plan to productionize it in a subsequent quarter
Open, In Progress, MediumPublic
Actions

Assigned To

Authored By

	Jdforrester-WMF
	Aug 7 2023, 2:40 PM

Description

Note: The scope of this work is pretty locked-down. Potential follow-on work is tracked in T402957: [26Q3] Rust evaluator in production.

Description

Some parts of the function evaluator have already been ported to Rust. This quarter, we can combine these parts into a Rust-based service to replicate the functionality of the current Node-based one.

Desired behaviour/Acceptance criteria (returned value, expected error, performance expectations, etc.)

new service can accept all current versions of the evaluator request format
new service can run function calls in the executor for the appropriate programming language
new service runs function calls in separate threads
- new service maintains a pool of "hot" executor interfaces in order to avoid incurring WASI startup costs at request time
new service returns all (appropriate/feasible) metadata supported by the current Node-based evaluator
new service implements a time limit
new service implements a rate limit

Motivation and Context

The function-evaluator is a critical part of the Wikifunctions stack. It is currently (January 2023) implemented in Node using Wikimedia's service-runner within the service 'template'. Although this is a flexible framework and standard for RESTful services across Wikimedia, it is not very performant and as a critical path through which every Wikifunctions call runs. Improving the speed and reliability of this is important.

Related Objects
Search...

Status	Assigned	Task
Open	Jdforrester-WMF	T402957 [26Q3] Rust evaluator in production
Open	None	T382433 Get the Rust prototype of the function-evaluator's executors into a state where it could be deployed
In Progress	cmassaro	T343720 If we create a working evaluator service in Rust, we can measure its performance and stability characteristics and plan to productionize it in a subsequent quarter
Resolved	cmassaro	T399344 Rust-based evaluator is at feature parity with the current Node-based one
Resolved	cmassaro	T399322 Rust-based evaluator service should maintain a pool of ready executors
Resolved	cmassaro	T399323 Rust-based evaluator can add all currently-supported metadata
Resolved	ecarg	T399345 Rust-based evaluator implements a timeout
Resolved	ecarg	T407590 Check that timeout is working properly
Resolved	ecarg	T399346 Rust-based evaluator implements rate limiting
Resolved	cmassaro	T399321 Rust-based evaluator service can run a function call in the appropriate executor and return a result
Resolved	cmassaro	T400820 Use dedicated abstract-wikipeda-rust image in test-rust (and, eventually, other Rust images)
In Progress	Jdforrester-WMF	T407588 Create a fail-fast for rate-limiting
Declined	None	T402478 Rectify double/long issue with remainingTime in different Avro schemata
Resolved	cmassaro	T402477 Support Avro schema semver 2.0.0 and move each Avro schema to a separate Rust module
Resolved	cmassaro	T402473 Improve testing of Rust Avro deserialization
Resolved	cmassaro	T399320 Rust-based service should accept all current versions of the evaluator request format
Resolved	cmassaro	T406625 Create and use a shared set of fully-expanded Z7s for testing the evaluators, so we can assert stable performance and equivalence between Rust and non-Rust evaluators

Event Timeline

Jdforrester-WMF created this task.Aug 7 2023, 2:40 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 7 2023, 2:40 PM

Jdforrester-WMF added projects: Abstract Wikipedia team, function-evaluator.Aug 7 2023, 6:25 PM

Jdforrester-WMF triaged this task as Medium priority.Aug 31 2023, 6:56 PM

Jdforrester-WMF moved this task from To Triage to Backlog on the Abstract Wikipedia team board.

Jdforrester-WMF added a project: Epic.Oct 5 2023, 5:18 PM

Jdforrester-WMF moved this task from Backlog to 25Q2 (Oct–Dec) on the Abstract Wikipedia team board.Nov 6 2024, 5:51 PM

Jdforrester-WMF edited projects, added Abstract Wikipedia team (25Q2 (Oct–Dec)); removed Abstract Wikipedia team.

Jdforrester-WMF moved this task from Incoming to In Engineering on the Abstract Wikipedia team (25Q2 (Oct–Dec)) board.Nov 13 2024, 5:44 PM

Jdforrester-WMF moved this task from 25Q2 (Oct–Dec) to Engineering Backlog on the Abstract Wikipedia team board.Dec 18 2024, 6:26 PM

Jdforrester-WMF edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (25Q2 (Oct–Dec)).

cmassaro renamed this task from Explore re-writing the function-evaluator into a more performant, strongly-typed language (TypeScript/rust/golang/etc.) to If we create a working evaluator service in Rust, we can measure its performance and stability characteristics and plan to productionize it in a subsequent quarter.Jul 11 2025, 10:23 PM

cmassaro updated the task description. (Show Details)

cmassaro edited projects, added Abstract Wikipedia team (26Q1 (Jul–Sep)); removed Abstract Wikipedia team.

cmassaro added a project: Essential-Work.

cmassaro added a project: function-schemata.

cmassaro moved this task from Incoming to Epics for this Quarter on the Abstract Wikipedia team (26Q1 (Jul–Sep)) board.

cmassaro added a subtask: T399344: Rust-based evaluator is at feature parity with the current Node-based one.Jul 11 2025, 10:35 PM

cmassaro claimed this task.Jul 11 2025, 10:48 PM

cmassaro added a subtask: T402478: Rectify double/long issue with remainingTime in different Avro schemata.Aug 21 2025, 12:58 PM

cmassaro added a subtask: T402477: Support Avro schema semver 2.0.0 and move each Avro schema to a separate Rust module.

cmassaro added a subtask: T402473: Improve testing of Rust Avro deserialization.

cmassaro added a subtask: T399320: Rust-based service should accept all current versions of the evaluator request format.Aug 26 2025, 1:56 PM

cmassaro added a subtask: T402956: Build the JS executor when compiling the Rust image rather than relying on the checked-in blob.Aug 26 2025, 5:58 PM