Make the evaluator fail early and respond with the appropriate Z500 and HTTP status code when WASI runners are unavailable
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	cmassaro
	Oct 30 2025, 4:23 PM

Description

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

evaluator fails in <1s when WASI runner cannot be acquired
evaluator returns the appropriate Z500 and HTTP status code

Completion checklist

Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist

Related Objects
Search...

Status	Subtype	Assigned	Task
Open	BUG REPORT	None	T406848 wide-scale Python failure
Resolved		cmassaro	T408823 Fail Early if Evaluator Can't Acquire a WASI Runner
Resolved		cmassaro	T409071 When evaluator fails due to WASI unavailability, consider queuing the runner back to the executor pool
Resolved		cmassaro	T408826 Make the evaluator fail early and respond with the appropriate Z500 and HTTP status code when WASI runners are unavailable
Resolved		cmassaro	T408825 Make the orchestrator react appropriately when the evaluator fails to acquire WASI runners
Resolved		cmassaro	T408824 Allocate new Z500 and HTTP error codes for failure to acquire a WASI runner

Event Timeline

cmassaro created this task.Oct 30 2025, 4:23 PM

cmassaro added a parent task: T408823: Fail Early if Evaluator Can't Acquire a WASI Runner.

cmassaro added a subtask: T408825: Make the orchestrator react appropriately when the evaluator fails to acquire WASI runners.

cmassaro claimed this task.Nov 1 2025, 1:47 AM

cmassaro added a subtask: T408824: Allocate new Z500 and HTTP error codes for failure to acquire a WASI runner.

cmassaro edited projects, added Abstract Wikipedia team (26Q2 (Oct–Dec)); removed Abstract Wikipedia team.

cmassaro moved this task from Incoming to In Code review on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.

cmassaro edited parent tasks, added: T409071: When evaluator fails due to WASI unavailability, consider queuing the runner back to the executor pool; removed: T408823: Fail Early if Evaluator Can't Acquire a WASI Runner.Nov 3 2025, 1:57 PM

cmassaro closed subtask T408824: Allocate new Z500 and HTTP error codes for failure to acquire a WASI runner as Resolved.Nov 5 2025, 3:43 PM

Is this now fixed?

Jdforrester-WMF renamed this task from Evaluator fails early and responds with the appropriate Z500 and HTTP status code when WASI runners are unavailable to Make the evaluator fail early and respond with the appropriate Z500 and HTTP status code when WASI runners are unavailable.Nov 5 2025, 6:45 PM

Jdforrester-WMF triaged this task as High priority.Nov 5 2025, 6:48 PM