evaluate
Open, HighPublic
Actions

Assigned To

Authored By

	cmassaro
	Jan 4 2026, 7:17 PM

Description

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

Run a set of tests to ensure that running time, memory consumption, and other relevant metrics are better than or equal to /v1/evaluate
Enable /v2/evaluate in production (i.e., switch all callers off of /v1/evaluate)

Completion checklist

Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist

Details

Related Changes in Gerrit:

Subject	Repo	Branch	Lines +/-
OrchestratorRequest: Switch evaluations to v2 endpoint	mediawiki/extensions/WikiLambda	wmf/1.46.0-wmf.19	+1 -1
wikifunctions: Upgrade orchestrator from 2026-03-04-220825 to 2026-03-10-224034	operations/deployment-charts	master	+1 -1
OrchestratorRequest: Switch evaluations to v2 endpoint	mediawiki/extensions/WikiLambda	master	+1 -1
docs: Switch developer evaluations to v2 endpoint	mediawiki/extensions/WikiLambda	master	+3 -3

Customize query in gerrit

Related Changes in GitLab:

	Title	Reference	Author	Source Branch	Dest Branch
	Address outstanding issues with v2 functionality.	repos/abstract-wiki/wikifunctions/function-orchestrator!582	apine	apine-more-fixes	main

Customize query in GitLab

Related Objects
Search...

Status	Assigned	Task
Open	cmassaro	T412953 Create a composition interpreter to improve performance
Open	cmassaro	T413727 Validate and Productionize /v2/evaluate
Resolved	Jdforrester-WMF	T413881 Do not exclude src/transpilation/** from nyc coverage
Resolved	Jdforrester-WMF	T414589 Update check-wf-services.sh to also check the v2 endpoint
Resolved	ecarg	T417501 Support metrics and custom spans in composition language V2
Resolved	cmassaro	T418763 Improving modeling of types in composition language v2

Event Timeline

cmassaro created this task.Jan 4 2026, 7:17 PM

cmassaro added a parent task: T412953: Create a composition interpreter to improve performance.

cmassaro added a subtask: T413729: Ensure that all current unit tests pass with the new /v2/evaluate.Jan 4 2026, 7:20 PM

cmassaro added a subtask: T413840: Optimize composition language V2.Jan 6 2026, 12:17 AM

cmassaro added a subtask: T413881: Do not exclude src/transpilation/** from nyc coverage.Jan 6 2026, 1:46 PM

cmassaro added a subtask: T412955: Validate that compositions that previously were broken are now fixed by the new composition language.Jan 6 2026, 2:55 PM

cmassaro edited projects, added Abstract Wikipedia team (26Q3 (Jan–Mar)); removed Abstract Wikipedia team.Jan 6 2026, 6:41 PM

Jdforrester-WMF added a project: OKR-Work.Jan 7 2026, 6:49 PM

Jdforrester-WMF moved this task from Incoming to Ready on the Abstract Wikipedia team (26Q3 (Jan–Mar)) board.

DSantamaria triaged this task as High priority.Jan 8 2026, 3:16 PM

cmassaro added a subtask: T414589: Update check-wf-services.sh to also check the v2 endpoint.Jan 14 2026, 3:04 PM

cmassaro added a subtask: T416142: Consider adding stack trace to all Z5s, other kinds of nested metadata.Feb 2 2026, 9:58 AM

Jdforrester-WMF changed the status of subtask T414589: Update check-wf-services.sh to also check the v2 endpoint from Open to In Progress.Feb 10 2026, 12:47 PM

cmassaro added a subtask: T417501: Support metrics and custom spans in composition language V2.Sun, Feb 15, 10:21 AM

cmassaro removed a subtask: T416142: Consider adding stack trace to all Z5s, other kinds of nested metadata.Wed, Feb 18, 1:47 PM

Jdforrester-WMF closed subtask T413881: Do not exclude src/transpilation/** from nyc coverage as Resolved.Wed, Feb 18, 11:02 PM

cmassaro added a subtask: T418016: Banish circular refrences from Invariants.Fri, Feb 20, 5:29 PM

Jdforrester-WMF closed subtask T414589: Update check-wf-services.sh to also check the v2 endpoint as Resolved.Wed, Mar 4, 3:21 PM

ecarg changed the status of subtask T417501: Support metrics and custom spans in composition language V2 from Open to In Progress.Wed, Mar 4, 6:29 PM

cmassaro removed a subtask: T413729: Ensure that all current unit tests pass with the new /v2/evaluate.Wed, Mar 4, 6:39 PM

cmassaro removed a subtask: T413840: Optimize composition language V2.Wed, Mar 4, 6:42 PM

cmassaro removed a subtask: T412955: Validate that compositions that previously were broken are now fixed by the new composition language.

cmassaro removed a subtask: T418016: Banish circular refrences from Invariants.Wed, Mar 4, 6:45 PM

cmassaro added a subtask: T418763: Improving modeling of types in composition language v2.

DSantamaria changed the status of subtask T418763: Improving modeling of types in composition language v2 from Open to In Progress.Thu, Mar 5, 8:29 AM

cmassaro closed subtask T417501: Support metrics and custom spans in composition language V2 as Resolved.Fri, Mar 6, 11:56 PM

cmassaro claimed this task.Sun, Mar 8, 3:13 AM

apine opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/582

Draft: Address outstanding issues with v2 functionality.

cmassaro moved this task from Ready to In Code review on the Abstract Wikipedia team (26Q3 (Jan–Mar)) board.Mon, Mar 9, 4:27 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-orchestrator/-/merge_requests/582

Address outstanding issues with v2 functionality.

Change #1249990 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] docs: Switch developer evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1249990

Change #1249991 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] OrchestratorRequest: Switch evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1249991

Change #1249990 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] docs: Switch developer evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1249990

Change #1249991 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] OrchestratorRequest: Switch evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1249991

Change #1250051 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@wmf/1.46.0-wmf.19] OrchestratorRequest: Switch evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1250051

ReleaseTaggerBot added a project: MW-1.46-notes (1.46.0-wmf.20; 2026-03-17).Tue, Mar 10, 7:01 PM

Change #1250595 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2026-03-04-220825 to 2026-03-10-224034

https://gerrit.wikimedia.org/r/1250595

Change #1250595 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade orchestrator from 2026-03-04-220825 to 2026-03-10-224034

https://gerrit.wikimedia.org/r/1250595

Change #1250051 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@wmf/1.46.0-wmf.19] OrchestratorRequest: Switch evaluations to v2 endpoint

https://gerrit.wikimedia.org/r/1250051

Mentioned in SAL (#wikimedia-operations) [2026-03-11T21:35:56Z] <jforrester@deploy2002> Started scap sync-world: Backport for [[gerrit:1250051|OrchestratorRequest: Switch evaluations to v2 endpoint (T413727)]]

Jdforrester-WMF updated the task description. (Show Details)Wed, Mar 11, 9:38 PM

Mentioned in SAL (#wikimedia-operations) [2026-03-11T21:40:08Z] <jforrester@deploy2002> jforrester: Backport for [[gerrit:1250051|OrchestratorRequest: Switch evaluations to v2 endpoint (T413727)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-03-11T21:54:15Z] <jforrester@deploy2002> Finished scap sync-world: Backport for [[gerrit:1250051|OrchestratorRequest: Switch evaluations to v2 endpoint (T413727)]] (duration: 18m 19s)

Jdforrester-WMF updated the task description. (Show Details)Wed, Mar 11, 9:58 PM

Jdforrester-WMF closed subtask T418763: Improving modeling of types in composition language v2 as Resolved.

Jdforrester-WMF updated the task description. (Show Details)

Jdforrester-WMF moved this task from In Code review to Needs Sign-off on the Abstract Wikipedia team (26Q3 (Jan–Mar)) board.

ReleaseTaggerBot edited projects, added MW-1.46-notes (1.46.0-wmf.19; 2026-03-10); removed MW-1.46-notes (1.46.0-wmf.20; 2026-03-17).Wed, Mar 11, 10:00 PM

Maintenance_bot removed a project: Patch-For-Review.Wed, Mar 11, 10:34 PM