Page MenuHomePhabricator
Create Task
Maniphest T411333

Make function-evaluator tests much more resilient
Closed, ResolvedPublic
Actions

Description

Description

These tests fail constantly in CI, making MRs and image publication unreliable.

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • fix it

Completion checklist

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
wikifunctions: Upgrade evaluators from 2025-12-03-005631 to 2025-12-08-185405operations/deployment-chartsmaster+2 -2
wikifunctions: Upgrade evaluators from 2025-11-14-022545 / 2025-11-17-175029 to 2025-12-03-005631operations/deployment-chartsmaster+2 -2
Customize query in gerrit
Related Changes in GitLab:
TitleReferenceAuthorSource BranchDest Branch
Reorganize evaluator tests, relying as much as possible on the executor class fixtures.repos/abstract-wiki/wikifunctions/function-evaluator!452apineapine-reenable-testsmain
build: Update nyc from ^15.1.0 to pinned 17.1.0repos/abstract-wiki/wikifunctions/function-evaluator!447apineapine-hit-with-wrenchmain
Customize query in GitLab

Related Objects

Event Timeline

cmassaro created this task.Sun, Nov 30, 11:20 PM
CodeReviewBot added a comment.Sun, Nov 30, 11:21 PM

apine updated https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/447

Draft: Update package versions and make tests more robust.

Jdforrester-WMF mentioned this in rMSFE40063d083ddc: build: Update nyc from ^15.1.0 to pinned 17.1.0.Tue, Dec 2, 4:32 PM
Jdforrester-WMF mentioned this in rMSFEd7b874800f7c: tests: Temporarily drop most Python tests to pass CI.Tue, Dec 2, 7:37 PM
Jdforrester-WMF mentioned this in rMSFE28ef2634354d: build: Drop all linux/arm64 builds for now to reduce CI load.Tue, Dec 2, 8:09 PM
Jdforrester-WMF mentioned this in rMSFE7b69eb5ec010: tests: Temporarily drop most Python tests to pass CI.
Jdforrester-WMF mentioned this in rMSFE352da7f86e0a: build: Drop all linux/arm64 builds for now to reduce CI load.
Jdforrester-WMF mentioned this in rMSFE5a469b38f842: tests: Temporarily drop most Python tests to pass CI.
Jdforrester-WMF mentioned this in rMSFEff8179a0766f: tests: Disable the Python WASMedge memory limit test.Tue, Dec 2, 8:35 PM
Jdforrester-WMF mentioned this in rMSFE46f67337400d: tests: Disable the Python WASMedge memory limit test.Tue, Dec 2, 8:43 PM
CodeReviewBot added a comment.Tue, Dec 2, 9:04 PM

apine merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/447

build: Update nyc from ^15.1.0 to pinned 17.1.0

gerritbot added a comment.Wed, Dec 3, 1:46 PM

Change #1214517 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2025-11-14-022545 / 2025-11-17-175029 to 2025-12-03-005631

https://gerrit.wikimedia.org/r/1214517

gerritbot added a project: Patch-For-Review.Wed, Dec 3, 1:46 PM
gerritbot added a comment.Wed, Dec 3, 3:04 PM

Change #1214517 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2025-11-14-022545 / 2025-11-17-175029 to 2025-12-03-005631

https://gerrit.wikimedia.org/r/1214517

Jdforrester-WMF triaged this task as High priority.Wed, Dec 3, 6:42 PM
Jdforrester-WMF edited projects, added Abstract Wikipedia team (26Q2 (Oct–Dec)), Essential-Work; removed Abstract Wikipedia team.
Jdforrester-WMF changed the task status from Open to In Progress.Wed, Dec 3, 6:49 PM
Jdforrester-WMF moved this task from Incoming to In Engineering on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.
CodeReviewBot added a comment.Sun, Dec 7, 2:54 AM

apine opened https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/452

Draft: Reorganize evaluator tests, relying primarily on the executor class fixtures.

cmassaro moved this task from In Engineering to In Code review on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.Mon, Dec 8, 4:13 PM
CodeReviewBot added a comment.Mon, Dec 8, 4:28 PM

jforrester merged https://gitlab.wikimedia.org/repos/abstract-wiki/wikifunctions/function-evaluator/-/merge_requests/452

Reorganize evaluator tests, relying as much as possible on the executor class fixtures.

Jdforrester-WMF mentioned this in rMSFEf69d116cfb35: Reorganize evaluator tests, relying as much as possible on the executor class….Mon, Dec 8, 4:30 PM
Jdforrester-WMF moved this task from In Code review to Needs Sign-off on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.Mon, Dec 8, 4:35 PM
Jdforrester-WMF subscribed.

Code re-org, not deployable code, so lets immediately sign off.

cmassaro closed this task as Resolved.Mon, Dec 8, 4:44 PM
cmassaro moved this task from Needs Sign-off to Ready to deploy on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.
cmassaro moved this task from Ready to deploy to Needs Sign-off on the Abstract Wikipedia team (26Q2 (Oct–Dec)) board.
gerritbot added a comment.Wed, Dec 10, 2:56 PM

Change #1217214 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2025-12-03-005631 to 2025-12-08-185405

https://gerrit.wikimedia.org/r/1217214

gerritbot added a comment.Wed, Dec 10, 3:05 PM

Change #1217214 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2025-12-03-005631 to 2025-12-08-185405

https://gerrit.wikimedia.org/r/1217214

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits