Fix the Benchmark for the WASMedge-based Evaluators and Enable in CI
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	cmassaro
	Oct 2 2023, 7:06 PM

Description

This benchmark fails mysteriously!

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

fix the benchmark (not clear why it fails)
enable it in gitlab-ci.yaml

Remove all the non-applicable tags from the "Tags" field, leave only the tags of the projects/repositories related to this task

Completion checklist

Before closing this task, review one by one the checklist available here: https://www.mediawiki.org/wiki/Abstract_Wikipedia_team/Definition_of_Done#Back-end_Task/Bug_completion_checklist

Details

	Subject	Repo	Branch	Lines +/-
	wikifunctions: Upgrade evaluators from 2024-04-03-210033 to 2024-04-18-150843	operations/deployment-charts	master	+2 -2

Customize query in gerrit

	Title	Reference	Author	Source Branch	Dest Branch
	Turn benchmark tests back on and use new Avro version	repos/abstract-wiki/wikifunctions/function-evaluator!200	ecarg	grace/T347901/fix-benchmark-tests	main

Customize query in GitLab

Related Objects

Mentioned In: T360344: Create Benchmarks for the WASM Evaluators
T350284: Improve CI for Wikifunctions services to better test like reality
T349557: Clean up the evaluator/executor interfaces once we've simplified into WASM everywhere

Event Timeline

cmassaro created this task.Oct 2 2023, 7:06 PM

Jdforrester-WMF triaged this task as Medium priority.Oct 5 2023, 4:39 PM

Jdforrester-WMF moved this task from To triage to Backlog on the Abstract Wikipedia team board.

Jdforrester-WMF mentioned this in T349557: Clean up the evaluator/executor interfaces once we've simplified into WASM everywhere.Oct 23 2023, 8:33 PM

Jdforrester-WMF renamed this task from Fix the Benchmark for the JS WASM Evaluator and Enable in CI to Fix the Benchmark for the WASMedge-based Evaluators and Enable in CI .Nov 1 2023, 4:06 PM

sbassett subscribed.Nov 1 2023, 4:08 PM

Jdforrester-WMF mentioned this in T350284: Improve CI for Wikifunctions services to better test like reality.Nov 1 2023, 4:09 PM

Jdforrester-WMF mentioned this in T360344: Create Benchmarks for the WASM Evaluators.Mar 18 2024, 4:25 PM

cmassaro merged a task: T360344: Create Benchmarks for the WASM Evaluators.Mar 18 2024, 4:44 PM

cmassaro added a subscriber: Jdforrester-WMF.

Observations so far:

_mocha test/benchmarks/javascriptEvaluatorBenchmarks.js takes forever (10 minutes in, it's still going⏳)
_mocha test/benchmarks/pythonEvaluatorBenchmarks.js immediately returns:

(node:48369) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)


  0 passing (0ms)

Qs:

When did this begin? Or how long has this been an issue? Likely when this ticket was created...
(Could it have started when I consolidated all of those tests?) Likely no as those tasks were merged in January, long after this ticket was created.
I think I've noticed that non-wasm tests still get triggered/run when I run the script to only run wasm tests: check if this is true or if was only in Py || Js

Change #1021507 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-04-03-210033 to 2024-04-18-150843

https://gerrit.wikimedia.org/r/1021507

gerritbot added a project: Patch-For-Review.Thu, Apr 18, 3:34 PM

Change #1021507 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-04-03-210033 to 2024-04-18-150843

https://gerrit.wikimedia.org/r/1021507

Deployed.

Maintenance_bot removed a project: Patch-For-Review.Fri, Apr 26, 7:02 PM

Fix the Benchmark for the WASMedge-based Evaluators and Enable in CI Closed, ResolvedPublicActions

Description

Description

Completion checklist

Details

Related Objects

Event Timeline

Fix the Benchmark for the WASMedge-based Evaluators and Enable in CI
Closed, ResolvedPublic
Actions