Page MenuHomePhabricator

Protect All Writes to Stdout with Start and End Signals; Maintain Awareness of stdout State
Open, HighPublic

Description

Description

Executor stdout needs to account for a lot of different data types and RPCs now:

  • the final result;
  • debug logs;
  • reentrant calls;
  • wasmedge statistics;
  • wasmedge errors (early exit due to excessive resource consumption)

This means that stdout should provide clear start and end signals so that the executor interfaces can understand the nature of each piece of output. The structure might look like

start debug log
{ someLog: ... }
end debug log
start reentrant call
{ Z1K1: 'Z7', Z7K1: ... }
end reentrant call
start result
{ Z1K1: 'Z6', ... }
end result
[<datetime>] [info] ==========  Statistics  ===========  # these are the wasmedge statistics

Some amount of stateful awareness is necessary because wasmegdge's timeout/gas/memory limits could cancel execution at any time, e.g.

start debug log
{ someLog: ... }
end debug log
start reentrant call
[2024-03-29 00:24:09.675] [error] Create memory instance failed -- exceeded limit page size: 100

[2024-03-29 00:24:09.676] [error] Cost exceeded limit. Force terminate the execution.
[2024-03-29 00:24:09.676] [error]     In instruction: ref.func (0xd2) , Bytecode offset: 0x00000000
[2024-03-29 00:24:09.676] [error]     At AST node: expression
[2024-03-29 00:24:09.676] [error]     At AST node: element segment
[2024-03-29 00:24:09.676] [error]     At AST node: element section
[2024-03-29 00:24:09.676] [error]     At AST node: module
[2024-03-29 00:24:09.676] [info] ====================  Statistics  ====================
...

Desired behavior/Acceptance criteria (returned value, expected error, performance expectations, etc.)

  • executors send start/end signals before and after every discrete RPC or return value
  • executor classes maintain awareness of current state and delegate accordingly
  • executor classes' stdout parsing can identify anomalous exit conditions and respond accordingly

Completion checklist

Event Timeline

Mcastro triaged this task as High priority.Apr 4 2024, 4:35 PM
Mcastro moved this task from To triage to Upcoming Epics on the Abstract Wikipedia team board.

Change #1021507 had a related patch set uploaded (by Jforrester; author: Jforrester):

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-04-03-210033 to 2024-04-18-150843

https://gerrit.wikimedia.org/r/1021507

Change #1021507 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Upgrade evaluators from 2024-04-03-210033 to 2024-04-18-150843

https://gerrit.wikimedia.org/r/1021507