Use evaluation for validation
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	DVrandecic
	Jan 27 2021, 10:38 PM

Description

Every type will come with a function that validates the instances of that type. These functions are functions defined in Wikifunctions. This task is to actually use these functions to validate instances.

Details

	Subject	Repo	Branch	Lines +/-
	Enable validation by default, and defer to input when provided	mediawiki/services/function-orchestrator	master	+43 -42
	Enable validation on function calls	mediawiki/extensions/WikiLambda	master	+2 -4

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T278321 Provide a special page listing invalid ZObjects
Resolved	None	T290119 Phase η root task
Resolved	None	T290118 User-defined types work with validation
Resolved	None	T291028 Instances of a user-defined type which should not validate indeed do not validate
Open	None	T278318 Let users validate an unstored ZObject / edit on a ZObject
Stalled	None	T269177 ZObject DAO should have a isWellFormed method
Resolved	cmassaro	T260861 Use the orchestrator/evaluator to validate instances of each Z4/Type using on-wiki defined Z8/Functions instead of hard-coded
Open	None	T278316 Provide a validation API endpoint for arbitrary ZObjects from both MediaWiki and the orchestrator
Open	None	T292240 Save Un- (or Under-) Validated ZObjects, Validate Asynchronously via Orchestrator, Then Annotate ZObjects As Valid or Not
Open	None	T278325 Store that a page contains an invalid ZObject in a tracking table
Open	None	T278320 Allow to store an invalid ZObject explicitly, so we can have failure examples for Validator functions' Testers
Open	None	T294045 Create a "Validate" Endpoint in Orchestrator
Duplicate	None	T273125 Migrate hard-coded validators to validation defined on-wiki
Open	None	T278319 Provide functionality for users to trigger a re-validation of a stored object
Resolved	None	T273124 Use evaluation for validation
Resolved	cmassaro	T260321 Allow users to trigger function calls for built-in functions (orchestrator-only)
Resolved	DVrandecic	T261964 Agree on architecture for first evaluation engine
Resolved	Lindsaykwardell	T280558 Normalize Inputs to Orchestrator
Resolved	arthurlorenzi	T275095 Move normalization and canonicalization code to function-schemata
Resolved	None	T262770 Normalize ZObjects
Invalid	None	T280559 Retrieve Function Signature from Orchestrator
Resolved	cmassaro	T282843 Hook up orchestrator to the evaluator and pass through appropriately-dereferenced Z7 calls
Resolved	cmassaro	T271762 Create initial Function Evaluator service
Resolved	cmassaro	T269697 Turn Function Evaluator into a service
Resolved	cmassaro	T269698 Make Function Evaluator work async
Resolved	cmassaro	T272630 Define interface for the function-evaluator
Resolved	Jdforrester-WMF	T276478 Provide "semantically peel-able" list of programming languages that we allow
Resolved	cmassaro	T273515 Create Execution Environments for Supported Languages
Resolved	cmassaro	T273517 Create Execution Environment for Python
Resolved	cmassaro	T275944 Create Execution Environment for JavaScript
Resolved	None	T261470 Sandboxing and monitoring of evaluations
Resolved	SimoneThisDot	T291022 Validators for user-defined types can be defined and linked
Resolved	SimoneThisDot	T291021 Make the UX for creating a user-defined type straightforward
Resolved	SimoneThisDot	T292891 Provide a custom UX for Z4/Type
Resolved	cmassaro	T291030 Validator of a user-defined type can be implemented
Resolved	cmassaro	T291026 Let user implementations throw a user-defined error during a user-defined validation
Resolved	None	T291027 Users can create error types
Resolved	DVrandecic	T293423 Create a function that turns an error type into a type
Resolved	cmassaro	T289687 Allow users to create instances of errors in composition
Resolved	arthurlorenzi	T290346 Function model: Change the signature of validators from Type -> List to Type -> Type (pass-through)

Event Timeline

DVrandecic created this task.Jan 27 2021, 10:38 PM

DVrandecic added a parent task: T273125: Migrate hard-coded validators to validation defined on-wiki.Jan 27 2021, 10:48 PM

DVrandecic moved this task from To triage to Phase ε on the Abstract Wikipedia team board.Feb 3 2021, 5:22 PM

DVrandecic edited projects, added Abstract Wikipedia team (Phase ε); removed Abstract Wikipedia team.

DVrandecic mentioned this in T277925: Provide a validator for Z14/Implementation in the editing experience.Mar 19 2021, 10:10 PM

DVrandecic mentioned this in T269182: Figure out validation and well-formedness.Mar 24 2021, 1:33 PM

DVrandecic added a subtask: T260321: Allow users to trigger function calls for built-in functions (orchestrator-only).Mar 24 2021, 1:53 PM

DVrandecic added a parent task: T278316: Provide a validation API endpoint for arbitrary ZObjects from both MediaWiki and the orchestrator.

DVrandecic added a parent task: T278319: Provide functionality for users to trigger a re-validation of a stored object.Mar 24 2021, 1:56 PM

DVrandecic triaged this task as Medium priority.Apr 7 2021, 5:08 AM

I’m wondering whether it would make sense to think of a Z4K3/validator (function) as a constructor for validated instances of objects of the specified type. In practice, this would add an additional key into validated instances, with its value being the list of Z5/Errors (an empty list in the case of a valid object). For example, the valid positive integer represented by "2" would be the object {"Z1K1": "Z70", "Z70K1": "2", "Z70K2": []}. The unvalidated instance, on the other hand, would be a Z7/Function call to the Z70 validator/constructor function, passing "K1": "2" (and expecting type+value+errors – a validated Z70 instance – in return). This ensures that only the validator/constructor function can provide the Z70K2/errors value.

Jdforrester-WMF added a subtask: T271762: Create initial Function Evaluator service.May 5 2021, 5:21 PM

Jdforrester-WMF moved this task from Phase ε to To triage: Un-phased work on the Abstract Wikipedia team board.May 6 2021, 4:54 PM

Jdforrester-WMF edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (Phase ε).

Jdforrester-WMF closed subtask T260321: Allow users to trigger function calls for built-in functions (orchestrator-only) as Resolved.May 7 2021, 4:28 PM

Jdforrester-WMF added a parent task: T260861: Use the orchestrator/evaluator to validate instances of each Z4/Type using on-wiki defined Z8/Functions instead of hard-coded .May 11 2021, 8:33 PM

Jdforrester-WMF closed subtask T271762: Create initial Function Evaluator service as Resolved.May 25 2021, 6:04 PM

Jdforrester-WMF removed a subtask: T271762: Create initial Function Evaluator service.

Jdforrester-WMF closed subtask T282843: Hook up orchestrator to the evaluator and pass through appropriately-dereferenced Z7 calls as Resolved.May 26 2021, 8:21 PM

Jdforrester-WMF added a parent task: T278320: Allow to store an invalid ZObject explicitly, so we can have failure examples for Validator functions' Testers.Jun 2 2021, 5:22 PM

Change 720023 had a related patch set uploaded (by Lindsay Wardell; author: Lindsay Wardell):

[mediawiki/extensions/WikiLambda@master] Enable validation on function calls

https://gerrit.wikimedia.org/r/720023