When a Tester run fails for an Implementation, automatically de-approve said Implementation from its function
Open, HighPublic
Actions

Assigned To

None

Authored By

	Jdforrester-WMF
	Aug 3 2022, 8:49 PM

Description

When an "approved" (connected) Tester is run for a Function against an approved Implementation, it should always pass. If it fails (i.e. votes false, not returns an error that might be temporary), the system is in an inconsistent state. To fix this, we must automatically de-approve the failing Implementation.

[TBD: If the Tester fails all the Implementations, should it instead be disabled? How do we know what to do when?]

Related Objects

Mentioned In: T341199: Detach implementations/testers when editing function inputs/outputs should be handled in the PHP layer instead of the front-end
T319478: When a Tester runs for a Function's Implementations, automatically re-order the list of Implementations in correctness and speed order
Mentioned Here: T319478: When a Tester runs for a Function's Implementations, automatically re-order the list of Implementations in correctness and speed order

Event Timeline

Jdforrester-WMF created this task.Aug 3 2022, 8:49 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 3 2022, 8:49 PM

Jdforrester-WMF added a parent task: T310199: Select fastest correct implementation.Aug 3 2022, 8:49 PM

Jdforrester-WMF mentioned this in T319478: When a Tester runs for a Function's Implementations, automatically re-order the list of Implementations in correctness and speed order.Oct 5 2022, 9:19 PM

I am worried that due to a hiccup in the infra, we might end up descoping all implementations. Let's descope this one for now to get a better feel for how stable the tests run.

In order to allow for that, T319478 is being extended to sort by correctness and speed.

DVrandecic moved this task from Phase θ – Throttling to Phase κ – Clean-up on the Abstract Wikipedia team board.Jan 9 2023, 8:49 PM

DVrandecic edited projects, added Abstract Wikipedia team (Phase κ – Clean-up); removed Abstract Wikipedia team (Phase θ – Throttling).

Jdforrester-WMF removed a parent task: T310199: Select fastest correct implementation.Jan 23 2023, 7:19 PM

Jdforrester-WMF triaged this task as High priority.Mar 24 2023, 1:18 AM

Jdforrester-WMF moved this task from Incoming to Better before launch on the Abstract Wikipedia team (Phase κ – Clean-up) board.May 24 2023, 9:40 PM

Jdforrester-WMF moved this task from Phase κ – Clean-up to To triage: After launch on the Abstract Wikipedia team board.Jun 16 2023, 8:37 AM

Jdforrester-WMF edited projects, added Abstract Wikipedia team; removed Abstract Wikipedia team (Phase κ – Clean-up).

Jdforrester-WMF mentioned this in T341199: Detach implementations/testers when editing function inputs/outputs should be handled in the PHP layer instead of the front-end.Jul 6 2023, 4:22 PM

Jdforrester-WMF moved this task from To triage: After launch to Needs Product/Design on the Abstract Wikipedia team board.Aug 31 2023, 4:49 PM

DVrandecic moved this task from Needs Product/Design to No current plans / External on the Abstract Wikipedia team board.Mar 21 2024, 5:11 PM