Very likely related to T314621.

Note also that substantial work was done fairly recently on APIPerformTest.php, and also some changes in the Vue file that calls the API:
T309069. Happy to answer questions related to that work.

I can't currently reproduce the bug with this API method - every test I throw at the API is performed correctly. But the task to rework the API still stands.

Both wikilambda_perform_test&wikilambda_perform_test_zfunction=Z10014 (the documented example call) and wikilambda_perform_test&wikilambda_perform_test_zfunction=Z801 (the one we probably should actually document) work fine, as do the '<URL_ENCODED_JSON_FUNCTION>' ones in that documentation now that Edmund's great work is landed.

Is this task now just "update the on-wiki documentation"?

WikiLambda's ApiPerformTest.php uses the function-orchestrator /evaluate/ entrypoint to first run and then validate a tester

I did this work in June. What's the DoD here?

Hi @EWright-WMF and @Jdforrester-WMF , wikilambda_perform_test&wikilambda_perform_test_zfunction=Z801 does not appear to be working at present. Has there been a regression or is beta currently broken? The output is hard to read, but I still see "Key value not wellformed" in there. By visiting https://wikifunctions.beta.wmflabs.org/wiki/Z801?view=function-viewer, one can see 1 of the 4 tests is failing (and "Key value not wellformed" shows up in the metadata dialog).

Hi @DMartin-WMF - I can take a look at that. But given it's just one of the tests that's failing I suspect it's an issue with function orchestration or evaluation, rather than the perform test flow itself.

In T320509#8408300, @DMartin-WMF wrote:

Hi @EWright-WMF and @Jdforrester-WMF , wikilambda_perform_test&wikilambda_perform_test_zfunction=Z801 does not appear to be working at present. Has there been a regression or is beta currently broken? The output is hard to read, but I still see "Key value not wellformed" in there. By visiting https://wikifunctions.beta.wmflabs.org/wiki/Z801?view=function-viewer, one can see 1 of the 4 tests is failing (and "Key value not wellformed" shows up in the metadata dialog).

The broken test is "meant" to be broken (until we fix it, see T299194: Test Z8013 for Z801/Echo fails). I think we can declare this Resolved once this API is better documented.

Hi @Jdforrester-WMF - I agree this probably just needs documentation updating, which I can take on.

Taking this issue led to the idea of restructuring Tester - as proposed in this doc - but I think that work should probably be in a separate issue, which I've created in T323901.

In T320509#8423903, @EWright-WMF wrote:

Hi @Jdforrester-WMF - I agree this probably just needs documentation updating, which I can take on.

Thanks!

Taking this issue led to the idea of restructuring Tester - as proposed in this doc - but I think that work should probably be in a separate issue, which I've created in T323901.

Ack. Not sure that should be automatically accepted into the current Phase without discussion first though. :-)

@Jdforrester-WMF - apologies, I didn't mean to bypass processes. I've removed the tag and will leave it to to be triaged.

I've updated the documentation at https://www.mediawiki.org/wiki/Extension:WikiLambda/API#wikilambda_perform_test. I think we can now close this ticket.