Page MenuHomePhabricator

Provide pre-defined testers for pre-defined functions
Open, LowestPublic

Description

Lindsay suggested we do this in the Z8xxx range, with Z801 having up to ten pre-defined testers: Z8010, Z8011, … Z8019. This way we can run a test suite of the built-in function implementations.

  • Z801
  • Z802
  • Z803
  • Z805
  • Z808
  • Z810
  • Z811
  • Z812
  • Z813
  • Z821
  • Z822
  • Z844
  • Z866
  • Z868
  • Z881
  • Z882
  • Z885
  • Z886
  • Z888
  • Z899

Event Timeline

ArthurPSmith added a subscriber: ArthurPSmith.

I'll start taking a look at this!

I really like the idea of having the tests for Z801 be in the Z801x range, Z802 in Z802x etc.

This also means we are cutting the space of languages from Zxxxx to Z(1-7)xxx, but that should be OK for quite a while. We still have the Z9xxx space if needed.

Assigning this to @ArthurPSmith as discussed in today's meeting. If you have questions, feel free to ask, or if you change your mind feel free to unassign.

Thank you!

Ah, you already did claim it! Thanks!

I've started working on some of these - however, it would be nice to actually be able to run the tests to confirm what I'm doing makes sense. But when I click the button to do so right now I get a Failure with the following message:

Failure reason: wikilambda_perform_test-not-connected

I guess I'll dig into this but if anybody knows right at the moment what the problem is and how to fix that would be nice, thanks!

Ah, the problem is every function call is returning that same error - the orchestrator is not responding for some reason..

Tracked it down at least one step: the mediawiki container doesn't use the full container name 'mediawiki_function-orchestrator_1' to talk to the orchestrator, rather just the name provided in the compose file - 'function-orchestrator'. Is this perhaps because I'm using a different version of docker-compose? I think my docker versions are quite up to date...

And now I'm getting (for every function call) - "error type: wrong content type
type: wrong content type
String The provided object is not a function call" Hmm.

I've started working on some of these - however, it would be nice to actually be able to run the tests to confirm what I'm doing makes sense. But when I click the button to do so right now I get a Failure with the following message:

Failure reason: wikilambda_perform_test-not-connected

I guess I'll dig into this but if anybody knows right at the moment what the problem is and how to fix that would be nice, thanks!

Yeah, we just changed the default config to expect the names that docker-compose 2.x would generate (from _s to -s); previously it used the ones from docker-compose 1.x. Sorry!

The wrong content type error might be because we just last week changed fundamentally the Z2K1 validation step from a reference-as-string to an object reference. You might need to re-load all your data. :-(

You might need to re-load all your data. :-(

Actually when I started running into problems (including issues related to the changes with the Void and Unit ZObjects) I reloaded everything from scratch yesterday. So that can't be the problem right now.

But I just updated function-orchestrator again and things seem ok. So I guess a temporary problem with function-orchestrator. Anyway, all is working now!

Change 735622 had a related patch set uploaded (by ArthurPSmith; author: ArthurPSmith):

[mediawiki/services/function-schemata@master] Add some testers

https://gerrit.wikimedia.org/r/735622

Change 735622 merged by jenkins-bot:

[mediawiki/services/function-schemata@master] Add some testers

https://gerrit.wikimedia.org/r/735622

Change 736491 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/services/function-evaluator@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736491

Change 736492 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/services/function-orchestrator@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736492

Change 736493 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736493

Change 736493 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736493

Change 736491 merged by jenkins-bot:

[mediawiki/services/function-evaluator@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736491

Change 736492 merged by jenkins-bot:

[mediawiki/services/function-orchestrator@master] Update function-schemata sub-module to HEAD (1e03e46)

https://gerrit.wikimedia.org/r/736492

Change 740660 had a related patch set uploaded (by ArthurPSmith; author: ArthurPSmith):

[mediawiki/services/function-schemata@master] Add more builtin testers for basic builtin functions

https://gerrit.wikimedia.org/r/740660

Change 740660 merged by jenkins-bot:

[mediawiki/services/function-schemata@master] definitions: Add more pre-defined testers for basic built-in functions

https://gerrit.wikimedia.org/r/740660

Change 741104 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/services/function-evaluator@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741104

Change 741105 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/services/function-orchestrator@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741105

Change 741106 had a related patch set uploaded (by Jforrester; author: Jforrester):

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741106

Change 741106 merged by jenkins-bot:

[mediawiki/extensions/WikiLambda@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741106

Change 741105 merged by jenkins-bot:

[mediawiki/services/function-orchestrator@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741105

Change 741104 merged by jenkins-bot:

[mediawiki/services/function-evaluator@master] Update function-schemata sub-module to HEAD (2e912e6)

https://gerrit.wikimedia.org/r/741104

Jdforrester-WMF lowered the priority of this task from High to Lowest.Jan 18 2024, 5:51 PM

I've added some tests to a few of those still without tests. It would help if someone could link them.