Please also see https://www.wikifunctions.org/view/en/Z13725 , which I created to demonstrate the extent of this problem. It seems that a list of objects containing only Natural numbers will convert itself into a Typed list with Type Natural number.
{F42441441} (running Z12668 with an untyped list that is evaluated to a Typed list, despite the function’s signature.)

It also seems that an untyped list containing a Natural number along with any other type of object (which remains “untyped”/Type “Z1”) will fail in existing equality functions.

It looks like this is a pre-existing problem that also applies to Booleans.

See https://www.wikifunctions.org/view/en/Z13755
(The function’s return type is “Z1”.)

I found the semi-timeout in a simpler scenario, so am splitting that off as a separate bug: https://phabricator.wikimedia.org/T360385

I am running this locally (but using the wikifunctions.org API for ZID resolution). When I run the calls that time out, it takes 5066 ms.

We resolve 5 ZIDs: Z13717, Z13721, Z13720, Z13528, Z610.

We could avoid resolving Z610; this is the ZID corresponding to a programming language, and there is already work underway to use ZIDs instead of programming language literals.

We could also have a persistent cache in the orchestrator service itself (currently, we only have an ephemeral per-request cache).

What I can't figure out is how long the calls from the frontend to the orchestrator, and from the orchestrator to the evaluator, take in production. That could be a big bottleneck.

I don't understand why Z13528 (Test: 1+0=1) is on your list. That seems quite distantly related to this.

Looking at this again, I wonder if this is ultimately because lists of natural numbers are not yet fully supported.

T359233

No, this isn't related to type conversion. I'm able to run it successfully in my local environment, where the only real difference is the co-location of services (so no latency between the API and the orchestrator or between the orchestrator and evaluator).

Both tests in the initial report are now working for me. @GrounderUK can you check the ones you expected were related? Maybe we can close this?

In T359500#9963802, @99of9 wrote:

Both tests in the initial report are now working for me…

It depends what you mean by “working”… Your third point in “What happens” still applies: the function’s return type is a list of Z1s but the code implementations return a list typed as Natural number. I’m happy to pick that up separately and link back here but it’s not my highest priority yet (please see my reply to https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Typed_lists_now_open_beyond_Booleans_and_Strings).

Given what we've recently discovered in T370028, I don't actually understand how the python implementation is successfully passing Z13723.

In T359500#9988999, @99of9 wrote:

Given what we've recently discovered in T370028, I don't actually understand how the python implementation is successfully passing Z13723.

It succeeds because the replacement value list has Z1 as its type, so the numbers are not converted to integers. If you change the list’s type to Natural number then the test times out in the evaluator.

In T359500#9989069, @GrounderUK wrote:

In T359500#9988999, @99of9 wrote:

Given what we've recently discovered in T370028, I don't actually understand how the python implementation is successfully passing Z13723.

It succeeds because the replacement value list has Z1 as its type, so the numbers are not converted to integers. If you change the list’s type to Natural number then the test times out in the evaluator.

I've changed the list's type (https://www.wikifunctions.org/w/index.php?title=Z13723&diff=168391&oldid=139443), but now the errors on https://www.wikifunctions.org/view/en/Z13723 are not timeouts. They seem like the usual issues with code conversion of lists?

In T359500#10562111, @99of9 wrote:

In T359500#9989069, @GrounderUK wrote:

In T359500#9988999, @99of9 wrote:

Given what we've recently discovered in T370028, I don't actually understand how the python implementation is successfully passing Z13723.

It succeeds because the replacement value list has Z1 as its type, so the numbers are not converted to integers. If you change the list’s type to Natural number then the test times out in the evaluator.

I've changed the list's type (https://www.wikifunctions.org/w/index.php?title=Z13723&diff=168391&oldid=139443), but now the errors on https://www.wikifunctions.org/view/en/Z13723 are not timeouts. They seem like the usual issues with code conversion of lists?

That was what I was expecting (rather than the timeout). It “works” with a Z1-list input because there is no type conversion, so the replacement Natural numbers come back as Natural numbers. It fails with an N-list input because there is no identifiable type converter from code to convert the Python integers into Natural numbers (because the function specifies a Z1-list as output).