My guess here is that this is not an error at all; there is simply a timeout in the comparison of the (correct) actual results with the (correct) expected results. We could clarify this by having a different error according to where the timeout occurs, but if it occurs before the comparison, we get an error in the actual result, so then they’re not the same.

In T359664#9709817, @AAlhazwani-WMF wrote:

In T359664#9709254, @GrounderUK wrote:

Thank you 🙏

In T359664#9706260, @AAlhazwani-WMF wrote:

…we also explored a drag-and-drop interaction…

That looks pretty classy!
🤔…but a purely drag-and-drop solution has accessibility implications, so “insert” should be the priority.

totally agree! that was one of the reasons that made us pause a little longer on drag-and-drop, and wait for a more solid and accessible solution from Codex. as far as i can tell there are ways to make this interaction accessible via keyboard-only[1][2], but we'd like to test this extensively, and leverage the design system team extensive know-how on a11y best practices.

[1] https://dev.opera.com/articles/accessible-drag-and-drop/
[2] https://pressbooks.library.torontomu.ca/wafd/chapter/sortable-lists/

Thank you 🙏

In T359664#9706260, @AAlhazwani-WMF wrote:

…we also explored a drag-and-drop interaction…

That looks pretty classy!
🤔…but a purely drag-and-drop solution has accessibility implications, so “insert” should be the priority.

In T362278#9708152, @DVrandecic wrote:

…The implementation in https://www.wikifunctions.org/view/en/Z15723 should obviously not exist: it is merely a wrapper around the function itself

Please consider supporting “insert item”. It looks like adding a new item near the start of a list would not be a great experience otherwise.

In T361654#9695734, @cmassaro wrote:

Hmm, okay. So what are the ZIDs of a function and implementation that should currently work but don't? How can I reproduce the bug right now?

So I created a sandbox composition with echo, and this works correctly. https://www.wikifunctions.org/view/en/Z15160

I created a comparable error with a sandbox implementation here: https://www.wikifunctions.org/view/en/Z15106.

Yes, it is strange. When https://www.wikifunctions.org/wiki/Z14717?uselang=en&oldid=95554 (which is mapped as you would expect) produced erroneous results, I changed the composition so that all eight elements were mapped from the first argument (set to True). Then I set the second argument to False and tried changing the mapping for each element in turn. This confirmed that when its value was mapped into any of the first three elements, the implementation evaluated correctly to False, but if its value was mapped to any element from the fourth to the eighth, the implementation evaluated incorrectly to True.

In T358589#9690869, @cmassaro wrote:

…For user-defined types and generic types, this will be true unless and until we decide to banish arbitrary function calls from the type calculus.

In T358589#9690869, @cmassaro wrote:

…I'll wait for others weigh in.

In T361654#9690806, @cmassaro wrote:

Ah! I see. I was running it with all of the Boolean argument values set.

In T290595#9690130, @Amire80 wrote:

Which problem does this solve? I can more or less guess some, but it needs a clearer definition.

In T360904#9689310, @Jdforrester-WMF wrote:

Now deployed, so should be able to re-test and get further details.

This function has seven implementations and five tests. All seven implementations pass all of the tests and each test passes all seven tests. The test results for the function have yet to complete, however (still “running” after five minutes – I looped back round and achieved 100%)

In T358589#9678132, @cmassaro wrote:

My two cents:

I would be careful about us doing too much type-checking in the frontend.

In T361654#9688889, @cmassaro wrote:

@GrounderUK , are you still seeing this behavior?

Please see https://www.wikifunctions.org/wiki/Wikifunctions:Project_chat#Question_regarding_identifying_failing_or_unconnected_tests. If that discussion goes no further, my feeling is that a reason for disconnection should always be given (whether it is a Test or an Implementation that is being disconnected).

In T282062#7074909, @Jdforrester-WMF wrote:

Is this properly a property of the Key or of the Type which uses it?

Yes. And no.
Please see my opinion at https://www.wikifunctions.org/wiki/Wikifunctions_talk:Representing_identity#Proposal_6:_Multi-typed_lists

We should include the implementation fail ratio for each test (simply the number of failing implementations over the total number of implementations, perhaps also expressed as the result of the division).
If we could also identify tests that had previously passed but currently fail, that would be especially interesting… maybe that would be a diff between successive reports or between the current view and a prior baseline? (So we could have a latest date for each test/implementation combination, by test, implementation and/or function… linked to the metadata — oh, just what we should have on the function page, really!)

Hint: as far as I can tell, the problem occurs only in Testers, not in straightforward evaluations.

May help with T359772 (but that also mentions descriptions)

I’ve added a paragraph at https://www.wikifunctions.org/wiki/Wikifunctions:Naming_conventions#Search_characters and linked to that from https://www.wikifunctions.org/wiki/Wikifunctions_talk:Best_practices#Wikifunctions:Naming_conventions, linking to this ticket.

Thank you. I’ll document this somewhere on-wiki (any ideas where?) and then we can close this.

I encountered this error trying (lazily) to join a Natural number to a string. It disappeared when I converted the Natural number to a string first.

Please publish this intention on https://www.wikifunctions.org/wiki/Wikifunctions:Status to save people reporting bugs in the mobile interface.

I think we remembered to do this. See https://www.wikifunctions.org/view/en/Z14195.

Is there a correct way for me to suggest that “Verify in production” partially failed?
@Jdforrester-WMF (please see comments above)

In T360001#9634127, @AAlhazwani-WMF wrote:

that could work! i would also like to explore how might we include the zid directly to about widget, rather than the mediawiki header.

In T335583#9632190, @Amire80 wrote:

…

Can they be changed into "label" and "short description", or better, just "description"?

In T359802#9631817, @Jdforrester-WMF wrote:

When you run the tests during the edit, those values are uncached (as the text isn't saved). This is expected behaviour, but maybe we could mention that it's not cached for the very few people that might notice?

Yeah, but we shouldn’t do only this. Most users will want to do this *without* switching to qqx (“see it; fix it!”).

Maybe enable right-click and mobile equivalent?

…and here:
https://www.wikifunctions.org/wiki/Z13398?action=edit&uselang=en

It looks like this is a pre-existing problem that also applies to Booleans.

See also T359500. Untyped lists are converted to Typed list with Type Natural number.

Please also see https://www.wikifunctions.org/view/en/Z13725 , which I created to demonstrate the extent of this problem. It seems that a list of objects containing only Natural numbers will convert itself into a Typed list with Type Natural number.
{F42441441} (running Z12668 with an untyped list that is evaluated to a Typed list, despite the function’s signature.)

Bug observed in the wild:
https://www.wikifunctions.org/wiki/Z12190?action=edit

Thanks, Denny. I don’t disagree with what you say, but I’m not sure about your new title.

That seems to have fixed the problem for a single pipe but not for more than one (separate or together). In https://www.wikifunctions.org/view/en/Z13371, there is a pipe in both the “English word” string (Z13254K1) and in the expected result (Z866K2), so the bug appears when refreshing test results in edit mode. If either pipe is removed the test fails correctly with details of the mis-matched strings.

Functions with no label or description in a given language
Functions with keys that have no label or description in a given language (see https://www.wikifunctions.org/wiki/Z12967?uselang=ig&oldid=83821, for example)
Functions with input labels that have no value entered in a given language.

We should also consider finding functions with or without labels in a particular language. Perhaps those with labels in a fallback language other than (prior to) English would be an interesting further division, indicating the fallback language.

In T358089#9569550, @GrounderUK wrote:

In T358089#9569134, @Jdforrester-WMF wrote:

This sounds like a horrendous problem…

That link currently exhibits the problem. I don’t actually see the pipe in there but it has always failed whenever I have entered that character.

I have saved Z13091 with a pipe in a comment. To be clear, the implementation always works; it’s just that all the tests show as Failed (erroneously and with no details and only during the edit). If I introduce the pipe during an edit, the tests will not fail until I click the refresh icon in the Tests box. Similarly, they eventually pass (on refresh) if I remove the pipe character.

Created https://www.wikifunctions.org/view/en/Z13371

In T358089#9569134, @Jdforrester-WMF wrote:

This sounds like a horrendous problem, and we've had issues in the past, but I can't replicate – if I edit https://www.wikifunctions.org/wiki/Z13091?action=edit to add a # | comment, or a a = 2|3 line, either way the function appears to work? Can you give a specific case that fails for you? (Or does it come and go?)

It would be especially helpful if I could do this while editing an Implementation. It wouldn’t need to be connected automatically but that is worth considering too (as an option).

“Cancel” simply cancels the edits. This flips you straight back into “read” mode on Wikipedia and Wikidata. On Wikifunctions, for existing functions, we get a “Discard Edits” confirmation dialogue first, then we are returned to the page in “read” mode. We cannot be consistent with this for a page that does not exist, but a reasonable expectation after such a dialogue prompt is that the edits are discarded and you are left on the page as it was before any edit. Anything else would imply a third option in the “Discard Edits” dialogue, which is effectively “Back”.

This seems to be a Vector (2022) problem; it does not apply to Vector legacy (2010). In legacy, the left sidebar has a Languages section with a gear icon that works correctly. In Vector 2022, this is replaced by a message linking to the non-functioning widget, so I guess that provides a good clue about what the widget should be doing. I could be wrong, but I think it is just the Universal Language Selector, in which case it should always appear (and work).

You can see some attempts to make edit summaries less mono-lingual in my more recent contributions.

Update.
The Z107 problem seems to have disappeared but the “Argument type mismatch” error persists on the Implementation page. (It is now only possible to fail to evaluate the function once on this page, since the button is disabled (and argument input boxes are not available). An associated Implementation is shown as “not associated”.)

In T306383#8201887, @gengh wrote:

This bug may prove difficult to replicate after T315448 has been fixed.

Note: I think T315233 fixed this.

I removed this:
“The Compositions that can execute successfully seem not to have the Evaluate Function box on their Implementation page (which is https://wikifunctions.beta.wmflabs.org/wiki/Z14). See, for example, https://wikifunctions.beta.wmflabs.org/wiki/Z10056.”

Comment received via Telegram:
<James_F> The page 'Z14' is actually a Z2/Persistent object encapsulating a Z4/Type representing the concept of Z14/Implementations; so Z14 has a top-level Z1K1 of 'Z2', and a second-level (within the Z2K2) Z1K1 of 'Z4'. Not sure if that explains?

Clarified here [https://meta.wikimedia.org/wiki/Talk:Abstract_Wikipedia/Function_evaluator_call#The_normalized_type]

Should this include other special cases where there is a single element that is effectively null? For example, if a list of strings contains only {"Z1K1": "Z6", "Z6K1": ""}, does this reduce to Z13/Empty?

Nesting of Z6/Strings seems perfectly legitimate whilst a “raw string” return type does not. An explicit nesting should probably be removed during normalization but the evaluator cannot rely on this unless it performs the same reduction itself (as I imagine it should) after evaluating nested function calls.

I’m wondering whether it would make sense to think of a Z4K3/validator (function) as a constructor for validated instances of objects of the specified type. In practice, this would add an additional key into validated instances, with its value being the list of Z5/Errors (an empty list in the case of a valid object). For example, the valid positive integer represented by "2" would be the object {"Z1K1": "Z70", "Z70K1": "2", "Z70K2": []}. The unvalidated instance, on the other hand, would be a Z7/Function call to the Z70 validator/constructor function, passing "K1": "2" (and expecting type+value+errors – a validated Z70 instance – in return). This ensures that only the validator/constructor function can provide the Z70K2/errors value.

Only for a built-in Z2?

To be consistent, we might normalize a function call that should return a string, for example, as the value for a Z6K1. That is (maybe): {“Z1K1”: “Z7”, ...} normalizes to {“Z1K1”: “Z6”, “Z6K1”: {“Z1K1”: “Z7”, ...}}, just as the simple string “Wikifunctions” normalizes to {“Z1K1”: “Z6”, “Z6K1”: “Wikifunctions”}.

It seems to me that the “guidelines” envisaged are not “for external experts” but for us: “guidelines on working with external experts for the Abstract Wikipedia team and community” (or guidelines for [us] on [that]).

I wonder... Are we talking only about errors here? Or should we consider “messages” more broadly? For example, a correct evaluation could be accompanied by information about the cost of evaluation (which might be information or a warning), or the use of a deprecated or suspect object (which might be an error in some situations and not in others).

I think that makes sense. For my first option, I should have stated that the change to the referenced function’s return type would be from the more general to the more specific (e.g. from Z6/String to Z?/2-string).

Let's consider lazy evaluation.

Thanks for that, James. I may come back to this through T269182, but Happy Holidays in any event!

In T260861#6702579, @Jdforrester-WMF wrote:

In T260861#6701825, @GrounderUK wrote:

Just a thought... Do we envisage a one-to-one relationship between Z4/Type and its Z4K3/validator, or can the same validator function be reused on multiple Z4/Types (which seems to imply a supertype, but perhaps we want to keep that concept implicit for now)?

I imagine you might re-use some of the sub-functions of the validators, e.g. the validator for the type Zn for floating point numbers will have a function checking that ZnK2 is a string that represents an IEEE double-precision float (or whatever), and that would also be useful for the type Zm for complex numbers on values ZmK2 and ZmK4 or whatever.

Just a thought... Do we envisage a one-to-one relationship between Z4/Type and its Z4K3/validator, or can the same validator function be reused on multiple Z4/Types (which seems to imply a supertype, but perhaps we want to keep that concept implicit for now)?