Ok, I got federation to work - sort of. From the main query service I can query the scholarly subgraph - but if I try to use the resulting values I always get a timeout.

Hi - how does the federation work? I'm experimenting with this by trying to get the list of names of authors on a scholarly article - the article data itself is in the scholarly article subgraph, but the human items for the authors are in the main one. So I need to do a federated query but it's not clear how? Can you provide an example? Do I start on the main graph and federate to the scholarly one, or vice versa?

It certainly would be good to get this fixed. However, I think this points up a fundamental problem with some of the more complex data structures supported by Wikidata (quantity ranges are a similar case, and probably some of the lexeme structures as well). Namely - this COULD have been implemented as just a qualifier on coordinate location (P625) - either using P376 or something more directly appropriate. But instead it was implemented as an internal structure within Wikidata that requires direct coding within Wikibase to properly support it, and still after many many years is not supported in the Wikidata UI. I think a better longer-term solution here is to plan to sunset these more complex data structures that could simply be handled with properties/qualifiers. One way or another this NEEDS to be fixed though!

I rely heavily on my Wikidata Watchlist. I generally just look at the last 7 days but longer is useful sometimes. I think the proposed 3 and 4 solutions above (splitting out autopatrolled changes) would be a good solution for my purposes - 99% of edits I check are the unpatrolled ones, so if we could keep 30 days of unpatrolled changes and cut the other to 14 days or less I think that would be fine for me. By the way I find ORES essentially useless to evaluate the quality of Wikidata edits.

Note - I've updated to the latest versions of everything as of today (Jan 10) and this is still broken. Moreover, due to changes related to lists (Z10) a bunch of the testers are not working, but I can't update or test them due to this problem!!

I see the merit of this idea at least for properties, but I'm wondering where you envision the property discussion to take place? On the talk page? Would that be preserved somehow (referring back to old proposal discussions is done very often).

@Jdforrester-WMF yes, this looks good to me now too. This can close.

In T296208#7521080, @Jdforrester-WMF wrote:

We are locked specifically to 5.3.6 because that's the version that passed security review; per https://libraryupgrader2.wmcloud.org/library/composer/symfony/yaml?branch=main there are a few other users in devdeps and deps but none that should hard-conflict. Did you manually add the dependency, or did you use a composer.local.json file?

It's possible this is only happening for types within a List? I just tried something similar for the types of values of a Pair, and type selection there worked fine.

Good points from @MisterSynergy and others above. One other case I often run into is problems caused by item merges; if both original items had P279 statements this can cause significant trouble (for example it is a common source of subclass loops).

Hmm - I agree with the above that P2860 should not be on this list. If we are including the "partitive" properties like P361 and P527 (taxonomic in the sense that they group parts of something with the whole), what about P355 (subsidiary) and P749 (parent organization), which are used that way for organizations, or other properties of that sort?
On the other hand your list does not include the truly taxonomic property P171 (parent taxon) - which is explicitly a subproperty of P279. P10019 (term in higher taxon) seems to also be a subproperty of P279.

Screenshot of a passed test:

But I just updated function-orchestrator again and things seem ok. So I guess a temporary problem with function-orchestrator. Anyway, all is working now!

You might need to re-load all your data. :-(

Actually when I started running into problems (including issues related to the changes with the Void and Unit ZObjects) I reloaded everything from scratch yesterday. So that can't be the problem right now.

And now I'm getting (for every function call) - "error type: wrong content type
type: wrong content type
String The provided object is not a function call" Hmm.

Tracked it down at least one step: the mediawiki container doesn't use the full container name 'mediawiki_function-orchestrator_1' to talk to the orchestrator, rather just the name provided in the compose file - 'function-orchestrator'. Is this perhaps because I'm using a different version of docker-compose? I think my docker versions are quite up to date...

Ah, the problem is every function call is returning that same error - the orchestrator is not responding for some reason..

I've started working on some of these - however, it would be nice to actually be able to run the tests to confirm what I'm doing makes sense. But when I click the button to do so right now I get a Failure with the following message:

I'll start taking a look at this!

@DVrandecic I just created an implementation on my copy (just updated with the latest versions of everything pulled from gerrit). Seems like maybe it was a temporary problem?

Of course this is a very general problem - how do you supply the arguments to a function that is returned from a function call (or perhaps just constructed in place), rather than being a persistent ZObject with an id? Do we have any kind of general solution for this now? I was looking through documentation on metawiki and couldn't find anything...

@DVrandecic I'm trying to work through an example of how, for example, a list of strings would be implemented; is there a document anywhere we have that goes into this in detail? The sticking point I'm on is what would the keys for a specific value of the generic type look like. If the generic list type function is Z9999, then its main key (Z9999K1) would be just the type of the list, right? But that specific value of the type would also have keys - for a list it has "Head" and "Tail" keys. Presumably when you do the function call with Z9999K1 = Z6 (string) then it returns a Z4 object that looks a bit like our current Z10, but with the type for key Z10K1 as Z6 instead of Z1, etc. But how would the "Z10K1" key be referred to now? That is, in this section of what is now in Z10:

In T282020#7339299, @Lindsaykwardell wrote:

Do we still want to do this? Or should we mark as resolved for now and wait for Aishwarya's designs?

This looks to me like it's done - Z802 for example links to Z8 both in view and edit modes - ok if I close it?

I think this is done now, right?

I think we're done here right? Thanks!

I'll take a look at this unless somebody else was itching to work on it...

Something like that. Also the link should be removed if all references to one ZObject from another are removed.

Actually currently it always says "ZObject: Z2" - because it's calling $zobject->getZType() and they are all Z2's at that level.

If I understand this correctly, the idea is to show the function signature, for example for Z813 ("empty") the Vue code displays "( list: List ): Boolean" - so we want to place that string somewhere near the top of the page under the title...

It's working on notwikilambda! Closing...

Maybe more fundamentally the regex matches should all be adjusted so it's looking for Z[1-9]\d* rather than Z\d+ as a valid Z id?

Hmm, another strange case is search for L:Kelly - the 3 current matches are for L404650, L361948 and L230178, none of which seem to have the string "kelly" in them. So there's some sort of stemming going on here in addition to the case insensitivity, which would be nice to be able to avoid if possible.

Denny's posted this notebook: https://public.paws.wmcloud.org/User:DVrandecic_(WMF)/Lexicographic%20coverage.ipynb which does pretty much the above for the language Wikipedia corpora. Results at https://www.wikidata.org/wiki/Wikidata:Lexicographical_coverage
However, I don't think he wants to keep running it, so can we move it somewhere that it will be run regularly by a bot or something? The coverage/completeness data is helpful, and the 'missing' page for each language is a great guide to editors, if it could be kept reasonably up to date.

From the discussion it sounded like we didn't want to (partially) address this by allowing null values for keys (JSON does allow 'null' so it could be done that way, though the UI currently doesn't have a mechanism for entering a null, and further changes would still be needed in ZObjectFactory anyway) - rather the optional key should just be absent from the ZObject. If key had an additional boolean attribute "optional" (itself optional, default false) that would partially address the problem, but not the case where we want exactly 1 of N specific keys to be not null (as for implementation). Eventually each type should have its own in-wiki validator that can take care of all this. There are possibly other mechanisms being considered also. So maybe simplest would be for ZObjectFactory to just treat every key as optional for now?

@Jdforrester-WMF Z501 is now loading, but Z13 still not - you can see this on notwikilambda right now for instance - https://notwikilambda.toolforge.org/wiki/Special:AllPages?from=&to=&namespace=2468

@Jdforrester-WMF the patch I just uploaded does also fix T276836: Cannot edit Z4/Type - the issue there was indeed the change in normalization. I'm not sure however if my fix for that (denormalizing strings and references in the key validation check) is ideal... but it does seem to work. I added tests that fail without these changes and pass now.

In T276841#6921778, @Jdforrester-WMF wrote:

I'm assuming this is the same issue as T276836: Cannot edit Z4/Type, and that the recent changes to the normalisation code in the front-end API means the operations made clientside are no longer valid?

I see the same problem. Looking into it now!

Just a note - this has been added to the preloaded ZObjects, but it does not load (missing on notwikilambda.toolforge.org for example) due to the reference to Z7, which does not exist yet. It would probably be helpful to have some indication during the update/load process that loading a particular ZObject like this failed...

After discussion James suggested keeping track of the validation context (basically I think option 4 above) - I'll look into this.

The above patch removes the type registry check (option 3). But maybe another solution would be better in the long run?

Ok, I've tracked down the source of the problem... It is indeed an infinite recursion issue, in the validation process.

Note that Z6K1 has value type Z6 and that doesn't cause this problem.

This is interesting - I'll take a look at it!

Some of my notes on other things the select widget should do that it doesn't do right now...:

TypeSelector has been replaced by SelectZobject, so this can close!

I think we're done here now? Thanks for all the help guys!

which was fixed in a patch. All this now deployed!

See also https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikiLambda/+/656176 which removes the remaining dependencies on the hard-coded types list except for the following:

this.keylabel = editingData.ztypes.Z3;

in OtherKeys.vue.

Marking as resolved then, thanks!

@gengh does this problem seem fixed now?

Ok, I think the amended commit in gerrit now fixes this issue. I looked into the question of why the "otherkeydata" was used, and the problem is the way this component is designed we need something that has all the keys in zobject EXCEPT the Z1 and Z2 keys which are handled in the FullZobject component. If we decide that's unnecessary then we can look into replacing it simply with zobject in this component. Another alternative is maybe to create a computed object which is zobject minus those specific key-value pairs.

Able to reproduce now, thanks! It seems to be specific to string values? Not quite sure what's going on; as to why having two different objects representing the same structure; it's probably from the evolution of my thinking about how to do this and learning how vue works. I'll see if I can make the change as you imply, I guess using the zobject value directly?

Above change doesn't address the main question here.

Oh yeah, I had noticed this was broken and wasn't sure the cause... This gives me a clue on the cause but I'm not sure how to fix. Will think a bit about it but feel free to propose a specific solution!

Things still seem to be working properly and at least the 5xx errors have disappeared, so i think we can say this is resolved now. It would be nice to know what caused it though!

@Kizule and @dcaro - thanks for checking. Yes, the problem appears to have gone away now (8:52 AM EST). But the 11 min load @dcaro saw sounds like a real problem - the examples come from a simple Wikidata Query Service query and it does nothing with them other than list them as links, so there's no way they cause trouble. I'm still wondering why there seem to be a large number of 4xx errors shown in grafana for the last day when I see only 3 in the access log on toolforge.

@gengh this is great, thanks! I was hoping we'd get vuex pieces some time, definitely needed. I'm a little stalled right now on what I was working on because I need to convert a zid to a label (also a very general problem, but this is needed to remove TypeSelector.vue and the dependency on the hardcoded type list); ok if I look at adding that to your key label support?

Or just test a json array directly - it gets entered (because it is a valid zobject) but causes trouble because it's not a valid *persistent* zobject:

var api = new mw.Api(); var test = api.postWithEditToken({action:'edit',format:'json',title:'ZObject:Z12366', text:'["test", "test2"]', contentformat:'application/json', contentmodel:'zobject'});

I.e. if you try to save an object that looks like {"Z1K1": "Z10", "Z10K1": ...} that would also fail because it's not in the persistent object format (Z2) but rather a list (Z10) at base. Or any other type that's not Z2.

hi @Jdforrester-WMF - don't we require the base persistent object always to look like {"Z1K1": "Z2", "Z2K2": (rest of object)} ? Also I'd prefer that the "Z2K1": id key always be defined and match the page name...

Although actually trying to do this, it appears the types listed in the table are not the ZObject id's, but string values - so instead of Z4 you have to query for "ZType". Hmm. That doesn't seem like the right choice long term but I guess we can work with it for now.

So having worked on an autocomplete interaction for the reference type, it seems to me that API (list=wikilambda_searchlabels within the mediawiki query API) could be usefully extended to restrict search results by type; with no query string that could return all ZObjects (up to a limit) of a given type. It already returns id, label combinations for a given language. With type = Z4 that would satisfy this ticket...

Change 633560 in gerrit has this adjustment - I'm not sure how to link it to this ticket though?

From stand-up today I'll look at modifying the Vue components to allow a "view" mode as opposed to edit mode.

I think this should be dealt with by having the default value for Z2K2 be null, not '' (an empty string). Alternatively, just remove the Z2K2 from the default Persistent ZObject.