Page MenuHomePhabricator

wikibase RepoApi disappeared
Closed, DeclinedPublic

Description

wikibase.api.RepoApi is essential JS module for accessing Wikidata from JS API.
It is basically a wrapper for ForeignApi allowing client side scripts to query Wikibase instances.
I'm not sure why - but it just doesn't appear anymore.

How to reproduce

  1. Goto enwiki
  2. Ask for RepoApi
mw.loader.getState(  'wikibase.api.RepoApi')

It doesn't appear anymore and is being used across wikis. Including, but not limited to:
https://fr.wikipedia.org/wiki/MediaWiki:Gadgets-definition
https://he.wikipedia.org/wiki/%D7%9E%D7%93%D7%99%D7%94_%D7%95%D7%99%D7%A7%D7%99:Gadget-Checkty.js

Event Timeline

Probably regression following T256054
Doesn't it appear in error logs of RM in the server side?

Hey,
Technically it's intentional. RepoApi is a bad module. It's confusing (which repo? Commons is also a repo) and doesn't bring much value (you can use mw.Api instead in WMF, as the URL to wikidata.org likely won't change) and having an extra module brings its own cost. (Read more), basically causing 30G/day extra traffic for all of Wikipedia readers. Also, it's worth noting the code behind is pretty old and buggy and we would like to get rid of it altogether but in the mean time you can still mark jquery.wikibase.linkitem as the dependency instead and use the global variable but I don't recommend it.

In non-repo wikis:

ladsgroup@mwmaint1002:~$ mwgrep RepoApi
frwiki              MediaWiki:Gadget-Wbi-Item.js
frwiki              MediaWiki:Gadgets-definition
hewiki              MediaWiki:Gadget-Checkty.js
hewiki              MediaWiki:Gadget-Rechtschreibpruefung.js
hewiki              MediaWiki:סקריפטים/105.js
hewiki              MediaWiki:סקריפטים/106.js
hewiki              MediaWiki:סקריפטים/98.js
ruwiki              MediaWiki:Gadget-wefinfobox.js

ladsgroup@mwmaint1002:~$ mwgrep RepoApi --user
dewiki              User:Fomafix/wikidata-redirect-linker.js
dewiki              User:ערן/spellcheck.js
enwiki              User:ערן/TemplateDataI18n.js
enwiki              User:ערן/spellcheck.js
hewiki              User:1Or/sandBox.js
hewiki              User:1Or/wikibase.js
hewiki              User:בורה בורה/מדיה ויקי:Gadget-Checkty.js
hewiki              User:חסר כינוי/common.js
hewiki              User:ערן/HarvestLabel.js
hewiki              User:ערן/common.js
metawiki            User:Jeblad/spellchecker/script.js
metawiki            User:Manu1400/AuthorityControl.js
metawiki            User:ערן/spellchecker.js
nnwiki              User:Jeblad/spellchecker/script.js
ruwiki              User:Vlsergey/common.js

I highly recommend not using this module anymore. Also according to stable interface policy, javascript code is unstable and prone to change with prior notice

I don't find RepoApi as bad module - it get in the constructor the repository - repository could be Wikidata, or Commons or other Wikibase site.

Also I don't agree with the arguments on heavy load - RepoApi can be lazy loaded on user action - upon pressing on linkitem.

Also I don't agree with the arguments on heavy load - RepoApi can be lazy loaded on user action - upon pressing on linkitem.

Please read the blog post, the cost of registering a module is big specially on all of wikipedia even if it doesn't end up loading the module on every request. Because the startup module doesn't get cached and has to be sent in every request. It's just a name and dependencies and a hash but registering a module in all of wikis adds up to 30 GB a day extra traffic (zipped).

Could you please help me understand what you'd like to do that is no longer possible now to achieve what? That'd at least help me figure out how to move on with this. Thanks!

Intentional removal of a never-supported piece of code is not a Regression imho.

  • Wikidata (wikidata client only) and communities (frwiki, hewiki, ruwiki) code is misaligned by removing JS API interface to wikidata that is used by gadgets -> functionality broken -> regression. to get it resolved we need to re-align them
    • communities code can adapt to new API-less wikidata (develop their own wrappers to API)
    • wikidata code can return removed code
    • (or anything in between)

Amir - the point that I think that is missed here is that API, unlike other modules is intended to expose functionality for developers for extending capabilities.
If API is useful for Wikidata itself (entityChangers, edittoolbar, formatters, parsers, PageConnector) - it is useful.
If API is deprecated for Wikidata itself - e.g in the long term we would like to get rid of it also in Wikidata repository itself - communities should be aligned with that

There could be pros and cons for each direction (less JS code to maintain - keeping only mw.Api, more easily to locate code that use API etc - having mw.RepoApi, alignment to mw ecosystem with core/resources/src/mediawiki.api resources that expose API to JS). and I think a good discussion on this is missing, and this is not just "performance" which doesn't seem to be good argument in regards to API resource.

Wikidata code that use RepoApi:

client/resources/jquery.wikibase/jquery.wikibase.linkitem.js
client/resources/wikibase.client.PageConnector.js
lib/resources/wikibase-api/Gruntfile.js
lib/resources/wikibase-api/README.md
lib/resources/wikibase-api/src/FormatValueCaller.js
lib/resources/wikibase-api/src/ParseValueCaller.js
lib/resources/wikibase-api/src/RepoApiError.js
lib/resources/wikibase-api/src/RepoApi.js
repo/resources/parsers/getApiBasedValueParserConstructor.js
repo/resources/parsers/getStore.js
repo/resources/Resources.php
repo/resources/wikibase.ui.entityViewInit.js
view/resources/jquery/wikibase/jquery.wikibase.entityselector.js
view/resources/jquery/wikibase/snakview/snakview.js
view/resources/jquery/wikibase/toolbar/jquery.wikibase.edittoolbar.js
view/resources.php
view/resources/wikibase/entityChangers/AliasesChanger.js
view/resources/wikibase/entityChangers/DescriptionsChanger.js
view/resources/wikibase/entityChangers/EntityChangersFactory.js
view/resources/wikibase/entityChangers/EntityTermsChanger.js
view/resources/wikibase/entityChangers/LabelsChanger.js
view/resources/wikibase/entityChangers/SiteLinksChanger.js
view/resources/wikibase/entityChangers/SiteLinkSetsChanger.js
view/resources/wikibase/entityChangers/StatementsChanger.js
view/resources/wikibase/store/store.ApiEntityStore.js
view/resources/wikibase/view/ToolbarViewController.js

In order to move forward with this I'd like to understand the specifics of what people are trying to achieve. Are there specific gadgets for example that are broken now? What data do they need access to? For what?

Are there specific gadgets for example that are broken now?

See Amir’s comment (T259211#6347466) – it looks like two frwiki gadgets, two hewiki gadgets, and one ruwiki gadget is affected, as well as several hewiki scripts where I don’t know where they’re used, and fifteen user scripts.

Also, as Amir wrote in that comment, the gadgets can declare a dependency on the jquery.wikibase.linkitem module instead of wikibase.api.RepoApi, and then the wb.RepoApi class should appear again. It will still be an unsupported, unstable interface, but that should be a relatively low-effort way to make the gadget work.

I’ll let Eran answer the other questions.

but in the mean time you can still mark jquery.wikibase.linkitem as the dependency instead and use the global variable but I don't recommend it.

This by itself won't work as the dependencies aren't defined correctly

mw.loader.load('jquery.wikibase.linkitem')
Exception in module-execute in module jquery.wikibase.linkitem: load.php:2:530
TypeError: "repoConfig is null"

Hey,
Technically it's intentional. RepoApi is a bad module. It's confusing (which repo? Commons is also a repo) and doesn't bring much value (you can use mw.Api instead in WMF, as the URL to wikidata.org likely won't change) and having an extra module brings its own cost.
...
I highly recommend not using this module anymore. Also according to stable interface policy, javascript code is unstable and prone to change with prior notice

I don't find RepoApi as bad module - it is just convenient JS wrapper for PHP api and aligns with how MW core is doing with mediawiki.api. From client side (client in the meaning of browser, not in the meaning of Wikibase client) it is really useful as you can find it in entityChangers or any other client side Wikidata code. When there are multiple consumers for API module, it is better to split it from the UI module IMO.

In order to move forward with this I'd like to understand the specifics of what people are trying to achieve. Are there specific gadgets for example that are broken now? What data do they need access to? For what?

For setting labels, getting entities and create claims (which were removed in https://gerrit.wikimedia.org/r/c/wikibase/javascript-api/%2B/609466 ).
Classic example of data access is to labels - and to translations

Change 621498 had a related patch set uploaded (by Eranroz; owner: Eranroz):
[mediawiki/extensions/Wikibase@master] Revert "Migrate wikibase.api module to repo and use it directly in client"

https://gerrit.wikimedia.org/r/621498

@Ladsgroup @daniel I would like to understand what is the approach we would like to go for the long run:

  • Do we want to have JS api for other extensions (the current and future use cases I can think of are Content translation such as T249458 and Client_editing_input) and for gadgets?
  • Do we want to have bare light API without JS wrapping? in which case, when do Wikibase plan to adapt to this approach? (rm RepoApi and change all the uses)

Just because we don’t advertise the JS API for other extensions, gadgets, or user scripts does not mean we have to remove it internally and change all its uses.

Change 621498 abandoned by Eranroz:
[mediawiki/extensions/Wikibase@master] Revert "Migrate wikibase.api module to repo and use it directly in client"

Reason:

https://gerrit.wikimedia.org/r/621498

it's disappointing that JS API that was just exposing the PHP API (part of it) and that is used used by Wikibase itself, isn't available anymore for other consumers :-(
Anyway it seems like JS consumers of wikibase should not relay on that anymore and should use re-implement it themselves or directly access using MW API for now.

Indeed, JS consumers of Wikibase should never have relied on it in the first place, because Wikibase’s JS code is not a stable interface.

Just pointing out (as it has not been stated above) but you can still talk to the Wikidata / REPO API without using the code in RepoApi, so losing access to this shouldn't block you from doing things.
Specifically re adding such code as a feature to be maintained in some compatible way would have to be a product decision.
In the future, the UI / JS landscape here is likely to look very different, so it probably best top not bind to something like this for now.

matej_suchanek subscribed.

I highly recommend not using this module anymore.

Does this concern Wikidata, too? There are some gadgets using RepoApi. They work since the module is available there. Should these be changed nevertheless?

Does this concern Wikidata, too? There are some gadgets using RepoApi. They work since the module is available there. Should these be changed nevertheless?

They are fine at this point. This might change in the future but nothing concrete at this point.