I think all the items that were updated during the breakage, and were not updated since then, would be broken now – their data was imported using the swapped Ps and Qs, and then after the fix it’s swapped the wrong way. We should be able to get a list of all potentially affected items from recentchanges – @Smalyshev is it possible to tell the updater to re-import those specific entities?

Scholia also looks fine again – lowering priority since only the investigation is missing. Thanks a lot for the quick response Stas!

?: Podcasts. I ask Wikimedians questions like: What would be your first orders as King (or Queen) of Wikipedia?

Those errors usually happen when the query returns 0 results and the visualization is not able to handle it, and that seems to be what’s happening.

In EntitySchema (package.json) we’ve been very happy with husky and lint-staged – I think @Michael is the expert there. (Unfortunately, PHPCS/PHPCBF’s exit codes don’t play nicely with it, but for the JS part it’s really wonderful.)

Well, there’s no train this week (Wikimania), but we could add it to next week’s train blockers already.

In T230135#5404642, @OlafJanssen wrote:

One more thing: How should be go about implementing the updated line of code

sparql = SPARQLWrapper(endpoint_url, agent='Some uniquely identifyable user agent')

in the outputs in the WDQS-interface, like so
(snip)

You probably need to set a good User-Agent – the default Python Requests one is blocked since about a month ago (announcement). (Though that doesn’t explain why it would still have worked yesterday…)

Even Pywikibot, which still calls Statements “Claims”, doesn’t read the type when deserializing a Statement (Claim.fromJSON), although it does emit a type when serializing it (Claim.toJSON).

Hm, that’s tricky… KrBot automatically fixes the format, at least sometimes (example edit), but I don’t know if it also does that in reference snaks (or would Citoid create new items for each ISBN in a reference, so that it would always be a main statement?), and I’m not sure if it’s acceptable for Citoid to make edits that need to be cleaned up by a bot later.

In T230050#5401241, @Smalyshev wrote:

Another this is - do we even need $this->prefixes? WDQS understands all those by default already.

Okay, I did have time for a quick git bisect, and according to that the culprit is Ie8ef676d9e, linked to T211799, T214557, T222306.

I assume this refers to the final TODO in init.js / “shows the current targetValue”, where we currently don’t check that the value is what we expect?

As far as I’m aware, this is done.

In T75181#1052495, @TTO wrote:

DELETE FROM change_tag WHERE ct_tag = "HHVM";
That's easy enough, but it also needs to be removed from tag_summary, which will require a maintenance script of some sort. Essentially just copy and paste the code of ChangeTags::deleteTagEverywhere and alter it to work in batches.

The petscan3 instance is one of those brought down due to Cloud VPS maintenance yesterday and today (announcement) – if the VM is set up such that it doesn’t need any manual action after a reboot, then this might resolve itself within a few hours.

Actually, let’s split this task up after all. Disregard the gerritbot comments, I’ll move those patches to one of the subtasks.

To clarify: for Wikidata Bridge we likely only need support in wbeditentity, but the most reasonable implementation (as far as I can tell) ends up adding support to almost all other modules as well. The exception are modules that create a redirect rather than a regular entity edit – I’ll leave those aside for now.

@darthmon_wmde I’ve already started working on this (as part of Wikidata Bridge task T227003), so I’m not sure it makes much sense to unassign me…

In T229221#5386347, @Ladsgroup wrote:

So I think we should inject terms as a mw.config variable and use it in termbox v1 and v2 while lazy loading the hook when someone tries to edit and stop using in every page view. Does it make sense to you?

Late update – I just found out that MediaWiki allows you to suppress the external link marker using the plainlinks class, so we could’ve gone with the external link approach after all. We don’t have to change anything for that now, but it might be useful if we ever need to come back to this for another reason.

The issue persisted for ca. 10 minutes after the scap finished, for reasons I don’t understand, but now it does seem to be resolved. Leaving the task open for verification by others.

Both of the changes attached to this task are merged, so I assume this can go to verification.

Tested with multiple instances across two projects (wikidata-dev and openrefine).

Done – I have backups on my laptop in case anybody needs them:

In T229604#5385566, @Moebeus wrote:

Can confirm: The problem started around 17:45 GMT +1 and affects all browsers (Firefox, Edge, Chrome and variants).

The wikibase.api module in wdio-wikibase has createItem() and getProperty() functions that we can use for this.

However, it is not clear why this should make a difference as our code is not supposed to modify $, even when called in the global scope.

In T220703#5383544, @Legoktm wrote:

You can use wiki pages or some other mechanism to share […] long URLs

Well, for the DB it wouldn’t make a difference because it always gets 2500 results anyways (and then sorts and limits them in PHP – I guess applying an offset here would save a bit of memory, discarding the unneeded terms earlier), but we definitely should pass the offset to ElasticSearch, yeah.

I looked a bit more into this, and it turns out that SearchEntities doesn’t support continuation all that well – basically, it asks the underlying search backend for offset + limit + 1 results, then returns the [offset, offset+limit) slice of that. Clearly, this isn’t very efficient for larger and larger offsets, which is why the API won’t return offsets higher than the standard API limit (50) for continuation. However, it won’t stop you from specifying larger limits yourself, potentially asking the search backend for arbitrarily large numbers of results.

I’ve proposed the API change in T229460: Support standard MediaWiki API continuation in wbsearchentities module / wbsearch list/generator.

Indeed – this seems to have been fixed in codemirror/CodeMirror#5936 (using the approach I mentioned above: hard-code the ranges of all Unicode letters into the regexp), which was released in 5.48.2 12 days ago, and I guess we installed that version?

It looks like the redirect is generated in Varnish, see text-frontend.inc.vcl.erb::mobile_redirect. We could try to disable it in some ways:

Well, jQuery would try to follow the redirect, but it doesn’t have the Access-Control-Allow-Origin header set, so it’s blocked by the Same Origin Policy. (That is, https://de.m.wikipedia.beta.wmflabs.org is not allowed to read the https://wikidata.beta.wmflabs.org redirect.)

In T229291#5377889, @Smalyshev wrote:

Alternatively, perhaps wbsearchentities should be changed to do continuation the usual way?

Would be fine with this too. Probably would have to support both elements, otherwise scripts that use current one would break?

It looks like we request https://wikidata.beta.wmflabs.org/wiki/Special:EntityData/Q11.json, and instead of directly returning the entity JSON, that redirects to https://m.wikidata.beta.wmflabs.org/wiki/Special:EntityData/Q11.json (mobile subdomain), and we don’t follow that redirect. I don’t really like Special:EntityData redirecting to a different domain, but fixing that sounds like it would be complicated (and involve changes in MobileFrontend, not just Wikibase), so it’s probably best to just follow the redirect.

I think we expect it to stay around for now.

Updater.java:289 is this line:

So if we purge the page via action=purge (probably with forcelinkupdate=1, though tbh I’m not quite sure what that controls) before reloading it, we shouldn’t have to worry about dispatch lag. That still leaves the issue of replication lag on the repo wiki, though: if the client wiki page is purged and re-rendered based on data from a replica database on the repo wiki that hasn’t seen the edit yet, there will still be stale data.

There currently seem to be at least two workarounds for this missing feature:

Alternatively, perhaps wbsearchentities should be changed to do continuation the usual way? That would probably improve compatibility with other ways to access the API as well (“continue” button on Special:ApiSandbox, continuation=True in python-mwapi, etc.).

Well, the original query doesn’t tweak the parameters, and by default we don’t want to follow too many empty continuations. This query returns results, though:

In T214884#5372933, @alaa_wmde wrote:

• How the RDF output of this look like? Should we not do it for MVP and add it later? It can have treat it as "StringValue" RDF output as well (for now)

I would've thought first of using the schema's url as a concept URI and used it here, but didn't look into it in details if that would work/make sense.