RE2 sounds nice… we could probably live without backreferences. But we’d need some way to use RE2 in PHP.

Good point, let’s discuss this in T176312: Don’t check format constraint via SPARQL (safely evaluating user-provided regular expressions).

For the wikidata-constraints test system (which doesn’t have enough RAM to run BlazeGraph), I’ve written a tiny server, minisparql, which listens for REGEX() SPARQL queries and evaluates them with PCRE. We could use a very similar service, except we can do away with the SPARQL wrapping (e. g. have the regex and text on two lines), and then we can write the whole thing in PHP (I thought the regex-parsing part would be easier to do in Python, but then I learned that Python doesn’t have built-in PCRE, so currently minisparql has Python run a PHP process just for preg_match, which is of course a bit silly).

Thanks, CI builds are working again :)

Setting High priority, everyone on the Wikidata team seems to be experiencing this.

One thing that could also be tweaked in that code is to track the last timestamp an entry was touched and prune ones older than a certain number of days. For busy regexes, that stops long-tail cruft from accumulating and reduce memcached I/O in bytes.

Copying this comment from Gerrit so it doesn’t get buried:

Interesting idea! It feels a bit weird to implement logic like this on top of the cache (I thought that’s the cache’s job?), but you’re the expert :) it sounds like it makes a lot of sense, at least, since the set of regexes is mostly static and the set of values is highly dynamic, with some very commonly used values.

Reopening because once more, I forgot some steps when adding a new result status.

The main change for this task is done, but based on the semi-related change https://gerrit.wikimedia.org/r/#/c/377267/1 (which I didn’t open a separate ticket for). Could someone perhaps review that too? :)

We’re not yet sure if the link should be right after the constraint type name or if it should have float: right.

I’ve tried to summarize the two options (without any pro/contra arguments) on-wiki (just for the syntax highlighting): https://www.wikidata.org/wiki/User:Lucas_Werkmeister_(WMDE)/External_identifiers

But perhaps it works if we have the TitleParser parse commons:Creator:Ludolf Bakhuizen…?

Copying Daniel’s comment from Gerrit:

If we assume spherical globes (which is already slightly inaccurate for Earth, but probably okay), the actual calculation is just “Earth distance × (globe radius / Earth radius)”, right? And I think a predefined set of globes is acceptable.

The URI is correct. The /wiki/ URL is a description of the map, but not the canonical URI of the map itself (comparable to the difference between http://www.wikidata.org/entity/Q42 and https://www.wikidata.org/wiki/Q42). The only problem is that the redirect from the canonical URI to the description isn’t live yet: see T163922: Create a URL rewrite to handle the /data/ path for canonical URLs for machine readable page content.

It looks like change 258519 implements the first of Daniel’s suggestions (add wdtn: and then reuse psn:, pqn:, prn:). But as Stas already pointed out, that’s problematic because it mixes up predicates for simple values (ps:) and for full values (psv:). Previously, psn: was the normalized variant of psv: (containing normalized full quantity values), but now it’s also the normalized variant of ps: (containing normalized simple external identifier URIs). I’m not sure if this is still something that OWL hates (technically, they’re both IRIs), but it’s inconsistent and confusing.

Okay, let’s say October 10th. If we send the announcement today, that’s three weeks and six days of advance notice, which should be plenty, considering that we don’t expect anyone to actually care about this :)

Thanks for your suggestions, Hanna! I like the upper row, specifically the ? / ! part, which seems to reflect the distinction between non-mandatory constraints (“are you sure this is right?”) and mandatory constraints (“this can’t be right!”) very nicely.

@Sjoerddebruin okay, it’s not as simple as I thought :D let’s discuss that in T175594, then.

Yeah, this query doesn’t crash:

SELECT (2 * ?distance AS ?circumference) WHERE {
  BIND("<http://www.wikidata.org/entity/Q111> Point(0 0)"^^geo:wktLiteral AS ?point1)
  BIND("<http://www.wikidata.org/entity/Q111> Point(180 0)"^^geo:wktLiteral AS ?point2)
  BIND(geof:distance(?point1, ?point2) AS ?distance)
}

But it reports the equatorial circumference of Mars as 40k km, which is incorrect (that’s Earth’s circumference (roughly)).

(But of course, just fixing that issue isn’t enough to properly calculate the distance, because the radius of the globe needs to be taken into account, right?)

It looks like BlazeGraph’s CoordinateDD type doesn’t like the longitudes of those coordinates… is it conventional to give them in range [0°, 360°) instead of [-180°, +180°) on bodies other than Earth? @thiemowmde I think you’re more familiar with that…

I’ve uploaded a second change to add a margin. Here’s what it looks like just before a line break:

@Krinkle thanks for your comments!

Change merged, but @thiemowmde disagrees with the change… do you want to discuss this here?

Because checking arbitrary regexes on arbitrary input opens us up to DoS attacks via malicious regexes (e. g. catastrophic backtracking – runtime of checking (x+x+)+y on xxx..xxx is quadratic in input length). This came up during the security review of the extension (T101467: Ex: WikibaseQualityConstraints - remove or sanitize regex for FormatChecker), and we eventually decided to check regexes on an external service instead, where queries are subject to a timeout (T102752: [RFC] Workaround for checking the format constraint).

The federated query used to work as of April

Aaron suggested hashing, which is an excellent idea. Moving back to Doing.

The new API format is now live on https://wikidata-constraints.wmflabs.org/.