We seem to have loops in P131, including entries which are P131 to themselves. This clearly needs fixing in data, but loop detection on the engine wouldn't hurt too.

Transitive properties aren't necessary a-cyclic -- for example, consider P460 "said to be the same as", which is transitive, currently used to link given names together, if they are considered to be variants of the same name in different languages.

The page at

https://www.wikidata.org/wiki/Wikidata:WikiProject_Names/given-name_variants

includes two columns of queries that find all the variants of each name (ie extract the whole equivalence class), using a P460* path query from a given starting name, and count up the number of occurrences of each one.

P460 is transitive, but it is not directed, and so there are huge numbers of loops in the equivalence classes. But this isn't a problem. The path search just keeps going until no new P460 link leads it to a node it has not already seen. So the engine can handles path loops, without a problem.

Similarly, looking at the query Daniel posted,

SELECT (COUNT(DISTINCT(?city)) AS ?count) WHERE {
  ?city wdt:P31/wdt:P279* wd:Q515 .    # find instances of subclasses of city
  ?city wdt:P131* wd:Q1202 .
}

if one comments out either one or the other of the two requirement lines, the query runs without a hitch. So I don't think there is anything to do with the nature of the data (eg whether it contains cycles or not) that affects whether or not Blazegraph can handle it.

The problem appears to arise with how Blazegraph combines the two requirements.

Related to this may be the fact that Blazegraph also fails with path searches of the form

SELECT (COUNT(DISTINCT ?a) AS ?count) WHERE { 
    BIND (wd:Q3305213 AS ?class) .    # paintings 
    ?a wdt:P31/wdt:P279* ?class .
}

even though the same search works perfectly if the variable ?class is eliminated, and wd:Q3305213 instead specified inline.

I filed the latter with Blazegraph as issue 1543 three weeks ago, but no commentary on it from them as yet

https://jira.blazegraph.com/browse/BLZG-1543

What's the status of this? The query above still errors out...

I believe a query like "instance of x in region y" is definitely something we should be able to answer.

I didn't have time for in-depth look into it as I was busy with the data duplication/update issues, but I did not forget about it. See also comments on https://jira.blazegraph.com/browse/BLZG-1543 - maybe trying to play with optimizer hints would make it better. It doesn't look like loops per se are the problem, as simple cases work, but there's definitely some query optimizer issue that breaks it for bigger loops. As soon as I am done with the duplication issue I'll get back to this.

BTW, looks like this query:

PREFIX gas: <http://www.bigdata.com/rdf/gas#>
SELECT ?item ?itemLabel  {
  SERVICE gas:service {
    gas:program gas:gasClass "com.bigdata.rdf.graph.analytics.SSSP" ;
                gas:in wd:Q1202 ;
                gas:traversalDirection "Reverse" ;
                gas:out ?item ;
                gas:out1 ?depth ;
                gas:maxIterations 12 ;
                gas:linkType wdt:P131 .
  }
  ?item wdt:P31/wdt:P279* wd:Q515 .
  SERVICE wikibase:label {bd:serviceParam wikibase:language "en" }
}

works much better for the same.

Earlier today abian on IRC mentioned that people are adding redundant P131 statements to get their queries to work.

One of the queries which is failing is this one.

I tried to figure out a way to get it to work and noticed that the selecting by P131 part takes less than a second and the selecting by P31 part takes a few seconds (usually between 2-5), but when I combine them together, it times out. I tried various things like subqueries, filters, turning off optimisation and nothing I tried would make it stop timing out. This makes no sense to me.

After finding this ticket, I tried the approach in the comment above and it can do the original query in a couple of seconds: see here

Bump.

Some further interesting examples in this thread on Project Chat of searches of this form that work and others that don't.

A search for all buildings in the county of Bedfordshire with Grade I heritage status works; but a search for all churches in the county of Bedfordshire with Grade I heritage status fails -- despite churches being a subclass of buildings, and so part of what gets returned in the first search.

It's clearly an optimiser issue, but just what is the optimiser's decision process here, that flips between buildings and churches ?

These are the sorts of queries that are going to need to work, if WDQS searches are e.g. going to replace or supplement categories in the Structured Data for Commons project.