Page MenuHomePhabricator

Use of xsd:duration throws an error in Wikidata Query Service
Closed, ResolvedPublic

Description

When running a query that uses xsd:duration, it terminates with an error.

Sample query:

# People who died in the last 7 days
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
SELECT ?person ?deathDate {
  ?person wdt:P31 wd:Q5 .
  ?person wdt:P570 ?deathDate .
  FILTER ( ?deathDate >= (now()-"P7D"^^xsd:duration) )
}

Short URL to this query: http://tinyurl.com/ztjo2hu

Error message:
com.bigdata.rdf.internal.NotMaterializedException

Event Timeline

Restricted Application added projects: Wikidata, Discovery. · View Herald TranscriptAug 25 2016, 2:20 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Alphos added a subscriber: Alphos.Aug 25 2016, 4:25 PM

Temporary workaround :

# People who died in the last week
# Well, more accurately, people whose death + 7 days is after the current day and time, but still !
SELECT ?human ?humanLabel ?date
WHERE
{
  ?human wdt:P31 wd:Q5;
         wdt:P570 ?date.
  FILTER(?date + "P7D"^^xsd:duration >= NOW() ).
  SERVICE wikibase:label { bd:serviceParam wikibase:language "en". }
}
ORDER BY ?date

Enjoy :-)

Smalyshev triaged this task as Normal priority.Aug 25 2016, 4:50 PM
Smalyshev added a subscriber: Smalyshev.

That's weird that it works with + but not with -. Looks like a bug somewhere in comparison or math implementation. I'll take a look.

It's weirder than you think :

SELECT ?now WHERE {
  BIND( ( NOW() - "P7D"^^xsd:duration ) AS ?now ) .
}

returns "Sep 1, 2016", which is 6 days from today (we're 2016-08-25T23:55:00.000+02:00, give or take a few minutes)

SELECT ?now WHERE {
  BIND( ( "2016-08-26T00:13:14.567Z"^^xsd:dateTime - "P7D"^^xsd:duration ) AS ?now ) .
}

returns "Sep 2, 2016".

Same goes for all values of date and time, including time values such as [...]T00:13:14.567Z, [...]T12:13:14.567Z, [...]T23:13:14.567Z to account for my timezone not being Zulu (I'm currently UTC+02:00) : all these subtractions of 7-day xsd:duration actually add 6 days, for some reason.

I'd look in the - operator implementation for xsd:dateTime and xsd:duration entities ;-)

Change 307347 had a related patch set uploaded (by Smalyshev):
Fix negative duration handling

https://gerrit.wikimedia.org/r/307347

Change 307347 merged by jenkins-bot:
Fix negative duration handling

https://gerrit.wikimedia.org/r/307347

The +/- part will be fixed soon, filed https://jira.blazegraph.com/browse/BLZG-2077 for the exception part.

Fix for duration arithmetics is deployed, leaving open for the exception issue.

Temporary fix:

# People who died in the last 7 days
SELECT ?person ?deathDate {
  ?person wdt:P31 wd:Q5 .
  ?person wdt:P570 ?deathDate .
  FILTER(sameTerm(dataType(?deathDate), xsd:dateTime))
  FILTER ( ?deathDate >= (now()-"P7D"^^xsd:duration) )
}

The bug is triggered by comparing date with non-date (string) literal that somehow ended up in the DB.

Esc3300 added a subscriber: Esc3300.EditedSep 1 2016, 9:08 AM

Good news about duration. Another possible workaround could be:

# People who died in the last 7 days
SELECT ?person ?deathDate
{
	?person wdt:P570 ?deathDate .
	FILTER ( ?deathDate > "2016-08-20T00:00:00Z"^^xsd:dateTime)
	FILTER ( ?deathDate >= (now()-"P7D"^^xsd:duration) )
	?person wdt:P31 wd:Q5 .
}
Alphos added a comment.EditedSep 1 2016, 9:39 AM

Or, to handle the string literals too :

# People who died in the last 7 days
SELECT ?person ?deathDate {
  ?person wdt:P31 wd:Q5 .
  ?person wdt:P570 ?deathDate .
  FILTER( IF ( sameTerm( dataType( ?deathDate ), xsd:dateTime ), ?deathDate >= ( NOW() - "P7D"^^xsd:duration ), STRDT( ?deathDate, xsd:dateTime ) >= ( NOW() - "P7D"^^xsd:duration ) ) )
} ORDER BY ?deathDate
Esc3300 added a comment.EditedSep 1 2016, 9:47 AM

Not really (execution time 12022 ms compared to 1859 ms). Besides, the culprit seems to be https://www.wikidata.org/wiki/Q5186062 with year "0"

Found with:

SELECT ?person ?deathDate
{
  ?person wdt:P570 ?deathDate .
  FILTER(!sameTerm( dataType( ?deathDate ), xsd:dateTime ) )
  ?person wdt:P31 wd:Q5 .
}

Change 308012 had a related patch set uploaded (by Smalyshev):
Fix NotMaterializedException on date comparisons with strings

https://gerrit.wikimedia.org/r/308012

Change 308012 merged by jenkins-bot:
Fix NotMaterializedException on date comparisons with strings

https://gerrit.wikimedia.org/r/308012

It seems that something like this stopped working (similar to the "temporary workaround" above T143897#2582831):

SELECT ?person ?deathDate
{
  ?person wdt:P31 wd:Q5 .
  ?person wdt:P570 ?deathDate .
  FILTER ( now() < (?deathDate+"P1Y"^^xsd:duration) )
}

It throws a "Vocab(-7)" error.

Change 309088 had a related patch set uploaded (by Smalyshev):
Additional fixes for T143897

https://gerrit.wikimedia.org/r/309088

Change 309088 merged by jenkins-bot:
Additional fixes for T143897

https://gerrit.wikimedia.org/r/309088

Smalyshev closed this task as Resolved.Sep 12 2016, 7:51 PM