Page MenuHomePhabricator

Implement searching of 'depicts' on commons with the 'relative position within image' qualifier
Open, LowPublic

Description

The relative position within image property, when used with 'depicts', indicates the position within the image of the thing-that-is-depicted

Example from wikidata https://www.wikidata.org/wiki/Q669994 - see depicts > barque > relative position within image. The data is stored in the format pct:x,y,w,h where "x represents the number of pixels from the 0 position on the horizontal axis, calculated as a percentage of the reported width. w represents the width of the region, also calculated as a percentage of the reported width. The same applies to y and h respectively. " (see http://iiif.io/api/image/2.0/#image-information-request-uri-syntax)

Fulltext indexing with elasticsearch won't work, because exact matching is not useful - we'd need to allow the user to select an area they're interested in and match if their specified area intersects with the area described in the qualifier.

As suggested for T194185 we could use the Wikidata Query Service (WDQS) to run a SPARQL query, and then use the returned ids as a filter for an elasticsearch query - basically we'd construct a SPARQL query , take all the resulting IDs, and then search elasticsearch for anything else we wanted to search for but only among the (max 1000) IDs we got from WDQS.

The SPARQL query would be pretty involved though - we need to return items where there is an intersection between a user-specified area of the image and the area specified in the qualifier (and because the qualifier is stored as a string we have to convert it into 4 decimals). Here's an example SPARQL for items depicting a boat (Q35872) where the boat in is the central box of a 3x3 grid superimposed on the image

SELECT DISTINCT ?item ?itemLabel ?x2 ?y2 ?w2 ?h2 WHERE {
  ?item p:P180 [ ps:P180 wd:Q35872; pq:P2677 ?iiif ].
  # split iiif string into its components
  BIND( STRAFTER( ?iiif, ':') as ?coords).
  BIND( xsd:decimal(STRBEFORE( ?coords, ',')) as ?x2).
  BIND( STRAFTER( ?coords, ',') as ?ywh).
  BIND( xsd:decimal(STRBEFORE( ?ywh, ',')) as ?y2).
  BIND( STRAFTER( ?ywh, ',') as ?wh).
  BIND( xsd:decimal(STRBEFORE( ?wh, ',')) as ?w2).
  BIND( xsd:decimal(STRAFTER( ?wh, ',')) as ?h2).
  # intersect with the coordinates pct:33,33,33,33 (the middle box of a 3x3 grid over the image)
  # x1=33,y1=33,w1=33,h1=33
  FILTER(
    (
      (?x2 <= 33 && 33 <= ?x2 + ?w2)
      ||
      (?x2 <= 66 && 66 <= ?x2 + ?w2)
    )
    &&
    (
      (?y2 <= 33 && 33 <= ?y2 + ?h2)
      ||
      (?y2 <= 66 && 66 <= ?y2 + ?h2)
    )
  ).
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE]". }
}

FYI the above takes 440ms to run on http://query.wikidata.org

Note that because of limitations passing data between WDQS and elasticsearch there will be edge cases where no results will be returned even if appropriate results exist.


This solution depends on T194401


Note that there are currently 130 items in wikidata with 'depicts' statements with a 'relative position within image' qualifier out of a total of ~70k items with depicts statements (~0.2%)

Related Objects

Event Timeline

Cparle triaged this task as Normal priority.May 9 2018, 11:52 AM
Cparle created this task.
Cparle updated the task description. (Show Details)May 9 2018, 11:55 AM
Cparle updated the task description. (Show Details)May 10 2018, 3:13 PM
Cparle updated the task description. (Show Details)May 11 2018, 11:14 AM
Cparle renamed this task from Investigate searching of 'depicts' on commons with the 'relative position within image' qualifier to Implement searching of 'depicts' on commons with the 'relative position within image' qualifier.May 11 2018, 11:19 AM
Cparle updated the task description. (Show Details)
Vvjjkkii renamed this task from Implement searching of 'depicts' on commons with the 'relative position within image' qualifier to cbdaaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii removed Cparle as the assignee of this task.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Wong128hk renamed this task from cbdaaaaaaa to Implement searching of 'depicts' on commons with the 'relative position within image' qualifier.Jul 1 2018, 3:17 AM
Wong128hk assigned this task to Cparle.
Wong128hk lowered the priority of this task from High to Normal.
Wong128hk updated the task description. (Show Details)
Wong128hk added a subscriber: Aklapper.
Addshore moved this task from incoming to monitoring on the Wikidata board.Sep 19 2018, 7:08 AM
Cparle removed Cparle as the assignee of this task.Nov 9 2018, 12:22 PM
Cparle lowered the priority of this task from Normal to Low.Nov 15 2018, 5:17 PM