Page MenuHomePhabricator

Implement searching of 'depicts' on commons with the 'relative position within image' qualifier
Open, LowPublic

Description

The relative position within image property, when used with 'depicts', indicates the position within the image of the thing-that-is-depicted

Example from wikidata https://www.wikidata.org/wiki/Q669994 - see depicts > barque > relative position within image. The data is stored in the format pct:x,y,w,h where "x represents the number of pixels from the 0 position on the horizontal axis, calculated as a percentage of the reported width. w represents the width of the region, also calculated as a percentage of the reported width. The same applies to y and h respectively. " (see http://iiif.io/api/image/2.0/#image-information-request-uri-syntax)

Fulltext indexing with elasticsearch won't work, because exact matching is not useful - we'd need to allow the user to select an area they're interested in and match if their specified area intersects with the area described in the qualifier.

As suggested for T194185 we could use the Wikidata Query Service (WDQS) to run a SPARQL query, and then use the returned ids as a filter for an elasticsearch query - basically we'd construct a SPARQL query , take all the resulting IDs, and then search elasticsearch for anything else we wanted to search for but only among the (max 1000) IDs we got from WDQS.

The SPARQL query would be pretty involved though - we need to return items where there is an intersection between a user-specified area of the image and the area specified in the qualifier (and because the qualifier is stored as a string we have to convert it into 4 decimals). Here's an example SPARQL for items depicting a boat (Q35872) where the boat in is the central box of a 3x3 grid superimposed on the image

SELECT DISTINCT ?item ?itemLabel ?x2 ?y2 ?w2 ?h2 WHERE {
  ?item p:P180 [ ps:P180 wd:Q35872; pq:P2677 ?iiif ].
  # split iiif string into its components
  BIND( STRAFTER( ?iiif, ':') as ?coords).
  BIND( xsd:decimal(STRBEFORE( ?coords, ',')) as ?x2).
  BIND( STRAFTER( ?coords, ',') as ?ywh).
  BIND( xsd:decimal(STRBEFORE( ?ywh, ',')) as ?y2).
  BIND( STRAFTER( ?ywh, ',') as ?wh).
  BIND( xsd:decimal(STRBEFORE( ?wh, ',')) as ?w2).
  BIND( xsd:decimal(STRAFTER( ?wh, ',')) as ?h2).
  # intersect with the coordinates pct:33,33,33,33 (the middle box of a 3x3 grid over the image)
  # x1=33,y1=33,w1=33,h1=33
  FILTER(
    (
      (?x2 <= 33 && 33 <= ?x2 + ?w2)
      ||
      (?x2 <= 66 && 66 <= ?x2 + ?w2)
    )
    &&
    (
      (?y2 <= 33 && 33 <= ?y2 + ?h2)
      ||
      (?y2 <= 66 && 66 <= ?y2 + ?h2)
    )
  ).
  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE]". }
}

FYI the above takes 440ms to run on http://query.wikidata.org

Note that because of limitations passing data between WDQS and elasticsearch there will be edge cases where no results will be returned even if appropriate results exist.


This solution depends on T194401


Note that there are currently 130 items in wikidata with 'depicts' statements with a 'relative position within image' qualifier out of a total of ~70k items with depicts statements (~0.2%)

Related Objects

Event Timeline

Cparle triaged this task as Medium priority.May 9 2018, 11:52 AM
Cparle created this task.
Cparle renamed this task from Investigate searching of 'depicts' on commons with the 'relative position within image' qualifier to Implement searching of 'depicts' on commons with the 'relative position within image' qualifier.May 11 2018, 11:19 AM
Cparle updated the task description. (Show Details)
Vvjjkkii renamed this task from Implement searching of 'depicts' on commons with the 'relative position within image' qualifier to cbdaaaaaaa.Jul 1 2018, 1:10 AM
Vvjjkkii removed Cparle as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed a subscriber: Aklapper.
Wong128hk renamed this task from cbdaaaaaaa to Implement searching of 'depicts' on commons with the 'relative position within image' qualifier.Jul 1 2018, 3:17 AM
Wong128hk assigned this task to Cparle.
Wong128hk lowered the priority of this task from High to Medium.
Wong128hk updated the task description. (Show Details)
Wong128hk added a subscriber: Aklapper.
Cparle lowered the priority of this task from Medium to Low.Nov 15 2018, 5:17 PM
MPhamWMF subscribed.

Closing out low/est priority tasks over 6 months old with no activity within last 6 months in order to clean out the backlog of tickets we will not be addressing in the near term. Please feel free to reopen if you think a ticket is important, but bare in mind that given current priorities and resourcing, it is unlikely for the Search team to pick up these tasks for the indefinite future. We hope that the requested changes have either been addressed by or made irrelevant by work the team has done or is doing -- e.g. upgrading Elasticsearch to a newer version will solve various ES-related problems -- or will be subsumed by future work in a more generalized way.

RhinosF1 removed a project: Discovery-Search.
RhinosF1 subscribed.

Re-opening tasks and removing from team workboard per IRC feedback given yesterday and discussion with MPham.