Page MenuHomePhabricator

Add full-text search support to Query Service
Closed, DeclinedPublic

Description

Currently, Query Service has limited capability to search through the data, due to absence of support for FTS engine. We did not add this capability, since we already have FTS implementation in ElasticSearch - however, we do not actually use it from Blazegraph.

Blazegraph allows to plug in external search engines: https://wiki.blazegraph.com/wiki/index.php/ExternalFullTextSearch

We should develop a capability to use our search from inside Blazegraph, there are a number of requests for queries that include searching and right now they are not running efficiently due to inability to leverage ElasticSearch.

Current workaround: use SQL on quarry.wmflabs.org

Related Objects

StatusAssignedTask
DeclinedNone
ResolvedSmalyshev
InvalidNone
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
Resolveddcausse
Resolveddcausse
ResolvedSmalyshev
Resolveddebt
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
ResolvedSmalyshev
Resolveddcausse
ResolvedSmalyshev

Event Timeline

Smalyshev created this task.Aug 1 2016, 9:13 PM
Restricted Application added projects: Wikidata, Discovery. · View Herald TranscriptAug 1 2016, 9:13 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Smalyshev moved this task from Needs triage to WDQS on the Discovery board.Aug 2 2016, 6:45 PM

Sample syntax from T144280:

  SERVICE wikibase:sitesearch { 
		?item wdt:P31 ?instance .
      bd:serviceParam wikibase:searchsite "wikidatawiki"  . 
      bd:serviceParam wikibase:searchterm "John Doe"  . 
      bd:serviceParam wikibase:searchlang "en"  . 
      bd:serviceParam wikibase:searchresultlimit "500" . 
      bd:serviceParam wikibase:searchresultoffset "10" . 
  }
thiemowmde triaged this task as Low priority.Aug 31 2016, 7:40 AM
thiemowmde added a subscriber: Jonas.
Smalyshev raised the priority of this task from Low to Normal.Sep 12 2016, 10:46 PM
Abbe98 added a subscriber: Abbe98.May 4 2017, 1:07 PM

The current solution works for me. Thanks!

Restricted Application added a subscriber: PokestarFan. · View Herald TranscriptJul 26 2017, 8:20 AM
Lydia_Pintscher moved this task from incoming to monitoring on the Wikidata board.Mar 6 2018, 9:23 AM

Given that you can use MWAPI to run fulltext search over any wiki (including Wikidata) I am not sure we need another implementation in Blazegraph...

Lydia_Pintscher closed this task as Declined.Apr 15 2018, 3:31 PM
Lydia_Pintscher added a subscriber: Lydia_Pintscher.

Agreed. Let's close this and if someone has a good reason to reopen it please do so.