Page MenuHomePhabricator

setup elastic search for main phabricator
Closed, ResolvedPublic

Description

Talked to d^ about this:

Notes:

'search.svc.eqiad.wmnet:9200'
reindex
success?

Details

Reference
fl565

Event Timeline

flimport raised the priority of this task from to Normal.
flimport set Reference to fl565.

Rush wrote on 2014-08-28 21:23:53 (UTC)

https://gerrit.wikimedia.org/r/#/c/156886/
https://gerrit.wikimedia.org/r/#/c/156891/

so far no luck:

{"error":"IndexMissingException[[phabricatormain] missing]","status":404}

I talked to ^d and he said he is on the case

demon wrote on 2014-09-02 17:19:30 (UTC)

curl -s -XPUT search.svc.eqiad.wmnet:9200/phabricatormain

Swap phabricatormain for any other index we need created. Trying to get this added to bin/search upstream. https://secure.phabricator.com/T5990 has way more info on what's going on.

Qgil added a subscriber: Qgil.

If this is a blocker, I believe it is a blocker of the Bugzilla-Migration, not of the pure setup of this instance.

Qgil set Security to None.
Aklapper moved this task from Backlog to Doing on the Bugzilla-Migration board.Sep 18 2014, 3:33 PM
demon added a subscriber: demon.Sep 22 2014, 11:37 PM

This should all be done already, we're using Elasticsearch in this instance.

Maybe my expectations need to get re-adjusted, but searching for "Contains words: delete" on https://phabricator.wikimedia.org/maniphest/query/advanced/ the search results do not include T191 which has a comment containing the word "deleting". Same 'missing' result when entering "delete" in the generic Phabricator search field in the upper corner.

Either my expectations towards stemming are too high, or I misunderstood the skills of ElasticSearch, or I misunderstand how to use the advantages of ElasticSearch in Phabricator?

Qgil added a comment.Sep 24 2014, 9:15 PM

fwiw I just made a typo: "baner", which offered zero results. I realized the problem and I searched "banner", getting 3 results.

demon added a comment.Sep 24 2014, 9:18 PM
In T378#15, @Aklapper wrote:

Maybe my expectations need to get re-adjusted, but searching for "Contains words: delete" on https://phabricator.wikimedia.org/maniphest/query/advanced/ the search results do not include T191 which has a comment containing the word "deleting". Same 'missing' result when entering "delete" in the generic Phabricator search field in the upper corner.

Either my expectations towards stemming are too high, or I misunderstood the skills of ElasticSearch, or I misunderstand how to use the advantages of ElasticSearch in Phabricator?

ES is only as good as the app using it :) The Phabricator/ES shim is pretty rudimentary, so it wouldn't surprise me if we're not using it as ideally as possible. This is a point of iteration (but should probably be its own task).

In T378#17, @Qgil wrote:

fwiw I just made a typo: "baner", which offered zero results. I realized the problem and I searched "banner", getting 3 results.

I'm almost certain Phabricator/ES doesn't implement any sort of spelling suggestions. Again we could improve here.

Qgil added a comment.Sep 24 2014, 9:38 PM

As Andre said, this is a problem of expectations. If this is what we have, then this is what we have, and the rest are feature requests. The requirement was to have Elasticsearch for Day 1 and this is in place. I also think we can resolve this task.

@Aklapper: It wouldn't surprise me at all if phabricator isn't indexing comments.

Comments are indexed, but it's indexing full words instead of (sub)strings. I got used to entering "delet" as a (sub)string search in Bugzilla. In Phab I am left with only "Contains Words", so I'd have to run three separate searches for "delete", "deletion" and "deleting". Not good, but likely not the fault of ES but Phab as explained above by Chad. I guess I'll go upstream...

Oh crap, I did already of course: https://secure.phabricator.com/T4818 and Evan said that ElasticSearch does support stemming, which was exactly the reason why I requested using ES. So that still leaves me with the question: Where is the bug in the code that stemming does not work as I had expected it? ES config or Phab?

chasemp closed this task as Resolved.Oct 7 2014, 8:37 PM
chasemp claimed this task.
In T378#5633, @Aklapper wrote:

I did already of course: https://secure.phabricator.com/T4818 and Evan said that ElasticSearch does support stemming, which was exactly the reason why I requested using ES. So that still leaves me with the question: Where is the bug in the code that stemming does not work as I had expected it? ES config or Phab?

This is still a valid problem. Shall I move this to a separate ticket (though I'd prefer if someone told me first whether it's us or upstream Phab to blame)?

Qgil added a comment.Oct 8 2014, 10:03 PM

Please create a separate ticket under Phabricator, and not Bugzilla-Migration. The current search is not amazing but it works.

In T378#9732, @Qgil wrote:

Please create a separate ticket under Phabricator, and not Bugzilla-Migration. The current search is not amazing but it works.

-> T679