aklapper wrote on 2014-04-24 09:17:51 (UTC)

I'm saying that Phab's current and default search is weaker/inferior than Bugzilla's search as it only covers words and not strings.

qgil wrote on 2014-04-24 13:55:33 (UTC)

Yes, after some days using defaults Phabricator's search it is clear that it is not as good. The reason why have moved this task to (Not critical for the RfC) is the availability of an ElasticSearch backend that in theory would fix this problem, combined with the fact that ElasticSearch is a familiar technology already in use in Wikimedia.

Still, proper planning and resourcing is needed to use this backend. I will add a dependency to the related task.

qgil wrote on 2014-04-24 13:57:23 (UTC)

You can not create that dependency, because it would create a circular dependency: T159: Use ElasticSearch backend for Maniphest to get stemming feature → T41: Allocate resources for the migration and maintenance → T43: Define what features existing in our current tools are really missing in Phabricator → T46: Identify features Bugzilla users would miss in Phabricator → T159: Use ElasticSearch backend for Maniphest to get stemming feature.

Interesting feature. :)

demon wrote on 2014-04-24 14:04:54 (UTC)

We've got plenty of capacity to handle it in prod.

Let's look at setting up a labs Elasticsearch for Phabricator here in labs to connect to so we can see it in action.

demon wrote on 2014-05-15 18:50:30 (UTC)

This is done on an ad-hoc basis so we can test on the labs install.

`apt-get install openjdk-7-jre elasticsearch`
`/etc/init.d/elasticsearch start`
`cd /phab/phabricator`
`./bin/config set search.elastic.host 'http://localhost:9200'`
`./bin/search index --all`

When we move to production, we won't have to install Elasticsearch on the Phabricator box, we'll just use the production instance. Only the last 2 steps will be necessary then.

qgil wrote on 2014-05-16 04:01:58 (UTC)

Is this why the search is broken now?

Unhandled Exception ("HTTPFutureResponseStatusHTTP")
[HTTP/400] 
{"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures   
{[waqAUwXISqqUm85td2JDmg][phabricator][1]: SearchParseException[[phabricator][1]: from[-1],size[-1]: 
Parse Failure [Failed to parse source [{\"query\":{\"filtered\":{\"query\":{\"bool\":{\"must\":[{\"field\":
{\"field.corpus\":\"https\\\\:\\/\\/secure.phabricator.com\\/D9148\"}},{\"bool\":{\"should\":[{\"text\":{\"relationship.proj.phid\":
{\"query\":\"PHID-PROJ-lykw342rrq42z6t2osbf\",\"type\":\"phrase\"}...

mmodell wrote on 2014-05-16 11:41:20 (UTC)

it looks like phab's elasticsearch instance is broken or out of sync with what phabricator is requesting from it...

though I could be misreading that and it might just be a bug in phabricator.

aklapper wrote on 2014-05-16 13:32:35 (UTC)

Reopening as the search going to http://fab.wmflabs.org/maniphest/query/advanced/ and entering anything for "Contains Text" creates above error, and timing fits reasonably well to assume that it's because of this change.

@demon: Could you look into this (or temporarily revert if it's more complicated, so the search works again)?

demon wrote on 2014-05-16 13:36:21 (UTC)

Yeah I ran out of space trying to index things so it screwed up. Gonna fix it today.

demon wrote on 2014-05-16 18:21:04 (UTC)

Fixed by moving the data to project storage and off of /var. Mostly reindexed now. Won't be a problem in production :)

demon wrote on 2014-05-16 18:23:59 (UTC)

And yet we're still getting parse exceptions. What's up Phabricator?

qgil wrote on 2014-05-19 22:21:52 (UTC)

Can we go back to the default search engine in this instance, please? I miss functional search here. As long as Elasticsearch works in the production instance all is fine.

aklapper wrote on 2014-05-21 10:12:58 (UTC)

@mmodell, @demon: Assuming this problem is related to not being able to edit dependencies (T347), could somebody please reset the search to the default backend?

demon wrote on 2014-05-21 21:42:47 (UTC)

Also, we don't know it'll work in production. Part of having a dev install is to test things we want to work in production.

I doubt it's related.

demon wrote on 2014-05-21 21:51:39 (UTC)

It's broken in 1.x+. See https://secure.phabricator.com/T4446

Going to see what we can do to help. Turning it off in the meantime.

aklapper wrote on 2014-05-23 06:01:30 (UTC)

Thanks @demon!

demon wrote on 2014-05-28 22:43:09 (UTC)

Upstream issue fixed, local install updated, Elasticsearch turned back on.

qgil wrote on 2014-06-05 17:35:30 (UTC)

Search is broken again.

aklapper wrote on 2014-06-15 12:13:25 (UTC)

Search works for me - if it's broken, steps to reproduce are welcome.

qgil wrote on 2014-06-16 19:13:39 (UTC)

Introduce any text string in the search that is not a project and click Enter.

For instance, typing "calendar": http://fab.wmflabs.org/search/query/l7k1ltV5ZSVP/#R

Nemo_bis wrote on 2014-06-17 20:32:33 (UTC)

Currently I'd have to run three separate queries in Maniphest as the advanced search only supports full words by default.

I suppose this means there is no regex search? Is there a ticket about that? (Links to docs appreciated in comment 0.)

aklapper wrote on 2014-06-18 10:16:35 (UTC)

In T159#43, @Nemo_bis wrote:

I suppose this means there is no regex search?

No idea if the default search backend supportes regexes. ElasticSearch (which this ticket is about) supports regexes: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-regexp-query.html but no idea if Phabricator's search UI allows entering regexes.

qgil wrote on 2014-07-09 14:05:25 (UTC)

(The search here in fab.wmflabs is still broken. Maybe we should just run the default search engine here, and then focus on Elasticsearch only in the official instance?)

qgil wrote on 2014-08-14 12:03:17 (UTC)

Search works again in this instance. Thank you!

aklapper wrote on 2014-08-25 12:43:02 (UTC)

To summarize what needs to be done here:
Once the Phabricator production instance is up, the last two steps of http://fab.wmflabs.org/T159#17 need to be executed on it.

qgil wrote on 2014-08-27 20:13:04 (UTC)

This has been created: T565: Recruit a Technical Writer for the Engineering Community team. Here in Labs we have already Elasticsearch backend (right?). Therefore, can we close this task?

Rush wrote on 2014-08-27 20:15:53 (UTC)

unless there is more to do? I wasn't sure so I let it be

importing issue status