Page MenuHomePhabricator

When running updateSearchIndexConfig.php for test.wikidata, the script chokes on the analyzers
Closed, ResolvedPublic


mwscript extensions/CirrusSearch/maintenance/updateSearchIndexConfig.php --wiki testwikidatawiki
content index...
	Fetching Elasticsearch version...1.7.1...ok
	Scanning available plugins...
		langdetect-, wikimedia-extra, analysis-icu, swift-repository, experimental highlighter
		head, whatson
	Infering index identifier...testwikidatawiki_content_1415316749
	Picking analyzer...english
	Index exists so validating...
		Validating number of shards...ok
		Validating replica range...ok
		Validating shard allocation settings...done
		Validating max shards per node...ok
	Validating analyzers...cannot correct
This script encountered an index difference that requires that the index be
copied, indexed to, and then the old index removed. Re-run this script with the
--reindexAndRemoveOk --indexIdentifier=now parameters to do this.

on beta wikidata, I had no problems with this script and locally, normally it's ok. is a patch from the Cirrus team that adds an option of updating/validating only the mapping, so could be a work around.

running --reindexAndRemoveOk would probably also be ok in the case of test.wikidata, but if we have the same problem with wikidata then reindex can be more complex and not a great option.

Event Timeline

aude raised the priority of this task from to High.
aude updated the task description. (Show Details)
aude added projects: Wikidata, CirrusSearch.
aude subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
aude renamed this task from When running updateSearchIndexCzyonfig.php for test.wikidata, the script chokes on the analyzers to When running updateSearchIndexConfig.php for test.wikidata, the script chokes on the analyzers.Oct 21 2015, 4:00 PM
aude set Security to None.

From the CirrusSearch README:

B.  Build a copy of the index, reindex to it, and then force a full reindex from source documents.  Uses
more disk space but search should be up the entire time:
 php updateSearchIndexConfig.php --reindexAndRemoveOk --indexIdentifier now
 php forceSearchIndex.php

So this seems to be as expected and should work just fine, unless that readme is wrong.

the patch that adds the --justMapping option got split up into two patches, one that adds the option to the script, and the second patch for "allowing mapping customization with numeric fields"

this is the patch that adds the option:

backported the --justMapping patch and the problem is resolved now

aude moved this task from Proposed to Done on the Wikidata-Sprint-2015-10-13 board.