Page MenuHomePhabricator

Fix ExternalIndex to support groups & replica topology
Closed, ResolvedPublic

Description

The multi-instance support code added for ExternalIndex was designed without the group+replica concepts in mind.
It works with a new config var CirrusSearchExtraIndexClusters but is now ambiguous and overkill as with group&replicas we now "force" the cluster to have a consistent shape. Additionally we can no longer reference a cluster directly by its name in CirrusSearchClusters because for writing we now use the replica (DC).

In short the way to configure ExternalIndices on WMF setup will just be to change:

CirrusSearchExtraIndexes: { NS_FILE: "commonswiki_file" }

to

CirrusSearchExtraIndexes: { NS_FILE: "chi:commonswiki_file" }

We also need to find a way to disable OtherIndex updates for a particular cluster, this will be needed during the transition phase where we want to write docs to their wiki index but not duplicate the writes to a commonswiki_file index that does not exist on psi & omega...

Event Timeline

dcausse triaged this task as Medium priority.
dcausse moved this task from needs triage to Current work on the Discovery-Search board.

Change 480105 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Fix OtherIndex to comply with replica+group concepts

https://gerrit.wikimedia.org/r/480105

Change 480105 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Fix OtherIndex to comply with replica+group concepts

https://gerrit.wikimedia.org/r/480105