Page MenuHomePhabricator

Fix ExternalIndex to support groups & replica topology
Closed, ResolvedPublic

Description

The multi-instance support code added for ExternalIndex was designed without the group+replica concepts in mind.
It works with a new config var CirrusSearchExtraIndexClusters but is now ambiguous and overkill as with group&replicas we now "force" the cluster to have a consistent shape. Additionally we can no longer reference a cluster directly by its name in CirrusSearchClusters because for writing we now use the replica (DC).

In short the way to configure ExternalIndices on WMF setup will just be to change:

CirrusSearchExtraIndexes: { NS_FILE: "commonswiki_file" }

to

CirrusSearchExtraIndexes: { NS_FILE: "chi:commonswiki_file" }

We also need to find a way to disable OtherIndex updates for a particular cluster, this will be needed during the transition phase where we want to write docs to their wiki index but not duplicate the writes to a commonswiki_file index that does not exist on psi & omega...

Event Timeline

dcausse created this task.Dec 17 2018, 1:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 17 2018, 1:12 PM
Restricted Application added a project: Discovery-Search. · View Herald TranscriptDec 17 2018, 1:13 PM
dcausse claimed this task.Dec 17 2018, 1:21 PM
dcausse triaged this task as Normal priority.
dcausse moved this task from needs triage to Current work on the Discovery-Search board.

Change 480105 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Fix OtherIndex to comply with replica+group concepts

https://gerrit.wikimedia.org/r/480105

dcausse updated the task description. (Show Details)Dec 17 2018, 9:45 PM

Change 480105 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Fix OtherIndex to comply with replica+group concepts

https://gerrit.wikimedia.org/r/480105

debt closed this task as Resolved.Jan 14 2019, 11:48 PM