Page MenuHomePhabricator

Upgrade to elasticsearch 6.5.4 for cirrus / codfw
Closed, ResolvedPublic

Event Timeline

Gehel triaged this task as High priority.Mar 21 2019, 1:17 PM

Change 498079 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] elasticsearch: upgrade to elastic 6.5.4 for cirrus / codfw

https://gerrit.wikimedia.org/r/498079

Change 498080 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] elasticsearch: deploy elasticsearch config for ES6 cirrus / codfw

https://gerrit.wikimedia.org/r/498080

Change 498318 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/mediawiki-config@master] [cirrus] switch low volume wikis to eqiad (elastic 6.5.4)

https://gerrit.wikimedia.org/r/498318

Change 498319 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/mediawiki-config@master] [cirrus] switch all wikis to eqiad (elastic 6.5.4)

https://gerrit.wikimedia.org/r/498319

Change 498318 merged by jenkins-bot:
[operations/mediawiki-config@master] [cirrus] switch low volume wikis to eqiad (elastic 6.5.4)

https://gerrit.wikimedia.org/r/498318

Mentioned in SAL (#wikimedia-operations) [2019-03-25T17:10:30Z] <dcausse@deploy1001> Synchronized wmf-config/InitialiseSettings.php: T218878: [cirrus] switch low volume wikis to eqiad (elastic 6.5.4) (duration: 00m 49s)

Change 498319 merged by jenkins-bot:
[operations/mediawiki-config@master] [cirrus] switch all wikis to eqiad (elastic 6.5.4)

https://gerrit.wikimedia.org/r/498319

Change 498079 merged by Gehel:
[operations/puppet@production] elasticsearch: upgrade to elastic 6.5.4 for cirrus / codfw

https://gerrit.wikimedia.org/r/498079

Change 498080 merged by Gehel:
[operations/puppet@production] elasticsearch: deploy elasticsearch config for ES6 cirrus / codfw

https://gerrit.wikimedia.org/r/498080

Mentioned in SAL (#wikimedia-operations) [2019-03-28T08:38:42Z] <gehel> retry shard allocation on elasticsearch codfw all clusters (curl -k -XPOST 'https://localhost:9243/_cluster/reroute?pretty&explain=true&retry_failed') - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:00:43Z] <gehel> restarting elasticsearch-psi on elastic2036 (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:05:59Z] <gehel> restarting elasticsearch-psi on elastic20[35,36,53] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:09:11Z] <gehel> restarting elasticsearch-omega on elastic2050 (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:19:13Z] <gehel> restarting elasticsearch-omega on elastic20[38,50] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:28:30Z] <gehel> restarting elasticsearch on elastic20[25,27] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:37:27Z] <gehel> restarting elasticsearch-psi on elastic20[39,40] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:42:10Z] <gehel> restarting elasticsearch on elastic20[28,29,41] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T09:56:36Z] <gehel> restarting elasticsearch-omega on elastic2031 (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T10:11:23Z] <gehel> restarting elasticsearch-omega on elastic2050 (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T10:22:54Z] <gehel> restarting elasticsearch on elastic20[34,36,50] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T10:54:09Z] <gehel> restarting elasticsearch-psi on elastic20[35,36,53] (shards stuck in recovery) - T218878

Mentioned in SAL (#wikimedia-operations) [2019-03-28T14:07:00Z] <gehel> reindexing changes from '2019-03-26T12:00:00Z' to '2019-03-28T12:00:00Z' into cirrus / elasticsearch - T218878