Page MenuHomePhabricator

Reshard commonswiki_file elasticsearch index
Closed, DeclinedPublic8 Estimated Story Points

Description

commonswiki_file shards have crossed our 50GB threshold in codfw and cloudelastic.

Change primary shard count in mediawiki-config and deploy it, then kick off reindex

Event Timeline

CBogen set the point value for this task to 3.Aug 17 2020, 5:17 PM

Change 628980 had a related patch set uploaded (by Ryan Kemper; owner: Ryan Kemper):
[operations/mediawiki-config@master] cirrussearch: increase up commonswiki_file shards

https://gerrit.wikimedia.org/r/628980

Increasing the number of shards for commons wiki is starting to be an issue. We need a better strategy.

Change 634381 had a related patch set uploaded (by Ryan Kemper; owner: Ryan Kemper):
[operations/puppet@production] Bring 3 new eqiad wdqs nodes into service

https://gerrit.wikimedia.org/r/634381

Change 628980 abandoned by Ryan Kemper:
[operations/mediawiki-config@master] cirrussearch: increase up commonswiki_file shards

Reason:
we ended up just bumping the limit, so there is no need to re-shard this index anymore

https://gerrit.wikimedia.org/r/628980

Closing this ticket because we ended up changing the alert thresholds which removes the need to re-shard the index

Housekeeping note: see https://phabricator.wikimedia.org/T265908 for the patch that changes the alert thresholds, which should clear the alerts that led to this ticket originally being created.

dcausse removed RKemper as the assignee of this task.
dcausse moved this task from In Progress to Incoming on the Discovery-Search (Current work) board.
dcausse subscribed.

Re-opening as it's alerting again

Joe triaged this task as Medium priority.
Joe subscribed.

@RKemper I'm assigning this task to you, since it seems you're acting on it. Please remove yourself as assignee if that's not ok.

@RKemper I'm assigning this task to you, since it seems you're acting on it. Please remove yourself as assignee if that's not ok.

Thanks, your assumption is correct!

CBogen changed the point value for this task from 3 to 8.Jan 4 2021, 4:39 PM
CBogen moved this task from Incoming to In Progress on the Discovery-Search (Current work) board.

We could probably cancel this? In T271493 we are fixing the data size issues which will remove the need to re-shard.

Agreed, we should re-assess the shard sizes end of March 2021.