Page MenuHomePhabricator
Create Task
Maniphest T231828

Change HDFS balancer threshold
Closed, ResolvedPublic2 Estimated Story Points
Actions

Description

HDFS balancer default threshold is 10, meaning the variation of DFS-used is between minus and plus 10 percent. As the cluster is relatively full (70%+), this can lead to hosts being 85% full.

Details

SubjectRepoBranchLines +/-
Run hdfs balancer with threshold of 5%operations/puppetproduction+2 -2
Customize query in gerrit

Event Timeline

JAllemandou created this task.Sep 2 2019, 3:57 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 2 2019, 3:57 PM
gasserandreas moved this task from Incoming to Event Platform on the Analytics board.Sep 12 2019, 2:13 PM
Ottomata moved this task from Event Platform to Incoming on the Analytics board.Sep 20 2019, 8:47 PM
• fdans renamed this task from Should we change HDFS balancer threshold? to Change HDFS balancer threshold.Sep 23 2019, 3:25 PM
• fdans moved this task from Incoming to Operational Excellence on the Analytics board.
gerritbot added a comment.Oct 2 2019, 6:35 PM

Change 540457 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Run hdfs balancer with threshold of 5%

https://gerrit.wikimedia.org/r/540457

gerritbot added a project: Patch-For-Review.Oct 2 2019, 6:35 PM
Ottomata claimed this task.Oct 2 2019, 6:37 PM
Ottomata triaged this task as Low priority.
Ottomata added a project: Analytics-Kanban.
Ottomata moved this task from Next Up to In Code Review on the Analytics-Kanban board.
gerritbot added a comment.Oct 3 2019, 8:38 AM

Change 540457 merged by Elukey:
[operations/puppet@production] Run hdfs balancer with threshold of 5%

https://gerrit.wikimedia.org/r/540457

Stashbot added a comment.Oct 3 2019, 8:43 AM

Mentioned in SAL (#wikimedia-analytics) [2019-10-03T08:43:38Z] <elukey> apply 5% threshold to the HDFS balancer - T231828

Maintenance_bot removed a project: Patch-For-Review.Oct 3 2019, 9:10 AM
elukey moved this task from In Code Review to Done on the Analytics-Kanban board.Oct 4 2019, 1:42 PM
• Nuria closed this task as Resolved.Oct 24 2019, 7:09 PM
• Nuria set the point value for this task to 2.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits