Page MenuHomePhabricator

Automate run of refreshNodes on masters
Closed, ResolvedPublic

Description

When we exclude a host from hadoop we have to manually run below commands once puppet has run
Once done, run puppet on both Master nodes (an-master1001 and an-master1002) to update the exclude file. Then run hdfs dfsadmin -refreshNodes command for each NameNode FS URI (run these command on an-master1001 for example):

sudo -u hdfs hdfs dfsadmin -fs hdfs://an-master1001.eqiad.wmnet:8020 -refreshNodes
sudo -u hdfs hdfs dfsadmin -fs hdfs://an-master1002.eqiad.wmnet:8020 -refreshNodes

Run this on each ResourceManager host (an-master1001 and an-master1002 for example):

sudo -u hdfs yarn rmadmin -refreshNodes

We should update puppet code to automatically trigger those commands if hosts.exclude file is updated

Event Timeline

Change 893999 had a related patch set uploaded (by Nicolas Fraison; author: Nicolas Fraison):

[operations/puppet@production] hadoop: automate refresh of exclude nodes in NN adn RM

https://gerrit.wikimedia.org/r/893999

Change 893999 merged by Nicolas Fraison:

[operations/puppet@production] hadoop: automate refresh of exclude nodes in NN and RM

https://gerrit.wikimedia.org/r/893999

nfraison claimed this task.