Page MenuHomePhabricator

Add JMX and Prometheus monitoring to Hadoop Journal nodes
Closed, ResolvedPublic3 Story Points

Description

The HADOOP_JOURNALNODE_OPTS variable seems to work if added to hadoop-env.sh to add JMX config to the HDFS Journal Node opts. After that we should also add the Prometheus JMX agent to monitor its metrics.

Event Timeline

elukey created this task.Apr 24 2018, 1:10 PM
elukey triaged this task as Normal priority.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2018, 1:10 PM

Change 428632 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet/cdh@master] Allow the configuration of JMX and extra jvm opts for journal nodes

https://gerrit.wikimedia.org/r/428632

elukey renamed this task from Add JMX and Prometheus monitoring to Hadoop Journal nodes' JVMs to Add JMX and Prometheus monitoring to Hadoop Journal node JVMs.Apr 24 2018, 1:37 PM
elukey renamed this task from Add JMX and Prometheus monitoring to Hadoop Journal node JVMs to Add JMX and Prometheus monitoring to Hadoop Journal nodes.

Change 428632 merged by Elukey:
[operations/puppet/cdh@master] Allow the configuration of JMX and extra jvm opts for journal nodes

https://gerrit.wikimedia.org/r/428632

Change 428636 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::hadoop::common: allow jvm opts for journalnodes

https://gerrit.wikimedia.org/r/428636

Change 428640 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::hadoop::worker: deploy the journalnode prometheus config

https://gerrit.wikimedia.org/r/428640

Change 428636 merged by Elukey:
[operations/puppet@production] profile::hadoop::common: allow jvm opts for journalnodes

https://gerrit.wikimedia.org/r/428636

Change 428640 merged by Elukey:
[operations/puppet@production] profile::hadoop::worker: deploy the journalnode prometheus config

https://gerrit.wikimedia.org/r/428640

Change 428669 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] role::analytics_cluter::hadoop::worker: enable jmx metrics for journalnodes

https://gerrit.wikimedia.org/r/428669

Change 428669 merged by Elukey:
[operations/puppet@production] role::analytics_cluter::hadoop::worker: enable jmx metrics for journalnodes

https://gerrit.wikimedia.org/r/428669

Change 428695 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::hadoop::monitoring::journalnode: correct port

https://gerrit.wikimedia.org/r/428695

Change 428695 merged by Elukey:
[operations/puppet@production] profile::hadoop::monitoring::journalnode: correct port

https://gerrit.wikimedia.org/r/428695

elukey set the point value for this task to 3.Apr 24 2018, 5:39 PM
elukey moved this task from Next Up to Done on the Analytics-Kanban board.
elukey moved this task from Backlog to Done on the User-Elukey board.Apr 25 2018, 10:19 AM
Nuria closed this task as Resolved.May 8 2018, 10:44 PM
Vvjjkkii renamed this task from Add JMX and Prometheus monitoring to Hadoop Journal nodes to uceaaaaaaa.Jul 1 2018, 1:14 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii raised the priority of this task from Normal to High.
Vvjjkkii removed elukey as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed the point value for this task.
Vvjjkkii removed subscribers: gerritbot, Aklapper.
Lofhi renamed this task from uceaaaaaaa to Add JMX and Prometheus monitoring to Hadoop Journal nodes.Jul 1 2018, 7:03 AM
Lofhi closed this task as Resolved.
Lofhi lowered the priority of this task from High to Normal.
Lofhi assigned this task to elukey.
Lofhi updated the task description. (Show Details)
Lofhi set the point value for this task to 3.
Lofhi added subscribers: gerritbot, Aklapper.