Page MenuHomePhabricator

Clean up graphite1001.eqiad.wmnet, now that coal has been moved
Closed, ResolvedPublic

Description

Now that coal and coal-web have been moved to the webperf machines, a few cleanup steps are needed on both graphite1001 and graphite2001. These require root access (which I don't have), so leaving it to someone who does have that access to run them.

These can be run any time.

  • Stop coal services. Coal should be already, just try again in case.
sudo systemctl stop uwsgi-coal
sudo systemctl stop coal
  • Disable coal services
sudo systemctl disable uwsgi-coal
sudo systemctl disable coal
  • Remove the systemd control files
sudo rm /lib/systemd/system/uwsgi-coal.service
sudo rm /lib/systemd/system/coal.service
  • Reset the systemd daemon to remove those services
sudo systemctl daemon-reload
  • Remove the symlinked coal whisper storage dir, create the new dir, and copy the data files
  • note: leaves a copy of the files in /var/lib/coal, just in case.
sudo rm /var/lib/carbon/whisper/coal
sudo mkdir /var/lib/carbon/whisper/coal
sudo chown _graphite:_graphite /var/lib/carbon/whisper/coal
sudo cp -r /var/lib/coal/*.wsp /var/lib/carbon/whisper/coal/
sudo chown -R _graphite:_graphite /var/lib/carbon/whisper/coal/
  • Remove nagios/nrpe configs:
  • note: slightly adjusted reload command
sudo rm /etc/nagios/nrpe.d/check_uwsgi-coal.cfg
sudo systemctl reload nagios-nrpe-server.service
  • Remove uwsgi config
  • note: this was already done, it was "no such file or directory" already when i ran it
rm /etc/uwsgi/apps-enabled/coal.ini
rm /etc/uwsgi/apps-available/coal.ini

Give me a shout if there are any questions about this - marlier on IRC.

Event Timeline

Mentioned in SAL (#wikimedia-operations) [2018-05-09T19:23:07Z] <Krinkle> Disable coal service on graphite1001/graphite2001 (T194283)

Mentioned in SAL (#wikimedia-operations) [2018-05-09T19:23:14Z] <Krinkle> Stop and disable coal-web (uwsgi-coal) service on graphite1001/graphite2001 (T194283)

I've performed this subset of commands on graphite1001 and graphite2001, as I noticed coal-web running there.

$ sudo systemctl stop uwsgi-coal
$ sudo systemctl stop coal

$ sudo systemctl disable uwsgi-coal
> Removed symlink /etc/systemd/system/multi-user.target.wants/uwsgi-coal.service.

$ sudo systemctl disable coal
> Removed symlink /etc/systemd/system/multi-user.target.wants/coal.service.

Mentioned in SAL (#wikimedia-operations) [2018-05-09T19:36:52Z] <mutante> graphite1001, graphite2001 - deleting uwsgi-coal and coal sytemd unit files; systemctl daemon-reload (T194283)

Dzahn updated the task description. (Show Details)

Mentioned in SAL (#wikimedia-operations) [2018-05-09T19:49:18Z] <mutante> graphite1001/2001 - rm check_uwsgi-coal NRPE check config, reloading nagios-nrpe-server (T194283)

Dzahn updated the task description. (Show Details)
Dzahn updated the task description. (Show Details)

all done, see checkboxes in the description and notes, i appreciate the detailed ticket with command lines :)

Vvjjkkii renamed this task from Clean up graphite1001.eqiad.wmnet, now that coal has been moved to kadaaaaaaa.Jul 1 2018, 1:11 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed Dzahn as the assignee of this task.
Vvjjkkii triaged this task as High priority.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii edited subscribers, added: Dzahn; removed: Aklapper.
CommunityTechBot renamed this task from kadaaaaaaa to Clean up graphite1001.eqiad.wmnet, now that coal has been moved.Jul 1 2018, 2:58 PM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to Dzahn.
CommunityTechBot raised the priority of this task from High to Needs Triage.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot edited subscribers, added: Aklapper; removed: Dzahn.