Improve documentation about Toolforge checker services
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• GTirloni
	Dec 5 2018, 1:55 PM

Description

Document all services running on tools-checker-* nodes.

Explain how the checks are made, how they are made available to monitoring and how Icinga uses them.

Add possible troubleshooting steps.

Event Timeline

• GTirloni created this task.Dec 5 2018, 1:55 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 5 2018, 1:55 PM

There is a toolschecker tool that has the following crontab:

*/5 * * * *  /usr/bin/jsub -N cron-tools.toolschecker-1 -once -quiet touch /data/project/toolschecker/crontest.txt

toolschecker.py has this about it:

@check('/toolscron')
def cron_check():
    ''' A tools cron job touches a file every five minutes.  This test verifies
        that the mtime is appropriately recent.'''
    filepath = '/data/project/toolschecker/crontest.txt'
    tenminutes = 60 * 10
    mtime = os.path.getmtime(filepath)
    if time.time() - mtime < tenminutes:
        return True
    return False

So this seems to be checking if: