<jgleeson> Jack Gleeson 
well that was tricky
11:11 AM 
after reading cstone's email about the zombie recurring charge job, I went through the usual motions of trying to restart it by killing and removing the lock file, but it didn't work. running `run-job -k` gave the impression that the job was stopped when it wasn't and then, when trying to manually remove the lockfile, we can't do that either, as it's owned by the user 'jenkins', so it was stuck. 
11:17 AM 
my first thought was that we could be sneaky and inject the `rm /tmp/recurring_smashpig_charge.lock` command into the recurring charges slow-start command job manifest and push that out, since it will run as Jenkins. I tried that only to find out that run-job won't run any job commands, standard and slow-start, if a matching lock file already exists, grrrr, so that failed. 
11:21 AM 
After looking at it for a bit longer, I realised that the only thing blocking us here was the job name matching the stubborn lock file, and we could get around this by using an unrelated job file, which doesn't have a lock file blocking us, so I temporarily updated the matching gifts sync job manifest to include: 
11:21 AM 
slow_start_command:
11:21 AM 
 rm -f /tmp/recurring_smashpig_charge.lock
11:23 AM 
I pushed that up, ran slow-start on that job, and voila! the lock file was gone