Page MenuHomePhabricator

Tool Bothasava writes too many cache files to /tmp on the bastion
Closed, ResolvedPublic


Hi @Kotz, your tool bothasava writes 1000s of cache files to /tmp on the bastion.

root@tools-bastion-03:/tmp# ls -l | grep BotHasava_cache_ | wc -l

The disk space on the bastion is tight and pretty full right now. Are these files needed? Could you clean these up and ensure the tool cleans up these files on a regular basis? (I may have to clean it up if I don't hear back soon)


Event Timeline

I am cleaning up now. You may delete these files on whim. Most are very old and the tool should not be creating so many. I will track it going forward. thanks

added scheduled deletion to crontab

jsub -N deltemp -once -quiet find /tmp -iname "*hasava*" -user tools.bothasava -mtime 14 -type f -delete

jsub -N deltemp -once -quiet find /tmp -iname "*hasava*" -user tools.bothasava -mtime 14 -type f -delete

jsub executes the command on grid, not bastion.


I understand.

I use cron to run the bot periodically with jsub. I was afraid it was creating too many tmp files in there. maybe it was wrong of me assume that. Is /tmp saved between invocations on the grid?

I removed these files from bastion /tmp manually.

Is /tmp saved between invocations on the grid?

Yes and no. There are several things to consider here:

  • If the temporary file was properly created via [[|tmpfile(3)]], the file shall be deleted automatically upon exit.
  • If the temporary file was properly created via [[|mkstemp(3)]] or similar, the file shall be in a temporary directory created by grid, such as /tmp/5979439.1.task/. This directory, along with all the temp files you created with this method, will be deleted automatically by grid engine upon exit.
  • If the temporary file was improperly and arbitrarily created, in /tmp but not in the grid-provided directory (such as /tmp/5979439.1.task/), no one will delete them for you, as far as I am aware. You should delete them yourself.
    • However, the grid has many execution hosts, you should not expect to see the file again if you do a jsub to a random execution host. (i.e. Every jsub, in the majority of the time, you get the /tmp on a different host).

TL;DR: If the files were created properly, they are deleted automatically; otherwise, they will be left in a random host not deleted, and you might have to go through every host in order to find and delete the file.

But this task is about the bastion. You simply should keep the bastion clean and not create a ton of temporary file in it.



The bot needs to save some caching information which is best to be available across invocations. Is the best place to save it would be /data/project/bothasava/cachedir then ?


Yes, if you need the data saved across invocations, /data/project/ can be a choice, although [[|/data/scratch]] would be a better choice.

OK. I moved the cache file creation to /data/project/bothasava/ . thanks @zhuyifei1999