GPCC Home

SSH Information

Login Service

Statistics and Computation Service

• SCS on Linux
• GIS Programs
• Math Programs
• Statistics Programs

Long-Running Jobs

The Problem: AFS Token Expiration

Your home directory is located in AFS, which is a secure, networked file system. AFS implements its security using kerberos. When you log in, you are automatically issued AFS kerberos tokens, which are set to expire after 25 hours, by default. In addition, when you log out of the system, your tokens are invalidated. So, any jobs that remain after you log out, or after your tokens expire will no longer be able to write into AFS.

There are ways to set jobs up so that they can be issued tokens to use and so they can renew them, but that adds an additional layer of complexity. Instead, our recommended solution to this problem is, rather, to use local disk space for long-running jobs, since access to local disk space is less of a problem and usually faster than access to AFS.

A Solution: Write to Local Disk

For purely temporary files, you can create a directory under /tmp that bears your uniqname. For example, if your uniqname was grundoon, the commands

mkdir /tmp/grundoon

will create a directory for your temporary files. If you wish to keep your data files private, you should set the permissions so that only you can read them. This can be done with the command (continuing the example)

chmod -R go-rwx /tmp/grundoon

Any directories that do not bear the name of their owner, and any files kept directly under /tmp will be deleted periodically.

The local disk is NOT backed up, so you should not view it as a replacement for IFS. Rather, it should be used as a supplement. Local disk is a great place for temporary files. In the course of analysis, many versions of data files may be created while the most appropriate coding is sought, while data cleaning is undertaken, or during the course of exploratory analysis. Once a preliminary change to your data has been made, you should make a copy of it back into IFS where it can be backed up.