On most clusters storage is split into 3 different types each of which has different advantages and limitations. Additionally source code can be stored on one of the version control servers run by the Faculty of Science Computing Unit.
|Name||Location||Can also be called|
|Cluster Home Drive||
|UNSW Home Drive||
Cluster Home Directories
Your home directory $HOME (e.g. /home/z1234567) is your personal storage area on the cluster and it is available on every node in the cluster. Any files stored in your home directory are regularly backed up, but a quota is imposed on each user. You can check your current usage with the quota command. The home directory is suitable for storing relatively small but important files such as documents, scripts and source code.
Your Cluster home directory is available on all of the nodes in the cluster.
Each user is also provided with a global scratch directory (e.g. /srv/scratch/z1234567). It is available on every node in the cluster, however, unlike home directories it is not backed up and there is no quota. The only limit imposed on global scratch is the physical capacity of the storage device. It is a shared resource and care must be taken by each user to avoid exceeding reasonable levels of usage. Everytime a user logs in they are presented with a usage summary to help monitor personal and total usage. Global scratch is not for long term storage of data; it is ideally suited to the temporary storage of data for running jobs.
Global scratch is available on all the nodes in the cluster and should be used for any large files.
UNSW H-Drive and Shared Drives
When you log on to a cluster your UNSW H-Drive and Shared Drives are not available. If you run the following command
then your drives will be available at
$HOME/maths (for example) respectively.
Your UNSW Network drives are NOT available on the cluster nodes.
CCRC data storage
If you are in the CCRC and have been given access to one of the data directories then you will find the storage under /srv/ccrc.
These drives are NOT available on cluster nodes.
Every compute node provides an amount of disk space as local scratch. The total capacity of local scratch is typically 200GB. When a job starts on a compute node, a directory within local scratch is automatically created for the sole use of the job, and it is automatically removed when the job finishes. The location of the local scratch directory is available via the $TMPDIR environment variable within the PBS job script. Local scratch space is ideally suited for I/O intensive jobs that read and write intermediate files. However, any files required beyond the life of the job must be copied elsewhere (e.g. global scratch) before the job finishes otherwise the data will be lost. If a job requires local scratch space then it is highly recommended to declare the local storage requirements in a resource request within the job script.
If you have I/O intensive jobs then you can automate the transfer of files to and from local scratch using the prologue and epilogue commands.