The UCT HPC cluster has a wide range of codes installed. However occasionally these need to be extended to add additional functionality by adding packages or libraries. In order to keep the base install clean we require researchers to add these packages or libraries locally as they do not have write access to /opt/exp_soft, the central software volume.
Important
Both methods outlined below induce high load on the server they are run on due to the fact that code compilation occurs. Please do not do this on the head node, rather start an interactive job and then exit the job once the packages are installed. To start an interactive job:
sint –account=<your_account> –time=10:00:00 –partition=ada –ntasks=1 –nodes=1
R
To install packages locally the following method can be used:
Start an interactive job and activate the version of R you require with module
module load software/R-3.6.0
Start R and install the package
install.packages("XYZ")
You will immediately be challenged by the installer as you do not have write rights to the software location
Warning in install.packages("XYZ") :
'lib = "/opt/exp_soft/R-3.6.0/lib64"' is not writable
Would you like to use a personal library instead? (yes/No/cancel)
Enter yes, the package will then download, compile and install. You can then load the library.
library(XYZ)
Once this is confirmed as working you no longer need to install the package each time you start R, just load the library.
Python\Anaconda pip
The pip package management tool can be used to install packages locally.
module load python/anaconda-python-3.7 pip install --user PackageName
If a version of the package you’re trying to install exists in /opt/exp_soft already then pip will not install locally as the package already exists from its point of view. To get around this add the –ignore-installed argument.
pip install --user PackageName --ignore-installed
Python\Anaconda virtual environment
Python has a specific tool used to install packages, conda. Python also comes with a mechanism to set up virtual environments. Below is an example of setting up the short read aligner bowtie:
Start an interactive job, then:
module load python/anaconda-python-3.7 conda create -y -n RelevantName source activate RelevantName conda config --add channels defaults conda config --add channels bioconda conda config --add channels conda-forge conda install bowtie conda deactivate exit
The above only needs to be done once. The term RelevantName should be a short text word describing your project\environment. Additionally the channel names are for bowtie specifically. Your package may require different channels. Now when you launch jobs your script should contain:
module load python/anaconda-python-3.7 source activate RelevantName