Compliments of the season. Many of you may be asking where I have been and what I have been up to. For the past year and a bit, I’ve been working with the Astronomy crowd on the deployment of their OpenStack compute cluster for various astronomy projects. It’s been a great learning experience and I am grateful for the opportunity to have been involved in the process. Over the next few months, I will be handing over operational duties to the two newly employed Cloud Infrastructure guys. I am looking forward to coming back to HPC and working with researchers to build better tooling for their research environments.
Right, now for business. Everyone who knows me understands that I am all about the container space and what exciting opportunities it brings for researchers especially when it comes to portability and reusability of software. Last week I started to introduce Singularity containers for our HPC application space. Now, we have been running singularity containers for many years on the HEX cluster but we never really committed to making it the defacto standard. With the next-generation HPC which we built for the university, we hope to achieve this ambitious task.
We started out with TensorFlow. TensorFlow is an open source software library for high-performance numerical computation.
To access and use TensorFlow, authenticate to the UCT HPC Cluster and include the following in your slurm job submission script.
module load software/tensorflow-cpu-py2
Python, which TensorFlow makes use of at its core, is loaded as an environment alias through module which points at the execution parameters of singularity. This makes it easier for researchers to incorporate into their pipelines as opposed to having to call the singularity commands.
We have a number of modules available for TensorFlow which include both Python v2/3 as well as CPU/GPU variants. You are welcome to choose one of the following:
module load software/TensorFlow-CPU-py2
module load software/TensorFlow-CPU-py3
module load software/TensorFlow-GPU-py2
module load software/TensorFlow-GPU-py3