Running containers with singularity
After having completed this chapter you will be able to:
- Login to a remote machine with
singularity pullto convert an image from dockerhub to the ‘singularity image format’ (
- Execute a singularity container
- Explain the difference in default mounting behaviour between
singularity shellto generate an interactive shell inside a
- Search and use images with both
- Singularity documentation
- Singularity hub
- An article on Docker vs Singularity
- Using conda and containers with snakemake
Login to remote
If you are enrolled in the course, you have received an e-mail with an IP, username, private key and password. To do the Singularity exercises we will login to a remote server. Below you can find instructions on how to login.
Open a terminal, and
cd to the directory where you have stored your private key. After that, change the file permissions of the key:
chmod 400 key_<username>.pem
Then, login like this:
ssh -i key_<username>.pem <username>@<IP>
Below you can find video tutorials and information to log in with MobaXterm.
MobaXterm is an SSH client for Windows. You can use it to connect to the remote host and edit remote scripts. With MobaXterm, you will automatically login to the remote server once you’ve started the SSH session. Set it up on your own computer using your own credentials and the video below.
Pulling an image
Singularity can take several image formats (e.g. a
docker image), and convert them into it’s own
.sif format. Unlike
docker this image doesn’t live in a local image cache, but it’s stored as an actual file.
Exercise: On the remote server, pull the docker image that has the adjusted default
CMD that we have pushed to dockerhub in this exercise (
singularity pull. The syntax is:
singularity pull docker://[USER NAME]/[IMAGE NAME]:[TAG]
singularity pull docker://[USER NAME]/ubuntu-figlet:v3
If you weren’t able to push the image in the previous exercises to your docker hub, you can use
geertvangeest as username to pull the image.
Executing an image
.sif files can be run as standalone executables:
And you can overwrite the default command like this:
./[IMAGE NAME].sif [COMMAND]
Exercise: Run the
.sif file without a command, and with a command that runs
figlet. Do you get expected output?
Running it without a command (
./ubuntu-figlet_v3.sif) should give:
__ __ _ _ _ | \/ |_ _ (_)_ __ ___ __ _ __ _ ___ __ _____ _ __| | _____| | | |\/| | | | | | | '_ ` _ \ / _` |/ _` |/ _ \ \ \ /\ / / _ \| '__| |/ / __| | | | | | |_| | | | | | | | | (_| | (_| | __/ \ V V / (_) | | | <\__ \_| |_| |_|\__, | |_|_| |_| |_|\__,_|\__, |\___| \_/\_/ \___/|_| |_|\_\___(_) |___/ |___/
Running with a another
./ubuntu-figlet_v3.sif figlet 'Something else'
____ _ _ _ _ / ___| ___ _ __ ___ ___| |_| |__ (_)_ __ __ _ ___| |___ ___ \___ \ / _ \| '_ ` _ \ / _ \ __| '_ \| | '_ \ / _` | / _ \ / __|/ _ \ ___) | (_) | | | | | | __/ |_| | | | | | | | (_| | | __/ \__ \ __/ |____/ \___/|_| |_| |_|\___|\__|_| |_|_|_| |_|\__, | \___|_|___/\___| |___/
Mounting with Singularity
Singularity is also different from Docker in the way it handles mounting. By default, Singularity binds your home directory and a number of paths in the root directory to the container. This results in behaviour that is almost like if you are working on the directory structure of the host.
Running the command
pwd (full name of current working directory) will therefore result in a path on the host machine:
Exercise: Run the above command. What is the output? How would the output look like if you would run a similar command with Docker?
A similar Docker command would look like (run this on your local computer):
docker run --rm ubuntu-figlet:v3 pwd
The output of
./ubuntu-figlet_v3.sif pwd is the current directory on the host: i.e.
/home/username if you have it in your home directory. The output of
docker run --rm ubuntu-figlet:v3 pwd (on the local host) would be
/, which is the default workdir (root directory) of the container. As we did not mount any host directory, this directory exists only within the container (i.e. separated from the host).
If you want to debug or inspect an image, it can be helpful to have a shell inside the container. You can do that with
singularity shell ubuntu-figlet_v3.sif
To exit the shell type
Exercise: Can you run
figlet inside this shell?
Singularity> figlet test _ _ | |_ ___ ___| |_ | __/ _ \/ __| __| | || __/\__ \ |_ \__\___||___/\__|
During the lecture you have learned that singularity takes over the user privileges of the user on the host. You can get user information with command like
Exercise: Run the
figlet container interactively. Do you have the same user privileges as if you were on the host? How is that with
A command like
whoami will result in your username printed at stdout:
Singularity> whoami myusername Singularity> id uid=1030(myusername) gid=1031(myusername) groups=1031(myusername),1001(condausers) Singularity> groups myusername condausers
With singularity, you have the same privileges inside the singularity container as on the host. If you do this in the docker container (based on the same image), you’ll get output like this:
root@a3d6e59dc19d:/# whoami root root@a3d6e59dc19d:/# groups root root@a3d6e59dc19d:/# id uid=0(root) gid=0(root) groups=0(root)
A bioinformatics example (extra)
Check out biocontainers.pro. This registry contains a large collection of containerized bioinformatic tools. These are very powerful for pipeline development. You can pull one of their images from dockerhub using
singularity like this:
singularity pull docker://biocontainers/fastqc:v0.11.9_cv7
You can directly pull a
singularity image like so (as shown on the
singularity pull fastqc_0.11.9.sif https://depot.galaxyproject.org/singularity/fastqc:0.11.9--0
Let’s test the image. Download some sample reads first:
mkdir reads cd reads wget https://introduction-containers.s3.eu-central-1.amazonaws.com/ecoli_reads.tar.gz tar -xzvf ecoli_reads.tar.gz rm ecoli_reads.tar.gz
Now you can simply run the image as an executable preceding the commands you would like to run within the container. E.g. running
fastqc would look like:
cd ./fastqc_v0.11.9_cv7.sif fastqc ./reads/ecoli_*.fastq.gz
This will result in
html files in the directory
./reads. These are quality reports for the sequence reads. If you’d like to view them, you can download them with
scp or e.g. FileZilla, and view them with your local browser.