Secure. Portable. Encryptable.Apptainer is a secure, portable, and easy-to-use container system that provides absolute trust and security. It is widely used across industry and academia in many areas…
Secure. Portable. Encryptable.
Apptainer is a secure, portable, and easy-to-use container system that provides absolute trust and security. It is widely used across industry and academia in many areas of work.
Apptainer allows unprivileged users to use containers and prohibits privilege escalation within the container; users are the same inside and outside the container.
The single-file SIF container format allows you to reproducibly build, share, and archive your workload from workstations to HPC to the edge.
Apptainer can encrypt containers and integrates with Vault and other secret management platforms to secure applications, models, and data.
We tried to use this on our compute cluster for silicon design/verification. We gave up in the end and just went with the traditional TCL (now Lua) modules.
The problems are:
1. You can't have apptainers that use each other. The most common case was things like Make, GCC, Git etc. If Make is in a different apptainer to GCC then it won't work because as soon as you go into Make then it can't see GCC any more.
2. It doesn't work if any of your output artefacts depend on things inside the container. For example you use your GCC apptainer to compile a program. It appears to work, but when you run it you find it actually linked to something in the apptainer that isn't visible any more. This is also a problem for C headers.
3. We had constant issues with PATH getting messed up so you can't see things outside the apptainer that should have been available.
All in all it was a nice idea but ended up causing way more hassle than it was worth. It was much easier just to use an old OS (RHEL8) and get everything to work directly on that.
So you are using container app and your biggest problem with it is, that it's doing exactly as advertised -> containers :D
If you want to be unnecessarily dismissive about the problems with containers, sure.
I think of using Apptainer/Singularity as more like Docker than anything else (without the full networking configs). These are all issues with traditional Docker containers as well, so I’m not sure how you were using the containers or what you were expecting.
For my workflows on HPC, I use apptainers as basically drop-in replacements for Docker, and for that, they work quite well. These biggest benefit is that the containers are unprivileged. This means you can’t do a lot of things (in particular complex networking), but it also makes it much more secure for multi-tenant systems (like HPC).
(I know Docker and Apptainer are slightly different beasts, but I’m speaking in broad strokes in a general sense without extra permissions).
You don't mix and match pieces of containers, just like you wouldn't mix and match binaries from different distributions of Linux.
You can use a container as a single environment in which to do development, and that works fine. But they are by definition an isolated environment with different dependencies than other containers. The result of compiling something in a container necessarily needs to end up in its own container.
...that said, you could use the exact same container base image, and make many different container images from it, and those files would be compatible (assuming you shipped all needed dependencies).
> you wouldn't mix and match binaries from different distributions of Linux.
You can absolutely mix and match lots of different binaries from different sources on one Linux system. That's exactly what we're doing now with TCL modules.
> and make many different container images from it
Well yes, that's the problem. You end up either putting everything in one container (in which case why bother with a container?), or with a combinatorial explosion of every piece and version of software you might use.
TCL modules are better. They don't let you cheat like containers do, but in return you get a better system.
Great to see Apptainer getting some attention. It generally excels over other container options (like Docker and Podman) in these scenarios:
- Need to run more than one activity in a single container (this is an anti-pattern in other container technologies)
- HPC (and sometimes college) environments
- Want single-file distribution model (although doesn't support deltas)
- Cryptographically sign a SIF file without an external server
- Robust GPU support
> Want single-file distribution model (although doesn't support deltas)
You can achieve that with docker by `docker save image-name > image-name.tar.gz` and `docker load --input image-name.tar.gz`.
It likewise doesn't support deltas but there was a link here on HN recently to something called "unregistry" which allows for doing "docker push" to deploy an image to a remote machine without a registry, and that thing does take deltas into account.
Apptainer and singularity ce are quite common in HPC. While both implementations fork the old singularity project, they are not really identical.
We use singularity in the HPCs (like Leonardo, LUMI, Fugaku, NeSI NZ, Levante) but some devs and researchers have apptainer installed locally.
We found a timezone bug a few days ago in our Python code (matplotlib,xarray,etc.), but that didn't happen with apptainer.
As the code bases are still a bit similar, I could confirm apptainer fixed it but singularity ce was still affected by the bug -- singularity replaces the UTC timezone file by the user's timezone, Helsinki EEST in our case in LUMI HPC.
> Apptainer and singularity ce are quite common in HPC. While both implementations fork the old singularity project, they are not really identical.
Apptainer is not a fork of the old Singularity project: Apptainer is the original project, but the community voted to change its name. It also came under the umbrella of the Linux Foundation:
* https://apptainer.org/news/community-announcement-20211130/
Sylabs (where the original Singularity author first worked) was the one that forked off the original project.
Luckily they’re still compatible with each others containers. Can use Apptainer to build the container then run it on Singularity and vice-versa.