First lightning talk is Christian.
Yeah, thanks, Dennis.
Corn has said that he has a relaxed talk.
I have only 10 minutes, so I need to speed up.
What I would like to talk today is about HPC container conformance, which is a project
that came out of the HPC container advisory council, which is every first Thursday.
And we try to provide guidance on how to build and annotate HPC containers.
So conformance, what you might ask, so what are we trying to achieve?
We focus on two applications, maybe a third, but mainly Gromax and PyTorch, and we want
to go through an exercise of providing best practices on how to build or shape the container
and also how to annotate the container.
And I think that's the most important part, is the annotation part, by the way, anyhow.
What we don't want to achieve is we don't want to boil the ocean by making everything
work everywhere.
So that's why we focus on these two applications.
And we want also to allow for generic and also highly optimized images and make with annotations,
make sure that people can actually discover those and also provide some expectation management
for those.
We are going to focus on OCI images, and most likely on Docker files.
I mean, if people throw a lot of singularity build recipes at me, then maybe I will change
my mind, but for starters, we are going with Docker files and OCI images.
And if we have a Docker file that is derived from other artifacts, like a spec YAML file
or an easy build recipe or an HPCCM recipe, then of course we also want to include those
to make it easy for people to reproduce and tweak the actual container.
When going through this research or this project, I was like, I'm in touch with the
biocontainer community, and they created a paper in 2019, which is pretty interesting,
where they provide some recommendation on how to package and containerize bioinformatics
software.
Of course, they don't compile for different targets, and they don't use MPI a lot.
So it's just a baseline, I think, for our work in HPC, but it's a good baseline, and
I highly recommend this paper to be read by people.
So the first thing in the HPC container conformance project is the expected image behavior.
So I think we have all been there, where we have different images, we want you to swap
out and then we realize, oh, the entry point is different, or the container does not use
an entry point, but the application name.
And so we want to make sure that at the end of the day, all the containers that we produce
in the HPC world are built in a way that they behave the same way, so that you can
just swap out the container, you want to run Gromax, you try out multiple different
containers, and you don't need to change your submit script but only the name.
So at the end of the day, the container should drop you into a shell, like you're logging
into an SSH node, and it should also have a very small, ideally small, or even no entry
point so that it's easy to debug as well.
So if the entry point takes forever or makes a lot of changes, then it's hard to debug
the container.
So the container should be, has a very small or even no entry point, and maybe it changes
some environment variables to pick up the application that is installed maybe by Easy
Build or Spec, but it should be very small.
The main part is annotations for this project, and why annotations are the basic ideas, and
we have all been there, so everyone who's done HPC containers, that we encode the information
about the specific implementations of the image in the tag or in the name, and we don't
want to do this anymore, right?
So we want the information to be annotated to the image and not part of the name, because
the name might change.
So what do we want to do with these annotations?
We want two things.
First, kind of describe the image, the content of the image, and how the image is expected
to be used so that sysadmins and end users know what to expect.
So what user land is provided by the image?
What tools are installed on the image?
How is the main application compiled, like for what target, for what microarchitecture
of the CPU, for which GPU, which MPI is used and so on, so that we can take this information
and make maybe configuration examples for different container runtimes that hooks can
react to those annotations, like potman and seros, for instance, they can already react
to annotations.
So depending on what the image provides as information, the runtime can adapt and say,
okay, I have an open MPI container, I do this hook, I have an MPI base container, I take
this hook.
So I think that would be great if we can agree on certain annotations and agreeing on certain
annotations.
I think it's a huge task, but I'm hopeful that we can achieve this, and then make sure
that the configuration is done so that the application is tweaked the right way.
And another piece that we can achieve here is that we create maybe a smoke test that
looks at the host that is running on, looks at the annotations of the container that you
want to run, and just tells you, okay, this thing will sack fault anyway, you are on a
send too and you have an application that's compiled for Skylake, it won't work.
So that you don't download 30 gigabytes of images of layers just to realize that your
image won't work.
So I think that's also a very important part that we can do this.
Another part as well is not just describe the image, but make it easy for end users
to discover what images are around.
So you want to run Gromax, and you know or don't know the system you are on.
So maybe you can just run a tool or have a website that tells you you want to run Gromax.
I have looked through all the annotations, I know a little bit about your system.
Here we go, this is the image that you want to use.
So also for discovery, I think that's important.
Of course, we will have mandatory and optional annotations.
So mandatory ones might be what CPU architecture is it compiled for, I think that's the obvious
one.
And optional ones, of course, if you want to add a CUDA version because your image has
CUDA installed, then of course that's an optional one.
Or you want to annotate the whole software bill of material.
Maybe it's too much information, but maybe not.
So there are optional and mandatory annotations, I think that's pretty clear.
And I created a couple of groups, like annotation groups that I think we should think about.
I won't go through every single line item here because I only have 10 minutes and it's
only 3 minutes left, so just maybe grab the slides afterwards and then go through it and
it's not written in stone, it's just a proposal, so happy to have feedback on this as well.
So the first big one, and I talked about it already, is of course hardware annotations.
So what is the target optimized for, the architecture, generic architecture or the real microarchitecture
and then a key version, a value for this.
As I said CUDA versions, driver versions and so on, I think that's obvious that we need
to annotate the container so that it defines what the actual execution environment should
look like.
Also obvious HPC things like the MPI and interconnect annotations so that you define
what the implementation of the container is, is it open MPI, is it image based, is it even
thread MPI because you only want to run single node.
Not framework is used, libfabrics, ucx, what have you and now I'm going through all the
line items so maybe I should stop, but at the end I think the last line is also important.
What is the container, 2 minutes left even, what is the container actually, how is it
expecting to be tweaked, so is the MPI being replaced, libfabric injected and so on, that's
also I think important so that the sysadmin or the runtime knows what to do with the container
to make it work on line speed.
Sysadmin annotations I think is also important so that we know what the container expects
from the kernel, what the modules are introduced and so on and also what the end user can expect
what tools are installed, is jq installed, is wget installed and so on.
Another annotation is of course documentation would be nice as well, base64 encoded markdown
would be great so that you can render how-tos and build tweaks and so on directly.
Okay, one minute, how to annotate, I think that's obvious as well that's a layered approach,
of course the base image should have annotations that we can carry over and if you build subsequent
images at the annotations that are important and after the image is already built you can
use things like crane or builder or podman I think or builder to annotate images at the
end without even rebuilding them, just repurposing them or we could also collect annotations offline
in another format and then annotate it.
Okay, ideally and that's like Kenneth and of course Todd as well, easy build, spec,
they should annotate it correctly so that we don't need to teach everyone to annotate
but the tools just annotate the image for us.
And that's the external piece so I created a tool MetaHub where we define images for
different use cases and we can also annotate those images without actually changing the
image but just with this.
So okay, 10 seconds, last one.
We need of course a fingerprint of the system to match the annotations against the host
itself so there needs to be a tool, time is up and yeah, so we need to discover the right
image, need to have a smoke test and help tweak the container.
That's like the last bit so I think that's it.
Thank you for the excellent example on how to do a lightning talk on time, we'll take
one question, any questions for Christian?
Do you need the clicker?
Thank you for your presentation, I would like to ask how does this relate to like existing
software supply chain method databases like GraphiS, does this complement their functionality,
is this completely something different?
I mean we are good at HPC to build our own thing and then just say that everyone should
adopt it.
We want to complement it, we want to use these two applications and go to the exercise and
then maybe learn from what we did with this project and try to push these ideas also in
other things.
But I think the AIML folks maybe didn't realize that they won't have this problem so we try
also to not only think about HPC here but also think about other communities as well.
So I'm open to everyone and the project is as well.
Thank you very much Christian, if you want to chat with Christian he'll be around probably
outside the door for the rest of the day or in the room and we'll switch it over to the
next.