Okay, next we have Dan Chudamak with testing containers with Python and PyTest.
Wow, thanks. You haven't heard the talk yet, but thank you. So, first, the boring part,
I'm Dan, I'm a software developer working for SUSE, I do other stuff, but since we only
have 15 minutes, I'm just going to jump right into the meet, and that's why should you test
containers? I'm not going to answer that, please test your containers if you deploy
applications or anything else. And the first question usually people ask is, why don't you
use shell scripts? Because I mean, shell scripts, they are super portable, they run everywhere,
and shell scripts are also pretty fast. And given that shell scripts run everywhere, and
they are so super-duper portable, everyone understands them. Apparently, I'm not everyone.
Because in my opinion, shell scripts are very brittle, especially once I have to do string
modeling, that's the point where I start to test my tests. And if I need to write tests
to test my tests, I think I'm doing it wrong. You can disagree, but so let me give you the
short sales pitch, why you should use Python, PyTest, and especially this is about a PyTest
plugin that I wrote that's called PyTestContainer. So, what this thing can do for you is, it
handles all the boring plumbing part of a test suite for containers, like pulling images,
building containers, launching everything, and cleaning up, and not leaving you with
terabytes of stale data. It uses the Python test-infra module, in case you know Python,
PyTest, this is just another convenience module to just access files, check whether there
are open ports, stuff that you can do with the Python standard library, but it's just
another convenience layer there. One part that took more time than I care to admit,
but that I'm moderately proud about is that the whole test suite is designed, it supports
parallel test runs. So, if you use the PyTest X-Test plugin, it allows you to execute all
your tests in parallel, so assuming you have 500 cores, you can run 500 tests in parallel.
And the whole thing also works if your container images expose ports, provided you don't open
a thousand ports on each and run 500 tests in parallel, then you'll run out of three
ports. But there's tools for that. If you're using Podman and not just Docker, it can also work
with Podman parts. You can also create abstractions to create container volumes and it will clean
up after itself. Also, if you're more into the area, I have an application and I want to check
whether it works not just on my box, but also on Fedora, CentOS, Debian, Arc Linux, Alpine,
and whatever else there is in the world. You can just define a set of tests and you tell the
plugin on which container images to execute them and it will do that for you. So, that allows
you to have the same set of tests and run that on different containers, which would be, that would
be more into the area that you are looking to test an application. It works with Podman and Docker,
you just change it by changing environment variable. And if you happen to be in the lucky
position to support enterprise-grade software that's very stable, hence very old, it still works
with Python 3.6 and on all the important architectures, which took also more time than I care to admit.
So, let's just take a look at a very simple example. This would be just a typical Python test
file with your imports, then you define your container image. In this case, it's just the
open-source-at-umbleweed image. And then you define a very trivial test. And in this case,
what you can see here is, so this stuff here, that's really what testing for shines and it just
takes a look at the O.S. release there. Very simple test, but you could do more elaborate
examples. So, what are possible use cases? Of course, you'll just base images, you could test
those, you could do applications inside containers, and another test, another possible use case would
be you have an application and you want to check whether the application works on multiple O.S.s,
but you don't need virtual machines for that. Then you could use PyTest container for that as well.
So, I guess if you're in this talk, you might know a bit about PyTest and as the name suggests,
it's a Python testing framework, otherwise the name would be very bad. All it really does is
assemble tests, so it's like unit test on steroids, executes all test functions. And one thing that
PyTest container uses extensively are fixtures. If you're not known to PyTest, you probably know
setup and tear down functions from all other testing frameworks and PyTest fixtures are kind of
the thing there. So, a fixture is really just a parameter for a test function and it can return
a certain value and before that do some setup, give you something in this very simple example.
This is from the PyTest docs, so it would give you, it would for instance create a mock SMTP connection
and for the PyTest container, it gives you a connection to the already created container.
And another cool thing that PyTest has is test parameterization, where you can just define
multiple parameters. So, in this case, you would get your, you would have a test and you want to
execute it for all combinations of those values, so it would run your test for all the, for the whole
Cartesian product, so all combinations of 0, 1 and 2 and 3. Let's just jump into a few usage
examples. So, in case you want to build new containers, you just define yourself the base
URL, you have the, you define yourself the docker file and you create the creatively named class
derived container and if you didn't already see, I shouldn't be in charge of naming things because
I'm terrible at it, but I'm not very creative. But so, what will happen now if you pass this,
if you pass this created class into your test function, the plugin will first pull your, pull
the space image, it will build the container on top of that, launch it, pass it into this, pass it
into the test function and once the test is executed, it will get cleaned up. You can also,
so you can also define pass in other already created containers into this as a base, that all
works. I have an example for that later. As I mentioned, binding free ports, you might not say,
why don't you just add a parameter somewhere, okay, expose port 8000 on the host and that works as
long as you only launch, as you don't launch tests in parallel. So, if you want to launch, if you
for instance want to test this specific container five times in parallel, you can't bind all of them
to the same port and for that, there's a relatively simple abstraction. So, you just create this
port forwarding class, pass it into the container and then it will get exposed in the test. There,
you will get the host port and this is inferred automatically on launch of the test.
If you want to test ports, so this is very apartment specific, works rather like this. So,
essentially a port is just a combination of containers and the only really interesting
part that you want to use it for is again port forwarding, works exactly the same like with
containers. One little catch. So, so far I was telling, I was claiming that your containers would be
launched after the test and destroyed after the test and that's not entirely true because most
tests don't modify the container and then you can get away with creating your container before
all tests and tearing it down after all tests and you save actually a substantial amount of time.
But if you decide to do tests like these where you try whether RM minus RF actually works,
then any subsequent, any subsequent test will fail and start burning. So, and therefore there's a
different fixture that's called just container per test and that will actually ensure that all,
that you just get a fresh container for every test but it costs extra. So, but then you can also
RM minus RF everything in your container and the subsequent container will still kind of work.
For the case where you decide, where you want to run a bunch of tests,
but you don't want to do the whole pie test parameterization before that, you can just dump all
your containers into a global variable that's called container images and pie test will do the
automatic parameterization. So, in this case, these, all the tests in the test module would get
executed with all these containers and that's for instance what, what we're doing in, in the,
in the, for Kiwi test functions where you just want to ensure, okay, they work on CentroStream,
Fedora, DBN, R-Clinux, etc. Pp. What I've, what I hinted on previously that's dependencies
between containers which is essentially just you want to build a container based on another one,
based on another one, which would essentially be just you, you split up your, you would split
up your Docker file. This might sound like maybe a weird idea at first but it can be,
it can be relatively useful if you have, if you want to check different base images and then
build stuff on top of them or you want to, or you decide to modify your base image. So,
we have used this relatively extensively in the BCI test suite and you can, you can simply create
containers that derive from others and that derive from others and you can do this relatively
extensively, just don't add loops. That, that will not work. In case you want to, want to check whether,
for instance, the environment in your container is still, is what you, what you expect or some
config and you don't want to mess with the JSON that Docker Inspector, Portman Inspector gives you,
that's also to a certain extent implemented. So, you'll get, you'll get a Python usable
version of the inspect of a container where you can, for instance, check what's the user in there,
what's the CMD of this container, if there's something in the environment and other stuff.
Since I'm nearly out of time, I'm going to jump over this since it's not really that interesting
actually. One important thing is if you are, so if you create an application in your container,
applications usually take time to launch, please use health checks. Health checks are cool. I know
they are not part of the OCI spec, that's a bummer, but please use health checks. Because if you start
using a test suite and you, and you yourself execute a test manually, you launch your container,
you try to curl where the application is there and it all works and then you automate it and it
suddenly fails because the machine is much faster than you are. And your application is simply not
up there. What PyTest container supports, if your app, if your container image has a health check
defined, it will wait until the container is healthy. And as long as it's not healthy, it will not
execute your test. If it becomes unhealthy, your test immediately fails. So, if you add a health check
into your container file or if it's just in the image, then it will wait and it will start
execute the test and you can always be sure that your container will be healthy. And if you don't
want that for whatever reason, then you can just say, okay, I don't care about health checks. Then
you just define the timeout to be minus one or you comment the health check out, but well. And then
you can, then your container might not, might still be starting or it might be unhealthy.
As I said, by default, it will use, it will pick Portman, but you can just set use Docker.
And what I'm, as I said, moderately proud is you can just run your tests in parallel. That also
works with port forwardings. So, that can save you a lot of time unless all your container builds
run themselves in parallel. So, then you're not going to save a lot. The thing cleans up very
well after itself. So, if you create containers of volumes or parts or temporary directories,
all that gets cleaned up, images and intermediate layers are retained because otherwise it would
just take forever and ever. There's a few people that use it. Most of them are those that I bullied
into that, but maybe you find this useful and you will be on this list. And since I'm out of time,
thank you.