Hi, everyone. How are you? How is hosting the weekend? Good?
Yes.
That's nice. I'm happy to be here. It's my first time in Europe and it's the first time
that I will talk in English for a first event in person. This is pretty nice. My name is
Edet Buja. I am a technology evangelist at Percona and this is a very basic and friendly
introduction about databases and containers. About me, I am from Peru in South America.
I am working as Six Months in Percona. It's an open source company. We create open source
databases free. I am a Google woman tech maker. I was nominated as a docker captain last year
and I am a database and container enthusiast. You can follow me on Twitter and LinkedIn.
I used to post about containers, Kubernetes, open source. For the agenda today, we are going
to see about containers. We will see docker architecture. We will see the workflow between
the components of docker. We are going to have two examples of how we are running a single
Percona server MySQL container and we are going to run multiple containers for Percona
server MySQL. We will see the docker volume, how this is important in this work of databases
on containers. We will see backups, restores of databases and best practices. Let's start
it. What's a container? How many of you knows what's with docker? Yeah, a lot. Okay. That's
nice. Docker or do you use other tools? Yeah, there are different kinds of tools for container
application. But a container is like a single unit, lightweight unit of software that package
everything that you need for your application. When we run application, when we build application,
we know that we need a lot of packages. If you are running, for example, if you are building
a Java application, you need libraries, dependencies, many things to run your application. So everything
have to be containerized in a single unit of software and this is going to be isolated
for other things like your infrastructure. And the good thing is that your container
can run on different platforms in your laptop, in your server, in your cloud. With this,
we end with a problem that we have when we say, hey, your program runs. Yes, this works
just on my computer. But no, it has to run in different platforms. We don't need to have
this problem to dependencies and other kind of things when we test our application in
other platforms. There are different tools, as I say, for containerization. We have container
interface, for example. We have container D and we have Docker that is the tool that
we are going to focus now. All these tools are also in the cloud native computing foundation
ecosystem. If you see the landscape, you will see a lot of tools there. There is a part
for containerization and there are more than three. There are a lot of tools for them.
The Docker architecture, it works like a client-server model. We have the Docker DMO, which is going
to process all the commands. It's going to start to listen to the client always and the
client is going to send a request to the DMO through the REST app. With this model, the
Docker DMO can also manage network containers, images, and Docker volumes. If we go more
in detail, we will see that we have the client, the DMO that is also called the engine of
Docker, and we have another component that could be your Docker registry, the public,
which is Docker Hub, where all the official images are published, and also we can have
our own private registry in case we don't want to share it with the public. In this
case, this is the flow of a component. For example, if we do a pull, we are going to
try to bring the image from the Docker Hub into the Docker DMO cache. If the Docker DMO
doesn't find the image in cache, it's going to bring it from the Docker Hub. But if this
is in cache, it's going to take it just that and start to process. The same with Docker
build. When we run Docker build from the client, the Docker DMO will try to take a Docker file.
A Docker file is a recipe with a lot of instructions where we put all the commands to run our application
and deploy it. So I'm going to, the Docker DMO is going to take the Docker file and build
it, build the image, and if you want, we can also run it. We run, we will create a container.
The container is our application that is already alive and is ready to make connections
of petitions. One more thing here is that we can have everything in our host or we can
have clients, remote clients that could make petitions to the Docker DMO. Container benefits.
There are pros and cons, but now I'm going to focus on these benefits, the containers
give us. So one of these is we can reduce costs with this because we can run several containers
in a single infrastructure. That's infrastructure that we have because of the technology of
containers is different than the virtualization. In virtualization, we use the hypervisor and
when you create virtual machines, it consumes more resources from your, from your infrastructure,
but when you use containers, it's very different. You are using that technology, a container
would make it possible to run different, a lot of containers in a single machine. So
for that reason, it's possible to reduce costs. Also, the containers are very friendly with
continuous integration and continuous delivery process. If you have like a big application,
a monolithic application, this, and you want to, you want to run container, you want to
integrate it in the DevOps process. This is going to be hard. We have to work like microservices
to make each service as a container and included in the continuous integration and continuous
delivery process. It's easy. When we build, when we build our application over a container,
it's easy to kill it. It's easy to create it again. It's easy to fail and the process
is faster. Another benefit is the multicloud compatibility with the time several companies
try to migrate to a hybrid cloud. They just don't, don't want to have everything on premise.
They also want to scale. They want to grow. So for a reason, they opt for cloud and containers
fit very good in this. You can install Docker. I know you did it. You can choose your distro.
You are, you use Debian, the CentOS, everything. So you can go to the official Docker documentation
and easily look all the steps. When you install this, it will install it, the Docker client,
the Docker DMO and other tools that you will need to use Docker in your local matching.
We already talk about containers, right? But this talk is about exploring database on containers.
We are going to talk about my SQL, which is at this base relational database. We know
that it's a database. And to run my SQL on containers, we need to understand how volumes
works because the most important thing running databases on containers is the data. If we
lost the data, we lost everything. For the next slides, we are going to focus in this
part. We will use the image of Percona server for my SQL. This Percona server for my SQL
is open source. It's like my SQL, but with more nice things. You can use it. It's open
source. It's in Docker Hub. So we will use this image and we will create a Docker container.
We will see how it works with all volumes. We will see the layers in Docker and then
we will create a persistent volume and we will see how it changes in the layers of Docker.
So just here to see that if you want to have an image, it's necessary to have a Docker file.
You can use a Docker file before by yourself. That's good. A Docker file is a recipe where
you will put everything for your application. So you need this to create an image. Then
you need an image to create your Docker container. There are three essential steps here to remember
how Docker works. We will run a single Percona server for my SQL container. We will use Docker
run to create the image. No. We don't use Docker run to create the image. We use Docker
run to create a container. So we use this to create a container. So we will do dash
D to say run this container in the background. I don't want to use the terminal. And I will
call it Percona server for my Percona server one. I will pass it like the environment variable,
for the root. This is not a good practice here. This is just to show how we are going
to create a container. And we will use this official Percona server for my SQL. With this
I am creating a container, right? I'm creating a container with this one. Okay? So if we
go to Docker image LS, this is going to pull the image of Percona server and then it will
create the container. That command is going to do two things. It's going to bring the
image from the official Dockerfab and it's going to create a container. So if we see
Docker container PS, our container is up. Okay. After we have the database, we need
to add data. We will add databases, we will add data, we will change registers, we will
have transactions, many things that we can do like a regular database. Okay. If we run
a single Percona server in my SQL container, we know how it works in layers. If we see
this in green, there are layers from Percona, Percona server image. This is the image that
we pull it, that we can change. This is just react only. We can change this, but in top
of that, it's going to be created a layer, a new layer. This layer, this layer is react
only. I can add data. This layer is the one that will contain all the things that I am
doing in Docker on that image, on that container. I added a new database. Yes. I create a new
registry. I delete it. I add the transactions. All this is going to save it here. But what
happens if I don't have volume? My container is ephemeral, right? It could die. It could
crash. My machine could crash. And all my data is going to be lost. I will, I will
lose all the data. We will see how it works with multiple containers. To run multiple
containers with the same image, if we see this is the same image, the same version of
the image, we will just change the name of this container. Also, we can change another
thing because this is a database, right? What thing we can change? They run in a port, right?
In which port my SQL used to run? Yeah. So I need to change the port for the other container
to avoid the conflict. Okay. How it works in layers. The same. We will use the same layer.
We will use the same layer for Percona, Percona server, which can, we can modify. But in top
of that, we are going to have two layers more. One of the first containers that I created
and the second for the other that I can add. I can add data. I can change things. But once
again, if I don't have volume, this is going to die. But this is how to work if we want
to create an application when it doesn't matter if we save the state of this application.
This is important. Persist data in databases is really important for this kind of application
because sometimes we think that, like Kubernetes, since it was created for a state less application,
but now we have options to use stateful applications on containers. And this is one of the reasons.
Create volumes. So it's pretty easy to create volume. We can create a volume just with dash
V or dash, dash volume. And we can say it, we can create a local volume with local run
and detach. We will call it Percona server. The same process. And when we say dash V,
we are saying, okay, this will be my volume in a host, in my local data directory. And
this one is going to be inside my container. So this is like a mirror from this image.
And how it works. In layers, we have the same, the layer that we can modify. And in top of
that, we are going to create another layer. But in this case, we are adding, we are creating
the mounted volume in BarLivMySQL. There are other directories that we can create the volume.
I am just adding, as an example, this, because in MySQL, we have configuration files. We
have logs. We have another things. But for that, we want to create these volumes for
all of that things. I am just adding, as an example, BarLivMySQL, which is also a directory
that is very important. And this local directory is the one that could be in my host. But it
is not recommended, because if your host crashes, everything crashes too with your volumes.
It is preferable to run it in a remote host. Okay. Two backups. Who here make backups?
Okay. I use the very easy way to make backups. I use it for logical backups, my SQL dump
used in the container. And for physical backups, we use in the company PerconextraVacup, which
is, have more features to have that physical backup. And for restore, I will use also my
SQL dump. And we don't use PerconextraVacup in this case, because it has a lot of pins.
For backup, I will execute a backup in a container that is already running. PerconaserverVacup
is already running. Let's see that we created. And we are executing Docker exit, it, to enter
into the Percona in that container and type that common, my SQL dump, to create a backup
of the database. So the backup is going to be in that file, dump SQL. And the same process
with restore, we can take that backup. And this is a different container. I'm going to
restore the dot SQL file in a different container. In this case, in PerconaserverRestore, using
my SQL, use that command, my SQL. Okay. Best practices or some recommendation to use containers
in database. Okay. And one of this is that we can keep constantly monitoring our database
and the whole system, because we don't know when we are going to don't have enough resources
for our containers. We should be aware of that or have notifications to say, hey, you
don't have a note disk, you don't have a note memory, so provision or try to scale in your
resources. So we should keep monitoring. Using some tools for that, for example, is PMM.
We can use open source monitors to monitor our databases on containers. And we can store
this data in persistent volume outside the container. It recommended no inside the container,
because it's easy to create plans for recovery. We can restore the data easily also and fast.
We should limit the resources of utilization of our containers. Our containers, we know
that they are small, but also we should limit when they are a lot. And we should regularly
have backups of the database and store these backups in a different location. And have
a plan of migration and disaster recovery is really great. In that case, having a monitoring
tool helps a lot. And what more? That's all. You can find me in LinkedIn and Twitter.
Okay, we have time for questions. If you absolutely need to leave and you can't wait until the
talk is over, please do so as quietly as possible so we can understand the questions. Thanks.
Hi. Thank you so much for your talk. It was really interesting. I'm wondering what kind
of limitations do you see when you're speaking about having a databases arriving in containers?
There is storage limitations, CPU, or something else? Guys, can you please be a little quiet
so we can understand the question? All right, I will try it with the microphone.
Yeah, you. The people can you. Thank you. I was wondering maybe, first of all, really
cool talk. Thank you so much. My question would be, could you maybe talk us through some kind
of limitations that you can see when you're running databases from containers? You didn't
understand it? Thank you so much for the talk. It was really cool. Maybe you can share with
us some kind of limitations that you see when you're running to the solution of running databases
inside containers, right? You cannot really run very big database. You probably will have
a problem with that. What kind of limitations do you see?
So, yeah, the question is about sorry, the question is about what limitations you can
run into with database containers? Yeah, I don't want to say this, but it depends really of the
business. Okay, if you want to invest a lot of money in infrastructure, but because at the end,
your database, the volume that you have is not going to be part of your container,
it's going to be outside. And this depends on you. You want to invest a lot of money
to save that data. It's good. You want to replicate it? Please try and be quiet while we
are asking questions. Are there any more questions?
There is one more question from the back, so please be quiet.
Thank you. Hello. I wanted to ask, did you notice any kind of performance issues?
Did you benchmark things? Did you identify some kind of overheads going on when you
containerize a database like MySQL or other kind of databases really? Sorry, I didn't get your
question. All right, I'm just going to ask you. When you containerize a database,
be it MySQL or Postgres or any kind of open source database that you may have tested on this kind
of setup, did you notice any kind of overheads, compute, memory, or disk, essentially, where
you can see that the database performance or operation is significantly affected by the fact
of being containerized? I'm not sure about that, but if you use open source to monitor
your containers on databases, you can have a visualization of these things if you don't have
enough resources so it can show you alerts or things like that where you can figure out where
exactly is your limitation. Okay, so for example, did you run Benchmark?
Could you help me? Okay, could you help me to answer? Okay, my friend is going to help me to
answer this. All right, thank you. Yeah, thank you to you. Hey, so usually the performance
degradation is around two, three, four percent. The issue is more about how you configure the
database, kind of storage, if it's local or network storage, but the virtualization is
minimal. It's like running on a EC2 instance. Okay, so there is an impact, miserable, at least you
say around four or five percent, but you say that's not going to be the, that there are configurations
we can do to try to avoid that. Do you have any kind of paper or any kind of resources that we might
use to avoid those kind of bottlenecks? If I got correctly, not much. The measure that we do in
databases is measuring TPS. So you will notice on, if we're running Benchmarks, we've seen Bench,
for example, three percent, like if you are running 1,000 credits per second, you will get
980, 990 credits per second when containerized. Okay, and do you have any kind of recommendations,
kind of generic recommendations you can do so that when you run a database in a container,
here is what you can do to try and negate some of the performance bottleneck that you guys have
noticed? To be honest, on real-day activities, I would say 99 percent of the performance will come
from how you configure my SQL, not the containerization is like just a small piece of the game.
You can make more effect by modifying the database configuration.
All right, thank you very much.
Thanks to you.