you
Hello FOSDEM 2021 and welcome to this talk on code reloading techniques in Python.
In this talk we'll be reviewing a few techniques that allow you to reload code and apply the
new changes you made to your program.
We'll look at a few different techniques to do so and we'll have an in-depth look at
the inner way of how they work precisely.
Few words about myself, so my name is Hugo Herter, I'm a software engineer consultant.
I discovered Python and Linux in 2003 and have been using them a lot since then.
My first FOSDEM was in 2004 and I think I attended almost every edition since.
When I'm passionate about free software, I'm really happy this year to be able to attend
all the dev rooms without missing any.
When I was starting to learn Python, I was quite amazed by how easy it is to play with
all the internals of the language and the constructs.
One thing that I found pretty interesting is this exact function that allows you to execute
any Python code found in a string that might come from anywhere.
As you can see in the example on the right, that just executes Python code from the network
and gives you a remote Python shell on any machine that runs this script.
It's basically the same idea you have in Jupyter notebook with more security and more advanced
features.
When I was learning Python, I started also to write my own web framework.
This was back in the times when Flask and Django even didn't exist.
We only had Zope and a few frameworks that do not exist anymore and on that web framework
I used a function called exec file which is the same as exec but for a file, it would
execute all the Python code in a file and I used this to be able to make my changes
up here immediately when I was changing some files in some web pages.
Which brought me to this idea of code reloading.
It's something that I have been playing with for a long time and I wanted to share this
with you because there are many interesting techniques here and you might not know about
all of them.
What I called code reloading here is the process of replacing part of a program with
a new version, part of all of it.
I'm focused here on the source code because it's the term that's used mostly for interpreted
languages.
When you are using compiled languages, there are other terms that might mean similar things
but they are slightly different.
I talk about cold reloading and what I mean by cold is that you take the process and you
stop it and then you restart it.
And hot code reloading means that you keep the process running and you patch it with
a new code without stopping it.
So as an illustration on the left, we have some kind of cold code reloading on a racetrack.
You stop the car, you have access to all the internals, you can change everything you want
but the car is out of the circuit, it's not running anymore.
The driver is out of the car as well.
On the right side you have hot code reloading where the driver is still in the car.
You may not want to change everything, you don't have access to the chassis, changing
the engine might be a complicated task here but you have access to quite a few pieces of
the car already and if you just want to change the color or type the wheels, it's pretty
easy to do.
You don't need to stop the car and it goes much faster.
It's going to be the same ID we have in programming with cold code reloading and hot code reloading.
So cold code reloading, you stop the process and then you restart it again.
It's easy, it's reliable, you've all done it if you did some Python code or any kind
of programming, it's the default way of doing it.
The issues you have with it is that you lose the states so getting that state back might
take time.
If you are programming a video game for example and you are a Vatorite in a special place
and it took you some time to get there with certain enemies and you want to trick the
behavior of the enemies, in that case restarting the entry game every time might be pretty
annoying and you would be interested in something that keeps the state of the whole program.
The easiest way on Linux to do cold code reloading is control C up arrow enter to just run the
same command again and it's a super easy way to do it because we all know this first shortcut
by reflex as we use it all the time in programming.
Let's have a look at how some web frameworks do this cold, some web frameworks do code reloading
and they used this cold approach of restarting everything but they do it in an automated
way.
Let's have a look at how they do it precisely.
The entry point is this function here run with reloader and you pass it a function.
It will run this main function and enable the reloader on the side and stop that function
if the code has changed to restart it.
The first thing we see is that it's calling here single.signal six-term lambda rx sys.exit
which means if the process receives the single six-term to terminate from the system then
it will exit to make sure that it doesn't hang if it receives this single.
This is a way to behave properly even in multi-threaded environments.
Sometimes when you have multiple threads the signals are not received by all threads and
you have to press control C a few times to stop it for example using some frameworks.
Then we see that it's initializing a reloader here using this function getReloader which
is defined a bit higher.
There are two reloader classes in Django.
One is the watchman which will watch for files on the file system and the other one is the
stat reloader which will just watch every second if the properties of the files have
changed and in that case say well the properties have changed so we should trigger a reload.
The watchman is faster and more powerful but the stat reloader works as a fullback to this.
And then it will pass this reloader as well as the main function here to this startDjango
function which is right here.
So that function basically starts our main function here in a thread so it creates a
thread to run this main function it sets it as a daemon it starts the thread so no
our main function is running but in a thread not in the main thread but in a side thread
which is controlled by this function.
Then it starts the reloader and it passes the thread to that reloader class which will
be in charge of stopping it and restarting it if something has changed and it will run
this in the loop as long as the reloader should not stop.
So this is the Django approach start the main function in a thread and then look for changes
on the file stem when they happen have this reloader class to just stop the thread and
restart it.
When looking at how Flask handles this reloading it's a bit more complicated because it's
not within Flask it's within WorkZook which is a web framework library used by Flask.
But we can find something similar we have this run with reloader function that takes
a main function as an argument it does a register to the same signal as Django and then it starts
a thread here with the main function and it launches the thread here and if we look at
the reloader this it comes from this reloader loops and when looking at it we can see that
it's also using something similar a stat reloader and a watchdog reloader and these are very
similar to those used in Django so we can assume that the behavior is identical even
if the codebase is different here.
So both Django and Flask use these watchman or watchdog reloaders under the hood but how
do these work?
Well there is something called iNotify on Linux and there are similar APIs on other
other platforms that allow you or your process is to watch for file system events and receive
a notification without having to constantly look if something has changed.
On Linux it's iNotify which you can use directly from the library piNotify if you're using
Python or there is this library called watchdog that you can use on all main platforms.
The way the interface to use watchdog looks like this so you can create an observer that
is in charge of receiving these signals and will run in a thread in the background and
you can then schedule some handlers on it and say well I want to register for example recursively
if you're looking for on a folder or not and then just start it and it will work using
a callback based approach.
Because it runs in the background I added this input at the end just to make the program
block and to be able to see something before Python exits.
Let's now look at hot code reloading so in this case we want to keep the process running
we want to replace the code in memory.
We hope it won't crash the program this might happen if we have inconsistencies and the new
code is not compatible with the existing one or the existing state and we want to take
advantage of the fact that it keeps the state and it's really fast to do this.
There are two challenges in this case one is we need to find and load the new code because
if you're just reloading everything you might as well restart the entire program and we
need to replace the references so in Python you can pass a lot of objects as variables
and you can have references to these objects in many places and we need to find all these
places to be able to make them use the new code instead of the old one.
There are other languages that also allow this kind of hot code reloading.
In Java for example you have this functionality called hot swap which basically allows you
via the debugger to specify a class and ask the virtual machine to replace the class with
the new compiled code from a class file.
In C and C++ you have DLL code reloading that allows you to reload a dynamic linked
library or shared library.
In this case as well they need to share the same interface, they need to expose the same
functions and classes and methods that the previous version did.
There are some changes that you're not allowed to because then it would break the compatibility.
In Python there are three ways of loading codes.
One is eval that allows you to evaluate a function and that one is not very useful in
our use case.
However the other two methods do work and they both have their advantages and disadvantages.
The first one is the import module that you are using when you are importing a library
and in a way similar to the DLL libraries you can reload a module that has already been
loaded and have the new version replace the old one and exec allows you to execute just
any Python code from string which can also be used in some cases.
What you see here is on the left a text editor with some Python code and on the right a Python
console.
So the standard way to load Python code is using import.
I will import my module and then I can call module.sayhello and it will just run it.
If I change the source code say hello will still be at the old version as expected.
There is however a library we can use with in Python to reload this module from import
lib and we can reload here module.
And now if we call our function again we can see that we have the new version.
However if we have a reference to this function then it's a bit trickier because that reference
will not be updated say we have say equals module.sayhello when we call say we will still
be calling this function.
Now let's update the source code and reload the code module.sayhello has the new version
however say is still using the old version.
So now we have one version that doesn't exist on the file system anymore the previous version
but it's still in memory it's still loaded and this is something we have to take care
about to be careful about when we're doing hot code reloading.
In practice however sometimes we are facing code that's not optimal for the reloading
that we were just seeing.
For example here we have a connection to a database that's initialized as a singleton
and this is a pattern we see in a lot of code where that connection is created just within
the module and so it's executed every time we import it.
So the first time we import our module it will take some time to initialize it.
We can now call our function say hello let's say we want to update the code of our function
we need to reload it so we can use importlib.reload module 2 and again we have to pay the cost
of the time to establish this connection and now we have the new value working.
So in this case this use of importlib.reload can be quite painful here we are just facing
a delay but sometimes there are also thresholds, limits, there are ports that are still appearing
as busy, there are other issues that we can face that make this approach more difficult.
If you can use it it's really nice because it's really simple it's all built-in in Python
and it's just one line so go for it but it's not always optimal in some kind of complex
software.
There is however another approach we can use to reload this function say hello without
going through the reloading of the database and this is the idea here is to go get the
new value of the source code and then to execute that using the function exec.
So for the first step we need to get access to the source code of a function and there
is a tool for that in Python in the library inspect so I will just import inspect and
then I can use inspect.getSource of module 2.sayHello.
I have the source code of the function and the really cool thing here is that if I do
add some code for example I add pass here and then I do a return here and let's do some
more changes, let's do it, call it again and we can see here that this function gets source
gets the new value of the code and not the old one so it's quite smart there which is
really handy in our use case.
So we now have the source code in a string and we could just try to execute it however
if we want to replace it within the module so that other functions that depend on it
would be able to access it we need to also give it access to things inside this module
for example db if we execute this module here it will not have access to the db variable
from that module.
So what we need to do is use inspect to get access to the module of that function.
So let's say module equals inspect.getModule of module 2.sayHello.
In this case we know which module it is, it's module 2 so it will just get the same thing
but sometimes we just have the function around and we don't know directly from which module
it is so we can use this and then what we can do is create, we want to be able to extract
the new value of the function so we'll create a local directory, I'll call it locals underscore
equals dictionary sorry not directory and then I can just execute the source code so
just copy this or copy the code here within the module that underscore and underscore addict
which is a dictionary representing the namespace of what's within that module and my local
dict here that I created just above.
And now let's look at locals, we have say hello and if we compare it to module 2.sayHello
you can see that they have the same identifier here and if I call one and call the other
so this is the old one and this is the new one and the identifiers look similar but if
we look here we can see they're not exactly the same so let's not be mistaken by the
fact that they look really close now we can just finally update this function if you want
so we can do module 2.sayHello equals locals.sayHello and finally module 2.sayHello call it and
there we have this we just reloaded the function without reloading the connection to the database
here.
If you want to use hotcode reloading in your projects and you don't want to write yourself
the methods to do it using the tools we have seen previously you can also use this library
called reloader from reload their import auto reload and here in this case you just decorate
your functions with this auto reload decorator and it will automatically replace them using
the proxy method we've seen above and the instance reference to the class method with
the new code when the code changes by watching the file system so this is a wrap up of all
the methods we've seen previously.
You can also manually specify when the code should be reloaded by changing the decorator
in this case you can manually reload the class or you can start a timer that will just reload
it every second or again look at the file system and as the file system changes trigger
the reloading of this class.
This on pypy so you can just install it using pip install reloader and try start using it.
The source code is pretty simple it just fits in one file and then you have a directory with
a few examples.
Thanks for watching and join me in the question and answer matrix room for if you want to
discuss any things you can find all the examples we've seen on this GitHub repository thank
you.
Thanks Hugo for your talk so I think we can now start with the questions.
First question is how does reloading using execs behave in terms of compiling to intermediate
forms like PyC and so on?
So it's using, Python internally is using bytecode so exec is a two steps process the
first step is it will compile it to bytecode it will just not store that bytecode on disk
and then it will execute that bytecode as like the rest of the bytecode.
And are there examples of applications that use hotcode reloading?
Usually it's a process that at least I use for development so it's not used that much
in production because then it can cause a lot of issues but it's the hotcode reloading
in general is used a lot by game developers because they're tweaking the dynamics of the
game while playing it and restarting the entire game every time you make a change to some
logic doesn't make sense in that case.
And how do you deal with side effects like things like shared resources and so on?
So the idea with hotcode reloading the way I presented it you keep these resources on
so you keep the state you keep these resources.
Of course if something changes outside of the scope of your changes then you may have
compatibility issues and then you just have to accept it and restart the whole process.
Okay any further questions from the chat?
What are the dangers that remain?
Could you fix them?
Well I think Python itself is not designed for hotcode reloading and other languages
have allowed this in a safer way so in order to make hotcode reloading easier in Python
I think there will be some big changes within Python would be required.
If you take the example of Erlang that's a language that's designed to allow hotcode reloading
and it's used a lot in network equipment and it's a feature built in the language in the
tooling.
If you take the example of Java there is a rule you can reload a class as long as its
interface does not change and your ID, your tooling will check that.
In Python there are no such checks so at the moment there are no guarantees that the new
version of the code would work with a high chance.
Okay thank you.
So, do you add decorators for reloading in your code base?
Is there a best practice to ignore them at the moment running your code in production?
I think it aligns with the other point that's adding decorators just for the sake of reloading
for half an hour doesn't make sense.
It's a trade-off.
I use decorators because that allows me to know exactly what is being hot reloaded and
what is not.
And also as a way to work with the references.
Another strategy I thought about was to try to replace in memory all the references to
the function with the new one within Python.
And that requires much lower access to the internals of Python.