Okay. Okay, now we have Linus de Meijer with shorter feedback loops with the live book.
Give it up.
All right. Thank you. Can everybody hear me? Yeah. To my surprise, I am the first and
only Belgian here presenting. Well, I can welcome you all here. Maybe the first question
I want to ask, which is the most important one, who is hungry right now? All right. Sorry,
I cannot help with that. But another question I would like to ask is, who has heard of Live
Book? Who knows what it is, more or less? Yeah. Okay. It's a lot of people who has worked
with Live Book professionally or has just, has it on their computer? Okay. Less people,
but there are some. If you want to follow along and if you already installed Live Book,
then you can go to my GitHub repository. I have a little notebook prepared. And I try
to switch back and forth between this presentation and the Live Book. All right. The goals of
today would be to introduce you to Live Book, to make sure you all understand what it is,
how you can get it, how you install it, the various options. And then I think the most
interesting part is how I used it in three different cases and what I learned from using
it in a real project. And underneath all this, I hope I can bring across the message that
Live Book really helps to start somewhere in the middle. So you don't spend time like
scaffolding an application. And then just after a few days or hours, gets to the most
interesting part. So Live Book enables you to start in the middle. That's my main message
here. Whenever you start Live Book, you're greeted with like an introduction page. At
the top you see your folder structure. There is a very nice learning section. And at the
bottom there you have your sessions. So you will import a Live Book often and then a session
will appear and you can hook into that. So you can, that's actually what we are going
to do here. I'm going to go to my notebook that I just prepared here. And yeah, I just
wanted to point out if you are just starting with Live Book, there is a very good learning
section. So please go through these. Also if you're learning Elixir, it's a very good
way to familiarize yourself with Elixir. And it also covers things like how you make
pretty graphs or how you would use the Kino Library, which is the one that is used to
actually interact with your Live Book. It's all just marked down. So it uses those, yeah,
you can see here, it uses these code fences with the Elixir annotation. So it's very easy
to check into your GitHub repository and make sure you can review it if you want. And GitHub
also recently added the feature that it nicely formats your Live Books. They have the extension
Live MD. So it integrates nicely with your version control system. The basics. We have
code cells which can be executed. They contain your codes. And the first one is a little
bit special in the sense that it often contains your setup. So you can pull in all your dependencies.
So you can use the mix install function. And I'm going to use a few here, not too many.
But I'll go over these once they become relevant. So right here we have our first code cell.
We can just execute it. It takes a while to just start up. But then we can go. It is
being evaluated. You can see the green dots. So it's being evaluated. And you have all
those nice features that you can expect from an IDE. So you can ask it to autocomplete.
If you control space one more time, you get all the documentation for that function. So
you get a lot of help editing your code here. The result is being print down below here.
And you also have the ability to, like I did here actually, to put stuff. And that's also
being printed underneath your code cell. So that's very nice. And yeah, maybe the most
or a very important feature at least, it's that you can interleave your code blocks with
just regular markdown. So it's a really nice way to do a little coding and then explain
what you have done and then go on to the code again. Yes, a few words about reproducibility.
It's very nice to have this notebook and to know what actually will happen if you execute
all those code cells. If you start from the beginning, it's very, very clear. You go from
top to bottom. But what if you are going to edit in the middle, make a change somewhere?
Well, Lifehook has recovered. It analyzes all those bindings that are being made in
those code cells. And it makes sure that if you change something, the relevant code cells
underneath them are also going to be executed again. So that's actually the way you often
build up states. You have a code cell that creates a binding and then in the next code
cell you can reuse that or you can use that binding so you can build upon when you go
through all the code cells. I can do a little demonstration how branching sections work.
So the sections are actually shown very clearly here on the side. I have a few of them, but
one has the little branch icon. So this is a branching section. And this is just to show
how the execution model actually works. So right here, I demonstrate that you can use
the bindings from before, so from the main flow of the notebook. And if I start an infinite
loop here, this is just going to stay printing in this little frame here. You will see that
if I execute a code cell below, it will be queued, but it will never run because the
other code is just blocking that one. But if we carry on to the next session, we can
see that all is well again. So this is still blocked, but this is the main threat of execution.
So we are not blocked here anymore. And just to show that we cannot access the bindings
from before, I just triggered this error because we cannot access this variable. Okay, this
is a pretty picture that I stole from Josef Alem. I'm not going to go into detail, but
I just want to point out that everything is based or is heavily using the airline distribution
mechanisms that we all know from the airline OTP ecosystem. So we have a central application
here. It's a live view application, actually, with a lot of JavaScript. And we can connect
to it through WebSockets. That's all being handled for us. And it does not run the code
actually on the live book application itself, but in normal mode, it will spawn a new node
and run your code on this new node. So we call it a runtime. This runtime is not aware
of anything live book related. It is just a plain node that can execute code and you
get the results back. So that's what's going on underneath. There are a lot of ways you
can get live book on your computer. Recently, well, I used to like e-script installation,
but since it's tied to your Elixir installation, I now switch to using the desktop application,
which is getting very good at this point. You also can run it in the cloud. You can
have it as a Docker image. That's all being covered. The various ways to start, not very
interesting, but I think what's more interesting is my story of how I used it to mitigate risks
early on, some projects that I've been doing. Yes, I just want to sum up here the benefits
that I see. So it allows you to start in the middle. If you're using live book, you can
jump straight into your problem space. It increases transparency, and you can use it
because you can use that markdown in between to document your process. So all your thoughts,
you can put them in between all those code cells, and it's, I think, way better than
those obscure scripts we sometimes write, and you can also very easily share this document.
So that's actually something that we did. I got some tasks to do. A client was asking
something. We were doing something with machine learning and artificial intelligence. We were
not aware or we did not know whether we could do it. So I sat down, made his live hook,
and then documented all the steps I did, and in the end, got a pretty graph out of it,
so I could convince the client that we could do it, actually, with Elixir.
Just a little bit of context. I work for a small company. We often switch in between
projects. The company is named Zenjoy, and we are often working as a team of two. So
documentation and collaboration is very important, and also the communication with the clients
is very, very important. So in this first case, we were tasked to
interoperate with or to call an undocumented legacy API. It was very low level. It was not
as low level as tools explained to us, so it was not like we had to do the pattern matching
on the bit level, but we had to use the GenTCP module straight from Erlang, but it was very
nice to have this live book environment where we could just throw the commands at this server
that we could somehow use and see what came back. So in this way, we were able to create
a notebook that documented all the commands we could see or how it reacted, and you see
some pattern matching going on here. You also see some magic variables. So this was given
to us, so we could not change this, but at least we could document it, and this became
a very long document to refer back to. So this is another demonstration I wanted to
do. It's not because you're in the browser that you're constrained by any way. You can
still use all the process magic and all the GenServers you like, and this is just a demonstration
of how you would go around and spawn a TCP server. I'm using Thousand Island here. In
reality, I was using the other one, the older one, Cowboy and Ranch. Yes, thank you. But
now for this demonstration, I got to use Thousand Island, and it's super nice, so you can
just define your handler. It's just going to echo back whatever we send to it, and here
I started up. The only caveat or the thing you have to be aware of is that you can start
your children, your processes, and your supervisor tree under the Kino code cell. So whenever
you reevaluate here, you see that you can't see, but another process is started or a whole
tree starts. So this is a nice interop with the Lightbook environment. And once we have
those, we can even... Yeah. Yeah, I guess that the gods are not with me today, but at
least what I wanted to show is that you can actually draw a pretty picture of the supervision
tree right here. But still, I think the server got started, and so now I can net cut into
my local host on this given port, and I can see how some stuff is being echoed back to
me, so at least that works. All right. Back to the presentation. So it is a nice environment
to stub out a server and set up a situation where you can then use your application to
interact with this stubbed version of your API. All right. I'm going to show, or this
is just an example of how you would integrate your Livebook with a regular mixed project.
It's all just sitting next to each other. Oftentimes, I just make a folder where the
notebook lives, and then your mixed project, whatever it is, it can be a Phoenix application.
You can access it if you use the part way of referencing your dependencies. A few words
about a typical lifecycle that I've observed, you often start to experiment in your Livebook.
On good days, you add tests, and then you move all that code into the regular application
or in the regular mixed project, and you reference it from there on in the way that I just described.
So you promote reusable code, and that's often a way that worked very well for me. The second
case that I want to discuss is how I set up or created concurrent ETL pipeline, which
is a fancy word for just loading CSV files and then maybe transforming them and dumping
them into Postgres. So I really got to learn a lot about how concurrent data processing
actually happens. I got to play around with Flow, which is a very nice library which builds
on top of GenState. No, not GenState, the other one. GenStage, that's the one. And you can
still use all the power of processes that are available in Euler and Elixir. To demonstrate,
I've prepared or I want to show how you can use ECTO and then this Flow library right
within your like hook application. To start off, I create a repository, just like you would
do in a Phoenix application. Don't worry if you do not recognize this. This is kind of
standard stuff. You have to specify the adapter, and then you can emulate whatever mixed ECTO
create would do. So you make sure that your storage is up. In my case, it was already
up, so that's what it reports. And then you can even make your migrations like you would
if you used ECTO together with a Phoenix application. In this case, I also made sure there was
an item in the database so we can query it later on. So I have to make sure that this
repository actually runs. And then I migrate, well, I do a rollback to make sure nothing
is being left over, and then I migrate again. So we have the end situation that I can query
right here. So you can see that our fresh entry is just inserted with this new timestamp.
And then we can build upon this. This is another demonstration, very short. This is a definition
of a flow. Also, don't worry if you do not recognize this. That's not the key here. I
just want to show that you can use all those goodies, and you're not constrained in any
way. Right here, we are just emitting a value every second, and we're going to wait for
three seconds and then insert an item in the database again. So you get the logging, and
if we query it again, so you might recognize the nice ecto query syntax. You see I've
wrapped it in a data table, and you can now see the three new items appearing. So that's
very nice. Here you see me playing around and actually visualizing this ETL pipeline
where every color actually is another class of objects or is being inserted in another
table. It went very quickly, but when making this presentation, I also saw there is some
room for improvement. So not all cylinders are firing together, but at least it was fast
enough for our purposes. Another case I want to share with you is that we used the live
hook to actually connect to a live running instance. So remember, as I have shown in
the beginning, it's all just Erlang distribution under the hood. So instead of using the regular
setup where you do an LXR standalone setting, you can also do an attached node configuration.
The only thing you have to know is your node's name or the short name and a cookie which
you have to agree upon. And it's very good for doing one of the tasks. Maybe you don't
have a UI for something yet, and you want to do it in a live book, then this is a nice
way to actually have like a super admin interface, but be aware that this is still a live environment.
So if you do this, make sure to put a big disclaimer on top of your notebook to remind
you of the risks involved. All right. The last thing that I want to share or show is
how you would do tests in live hook. Like I said, on a good day, you write tests. And
we have seen some examples in the previous presentations, how you would do like a doc
test where you attach some kind of a formatted test and it's expected output. Well, since
a few versions, these tests are actually automatically run. So if you define a module, in this case
Christmas, you see that the doc tests are failing. I think I can easily fix it by changing
the expectation. And if you run it again, the doc tests are green again. But you can
also do just your regular testing. The only thing you have to think about is that you
have to disable your auto-running. But then again, you can do your testing and you have
to make sure you call the run function on the X unit module. So there is no excuse not
to test, actually. I want to end with reference to these two resources. There is an initiative
by Dockyard Academy. It's an open source curriculum to learn Elixir. And they have used the notebooks
or the live books to actually teach this to students. And the other thing you might have
heard about in the Elixir news is the Project Bumblebee, which allows you to actually play
around with these new neural networks like GPT2 and stable diffusion. And you can just
do it locally. So it's a very nice way. It integrates very nice into your live book notebook.
All right. That's it for me. Thank you very much. Thank you very much. Is there any question?
Could you maybe compare and contrast live books with Jupyter notebooks?
Yes. That's actually a reference. Sorry. So the question was how this relates to the
Jupyter notebooks, which we might also know. I think it's very much inspired by it. So
it's also a computational notebook. But I also see a lot of differences, although I
do not know Jupyter notebooks very well. But I think, for example, like the dependencies
in the first cell, I do not think there is such a system in the Jupyter notebooks. You
would have to use like Comda or Anaconda to set up your dependency. So it's a little
bit less integrated. But I cannot say more about differences. But you're very right.
There is a strong inspiration there. Yes. Thank you. Any other question?
Cool. Thanks for the talk. I wanted to ask, actually, whether there is an option as well
for a live book being available as an UI within the IDE, so kind of connected closer to the
development environment? No. Not that I know of. No. No. It runs in the browser, and that's
where it lives. So you can install it as a standalone application, but it's still something
that lives in the browser. But you're right in the sense that it is not a full-blown IDE,
and that's also one of the nuisances that I have noticed is that if you have very large
code cells, for example, you are missing some features. And if you're used to VI bindings,
for example, you will not find them there. Yeah. Cool. Thanks. Yes.
Okay. Last question. Does this work for multiple users collaborating on things? Yes. And I
should have shown this. It is one of the nicest features. Thank you for opening that door.
If you're using multiple sessions or, for example, multiple users in multiple locations,
you, for example, see the selections they made. You see a little cursor where they are
editing or you are editing, and you're actually editing the same notebook. So, yes, it's kind
of a live coding environment. Yes. I don't know. No. It's building on top of these goodies
we have. Yeah. Okay. Thank you again.