[00:00.000 --> 00:14.480]  Okay. Okay, now we have Linus de Meijer with shorter feedback loops with the live book.
[00:14.480 --> 00:15.480]  Give it up.
[00:15.480 --> 00:27.120]  All right. Thank you. Can everybody hear me? Yeah. To my surprise, I am the first and
[00:27.120 --> 00:36.600]  only Belgian here presenting. Well, I can welcome you all here. Maybe the first question
[00:36.600 --> 00:44.280]  I want to ask, which is the most important one, who is hungry right now? All right. Sorry,
[00:44.280 --> 00:52.320]  I cannot help with that. But another question I would like to ask is, who has heard of Live
[00:52.320 --> 00:58.160]  Book? Who knows what it is, more or less? Yeah. Okay. It's a lot of people who has worked
[00:58.160 --> 01:03.920]  with Live Book professionally or has just, has it on their computer? Okay. Less people,
[01:03.920 --> 01:10.040]  but there are some. If you want to follow along and if you already installed Live Book,
[01:10.040 --> 01:16.680]  then you can go to my GitHub repository. I have a little notebook prepared. And I try
[01:16.680 --> 01:24.480]  to switch back and forth between this presentation and the Live Book. All right. The goals of
[01:24.480 --> 01:33.200]  today would be to introduce you to Live Book, to make sure you all understand what it is,
[01:33.200 --> 01:37.880]  how you can get it, how you install it, the various options. And then I think the most
[01:37.880 --> 01:44.560]  interesting part is how I used it in three different cases and what I learned from using
[01:44.560 --> 01:53.240]  it in a real project. And underneath all this, I hope I can bring across the message that
[01:53.240 --> 02:02.160]  Live Book really helps to start somewhere in the middle. So you don't spend time like
[02:02.160 --> 02:09.600]  scaffolding an application. And then just after a few days or hours, gets to the most
[02:09.600 --> 02:15.400]  interesting part. So Live Book enables you to start in the middle. That's my main message
[02:15.400 --> 02:25.320]  here. Whenever you start Live Book, you're greeted with like an introduction page. At
[02:25.320 --> 02:32.480]  the top you see your folder structure. There is a very nice learning section. And at the
[02:32.480 --> 02:42.160]  bottom there you have your sessions. So you will import a Live Book often and then a session
[02:42.160 --> 02:47.480]  will appear and you can hook into that. So you can, that's actually what we are going
[02:47.480 --> 03:00.600]  to do here. I'm going to go to my notebook that I just prepared here. And yeah, I just
[03:00.600 --> 03:04.360]  wanted to point out if you are just starting with Live Book, there is a very good learning
[03:04.360 --> 03:10.160]  section. So please go through these. Also if you're learning Elixir, it's a very good
[03:10.160 --> 03:17.520]  way to familiarize yourself with Elixir. And it also covers things like how you make
[03:17.520 --> 03:24.320]  pretty graphs or how you would use the Kino Library, which is the one that is used to
[03:24.320 --> 03:36.280]  actually interact with your Live Book. It's all just marked down. So it uses those, yeah,
[03:36.280 --> 03:44.840]  you can see here, it uses these code fences with the Elixir annotation. So it's very easy
[03:44.840 --> 03:53.720]  to check into your GitHub repository and make sure you can review it if you want. And GitHub
[03:53.720 --> 04:02.800]  also recently added the feature that it nicely formats your Live Books. They have the extension
[04:02.800 --> 04:14.480]  Live MD. So it integrates nicely with your version control system. The basics. We have
[04:14.480 --> 04:20.400]  code cells which can be executed. They contain your codes. And the first one is a little
[04:20.400 --> 04:28.720]  bit special in the sense that it often contains your setup. So you can pull in all your dependencies.
[04:28.720 --> 04:36.160]  So you can use the mix install function. And I'm going to use a few here, not too many.
[04:36.160 --> 04:43.320]  But I'll go over these once they become relevant. So right here we have our first code cell.
[04:43.320 --> 04:49.360]  We can just execute it. It takes a while to just start up. But then we can go. It is
[04:49.360 --> 04:54.760]  being evaluated. You can see the green dots. So it's being evaluated. And you have all
[04:54.760 --> 05:02.360]  those nice features that you can expect from an IDE. So you can ask it to autocomplete.
[05:02.360 --> 05:12.000]  If you control space one more time, you get all the documentation for that function. So
[05:12.000 --> 05:21.120]  you get a lot of help editing your code here. The result is being print down below here.
[05:21.120 --> 05:26.440]  And you also have the ability to, like I did here actually, to put stuff. And that's also
[05:26.440 --> 05:32.480]  being printed underneath your code cell. So that's very nice. And yeah, maybe the most
[05:32.480 --> 05:39.520]  or a very important feature at least, it's that you can interleave your code blocks with
[05:39.520 --> 05:45.280]  just regular markdown. So it's a really nice way to do a little coding and then explain
[05:45.280 --> 05:59.640]  what you have done and then go on to the code again. Yes, a few words about reproducibility.
[05:59.640 --> 06:07.400]  It's very nice to have this notebook and to know what actually will happen if you execute
[06:07.400 --> 06:12.480]  all those code cells. If you start from the beginning, it's very, very clear. You go from
[06:12.480 --> 06:17.520]  top to bottom. But what if you are going to edit in the middle, make a change somewhere?
[06:17.520 --> 06:23.800]  Well, Lifehook has recovered. It analyzes all those bindings that are being made in
[06:23.800 --> 06:28.960]  those code cells. And it makes sure that if you change something, the relevant code cells
[06:28.960 --> 06:36.040]  underneath them are also going to be executed again. So that's actually the way you often
[06:36.040 --> 06:42.360]  build up states. You have a code cell that creates a binding and then in the next code
[06:42.360 --> 06:49.000]  cell you can reuse that or you can use that binding so you can build upon when you go
[06:49.000 --> 06:59.240]  through all the code cells. I can do a little demonstration how branching sections work.
[06:59.240 --> 07:05.720]  So the sections are actually shown very clearly here on the side. I have a few of them, but
[07:05.720 --> 07:14.480]  one has the little branch icon. So this is a branching section. And this is just to show
[07:14.480 --> 07:22.040]  how the execution model actually works. So right here, I demonstrate that you can use
[07:22.040 --> 07:32.160]  the bindings from before, so from the main flow of the notebook. And if I start an infinite
[07:32.160 --> 07:39.840]  loop here, this is just going to stay printing in this little frame here. You will see that
[07:39.840 --> 07:44.400]  if I execute a code cell below, it will be queued, but it will never run because the
[07:44.400 --> 07:49.920]  other code is just blocking that one. But if we carry on to the next session, we can
[07:49.920 --> 07:57.000]  see that all is well again. So this is still blocked, but this is the main threat of execution.
[07:57.000 --> 08:05.000]  So we are not blocked here anymore. And just to show that we cannot access the bindings
[08:05.000 --> 08:16.440]  from before, I just triggered this error because we cannot access this variable. Okay, this
[08:16.440 --> 08:23.640]  is a pretty picture that I stole from Josef Alem. I'm not going to go into detail, but
[08:23.640 --> 08:28.640]  I just want to point out that everything is based or is heavily using the airline distribution
[08:28.640 --> 08:36.520]  mechanisms that we all know from the airline OTP ecosystem. So we have a central application
[08:36.520 --> 08:43.880]  here. It's a live view application, actually, with a lot of JavaScript. And we can connect
[08:43.880 --> 08:50.200]  to it through WebSockets. That's all being handled for us. And it does not run the code
[08:50.200 --> 08:56.840]  actually on the live book application itself, but in normal mode, it will spawn a new node
[08:56.840 --> 09:04.560]  and run your code on this new node. So we call it a runtime. This runtime is not aware
[09:04.560 --> 09:10.200]  of anything live book related. It is just a plain node that can execute code and you
[09:10.200 --> 09:17.640]  get the results back. So that's what's going on underneath. There are a lot of ways you
[09:17.640 --> 09:26.160]  can get live book on your computer. Recently, well, I used to like e-script installation,
[09:26.160 --> 09:32.200]  but since it's tied to your Elixir installation, I now switch to using the desktop application,
[09:32.200 --> 09:38.400]  which is getting very good at this point. You also can run it in the cloud. You can
[09:38.400 --> 09:43.880]  have it as a Docker image. That's all being covered. The various ways to start, not very
[09:43.880 --> 09:53.640]  interesting, but I think what's more interesting is my story of how I used it to mitigate risks
[09:53.640 --> 10:02.560]  early on, some projects that I've been doing. Yes, I just want to sum up here the benefits
[10:02.560 --> 10:10.800]  that I see. So it allows you to start in the middle. If you're using live book, you can
[10:10.800 --> 10:18.880]  jump straight into your problem space. It increases transparency, and you can use it
[10:18.880 --> 10:25.000]  because you can use that markdown in between to document your process. So all your thoughts,
[10:25.000 --> 10:30.520]  you can put them in between all those code cells, and it's, I think, way better than
[10:30.520 --> 10:37.000]  those obscure scripts we sometimes write, and you can also very easily share this document.
[10:37.000 --> 10:45.520]  So that's actually something that we did. I got some tasks to do. A client was asking
[10:45.520 --> 10:51.080]  something. We were doing something with machine learning and artificial intelligence. We were
[10:51.080 --> 10:56.800]  not aware or we did not know whether we could do it. So I sat down, made his live hook,
[10:56.800 --> 11:02.520]  and then documented all the steps I did, and in the end, got a pretty graph out of it,
[11:02.520 --> 11:11.880]  so I could convince the client that we could do it, actually, with Elixir.
[11:11.880 --> 11:19.040]  Just a little bit of context. I work for a small company. We often switch in between
[11:19.040 --> 11:25.240]  projects. The company is named Zenjoy, and we are often working as a team of two. So
[11:25.240 --> 11:30.360]  documentation and collaboration is very important, and also the communication with the clients
[11:30.360 --> 11:39.320]  is very, very important. So in this first case, we were tasked to
[11:39.320 --> 11:49.720]  interoperate with or to call an undocumented legacy API. It was very low level. It was not
[11:49.720 --> 11:57.200]  as low level as tools explained to us, so it was not like we had to do the pattern matching
[11:57.200 --> 12:03.800]  on the bit level, but we had to use the GenTCP module straight from Erlang, but it was very
[12:03.800 --> 12:11.280]  nice to have this live book environment where we could just throw the commands at this server
[12:11.280 --> 12:19.360]  that we could somehow use and see what came back. So in this way, we were able to create
[12:19.360 --> 12:29.000]  a notebook that documented all the commands we could see or how it reacted, and you see
[12:29.000 --> 12:35.280]  some pattern matching going on here. You also see some magic variables. So this was given
[12:35.280 --> 12:40.280]  to us, so we could not change this, but at least we could document it, and this became
[12:40.280 --> 12:54.040]  a very long document to refer back to. So this is another demonstration I wanted to
[12:54.040 --> 13:01.440]  do. It's not because you're in the browser that you're constrained by any way. You can
[13:01.440 --> 13:07.960]  still use all the process magic and all the GenServers you like, and this is just a demonstration
[13:07.960 --> 13:19.400]  of how you would go around and spawn a TCP server. I'm using Thousand Island here. In
[13:19.400 --> 13:28.920]  reality, I was using the other one, the older one, Cowboy and Ranch. Yes, thank you. But
[13:28.920 --> 13:33.600]  now for this demonstration, I got to use Thousand Island, and it's super nice, so you can
[13:33.600 --> 13:40.760]  just define your handler. It's just going to echo back whatever we send to it, and here
[13:40.760 --> 13:47.120]  I started up. The only caveat or the thing you have to be aware of is that you can start
[13:47.120 --> 13:54.400]  your children, your processes, and your supervisor tree under the Kino code cell. So whenever
[13:54.400 --> 14:01.240]  you reevaluate here, you see that you can't see, but another process is started or a whole
[14:01.240 --> 14:08.400]  tree starts. So this is a nice interop with the Lightbook environment. And once we have
[14:08.400 --> 14:22.840]  those, we can even... Yeah. Yeah, I guess that the gods are not with me today, but at
[14:22.840 --> 14:29.280]  least what I wanted to show is that you can actually draw a pretty picture of the supervision
[14:29.280 --> 14:37.880]  tree right here. But still, I think the server got started, and so now I can net cut into
[14:37.880 --> 14:46.120]  my local host on this given port, and I can see how some stuff is being echoed back to
[14:46.120 --> 14:58.760]  me, so at least that works. All right. Back to the presentation. So it is a nice environment
[14:58.760 --> 15:07.160]  to stub out a server and set up a situation where you can then use your application to
[15:07.160 --> 15:20.360]  interact with this stubbed version of your API. All right. I'm going to show, or this
[15:20.360 --> 15:29.080]  is just an example of how you would integrate your Livebook with a regular mixed project.
[15:29.080 --> 15:33.520]  It's all just sitting next to each other. Oftentimes, I just make a folder where the
[15:33.520 --> 15:40.320]  notebook lives, and then your mixed project, whatever it is, it can be a Phoenix application.
[15:40.320 --> 15:53.080]  You can access it if you use the part way of referencing your dependencies. A few words
[15:53.080 --> 16:01.320]  about a typical lifecycle that I've observed, you often start to experiment in your Livebook.
[16:01.320 --> 16:08.280]  On good days, you add tests, and then you move all that code into the regular application
[16:08.280 --> 16:15.680]  or in the regular mixed project, and you reference it from there on in the way that I just described.
[16:15.680 --> 16:25.600]  So you promote reusable code, and that's often a way that worked very well for me. The second
[16:25.600 --> 16:32.320]  case that I want to discuss is how I set up or created concurrent ETL pipeline, which
[16:32.320 --> 16:39.880]  is a fancy word for just loading CSV files and then maybe transforming them and dumping
[16:39.880 --> 16:47.880]  them into Postgres. So I really got to learn a lot about how concurrent data processing
[16:47.880 --> 16:54.400]  actually happens. I got to play around with Flow, which is a very nice library which builds
[16:54.400 --> 17:05.080]  on top of GenState. No, not GenState, the other one. GenStage, that's the one. And you can
[17:05.080 --> 17:13.760]  still use all the power of processes that are available in Euler and Elixir. To demonstrate,
[17:13.760 --> 17:20.720]  I've prepared or I want to show how you can use ECTO and then this Flow library right
[17:20.720 --> 17:28.400]  within your like hook application. To start off, I create a repository, just like you would
[17:28.400 --> 17:33.040]  do in a Phoenix application. Don't worry if you do not recognize this. This is kind of
[17:33.040 --> 17:40.680]  standard stuff. You have to specify the adapter, and then you can emulate whatever mixed ECTO
[17:40.680 --> 17:45.240]  create would do. So you make sure that your storage is up. In my case, it was already
[17:45.240 --> 17:51.360]  up, so that's what it reports. And then you can even make your migrations like you would
[17:51.360 --> 17:59.400]  if you used ECTO together with a Phoenix application. In this case, I also made sure there was
[17:59.400 --> 18:05.840]  an item in the database so we can query it later on. So I have to make sure that this
[18:05.840 --> 18:12.640]  repository actually runs. And then I migrate, well, I do a rollback to make sure nothing
[18:12.640 --> 18:20.160]  is being left over, and then I migrate again. So we have the end situation that I can query
[18:20.160 --> 18:29.840]  right here. So you can see that our fresh entry is just inserted with this new timestamp.
[18:29.840 --> 18:37.520]  And then we can build upon this. This is another demonstration, very short. This is a definition
[18:37.520 --> 18:43.080]  of a flow. Also, don't worry if you do not recognize this. That's not the key here. I
[18:43.080 --> 18:48.720]  just want to show that you can use all those goodies, and you're not constrained in any
[18:48.720 --> 18:55.760]  way. Right here, we are just emitting a value every second, and we're going to wait for
[18:55.760 --> 19:02.560]  three seconds and then insert an item in the database again. So you get the logging, and
[19:02.560 --> 19:09.480]  if we query it again, so you might recognize the nice ecto query syntax. You see I've
[19:09.480 --> 19:18.880]  wrapped it in a data table, and you can now see the three new items appearing. So that's
[19:18.880 --> 19:25.440]  very nice. Here you see me playing around and actually visualizing this ETL pipeline
[19:25.440 --> 19:32.240]  where every color actually is another class of objects or is being inserted in another
[19:32.240 --> 19:39.280]  table. It went very quickly, but when making this presentation, I also saw there is some
[19:39.280 --> 19:46.480]  room for improvement. So not all cylinders are firing together, but at least it was fast
[19:46.480 --> 19:55.680]  enough for our purposes. Another case I want to share with you is that we used the live
[19:55.680 --> 20:05.960]  hook to actually connect to a live running instance. So remember, as I have shown in
[20:05.960 --> 20:13.040]  the beginning, it's all just Erlang distribution under the hood. So instead of using the regular
[20:13.040 --> 20:20.240]  setup where you do an LXR standalone setting, you can also do an attached node configuration.
[20:20.240 --> 20:27.520]  The only thing you have to know is your node's name or the short name and a cookie which
[20:27.520 --> 20:33.400]  you have to agree upon. And it's very good for doing one of the tasks. Maybe you don't
[20:33.400 --> 20:42.240]  have a UI for something yet, and you want to do it in a live book, then this is a nice
[20:42.240 --> 20:48.840]  way to actually have like a super admin interface, but be aware that this is still a live environment.
[20:48.840 --> 20:54.400]  So if you do this, make sure to put a big disclaimer on top of your notebook to remind
[20:54.400 --> 21:05.360]  you of the risks involved. All right. The last thing that I want to share or show is
[21:05.360 --> 21:11.680]  how you would do tests in live hook. Like I said, on a good day, you write tests. And
[21:11.680 --> 21:17.440]  we have seen some examples in the previous presentations, how you would do like a doc
[21:17.440 --> 21:28.880]  test where you attach some kind of a formatted test and it's expected output. Well, since
[21:28.880 --> 21:37.520]  a few versions, these tests are actually automatically run. So if you define a module, in this case
[21:37.520 --> 21:48.160]  Christmas, you see that the doc tests are failing. I think I can easily fix it by changing
[21:48.160 --> 21:53.200]  the expectation. And if you run it again, the doc tests are green again. But you can
[21:53.200 --> 21:57.320]  also do just your regular testing. The only thing you have to think about is that you
[21:57.320 --> 22:04.000]  have to disable your auto-running. But then again, you can do your testing and you have
[22:04.000 --> 22:10.720]  to make sure you call the run function on the X unit module. So there is no excuse not
[22:10.720 --> 22:23.480]  to test, actually. I want to end with reference to these two resources. There is an initiative
[22:23.480 --> 22:31.400]  by Dockyard Academy. It's an open source curriculum to learn Elixir. And they have used the notebooks
[22:31.400 --> 22:37.520]  or the live books to actually teach this to students. And the other thing you might have
[22:37.520 --> 22:44.800]  heard about in the Elixir news is the Project Bumblebee, which allows you to actually play
[22:44.800 --> 22:52.160]  around with these new neural networks like GPT2 and stable diffusion. And you can just
[22:52.160 --> 23:00.520]  do it locally. So it's a very nice way. It integrates very nice into your live book notebook.
[23:00.520 --> 23:19.520]  All right. That's it for me. Thank you very much. Thank you very much. Is there any question?
[23:19.520 --> 23:24.200]  Could you maybe compare and contrast live books with Jupyter notebooks?
[23:24.200 --> 23:31.840]  Yes. That's actually a reference. Sorry. So the question was how this relates to the
[23:31.840 --> 23:38.040]  Jupyter notebooks, which we might also know. I think it's very much inspired by it. So
[23:38.040 --> 23:44.160]  it's also a computational notebook. But I also see a lot of differences, although I
[23:44.160 --> 23:50.360]  do not know Jupyter notebooks very well. But I think, for example, like the dependencies
[23:50.360 --> 23:56.040]  in the first cell, I do not think there is such a system in the Jupyter notebooks. You
[23:56.040 --> 24:01.120]  would have to use like Comda or Anaconda to set up your dependency. So it's a little
[24:01.120 --> 24:07.960]  bit less integrated. But I cannot say more about differences. But you're very right.
[24:07.960 --> 24:21.040]  There is a strong inspiration there. Yes. Thank you. Any other question?
[24:21.040 --> 24:32.600]  Cool. Thanks for the talk. I wanted to ask, actually, whether there is an option as well
[24:32.600 --> 24:39.880]  for a live book being available as an UI within the IDE, so kind of connected closer to the
[24:39.880 --> 24:48.360]  development environment? No. Not that I know of. No. No. It runs in the browser, and that's
[24:48.360 --> 24:52.680]  where it lives. So you can install it as a standalone application, but it's still something
[24:52.680 --> 24:59.960]  that lives in the browser. But you're right in the sense that it is not a full-blown IDE,
[24:59.960 --> 25:05.520]  and that's also one of the nuisances that I have noticed is that if you have very large
[25:05.520 --> 25:11.280]  code cells, for example, you are missing some features. And if you're used to VI bindings,
[25:11.280 --> 25:16.000]  for example, you will not find them there. Yeah. Cool. Thanks. Yes.
[25:16.000 --> 25:26.840]  Okay. Last question. Does this work for multiple users collaborating on things? Yes. And I
[25:26.840 --> 25:33.200]  should have shown this. It is one of the nicest features. Thank you for opening that door.
[25:33.200 --> 25:40.760]  If you're using multiple sessions or, for example, multiple users in multiple locations,
[25:40.760 --> 25:44.680]  you, for example, see the selections they made. You see a little cursor where they are
[25:44.680 --> 25:52.640]  editing or you are editing, and you're actually editing the same notebook. So, yes, it's kind
[25:52.640 --> 26:03.160]  of a live coding environment. Yes. I don't know. No. It's building on top of these goodies
[26:03.160 --> 26:23.280]  we have. Yeah. Okay. Thank you again.