Now we have Arjan's Japanese and it's going to talk about how live view can be used to
keep you warm.
So give it up.
Alright.
Now thank you everybody.
Nice to see such a big turnout.
When we last organized this deaf room I think three years ago it was a much smaller room
and it was really packed but now it's not packed but it's obvious because it's a bigger
room so I'm very glad that everybody's here.
So my name is Arjan's Japanese and I'm going to talk a bit about knitting today with you
and also a bit about live view.
So I'm going to give a little bit of background about my project and about what I'm doing
and then I'm going to talk about live view and how I use it and then I'm going to wrap
up.
So let's start with some background.
I've been programming since I was a little kid yesterday in the when we had a beer together
I talked about what is your first OTP release that you used and mine was actually R13.
So that's a bit and you can guess my age a bit.
It's from 2009 so I've been using Erlang since 2009.
Before that I used PHP a lot but then I got introduced by somebody to Erlang and that
actually Mark Warrell had a bit the same story as Jose Valim had when he created Elixir.
Mark wanted to create also a web framework for Erlang.
Elixir was not yet born and so he created Zotonic a web framework.
And I'm still a contributor to that and it's still alive although it's not as popular as
Elixir or Phoenix for that matter.
But then later I got used to programming in Elixir and I've been doing that since for
quite a bit.
So my background is I studied AI I have a master's in that back when AI was not hot at all.
It was the middle of the winter of AI nobody wanted to do anything with it but I thought
hey why not.
But then it turned out I did not really do anything with AI for a long time so I just
became a regular web developer doing first PHP like I said and then Elixir.
But I'm still interested in AI in hardware and software and also in art actually because
after when I stopped or graduated from AI school I went to the Art Academy, the Rietveld Academy
in Amsterdam and I decided to proceed the career in art or at least try to do something
more creative outside of pure computer science.
So one of the let's talk about that a bit and because it gets me slowly to the knitting
stuff.
So I worked with Clazine van de Zans, Gilpe Lodz, she's a friend of mine and an artist
slash interaction designer and together we did quite a lot of projects that were related
to this kind of stuff so for instance we built an app where you could interact with a fake
social network from the Dutch golden age.
So you would walk around in a museum and then you would walk next to a Rembrandt painting
and then Rembrandt would send you a private message and wanted to become your friend.
And that way it actually told a story about that golden age that was also already like
a social network.
It told the story of history basically through the current situation and it's still used
by children mostly for education.
Other similar kind of project is where in Ghent where we created augmented reality installation
where there was actually a chat, a little chat going on here and there was an archaeologist
who was chatting to you about the objects that you would scan.
Another project that was really nice during COVID actually was Distance Disco.
And due to disco which is like an app where you dance silently with your headphones on
and you're matched to somebody else and then you have to basically mimic how you dance
and then based on if you find somebody who dances like you then you're probably dancing
to the same song because everybody listens to another song.
I gave a talk, that was written in Erlang actually, the back end for that, like with
processes for everything and matching people together.
That's another talk.
Another talk that I actually did three years ago here was with this printer, it was also
a project for Clazine where she created like some kind of interactive cooking installation
from the future.
So you first would have to interact with this Google speaker over here and then the speaker
would tell you, ask you a few very personal questions like do you believe in God and do
you value your privacy and what would you do if, I don't know.
And based on those information there would be a little recipe printed out and you would
get instructions on to make something in that installation.
So it was only logical that when Clazine got some new projects that she thought of me
and she thought of, hey I have this, I have now this customer which is our customer.
It's, I don't know, was some art collective of people who will approach Clazine to make
some kind of installation for a conference.
So like a conference like this where there would be somewhere a back channel with information
on how the people in the conference are doing.
So for instance a graph of the mood or tweets or pictures etc.
And she thought, hey why would we show it on the screen, why not show it in a knitting?
Which is logical, right?
And then she thought of me, this doesn't work, sorry, I have to do it like this.
Because actually ten years ago I already hacked a knitting machine once together with two
very talented people who did actually most of the knitting, I just did most of the software
around it.
But this was also a long time ago, you can I think still look this project up.
So Clazine thought, well I want to make like a giant knitting for a conference where everything
that is happening in the conference gets knitted out and then we have like this big carpet
that you can still look out after, like a big blanket of conference feedback.
So she asked me like do you want to do this project and I was like yeah why not.
Because it was for me, it's not like I do this kind of stuff full time, it's more that
I do it because I just like it and whenever Clazine has an idea I just do it.
So we went on to eBay or Markplatz actually and we bought this pass up electronic knitting
machine which is a machine from I think 30 or 40 years ago, Swiss made so very well made.
Some people describe this machine as the Rolls Royce under the knitting machines.
So I thought hey this is a nice machine to look at and to see if I can make it knit what
I want.
So I bought it and I put it in my home.
So now I have a room which is basically the knitting room because it's a large machine,
it's like I couldn't bring it unfortunately.
I would have and I thought well it's like a printer right, it has pixels, every knitting
is a printer, it's a pixel and I just write a printer driver for it basically.
Now well it's not that easy as it turns out, unfortunately and over the last couple of
months I've grown a lot of respect for the whole knitting industry, robotics things
because there's actually a lot more to it than just, now we as a software developer
we're very lucky to be in such a stable environment where we just write code and it does something
or it doesn't something but there's nothing in between, it's not hardware, it is kind
of, it's kind of interesting to learn and also I found out there's actually YouTube
videos of people operating this machine and those people are usually like 60 year old
women but they can do it so well there's like a lot of instructions how to do that and there's
a lot of parameters to tweak and a lot of weird tools that you have to use to get it
right but eventually I got it somehow working.
It's basically the parameter space of like you have to have a certain knit, you know
a certain thickness of the yarn and you have to like the proper tension of the yarn because
otherwise you get loops, it is unbelievable how much, it's like it's basically like trying
to learn to play the violin or trying to yeah something like that and like playing, trying
to play the violin without a proper instructor present or the instructor has died already
because it's such an old violin.
So I have a very small clip of me knitting if it's here, hello, oh there we go, it takes
a while and there's no release, no sound, oh that's very short but this is basically
how it goes so the machine is there, there's yarn coming in from the top and the machine
goes over the needles, there's a lot of needles here and then once this carriage goes over
it basically the needles hook into it and make it like a yarn or knitting and the knitting
comes out underneath.
And how did I want to automate this, I have to watch my time a bit and so I'm going okay.
I actually found online some Germans who actually used this machine before to hack it because
there's actually, damn it, this doesn't work.
So what I wanted to replace was this, how you used to program it was basically there
was this big flow chart in this manual where you would need to press the buttons in a certain
combination and then set the dials, everything and then upload the pattern and the way you
upload the pattern is you basically take a piece of matrix paper like with a grid and
you make some of the cells you make black and that's basically your pattern so you draw
pixel art on a paper and you feed the paper into this scanner because this is kind of
a scanner and then this thing somehow says okay well I've now remembered that pattern
and then this machine communicates with the thing that goes and the carriage that goes
back and forth and that actually then nits that pattern.
I did not try to do that because it sounded very hard and so instead I found this space
from the hackerspace Bamberg in which they used an Arduino because this is the connector
that you need to plug into the console so I basically replaced the whole console with
an Arduino so on one side there's the Arduino communicating with the carriage and then out
comes digital signals, just a serial protocol that goes back into the computer and then
the computer can tell you know knit this pattern, knit that pattern etc.
If you have more questions about this I can answer a lot, I'm not going to do it right
now but basically the new user interface that I'm working right now looks a bit like this
so it's just basically a browser and also because I wanted to use live view for something
so that's we're finally getting to that subject, yes.
So this knitting interface shows a bit, it shows well basically where the machine is
at in the knit process and it shows the current color that it is knitting, it shows whether
you have to move it left or right because there is a motor, you can enable the motor
so the knits automatically but you can also do it manually.
So there's a little counter, it has a start-stop button and it has several configuration things
like because it's a very big knitting machine so you have to specify or you have to knit
it here or knit it there, it's so wide and then you can upload a pattern so you can type
in 1 to 0's here basically literally you can type in 1 to 0's and it will then create
a pattern and then there's a state machine that will loop through that pattern to send
the proper instructions to the Arduino.
So I'm going to demo that a bit later because first I want to go a bit into the detail of
this live view, well it's not only live view, it's basically just a Phoenix or just an
elixir project that has several parts.
One of it of course is the user interface which is all the way over there and the other
part is the actual knitting machine that's over here so it's connected with an Arduino
like I showed and the Arduino is connected to the elixir so this middle part is the interesting
software part that I've built.
So there's a few components here and I've created some kind of color coding for that
I invented myself so basically green is an Erlang process or elixir process and orange
is like state, it's just data and without color is something that is not very interesting.
So basically whenever I used the NERVs UART library I think it was already mentioned that
the NERVs project is really nice for doing IoT kind of things with elixir so UART is
the protocol, it's basically a serial port so whenever you program an Arduino you can
tell it to send and receive serial commands and you can very easily listen to them with
elixir.
There's a monitor that basically looks for the serial port so I can hot plug basically
the knitting machine into my computer and pull it out.
And then basically whenever it receives some serial packet, just a line, basically just
a text line, it sends us over, Phoenix pops up to the rest of the system.
And then there are several other components that listen to those serial commands, one
of them is the control which is a gen server that basically has all this state, basically
Asia has all this state like where am I in the knitting etc.
So that gen server holds the state of the knitting machine itself so that basically
has the task of transforming this pattern plus the settings into a sequence of commands
that needs to be sent out to the knitting machine and also the state is also updated
whenever a new serial commands come in.
And then the control is also connected to the live view which actually shows everything
that is being done.
So what I'm going to talk the rest of is mostly this part because that's the title of the
talk, it's about a live view and how that works and what it is actually, how it works
in my case.
Are there any questions?
Oh, I have 10 minutes left, really?
Okay.
I have 10 minutes left.
So let's go a bit quicker now.
What is live view?
Well, from the docs it says live view provides rich real time experiences with user rendered
HTML and to understand or with server rendered HTML and to understand what that actually
means, let's dive into a little bit of the web history.
When this whole internet thing started, the first thing there was was just a browser with
HTML, right?
So you had a web server, you uploaded the HTML file too and you just view it.
That's basically it.
And when you want to go to another page, you just click a link and you view the other page.
That's basically it.
This is just what we now call a static website, very static.
So that worked pretty well for a while.
And then when at some point people thought, well, we can also not make the static, but
we can send something different back every time.
So that's when it became DHTML, dynamic HTML, PHP was born, other programming, I'm getting
feedback from the mic now, it's a bit irritating, whatever.
So basically, the HTML became dynamic.
So there was a lot of logic on the server, you would make a PHP file that rendered out
HTML.
The HTML was different for each user even because there were sessions, you have cookies
where you store basically the state like this user is logged in, this user has this stuff
in his cart, whatever.
So there's a lot of logic on the server that renders into HTML and then the HTML is just
sent over the wire and the browser displays it, win, done.
So that worked pretty okay.
But it was not really interactive because every time you had to do something on the
site, it would reload the page.
So then at some point people thought, hey, I can make little effects, I can do hovers
and animations.
So we make a JavaScript as born basically, I think with IE version, something free, it
started to became popular.
So there was a bit of JavaScript written to make things a bit more lively and a bit more
dynamic without having to reload the page every time.
Well, of course, you know what happened next.
So JavaScript became very big.
So a lot of the logic was actually moved to the client.
So the pages did not reload every time.
It's basically just load the page one time and then JavaScript basically takes care of
the rest.
It traces parts of the HTML with other parts, it even sends you to another URL without actually
reloading the page with push state kind of things.
So there was a lot of logic on the client suddenly.
And it would fetch underwater, not over HTML, but it would just use what's called Ajax,
but currently we have rest and GraphQL, all kinds of protocols to get data into the
client and then do stuff with the data.
So there was a lot of logic suddenly on the client.
And this is still the case.
I think when you write JavaScript, there's like any web project is quite heavy on the
server side or on the client side.
Now with LiveView, the pendulum has swung in the other way a bit again because they're
actually very interesting thing about LiveView is that we can do very interactive things.
We don't have to reload the page every time to do something interactive.
We can stay on the same page, but we can still dynamically change parts of the page without
having to do very heavy, create all kinds of APIs and do complicated things.
So suddenly with LiveView, the logic is again mostly for 99%, I would say, back on the server.
So it's actually, it's like a bit again from back in the old days.
You just render something, you just put it to the browser and then the browser displays
it.
It's as simple as that.
You don't need to write a lot of JavaScript unless you really want to.
That's actually one of the promises of LiveView that you can make UIs very quickly just staying
in Elixir and just templating from Elixir.
So how does that actually work?
Is there a diagram on the next slide?
Yes.
So basically what happens is, I can show this one first.
So there's one LiveView process.
So basically in Erlang, processes are very lightweight, five minutes left, oh no.
Templates are rendered on the server and they are rendered every time you update the state.
But it does not send the whole template over to the client, it just sends the things that
are changed.
So basically it works a bit like this.
So the first time it renders, we get some HTML and then it actually connects over a web
socket and then it says, hey, I'm a LiveView process and I can now interact with you.
So and then whenever some state changes, the browser is now connected to the process in
the corresponding process in the server.
So when the state changes, it actually re-renders something and it actually just sends the things
that have changed, sends it to the browser and then the browser is intelligent enough
that it can just patch small parts of the DOM tree to just change that part and not
change everything.
So that makes it very lightweight, very flexible.
And this is a bit, well, when you start with this, it is re, it's now integrated into
Phoenix, the Elixir web framework and I will give a little demo now, I think.
So let's make the knitting live.
So it would look a bit like, when you write an Elixir module, it looks a bit like this.
There's always something that you have to write, it's mount.
When you mount something, you just return, okay, I'm mounted and I have some assigns in
my socket.
It's a bit like normal Phoenix templating.
You assign things to something and then you can render, use those assigns to render something.
So back below here, there's actually the render function and in this case, it just renders
an image tag, image class movie.
Wonder what that is.
And the source is some kind of, is an image URL with a variable in it, a frame.
The frame is a sign that is assigned here.
So it renders a single image.
And whenever a serial command comes in from a serial port, basically it calculates a new
frame and then assign it to the socket again.
So this triggers another render here and it will probably change the frame number so it
will change the image.
So this basically connects the serial port to a live view.
I think that's better.
So I actually have a demo of that because I actually, well, I did not bring my knitting
machine, I actually brought my knitting machine emulator, which is an Arduino with a potential
meter attached.
And I can probably now, this will fail, but who cares, plug it in.
And then we go to this.
So if you look in the source, we see somewhere near this, this is the one.
We see the movie.
It's now at frame number 15.
And then if we, is it running, should be running.
If we now turn the knob, yes, okay.
So now I can knit.
See, I am knitting.
So I did not bring my whole machine, but I brought a virtual version of myself that is
now at home.
And I can control it through this little Arduino over here.
Yeah, I thought of this last night to demo it like this.
It was not really prepared, but yeah, so that, I hope that gets you a bit of the idea
how live view updates its state.
And in this case, it is very simple, but an actual live view, of course, is much more,
much bigger.
And actually, that is the rest of my talk that I still need to do.
Let's continue very quickly.
How much?
Oh, time's up.
Is it really up?
I think that you can take like one or two minutes, so I don't know where.
Okay.
I will quickly skip through the next slides then.
So you can imagine that writing a single elixir module with every logic in it, you get one
big assign with everything.
It's not really scalable.
So there are actually two things to make that scalable.
You make components.
So one of the components is function components, which are basically just rendering templates
inside functions.
And then the other ones are Phoenix live components, and those are basically like sub-live views
in your live view that have their own state and their own render function and their own
mount function.
So in this case, in my UI, these are, I just created a few components.
One component is settings components, which contains a form.
Another component is a row component that just renders a single row.
So this row component is very simple.
It just renders basically a set of divs, and then you can call it like this.
So it's basically, in the template syntax, using function components, basically the same
as a normal tag, but you prepend it with a dot, which is because it's basically a function
call.
And the live components are stateful.
So live components have their own states, or they have a mount, you can assign things
to there.
And what I already said, it's a live view inside a live view.
And so these signals directly communicate with the live components in this case.
Then there's some more things like slots you can create like different parts of your component
and make them into like separate things where you can put part of your DOM tree as well.
And I just wanted to say there's a lot out there, like there's a big community, and
I think live view is really getting a lot of traction.
And it's actually a shame.
I have not done anything really with live view in production, actually.
I wanted to make a disclaimer there.
But I really like where it's going, and there's a lot of projects popping up with component
libraries and people making stuff on top of it.
There's the storybook project, which is also very nice, which allows you to make a library
of components and then have like a live environment somewhere where you can document these components
and try them out and copy-paste the code for you to use inside your live view.
So there's a lot of things, there's JavaScript integration, which I'm not going to show.
There's live view native coming up, it's also very nice technology where you don't render
things to the browser, but you render things into a native app, so you actually build like
a native app, like what the React Native is to react is, well, you get the drift.
So thank you for listening.
Thank you, Arjen.