Hello everybody, can you hear me?
Kind of okay?
My name is Nelonavi, I'm an open source enthusiast and senior software engineer at the Succo Group.
I'm here to talk about screen sharing on Raspberry Pi with VNC.
And I don't know how many of you were at Fosnum 2023?
Alright, quite a lot of people. I have no idea if you've visited my talk, but I had a lightning talk
and it was about screen sharing and remote access with LDP.
And of course the most frequently asked question was, why don't you use VNC?
How can we do it with VNC?
So here is another talk, next episode.
And I'm going to share my experience in configuring Wayland and Western with VNC screen sharing.
The agenda focuses on Raspberry Pi 5, just because Raspberry Pi 5 is something new and we are all excited about it.
But actually at the end of the presentation you see that I tested this on several devices.
It's not hardware specific as long as you have Wayland and Western running on this particular hardware you are targeting.
The agenda includes obviously brief introduction to Wayland and Western, brief introduction to VNC,
the integration in the Yocto project and open embedded, and core image Western with VNC demonstration on Raspberry Pi 5.
And if you have no idea what core image Western is, you're going to learn in the slides that are coming.
The slides are already available on the website, fosn.org.
After the presentation I'm going to upload them to SlideShare as well.
And the slides are kind of exact steps or at least this was my goal.
So if you want to repeat this demo, following the information on the slides you should be able to do it.
So let's start with Wayland and Western.
How many of you are using Wayland?
Oh, that's fantastic. This makes my work really easy because this talk obviously is about Wayland.
Wayland is a display protocol that specifies the communication between a display server and the clients.
It's supposed to work with different windowing systems.
The main project was to replace the X windowing system.
The project is starting in 2008, so it's more than 15 years old now.
There's a security by design in terms of isolation, the input and the output of every window.
As you probably have noticed for many years this was kind of a problem when we want to take a screenshot.
But, you know, there's always advantages and disadvantages.
There are many different compositors for Wayland.
Western is the reference compositor and it's widely used in the embedded industry because it's small and simple.
There are other compositors that are much more convenient for desktop usage, of course,
but Western as a reference compositor for Wayland is very convenient for embedded devices.
What is important actually for this talk is that Western 13 was released in November last year.
It's important because the VLC backend was officially added in recent versions in Western.
We're going to talk about this in the next slides.
What are the remote desktop options that you have in Wayland and Western as of the moment?
One of the options is remote desktop protocol.
If you're interested in this, have a look at the recording of my lightning talk from last year.
How many of you are actually using CARE-DP?
And how many of you are using VLC in different setups?
All right, more people.
But it looks like the majority of the audience is not using any of them.
You should start using them. Let's speak.
So what is VLC?
It stands for Virtual Network Computing, and it's a graphical desktop sharing system based on the remote framework before protocol.
It was initially developed by Oliveri.
This was 25 years ago.
VLC has been around for a quarter of a century.
It's a pixel-based screen sharing protocol.
That's the big difference compared to LDP.
And it works with all windowing systems, including Microsoft Windows, macOS, GNOME and other distributions,
no matter if they're using X11 or Wayland.
Here's a matrix comparing LDP with VLC so you can understand the major differences in relation to this talk.
So obviously there are different types of protocols.
There are different types.
They work in a completely different way because LDP is a semantic protocol.
It's a way of control, forms and other graphical primitives, while VLC, as I mentioned, is pixel-based.
LDP has been in Western since 2016 and version 2.
VLC was relatively recently added.
It was added in Western 12.
This happened in 2022.
So the story of adding VLC to Western is a long story.
Keep in mind that I'm a later Wayland more Western developer.
I don't have contributions to Wayland and Western.
I'm an engineer working with the Yachter Project and Open Embedded.
So I'm kind of a user of Wayland and Western, and I'm doing integrations with Wayland and Western on various embedded devices.
But we need to give the credit to all these people who put a lot of efforts providing VLC back-end in Western.
The initiative, As Fires, I know, started by Stephen, and you can see that this happened almost five years ago.
It took actually three years ago, three or four years ago, but it took a while to be merged.
And finally, with some changes made by Philip, we're here.
This was merged and landed in Western 12.
The current version is Western 13, Western 14 is in development.
What I've seen in mailing lists and some emails that have changed are some new exciting things that are supposed to happen in new versions of Western.
But I'm talking about the things that are released and stable and you can use right now.
So just a few sentences to give you an understanding how VLC works with this back-end for Western.
It's available in Western 12 and newer versions, which means 12 and 13 as of the moment.
It's going to be available, of course, for the new versions as well.
It depends on a couple of open source projects.
Meet VLC is one of them.
This is an open source VLC server library with a clean interface started by Andriy.
The source code is available in GitHub in the ISC license.
So Meet VLC is built on top of other dependencies such as libdlm, mason, and package config.
And in the runtime, it depends on Andriy's main loop.
This is another open source project.
And as you can see from here, it's actually by the author of Meet VLC.
So they're tightly related.
And now we're moving on to what I do on a daily basis.
And this is the Yachter project and Open Embedded.
How many of you have experienced with the Yachter project and Open Embedded?
Oh, that's fantastic.
I have been telling everyone that the Yachter project and Open Embedded are becoming a de facto industry standard.
A lot of people were laughing, but now I see that there's so many hands, so it's good to see it.
So if you're new to the Yachter and Open Embedded in a nutshell,
the Yachter project is a collaborative project of the Linux Foundation.
It helps you create a custom embedded Linux distribution for your particular needs for the hardware that you are targeting.
It uses the Open Embedded build system, which includes BitBake and Open Embedded Core.
The most interesting part of the Yachter project, in my opinion, is actually Pocky.
This is the reference distribution of the Yachter project.
It's provided as a metadata.
There are no binaries.
Pocky gives you the opportunity to quickly bootstrap your own distribution,
so you're not starting from scratch.
If you want, you can start from scratch, of course,
but you can also take Pocky and build on top of Pocky, modify it, configure it, and so on.
So it really speeds up the process and simplifies the whole process.
The Yachter project has a steep learning curve,
but once you know how it works, it simplifies the process for building your products
and cuts the time to go to the market.
The Yachter project has twice per year a release,
and there are long-term support releases, which are covering at least two years.
Here is a short list of some of the recent Yachter project releases.
This one here is supposed to be released in April this year,
so in a couple of months, and this is going to be the next long-term release.
It's going to be maintained for actually four years now,
and the current LTS release is Kirkstone.
It was released in May 2022.
It will be maintained until April 2026, or at least until then.
The previous, actually the first long-term release was DINFO.
It was released four years ago, and it's still maintained.
This is quite convenient for the embedded industry,
because this way you can take a stable release with long-term support
and spend some time developing your project without having an outdated source
by the time when your product is ready to be shipped on the market.
West End and West End are provided as recipes by an open-embedded core.
Here you can see a screenshot of the Git repository of open-embedded core
containing Wayland and West End.
There are different versions of Wayland and West End recipes,
depending on the release of the Yachter project.
Here comes the tricky part when we're speaking about VLC,
because as I mentioned, VLC was relatively recently added
to Wayland and West End, so you have to pick a version of the Yachter project
which provides the required versions of Wayland and West End,
or otherwise you have to backport them to an old version,
which is... well, it opens the can of worms,
so I don't advise you to go this way.
You better migrate your product to a version of the Yachter project
that maintains it.
So, version 5 of the Yachter project is not released yet.
As you've seen on the previous slide, it's going to be released in April,
but as of the moment, it's targeting West End 13,
so maybe it will end up with slightly newer version of West End.
I don't know, but basically you have VLC there,
and you have it in the previous release with West End 12,
however, this is a short-term support release,
so if you are starting working to work on a new product,
I highly recommend you to go to version 5.
Other long-term releases such as Kirk's Room and DILFAL are with older Western versions,
which does not support VLC,
so if you want to take advantage of VLC in West End
and you are based on the Yachter project,
make sure you are using a release that's compatible.
How to bid-bake West End with VLC?
First, the VLC backend is not enabled in West End,
so those of you working with the Yachter project
has to create a bid-bake plan file for West End
to extend the package config variable and to add VLC to it.
This will trigger enabling the backend VLC argument to true
when you are building West End,
and also add need VLC as a runtime dependency.
I've explained that actually as a bid-time dependency,
and also you need to place a plan file.
However, I have created an example reference layer
which provides a demo integration of West End with enabled VLC.
This is the same open-source project that I started a year ago
for my previous talk, which was about RDP.
This year, actually the end of last year,
I just extended it by providing splitting it into two layers,
one layer for RDP, another layer for VLC.
I hope this demo layer will help other people quickly
get to speed with VLC or RDP, whatever they want.
Here, of course, this talks about VLC,
so the examples that we're talking about are VLC.
Behind the core test, what happens is that
when you do these configurations,
when you add VLC package configuration to West End,
you need a bunch of dependencies.
These dependencies are AML and need VLC.
As part of my work at Consulka Group,
I took the effort to upstream these,
and they are part of meta-open embedded.
The West End environment recipes are actually in open embedded core
that in order to build them with VLC,
you have to make sure that you are adding meta-open embedded meta-oE
to get all these two recipes and their dependencies.
The good thing is that actually the community liked it,
and I see that our people started contributing to these recipes here.
There have been several updates.
So these projects are obviously needed by a lot of people in the community
and in the industry, of course.
A little bit of information on how the baking of need VLC
for West End and VLC works.
So for need VLC, in order to work with the whole setup for West End and VLC,
we have to enable encryption with TRS.
This is how it works.
Keep in mind that different versions of West End depend on different versions of need VLC.
Here is a particular example for the things that are right now,
but if you are watching these slides or this recording in two years,
probably it's going to be different versions.
So keep in mind how you can adjust these things.
That's the point of this slide.
And this is how we do BB Append for West End Ine.
Basically, we are creating user West End and password West End.
This is the same user and password that we are going to use to log in remotely
to our targeted device.
In this case, it's a Raspberry Pi file,
but as I mentioned, you can do it with any other better device capable of running West End.
So this is standard VLC to staff.
You just deploy a directory that you need and so on.
So here is how you generate the certificate.
So a bunch of commands.
So obviously the important thing is here that you have to set up the name of the host
to make sure that all or most of the VLC clients will work.
Later on, I'll show you an example of what kind of a problem I had
because initially I didn't set it up properly.
Here I'm targeting Raspberry Pi 5.
This is the host of the device that I'm going to target.
So I'm setting it here.
Finally, of course, I have to copy the certificates to the target device.
In my case, to the Raspberry Pi 5.
And West End Ine is the main configuration file for West End.
We have to configure it there to make sure that the screen sharing is enabled.
So after reloading West End with this configuration,
in my demo setup, West End is started by a system-based service.
So I have to reload the system-based service to start West End.
And once it's started, I have to press Ctrl-Alt plus S in order to enable the screen sharing.
Of course, you know that many embedded devices don't have a keyboard.
So pressing Ctrl-Alt plus S is not an easy task sometimes.
It sounds easy if you're working on a desktop computer,
but sometimes you're not on a desktop computer.
And here comes a really convenient feature.
You can use the automatic startup of VNC just to say that the same is available for RDP.
So no matter if you're using VNC or RDP, you can do the same.
Startup. Here is the configuration.
Startup on startup. Start on startup. True.
And finally, when we have the Raspberry Pi working with Wayland,
West End enabled VNC from another computer in the same network using a VNC client,
which we can connect remotely.
In my case, I'm an open source enthusiast.
I'm trying to use pretty much open source everywhere I can.
So here I'm using a VNC client.
And here's how I started it.
Here's how I blocked in with user West End and password West End.
You saw how I configured that.
Here is the VNC demo. It's not super exciting.
Basically what you see here is Ubuntu 20.2.4.
This is the long-term release of Ubuntu that I'm running on my computer in the same network.
You see the VNAGR window.
And inside the VNAGR window, the VNC client, you see what is my Raspberry Pi 5 showing on the screen.
And a frequently asked question is what is the performance?
What is the frame rate?
I haven't made a lot of tests, but by running some simple demonstrations like on the screenshot here,
it runs with up to 20 frames per second, something like 18, 19, up to 20 frames per second.
I haven't performed any optimizations.
Basically you know that when you have screen sharing enabled,
the performance in terms of frames per second is dropping out.
A little bit of testing.
I have mentioned that I've tested it on several devices.
I made this talk about Raspberry Pi 5 because I was involved in the process of adding Raspberry Pi support
in Meta Raspberry Pi 5 BSP layer.
Now Raspberry Pi is there.
It's in the master branch.
It was recently merged by the maintainers.
And now I've opened the GitHub request to Backport, actually the support for Raspberry Pi to Kirkstone in Meta Raspberry Pi.
However, in terms of VNC, I have tested it with appropriate Western versions on other devices,
Raspberry Pi 4, RockPi 4, this is with a RockChip system on a chip,
Toradex version, IMX 8M plus system on a module.
So the tricky part about NXP, no matter if you're using IMX 8, 7 or 6,
is that you have to switch to the EtnaViv open source drivers
because otherwise with the Vivante proprietary drivers,
you have to use a fork of Western called Western IMX.
And Western IMX, it's always a version behind.
The last time I checked, which was a couple of days ago,
the version that was available was 11.
So we don't have in Western IMX the VNC back end.
So if you are running into this situation, consider switching to EtnaViv.
So we're wrapping up the talk with some conclusions.
VNC is a pixel-based graphical protocol.
It was added in Western 12.
It works on any device that supports the appropriate Western version.
It supports a TOS encryption.
And I have created this layer, Meta Western Remote Desktop,
which is available in GitHub, which allows you to build quickly a demo with RDP or VNC.
The slides are available on the Fosn.org website.
Thank you very much for your attention.
I think we have a couple of minutes for some questions.
Hi, thanks for the talk.
I had some issues with RDP as well.
I think it's when you mention performance,
I think it's to do with the screen share plugin and the full screen plugin.
It pins the single rendering thread of Western.
So when you have RDP and stuff enabled, even on a more powerful computer,
it does affect the presentation locally of the graphics.
Do you know if there's any work being done to improve that architecture,
maybe using shared buffers?
That's a very good question.
Unfortunately, I cannot provide a good answer to this
because as I mentioned, I'm not a Wayland or Western engineer.
I don't have upstream contributions there.
In terms of Wayland and Western,
I'm integrating it in the Yocto Project and Open Embedded.
So you probably have to ask in the mailing list
to compare the particular backhands available for Western.
Yeah, thank you very much for the talk.
This is a bit off topic and I'll keep it quick,
but on the screen sharing idea,
my boy keeps getting me to use this Parsec stuff,
which I think is encoding screens as 8.264 movies, basically.
I like the idea of leveraging GPU to the screen sharing.
Have you looked at that at all?
Is there anything you can comment on about that?
No, sorry. I'm not familiar with it,
but I can provide a comment. I'll have a look at it.
Okay, thank you very much. That was great.
All right. Thank you very much.
If you have questions, I'll be around.