Thanks. Welcome to my talk. So my first time at FOSDEMM, so questions, twists. So I will
do a summary of where we are now about Qualcomm SOC supporting in Mainline because I think
it's time to do a point. So I'm part of the Linao Qualcomm Longing Team. I joined one
year and a half ago. So my many daily work is actually platform Qualcomm support. I'm
maintainer with Calib, the commentator of UBoot Best Sports and I bring new platforms upstream
in Linux. I also maintain and develop other piece of Linux, namely the MLG SOCs,
DLM bridge, DLM panels. And I'm working only on upstream Linux and UBoot for the last few years.
And so I have a lot of patches upstream in UBoot and Linux. So I'm part of Linao. Linao has been
founded to actually enable and make Linux and any software on ARM work better. So we're basically
helping vendors and product makers actually make better products to work on ARM. So we have
plenty of services to help the whole stack of software running better on ARM. And open source
is at the heart of Linao. We mainly work on open source software mainly. So Qualcomm joined
Linao 10 years ago because they wanted to have better open source support at the time,
which was minimal. So they joined to support Linux, but it quickly collaborated plenty of other
places and so far so good. Even happy Linao and Qualcomm is happy with the situation.
So in the last 10 years, Linao and Qualcomm pushed a lot of really huge features for ARM
ecosystem in Linux, namely the power framework that the energy hour schedule really changed the way
Linux schedules correctly on cores. With help and Qualcomm participated in the standard and
software structure on our servers, the Dragon boards are the reference today to test Android,
ASP for example. We have called Linao, which is the principal kit storage for Qualcomm and for
Linao engineers. And namely for the last three years, we were pushing the flagship mobile platforms.
So this year, the last three months, I pushed the standard on Gen3 upstream and it was 98%
two months after the announcement, which is pretty cool. So the agenda, where we came from 10 years
ago, where we are now and the two supported devices, a demo and what's remaining. So we were 10 years
ago. So 10 years ago, Qualcomm and vendors using Qualcomm SOS ships with kernels with more like
three million change. So it was basically a separate kernel in a kernel. So this was a problem, but
it's a hard to solve problem. How do you integrate, how do you upstream so much change in main
Linux? That's why Linao started the learning team to fix this. And this is a graph I made to show
on the last years, the last 10 years, how Qualcomm managed their downstream kernels. So initially,
they used the long term kernel very long time and they kept accumulating new SOS support over time.
So each time frame and for the last four years, the company changed the strategy and they stopped
adding new code and simply changing existing code. I think the reason is first Android strategy with
GKI and because the main line Linux has enough support and has the principal architecture missing
over time. So this is what I posted nine years ago, eight years ago, which was true. I mean, it was
mostly nonexistent. It was the only SOS inverter that was not upstreaming almost nothing. So hopefully
it changed. So Linao worked on Qualcomm specific features in the last 10 years. So the biggest
feature was the remote proc to handle DSP because before Qualcomm had a complex custom solution to
speak to DSPs, which was like two million line of code only to speak to DSPs. And the biggest work
we did was to implement it correctly upstream. So we have now a fully integrated way to speak to DSPs
and it works really, really well. The other big feature of Qualcomm SOS is Interconnect because
the Qualcomm SOS are very complex and you can fine tune any data path in the SOS and you can
change the bandwidth. You can change the performance of any data path. So it was a huge feature. It
took a very long time to upstream it. All the Venus video decoder was complex because it needed
the other support before. The DSP audio support also needed the proper DSP handling. The DM driver
is a huge beast because the graphics display engine is really complex and supports a lot of features.
Lately the sound wire support was upstreamed for Qualcomm and other platforms and we worked on
plenty of very time consuming subjects but tiny. But all of these are needed to actually boot a platform.
So this is a graph of the upstream contribution. You can see it was quite a blow but all these
features are really complex to upstream because they are either Qualcomm specific or very complex but
it doesn't fit in any framework. So it took like seven or eight years to actually push the base
features to actually be able to boot a high-end device. And the last four years because we had a
complex support for all the small and very important features we are able to finally boot
high-end and commercial forms on it. So we had a lot of contribution from
Linao, Qualcomm and also the community. This explains a huge peak in the last two years.
So this is a graph of the supported board of the time. So 10 years ago we had only two boards
in 2D keyboards and now we have a lot of 300 boards which is huge. And most of them are community boards
and non-reference of the base boards. These are the new supported boards of our time. So for each
release the number of supported boards were added. So you can see in the last 10 release
there's a huge amount of new boards added which is great and the community actually helps a lot in
this case. So for the boards, supported boards, like Caleb says the historical dragon boards were the
first really publicly available boards in the SBC form factor and they really helped starting the
mainline development. And while they were like low-end SOCs, we supported a lot of features.
Those support camera and very high-end features so it helped develop the baseline support to
actually enable high-end SOCs at the end. So like Caleb said, these are the robotic boards.
They are quite expensive and it's the current Qualcomm offering in the IoT world
and they aim to support them fully upstream and
which is quite, it's quite each board as a mid-end and low-end. So it's quite diverse and it helps
supporting all the new features. So there is commercial phones which are running very, very well.
So you won't expect all the features for daily usage. So you don't have haptics, you don't have
camera but they work fine and you can boot and actually use it with Wi-Fi, Bluetooth and storage.
You can have a few tablet convertibles running on Linux, mainline like the Lenovo Yiga 640
and these are the Qualcomm high-end reference devices. So those are the devices we use
daily use to upstream high-end platforms. So this one is a one-year-old platform,
this one is a two-year-old platform and this one is actually running this presentation.
And those are the specific Qualcomm reference devices with test points used by Qualcomm engineers to
actually upstream develop Android and we upstream mainline Linux support with them.
So as I said, I was upstreaming the Gen3 support, the latest Qualcomm high-end SOC which the
Samsung phones were announced two weeks ago and in 6.1 RC1 that was announced like two days before
announcing the Samsung phones, we had already a display, UFS, PCI, USB, thermal, CPU fray,
QSPC, suspend-resume and crypto-working on mainline Linux, check out Linux master and it was working,
it works. And in the meantime, we developed audio, display portal mode, DSP, full DSP,
modem, compute and audio, USB PD and charger and GPU is the last remaining one and I won't talk about it.
So the flagship device you could use today is the Lenovo X14S. It's actually the best platform
to use Qualcomm devices. It's really powerful and you can use it daily. My colleagues are actually
developing mainline Linux on this platform. It supports, my colleague can use it about eight
hours time working and you have almost all everything supported. So this is an example what is
supported. You have a JPEU working storage keyboard, thermal, USB, suspend-resume, audio and you can
boot over EFI. So you can, but obviously they're still working process like every software stuff.
So the most important is the camera. Camera doesn't work. It's complex due to the sensor
putting raw data and Qualcomm not wanting to upstream the stuff. So it's in working progress.
We have something working. It's not public. We are working on it. There's plenty of other small
features are missing like the embedded controller, the power measurement. Power measurement is
infinite. It will never be perfect. So we're gaining amps every release. So it's a constant work.
There's always some small, wiffy and Bluetooth issues. Audio needs active speaker protection.
This is a big, modern feature. All the new, modern audio needs active speaker protection because
it's not no more included in the codecs. And some stuff are still missing like the fingerprint
or VDOC shale action. But we aim to support all this in the short frame. So today, if you want to test
Linux, many Linux on the expertise, you can use Fedora, Ambian, Nubuntu or Debian. We about changing
anything. It will install directly and boot and you could use it daily. So this is a great, great
advancement. So demo time. I mean no need for a demo because I'm running it. I'm running the
procession on a Qualcomm device. So yeah, for example, this is 8550. You can play a video,
for example. It works fine. You can switch. I'm still in full screen.
So you can see everything is fine. The video is still running.
So the GPU works very fine.
Up.
Demo effect.
Okay. So...
So to show it's really usable. You have Wi-Fi and Bluetooth working in GPU and this platform
is one year old. But I got hardware like two weeks ago. So it was great. And the support for the board
is actually on the... It's made by the Qualcomm ARM maintainer. So it should be part for 6.9.
So globally, what's remaining to support, properly support the Qualcomm SOCs, power optimization.
It's a long term, nearly infinite work because the Qualcomm is complex and we still need to gain
every time. So performance. Performance, like I said, each data path can be optimized.
And it's also a long, long journey to support power and performance and manipulation.
There are still missing some advanced graphic features, mainly for non-phones and non-laptops,
like HDR and multi-plane and so on. Video with decoding accelerator is working
progress. We're working with Qualcomm on it. Camera support is a big feature. Audio support,
we still need to support DisplayPort audio. To support audio over the HDMI or the DisplayPort.
Speaker protection, the sensor hub for the phones, feedback and the vibrator and the new
platform because each year we have between two and three platforms to support either in computers
or phones or IoT. Otherwise, it's keeping us working a lot. So we need help of the community
because we need testing and we need to support more devices. Thanks to the community, we have the
largest ARM 64 change in the last years. Every single release, we have a top change
because it's really actively changing. We are really supporting mainstream devices, the phones,
laptops, modems, accessories, converters. And we are working on new books. Qualcomm is
porting new devices. It will simplify installing new distributions. And if you want to know the
status of each SOC, you can go to this address in our GitHub IOMSM. It will give you a nice
overview of the support. So like the last line is the standard on Gen 3. So all the yellow lines
will be green in four weeks now. So for example, it's really kind of cool. We simply describe
each feature with and it generates automatically a website. So it's really cool.
So thank you for listening. I was happy to present the state of Qualcomm SST port and demo it
in live. And it works fine. So no demo effect. Thank you very much.
Very nice. Does anyone have any questions here?
Yeah, hi. When can we expect Qualcomm to start upstream in support for the Linux that runs
on the modems? On the modems? I have no idea. I'm sorry. Another question?
Thank you. The question is actually first, is Linao or Qualcomm considering doing any
upstreaming for legacy platforms? For what? For legacy platforms, for early edge upsets.
We do it daily. Okay. So this is also happening? Okay. Yeah, we continue adding features for all
platforms daily and the community helps a lot and we are testing it. And in fact, Qualcomm is
pretty consistent in the firmware interfaces and APIs and registers. So we in fact support all
devices quite constantly. And then the other thing you mentioned,
specifically on camera, there's a lot of work on Android. Second, you have a lot of out of
three drivers. Instead of a platform, Qualcomm, they actually get everything
supported directly in upstream Linux. I hope so. And the question here, one second.
Hello. Very nice talk. Any plans for Spectra ISP?
So yeah, it's the same question. I don't know. It's not in our hands.
Another one? Okay. I'll pass the mic.
You talked about many distros already working. If we had, for example, a root FS from another
distro, the boot loader situation, is it the same as in mobile phones and their SoCs? Or can we just
expect to boot from UEFI or similar? So for the laptop, they have a functional UEFI shipped with
the laptop. So there's no need for you boot. I think it's not perfect, but it works fine. So you
can directly install Fedora in UEFI on the laptop when you open it. So it works.
Thank you. So you mentioned something about video decoding. How will exactly that work?
Will there be a VA API driver or will it use something else?
Today, there is already a Venus v4l2 driver for the old platforms. And we are working to support
the new platforms using v4l2. So Qualcomm wants to push support for the platform. So we need to
find a way to merge it and make it more prettier. But yeah, v4l2. Okay. Thanks.
Another question, anyone? Yeah, yeah.
Hey, thanks for the talk. I had a question about availability of certain documents required to
write a lot of the drivers. Is Qualcomm making those documents available to the public?
So no, it's not the industry. They don't want to document the hardware publicly.
So for regular people that want to help, it would be like reverse engineering or?
Yeah, code. Okay. I mean, cool. I've implemented all the MLG support using code only. Almost no
documentation. So it's hard. So we need documentation for more complex features. But most of the
future, we use code, even us. Because documentation, you have registers, but you don't have the state
machine. You don't have the behavior of the hardware. So. Okay, we could fit in another
question if there is any. Otherwise, yeah. Okay. Yeah. I'd actually like to continue the question
that the teacher's raised. So how is it working then? So you signed an NDA with Qualcomm, get the
doc. Docs can write the code, but you're not allowed to document it. Yeah, speak about it. That's how
it works. Yeah. Yeah. Gotcha. Please give another round of applause for our speaker.
And it was really all running from this device here, the board. No laptop. Yeah.