[00:00.000 --> 00:11.560] as we did, so we try to cover as many, it's unfortunate, there's so many topics to cover [00:11.560 --> 00:16.200] when you do mobile, for Linux mobile phone stuff. [00:16.200 --> 00:21.760] Okay, welcome Luker, we have seen the first pair of phones being used as a presenting [00:21.760 --> 00:28.400] device this morning, so now we are going to learn how to put the kernel on it. [00:28.400 --> 00:40.040] Thank you, yeah so kind of very quickly who am I, my name is Luker Weiss, I've been learning [00:40.040 --> 00:45.720] phone since like 2017, I'm a post-marketers core team member and also my day job is Android [00:45.720 --> 00:51.520] platform engineer at Fairphone, kind of about the background of how the whole situation, [00:51.520 --> 00:57.040] so I mean Qualcomm has a lot of lot of SOCs like system on the chip, there's quite a lot [00:57.040 --> 01:03.480] of actually already supported, so you see all these wonderful numbers here, the ones [01:03.480 --> 01:08.320] launched since 2018, so like in the last four years, and they are already supported in Mainline [01:08.320 --> 01:12.800] as in they have a DTSI file and you can build something on top of this and it's booting. [01:12.800 --> 01:17.360] But of course there's also many, many others that aren't supported, especially mid-end [01:17.360 --> 01:22.240] ones like the high-end ones are normally very quickly supported by Linaro, so like for example [01:22.240 --> 01:29.000] the SM8550 is the newest one, it was basically supported in day one or the same with the [01:29.000 --> 01:36.600] SM8450, but yeah the other ones are not, but you can of course do it yourself. [01:36.600 --> 01:44.200] So the device, Fairphone 4, used the Snapdragon 750G, the SM725, yeah launched like a year [01:44.200 --> 01:52.800] and four months ago, running the 4.19 kernel, so which is already, I mean we had 6.2 nearly, [01:52.800 --> 01:59.920] and yeah like what I have so far working on the 4.9, on the 6.1 or 6.2 kernel is like [01:59.920 --> 02:06.320] all the basics that you can see here, USB including nearly the USB role switching, so [02:06.320 --> 02:10.240] you can actually plug in for example a keyboard into the device and not just use it like as [02:10.240 --> 02:16.680] a gadget, and internal storage in the SD card, so the UFS and other things. [02:16.680 --> 02:21.760] Display with backlight control which is separate components, touchscreen GPU, Wi-Fi, the remote [02:21.760 --> 02:28.600] prox which is like separate cores on the SoC, they are actually all booting, but at least [02:28.600 --> 02:32.960] for the modem one I'm actually not really able to communicate with it. [02:32.960 --> 02:36.880] Mobile data could also in theory work, so the Linux driver initializes it actually gets [02:36.880 --> 02:42.320] the remote prox up, it already does some initialization things there, but it's not really testable [02:42.320 --> 02:48.040] without actually having the modem up, so it's kind of untested, it's upstream already. [02:48.040 --> 02:53.000] Vibration motor, the flash and torch LED which is actually was upstreamed recently or is [02:53.000 --> 02:57.840] in the process of getting upstreamed, the camera I2C bus, so I can actually talk with [02:57.840 --> 03:02.840] like the I2C set and I2C get commands, I can talk with the camera, and like get the [03:02.840 --> 03:07.720] chip ID, so that works, but yeah, not really much more useful with the camera yet. [03:07.720 --> 03:12.720] And also lots of other plumbing which includes like, yeah, of course all the I2C bus, interconnects, [03:12.720 --> 03:19.800] like the bus scaling, cache scaling, and yeah, bunch of other stuff that's useful. [03:19.800 --> 03:23.800] So kind of what isn't working yet after this one year and four months that I've been sort [03:23.800 --> 03:30.840] of working on it, I have some parts that are actually sort of working, it's like the speaker, [03:30.840 --> 03:36.480] I do have, I can get sound out of the speaker, it is super quiet for some reason, I don't [03:36.480 --> 03:41.800] know why, and also for some of the audio formats it actually doesn't play at all for some reason, [03:41.800 --> 03:43.520] I know. [03:43.520 --> 03:47.200] One of the problems with the speakers also like not very many phones in Mainland actually, [03:47.200 --> 03:52.880] like Qualcomm phones have audio working, so it is still kind of a new area where this, [03:52.880 --> 03:56.440] yeah, where a lot of things are kind of unknown. [03:56.440 --> 04:04.080] In Bluetooth I have based on some patch set that I found, you can get it on the Bluetooth, [04:04.080 --> 04:08.720] you can make the phone discoverable, so you can see it on other devices, you can actually [04:08.720 --> 04:13.360] connect other devices to it, but when you try to like on Bluetooth CT area, do the scan [04:13.360 --> 04:16.360] on command, like it just fails. [04:16.360 --> 04:20.840] So which is a bit weird, I don't know why, probably need to spend some more time on it. [04:20.840 --> 04:24.480] And also like of course all the other parts that don't work, so the modem as I said before, [04:24.480 --> 04:30.280] I can talk with the modem via QMI, so the Qualcomm protocol, but when I say please enable [04:30.280 --> 04:36.040] yourself, it says nope, and doesn't say anything else, so it's kind of difficult. [04:36.040 --> 04:41.120] The microphones which are like also kind of, it's a different part of the audio stack. [04:41.120 --> 04:46.200] The camera sub system which is used for receiving image data from the sensors, it's not working, [04:46.200 --> 04:52.440] including the time of light sensors, like for autofocus it can be used. [04:52.440 --> 04:57.880] And the video encoding, decoding hardware which is for, so you can play MP4 for example, [04:57.880 --> 05:04.000] without actually doing the decoding on the CPU, NFC, the fuel gauge, so for battery percentage [05:04.000 --> 05:09.760] and the charging driver, they are not working, they are actually, I was able to port the one [05:09.760 --> 05:15.200] from the 419 kernel to mainline, like just import it. [05:15.200 --> 05:19.760] It does sort of work the fuel gauge driver, but apparently there's some weird, really [05:19.760 --> 05:24.200] weird things going on on Android where like a user space component writes something to [05:24.200 --> 05:29.880] the kernel driver, and without this like nothing works basically, it's super weird. [05:29.880 --> 05:35.640] And also this part of the USB-C, what Alfred already demonstrated before, like it works [05:35.640 --> 05:39.640] in the hardware, but it doesn't work with mainline, just with the downstream kernel. [05:39.640 --> 05:44.880] So kind of what is the things that you need to have when you're trying to get a new SoC [05:44.880 --> 05:50.320] network, it's like one of the first steps is kind of also figuring out how can you make [05:50.320 --> 05:53.720] this boot loader boot what you want to boot. [05:53.720 --> 06:00.720] Because in the Android case, like Google requires some special things going on, and also the [06:00.720 --> 06:06.080] way that many SoC manufacturers implement it is kind of sometimes working, it is working [06:06.080 --> 06:09.560] for Android, and that's good enough for them. [06:09.560 --> 06:15.320] And for example, the DTPO partition, which is device tree blob overlay, on some devices [06:15.320 --> 06:20.200] you can just fast-boot erase it, and then it doesn't try to apply some overlay for the [06:20.200 --> 06:24.400] old kernel on top of the new kernel, which doesn't work and doesn't make any sense. [06:24.400 --> 06:29.560] On some devices just crashes and burns, and yeah, it is not fun. [06:29.560 --> 06:35.800] So mostly you can do this, and there's also on new devices with GKI, the generic kernel [06:35.800 --> 06:40.320] image from Google, there's also the vendor boot partition, I actually have no idea what [06:40.320 --> 06:47.040] this one does, and how you need to wipe it to be able to do something. [06:47.040 --> 06:52.560] The serial console is actually quite useful if you can have access to it in the boot loader, [06:52.560 --> 06:54.240] if it doesn't boot. [06:54.240 --> 06:59.880] Like if you cannot even get Linux booting, normally on a serial console it will say what [06:59.880 --> 07:03.000] it's doing and why it's not doing the things that you want to do. [07:03.000 --> 07:07.960] It's like, yeah, on the Fairphone 4, on the new SoC, I got the first boot actually after [07:07.960 --> 07:14.720] some hours of working on it, which contains the early console, it's just basically already [07:14.720 --> 07:20.440] set up area where Linux driver can write to it, and you get serial output, and also the [07:20.440 --> 07:25.760] display via simple frame buffer, which is actually now way more easy than getting USB [07:25.760 --> 07:30.360] up or getting actually proper serial console up, so it's super nice. [07:30.360 --> 07:38.720] Simple frame buffers where the boot loader already sets up the display hardware correctly, [07:38.720 --> 07:44.040] so Linux actually just has to write to some memory area, the bytes for the pixels, and [07:44.040 --> 07:46.520] it will just magically appear on the screen. [07:46.520 --> 07:54.040] It is very nice, very useful, and yeah, the first boot was in like 180 lines of the DTSI [07:54.040 --> 07:59.600] for the SoC and 40 lines for the device, so yeah, total 220, and no single driver change [07:59.600 --> 08:04.600] was necessary for getting a completely new SoC booting anything, basically. [08:04.600 --> 08:12.840] Yeah, I was basically just following what Iskren wrote on his blog, mainline.dev, super nice, [08:12.840 --> 08:18.000] it really contains useful steps for the very, very first things that you need to do. [08:18.000 --> 08:24.000] Yeah, so if you want to go a bit further, you very quickly start to need the clock driver, [08:24.000 --> 08:29.960] which is GCC, global clock controller, driver, now you can basically just take whatever Qualcomm [08:29.960 --> 08:36.040] gives you, for example, for the 419 kernel, copy it over, modify a few small things, but [08:36.040 --> 08:37.040] then it works. [08:37.040 --> 08:43.680] You also, at least for the 419 kernel, also these power domains, which is like, is some [08:43.680 --> 08:50.560] concept in Linux, it's also called GDSCs, you need to, they were a bit differently [08:50.560 --> 08:55.440] implemented, not in the GCC driver, but you should put them in the GCC driver for mainline. [08:55.440 --> 09:01.120] Now there are more clocks with the RPM edge, also like various other bits you should add [09:01.120 --> 09:06.640] to the DTS, because otherwise it just won't, like random things won't work, which are dependencies [09:06.640 --> 09:11.520] that are not really expressed in the device tree, but the drivers still need, for example, [09:11.520 --> 09:15.520] access to the S-MEM for like doing various things. [09:15.520 --> 09:19.640] These definitions are basically all the same in downstream, so you can also mostly copy [09:19.640 --> 09:20.640] them over. [09:20.640 --> 09:24.920] Don't blindly copy them over, because it will be slightly different, but you can definitely [09:24.920 --> 09:27.720] get good inspiration of what you need to do. [09:27.720 --> 09:32.920] USB is, of course, kind of the next step, because just staring at the tiny text on the [09:32.920 --> 09:37.120] screen is not very good debugging, and you also don't have any input. [09:37.120 --> 09:42.320] You can do surprisingly much with simple framework, but at some point you, of course, want USB, [09:42.320 --> 09:47.680] at some point also a pin control driver, which is for the pin multiplexing. [09:47.680 --> 09:51.560] This really only starts getting used for once you get to more advanced components, let's [09:51.560 --> 09:57.440] say, for like I2C and other things, and also regulators are important at some point. [09:57.440 --> 10:02.800] I think that's actually, I don't know if you already need this for USB or not, but these [10:02.800 --> 10:06.480] are kind of the basic components that you need, and then you can start building actually [10:06.480 --> 10:11.640] enabling various components that you find, like the flash driver, the vibration motor, [10:11.640 --> 10:17.360] and things to talk I2C and things to talk other protocols. [10:17.360 --> 10:22.720] Of course, lots of things that can go wrong, the IMU is especially on new Qualcomm chips, [10:22.720 --> 10:23.720] it's kind of annoying. [10:23.720 --> 10:28.000] I mean, it's less annoying than old ones, but it's still annoying, because a lot of [10:28.000 --> 10:35.280] things, like some things that you do, or yeah, let's talk about IMU directly first, [10:35.280 --> 10:39.240] so like what's a bit different between downstream kernel and also mainline, is the bootload [10:39.240 --> 10:45.920] already initializes something in this memory seguration, or like SMMU is also called. [10:45.920 --> 10:50.480] I initialized some things for the bootloader to use, for example, for the internal storage, [10:50.480 --> 10:53.680] it already initialized it. [10:53.680 --> 10:59.160] This normally gets on downstream kernel, it just continues using those and adds some [10:59.160 --> 11:00.480] ones on top. [11:00.480 --> 11:06.360] On mainline, they actually get wiped completely, and they need to be reset up by Linux, which [11:06.360 --> 11:12.600] causes some problems if the downstream kernel doesn't express like which IMU to use, for [11:12.600 --> 11:19.280] example, for UFS, it's a very good example where it is bad. [11:19.280 --> 11:26.520] So you kind of need to find out there is a patch, there you can actually dump the mappings, [11:26.520 --> 11:29.680] and yeah, you can use this to figure out what it is. [11:29.680 --> 11:35.040] Also, the device really likes to reboot when anything is not really right. [11:35.040 --> 11:38.000] If you access some register in the clock, and some clock isn't on that it requires, [11:38.000 --> 11:39.000] it just reboots. [11:39.000 --> 11:41.560] It doesn't give you a kernel panic, it just reboots. [11:41.560 --> 11:46.680] If you're writing to a wrong register, it reboots, the IMU is defined wrong, it just reboots. [11:46.680 --> 11:51.000] Actually a thing for the IMU is that it sometimes gives you a message of why or at least that [11:51.000 --> 11:55.520] something isn't correct, but yeah. [11:55.520 --> 12:00.280] For printing, what I've actually used sometimes is just printing the current line where this [12:00.280 --> 12:05.040] in the driver, like sprinkling this everywhere, adding a sleep of like half a second, and [12:05.040 --> 12:09.960] then seeing like, oh, this is the last line that I was seeing, so it's probably messing [12:09.960 --> 12:10.960] up there. [12:10.960 --> 12:16.160] And maybe also increasing the sleeps, because sometimes the flushing doesn't happen, like [12:16.160 --> 12:19.640] printing it on the screen actually is a bit slow. [12:19.640 --> 12:26.320] Also, for like once you have like more USB up, you can actually also build various drivers [12:26.320 --> 12:33.040] as modules, and this way actually, yeah, it's not built in, and if it's built in, it loads [12:33.040 --> 12:40.280] like it's, I don't know, it like kernel locksack and 0.5, which is quite early, and if it then [12:40.280 --> 12:43.680] crashes immediately, you don't really have any time to be the debug, but if you build [12:43.680 --> 12:47.760] this module, you can load it later and actually have something set up already. [12:47.760 --> 12:55.440] Yeah, like what is important to do if you work on this, actually if you have anything [12:55.440 --> 13:02.000] working, if you have something working progress, just commit this into your repository already [13:02.000 --> 13:06.960] to have a reference point to go back to, because sometimes one single line change will fix [13:06.960 --> 13:13.160] everything or break everything, and like you can, your first commit doesn't have to be [13:13.160 --> 13:16.120] perfect, obviously. [13:16.120 --> 13:20.840] But also don't let this sketch branch that you have lying around want to have something [13:20.840 --> 13:24.560] working, don't let it sit around in your local repository on your GitHub fork or in your [13:24.560 --> 13:30.280] GitHub repository wherever, forever, and don't upstream it, because then it will just drop [13:30.280 --> 13:33.560] there and nobody will know that it's there, and they will probably, like this next person [13:33.560 --> 13:37.360] has to do exactly the same thing again, even though you have already got it working. [13:37.360 --> 13:41.480] So like already starts upstreaming your patches early, like if you have simple framework for [13:41.480 --> 13:43.800] booting on the device, upstream it. [13:43.800 --> 13:48.120] It would be very nice, because there's also a better overview of which SSCs have already [13:48.120 --> 13:52.840] been worked on, and it's very nice. [13:52.840 --> 13:56.000] Of course like when you upstream it, you also have to do some extra things, for example, [13:56.000 --> 14:00.800] adding the new compatible strings that are used in the device tree added to the documentation, [14:00.800 --> 14:06.120] and do some things there, but it's normally, it is, yeah, some extra work, but it's really [14:06.120 --> 14:08.000] not too bad. [14:08.000 --> 14:13.600] And also patches just because of how Linux development works, just takes some time to [14:13.600 --> 14:15.800] get upstream. [14:15.800 --> 14:19.920] So like two months later, if you go to, if you re-basement new version, your patches [14:19.920 --> 14:24.400] already there, so you can build on top, and don't have like 100 patches lying in your [14:24.400 --> 14:25.400] own tree. [14:25.400 --> 14:30.760] Or it's like get send email is not difficult, if there's a wonderful guide, get send email.io [14:30.760 --> 14:32.880] from the source developers. [14:32.880 --> 14:39.040] It explains it super nice, once you have configured once, it just works, yeah. [14:39.040 --> 14:42.040] Thanks for listening. [14:42.040 --> 15:00.920] We basically have one minute for questions. [15:00.920 --> 15:04.800] When you get GPU working, you should also actually get the display hardware working [15:04.800 --> 15:05.800] properly. [15:05.800 --> 15:12.840] But yeah, this was fortunately done for, for this SOC was done by Konrad, who is, who [15:12.840 --> 15:19.120] knows a lot there, like he got the display hardware completely up in the GPU also. [15:19.120 --> 15:21.920] This is used for actually, because simple framebuff, you cannot turn off the screen, [15:21.920 --> 15:26.880] you cannot basically do anything except just write pics of some memory, write data to, [15:26.880 --> 15:30.840] or write bytes to memory area, and that's it. [15:30.840 --> 15:34.680] Yeah, so you actually need to get the display hardware also up, but then you also get, can [15:34.680 --> 15:36.040] get the GPU up. [15:36.040 --> 15:38.760] And yeah, this one works really well in mainline. [15:38.760 --> 15:43.440] Like I run performance benchmark on it, it's actually, not too bad, it's actually relatively [15:43.440 --> 15:45.440] close to the downstream version. [15:45.440 --> 15:46.440] Yeah. [15:46.440 --> 15:47.440] I've contributed to the SDM 625. [15:47.440 --> 15:48.440] Mm-hmm. [15:48.440 --> 15:58.440] Like, you know, how to up the screen and manage the complexity of partners, generated partners? [15:58.440 --> 16:05.480] Are there panels? Yeah, the panel drivers are still, I think in general, a question of [16:05.480 --> 16:09.440] like how they should be handled upstream, because in theory, I think the panel drivers [16:09.440 --> 16:14.080] are not really generic, I mean, they are not generic. [16:14.080 --> 16:19.120] But in theory, they are relevant to the, to the display controller, and not actually [16:19.120 --> 16:25.360] a panel itself, which is two separate parts, but like, without having actually access to [16:25.360 --> 16:28.840] like all the documentation that are like internal to the company, you won't find out [16:28.840 --> 16:31.320] which, which driver this actually is. [16:31.320 --> 16:36.840] And currently, let them sit around in your tree, they are, I mean, most of them, you [16:36.840 --> 16:42.720] can also just generate from a downstream DTB, and it works, so good enough for, for now. [16:42.720 --> 16:48.080] At some point, we probably figured out, but the MSM8916 people also, they, they have like [16:48.080 --> 16:50.080] already like 20 or 30 panels there. [16:50.080 --> 17:04.400] I think trust is always running, so like the boot loader, which is like, it is a signed [17:04.400 --> 17:10.560] binary, and you cannot really replace it without having access to the, to the signing keys. [17:10.560 --> 17:15.200] It is running, I think, and it also, I think this is the thing that kills your, that kills [17:15.200 --> 17:17.760] the phone, like when you're doing something wrong. [17:17.760 --> 17:21.440] I don't know, you cannot get rid of it, I, you can probably somewhat communicate with [17:21.440 --> 17:22.440] it. [17:22.440 --> 17:25.760] I know that normally the fingerprint sensor is handled via Trustone. [17:25.760 --> 17:30.760] I was like, you actually talk to Trustone for the fingerprint, but I actually don't [17:30.760 --> 17:31.760] know how this works. [17:31.760 --> 17:32.760] Okay. [17:32.760 --> 17:33.760] Thank you very much. [17:33.760 --> 17:34.760] Thank you. [17:34.760 --> 17:48.760] Thank you very much.