Okay, thank you for showing up or for staying for my talk.
Scratching an itch by patching KMS con or put another words how knowing open GL can actually
to systems program, believe it or not.
Before I continue, I want to set the stage.
I see this talk as a partly being an experience report by somebody who did a drive by contribution
to KMS con.
Well, yeah, that's me.
And also to some degree, call for help with testing and maybe poking upstream to review
my pull requests.
And on top of all that, the whole thing is running on KMS con, the patch version.
So it's now a graphical output and it's basically one big live demo.
So fingers crossed nothing is wrong.
Here, there's a quick overview of what I'm going to be talking about the next 20, 25
minutes.
And so what is KMS con?
For those who don't know, it is a systems console replacement meant to completely live
in user space.
It has a nice plugin architecture.
It uses these plug-in architectures for actually implementing all its major features as shared
library objects.
It is unicode capable, meaning it has seroic support, Korean, Farsi, no emojis.
But if you are bored and would like to contribute, I guess there might be five persons on this
planet who would really welcome that.
Two plugins I want to point out.
One is the free type font support, which means you have vector-based fonts, open type, free
type.
And the other one, the most important one to me, is the rendering back end being based
on OpenGLES, which actually allowed the stuff to be done to it that I wanted it to do to
scratch my itch.
It's kind of old.
It was started 2012 by David Herrmann.
Wanted to basically replace the in-curnal system console that's hiding behind config
on a score YT.
And it has a side of project where David extracted the whole terminal handling, sorry, the terminal
handling stuff in its separate library.
It's basically this Lipt.
TSM is a toolkit, a neutral version of the Lipt.
VTE that you might be knowing from GDK or GNOME.
It was developed by David until about 2014, and then he magically probably vanished somewhere
in the depths of Redhead.
He didn't update his range anymore.
And then from that point on, it kind of organically grew MnT to some other place, which is there
at GitHub, AETF.
That's the current most active fork of KMS-Con.
So who am I?
Usually when I do these talks, I just say, hey, I'm Xlo.
I love computer graphics.
Considering the circumstances, the venue, a little bit more context is in order, I think.
I used to be a bit more visible and active in the open source community.
Back in the day, I write Kyro Clock, Kyro Dock.
I worked at Canonical.
I implemented notify OSD.
I always mess around with computer graphics.
And you can contact me or ask questions if you don't get them in today.
Later by email, the master don't thing on Twitter.
And when I'm not hacking on computer graphic stuff, I like to race my motorcycles and skateboard.
Enough about me.
Motivation.
Why I'm standing here.
Why did I do what I did to KMS-Con?
I have a kind of unusual monitor setup for a regular user, maybe not so unusual for a developer.
I've got two 24-inch widescreen monitors, turn sideways, next to each other in portrait mode.
That's not much an issue if you're running KDE agnome because they can easily rotate the output.
When you have these moments where you don't want to be distracted by anything, you switch up
every graphical output and just go to the systems console.
That with a normal system console doesn't work very well.
You see that in the picture in the corner.
You sit there like that and after a minute you go like, fuck me.
So I thought I can't be the only person on the planet who has this problem or use case.
There's got to be something that solves my issue.
I looked around.
I couldn't find anything.
So crap.
During my research, I stumbled across KMS-Con.
I said, ah, KMS-Con.
Sounds interesting, intriguing.
It's open source.
Oh, that's good, sir.
It has a pluggable rendering infrastructure and one of them is OpenGL.
I know OpenGL.
Rotating the output by 90 degrees with OpenGL.
That's a piece of cake.
I do that in the evening, maybe in the weekend best.
Famous last words.
It took me maybe one and a half weeks of some evenings.
But that was last year, end of January, start of February.
And I got it working.
I said, oh, awesome.
That's good topic for a lightning talk at FOSTA.
Wrote the folks, I mean, two or three days before February.
Ah, great idea for a lightning talk.
I said, now, sorry, Merkel.
Nice idea that everything's taken next year.
So here we are.
OK, so patch number one, the main edge to scratch.
The OpenGL rendering back end that I touched, I didn't touch the rendering back ends like
PIXMAN because, yeah, it technically would work, but it's not so nice.
It's not so fast.
I extended that with two entries in its V-table for all the methods that this plug-in provides.
And one of them is the GL-Tex rotate that allows, oh, wait.
Oops.
So there.
That allows to recalculate the aspect ratio, depending on the orientation that you want.
And then it also recalculates the amount of columns and rows for the characters to fill the screen.
And it does all that dynamically.
Dynamically because of the third patch that I'll be talking about, but you'll see.
And of course, talk is cheap.
Let's do some more demos.
I switch to another session.
Log in again.
T-Max.
So I'll create two paints and make something happening in these paints.
Oops.
Oops.
Hmm.
Just as a stand-in, so there's stuff happening.
And now I have a pre-configured hotkey that is super plus.
And I'll rotate the output.
It might make sense with this, but imagine your thing is turned around, right?
So clockwise, counterclockwise.
You can, of course, while you're doing that, resize the characters.
Bigger.
Keep rotating stuff.
And make it smaller again.
Oops.
Ah, okay, something.
Okie dokie.
Come on.
Hmm.
So, so much for the output rotation.
That was the main itch to scratch.
Then I thought, well, okay, now I spent the time to learn the code base, its architecture,
how stuff works.
I also really like this GPM tool.
If you maybe know GPM is this general pointer mouse deamon thing with the old console, it
provides a text cursor, which is attached to your mouse.
And then you can just select text and copy and paste stuff when you're really too lazy
to type.
That doesn't work with KMSCon.
I don't know, fuck.
I really like that.
It cannot be that hard, right?
So, okay, read the code again.
And then, since KMSCon tries to keep its dependency over external libraries really, really slow,
low, it implements its own event loop and timer system.
So, you read up that.
It's kind of documented in the code, but not like properly.
But, yeah.
So, I figured it out.
I'm using the kernel input system for mice and basically implemented a new additional
plug-in.
Right now, I still compile it directly into the binary because I'm a bit lazy, but that
needs to be extracted at its own shared library.
So, it works then with mouse track points, the Bluetooth-based Apple Magic mousepad,
maybe also with the first touch point for touch screens, but I don't have that hardware.
So if anybody has that hardware, please download it, report to me.
I'll try to make it work if it doesn't.
Yeah, there's still this busy polling issue.
I didn't hook that up to the event loop and the timer, so that's something I need to clean
up.
And the mouse plug-in is also the one that actually uses the TSM directly because the
current paste is basically implemented in lib TSM.
So, it allows you to select, oh, this is the start of the selection.
There's the end.
Copy it to a clipboard buffer, which TSM handles.
You don't have to deal with that in your own application code.
And then you just paste it wherever the cursor ends up being.
And we can demo that too.
So, let's cut and paste some file names.
Okay, okay, here's the mouse pointer.
So, I can just, ah, I think I found a new bug.
That is interesting.
Usually, that should not happen at that spot, but, yeah, pasting does work.
So, I move it up to the fan above.
The problem is I'm missing some recalculation when I resize the font.
So, usually, if I would scale it down one more, then it should work again.
But that works with regular Latin text.
But, I mean, that's cheap.
Okay, Scon supports Unicode.
So, let's look for some, oh, that's, I think that's Greek.
Yeah, I think that's Greek.
You select the same way, yeah, okay.
Imagine that would be correct.
Correct.
So, I drags there.
I have no idea what type of language that is.
So, yeah, it's Unicode, but it feels like a regular character.
And it just works.
That's the magic of K-Maskon.
I really love it.
I don't know why it's not shipped by default for every distribution.
I don't know.
It's really awesome.
So, that is patch number two.
The third patch is to make it a little bit more consistent with the way stuff works
in a typical decimal environment, a graphical environment.
I know from GNOME, when you have a driver sensor in the system
and you turn the laptop, it recognizes that.
And then it turns, if you allow it to be turned automatically, it turns your display.
And I thought, well, if you have the sensor, well, if the graphical environment, GNOME, KDE,
can read that, then, well, the system console should, too, right?
I mean, be consistent.
So, I sat down and looked around and implemented that.
That was a bit more tricky.
It was not that much work, but it was a little bit tricky.
And I thought to myself, well, since the amount of external dependencies needs to be really low,
I think I'm going to stick with the lowest level I can get along with,
and that is the low-level D-Bus API, which even the D-Bus maintainers discourage you from using.
I used it anyway.
It works.
I cannot show it on this laptop.
I have another laptop with me, which has such a driver sensor.
After the talk, either here or in the hallway, if you want to see that live come by,
run into me, and I'll demo that.
It's really fancy.
It's like extra lead points for us when you show that to your geek friends.
And then, at the end, there will be links to screencasts,
where I show that for anybody who's not here or sees that as a recording in the future.
Or just download the code, compile it, and use it yourself.
So, the last patch, that's basically just messing around.
The last patches kind of make sense.
They improve existing functionality or add new functionality.
Since I'm a computer graphics nut and I like to mess around with stuff,
I thought, now I want to have fun.
The best thing you can do with just a terminal-based text interface usually is, well, ASCII-Art.
And there's this famous demo, and I think everybody knows that, this ASCII-Art stuff.
It's basically a very clever way of doing very, very lower solution,
ray tracing on the CPU, and then using the characters as the, let's say, brightness pixels.
And then this creates the illusion of a 3D object running around.
But with KMSCon, and we have openGL rendering back-end,
surely we can do better these days.
And so I did.
So, I'll give you live GL SL shader backgrounds.
This is a little bit my, I'm also a little bit active in the demo scene.
I like to mess around with some demo events,
and this type of scientist fields rendering is typically the stuff that you do.
So you have now live shader backgrounds.
And this actually, although taken to the next level, also implements
an old initial to-do item from David Herrmann,
because he wanted to have background images.
Okay, it's animated. It's real time. Good.
Okay, now, after all this, to upstream or not to upstream?
Of course, to upstream, right?
I mean, be a good open-source citizen.
Upstream your work. You found something that you like.
You improved a little bit to a scratcher edge.
You put in the work to make it as clean as possible,
fit with the coding style and the system software design and everything.
Okay, so initially I brought my patches against the branch from David.
Then I was contacted by a Debian maintainer, developer, Victor Vesterhuis,
or something, Vicky, and he said,
ah, no, no, don't do that.
Use this fork here from somebody else.
This is much more actively maintained.
It has also a Mason build system.
It's much nicer.
Then the likelihood of your stuff being reviewed
and actually moved upstream are much greater.
I did that too, and then it sits there since about six months ago.
Not anybody looked at it and touched it.
I got two requests or questions on GitHub issues.
Yeah, it's just sitting there.
So this is also my hope to, let's say, mobilize the community.
You say, if you really like that, if you use KMSCon, you like these features,
please poke them so they review my stuff.
I'm willing to put in the work to make it fit the coding style
and everything as much as possible.
And I've still got future ideas with truly multi-hat capable KMSCon.
So it's not just mirroring the stuff.
So I'm seeing the same here and here,
but having actually per screen connected display one terminal session with KMSCon.
And then, of course, you need the information of how other screens arranged.
You could gather that information from any settings saved from KDE, Eugnome.
To read that with KMSCon, you need the libraries from KDE, Eugnome to link.
That's not so nice.
Nice would be kind of an end-curses interface where you have then ASCII art graphics
to arrange your layout of your screens.
But I started actually doing that.
It's not finished yet, but it would be really nice.
So maybe it would be accepted upstream too.
So links to the source code.
Some blog posts where I talk about what worked, what didn't work, what was hard,
some small milestones.
Lots of screencasts where I try to spread the word a little bit,
but the thing like and subscribe doesn't work because I'm always watching my stuff.
So I put it there anyway.
Are there any questions?
Go.
So just one one, I guess, with the mouse support.
Yes.
Were you actually like double-click to select?
Is everything character is possible?
Yes.
So the question is to select, double-click.
Yes, that does work.
I implemented that, but it's a bit iffy.
But usually it works.
But there's some things I still need to iron out.
But yes, yeah, it works.
Is my phone your scene handle?
Yes, basically that is my nickname forever and ever.
It's not a pun on Apple systems.
It's just make slow.
Yeah, that is my scene.
And you're part of a demo group?
No, I'm a single person.
I know the demo scene from forever,
but I only got active about three, four years ago.
And I'm not part of a demo group.
I ran two times at revision and evoke into the folks
from Logicoma.
I'm kind of sometimes jumping on the stream from Hoffman.
He's basically doing Amiga assembler stuff.
And he's an awesome musician, but I'm not part of a demo group.
But I love the demo scene.
If you like computer graphics and you don't own a demo scene,
you're seriously missing out.
Can I just have a part of this as we make that?
Yeah, I know.
There's the gathering assembly and all that.
No, wait, assembly is Finland.
I still need to get to Scandinavia for sure, for demo scene.
Can I ask on now?
Yeah, I mean, it would be nice to make a bosomatic variation
into KMSCon.
Yeah.
Sorry, your hand was a little bit fast.
Does it support subpixel rendering for rendering the fonts?
Does it support subpixel rendering for fonts?
Yes.
Well, whatever free type does.
So it's using free type and through Pango.
When you rotate the screen, then it takes into consideration
or then it just broke it?
It's rendering.
No, it should be.
It should work.
If the app doesn't do the rotation, you do the rotation.
I do the rotation.
It's broken, maybe.
I need, actually, that's good.
I need to look, but I think it's not broken.
But I need to look into that.
That's good.
Thanks a lot.
Yeah.
I mean, you missed the presentation,
but is there any minimum version of OpenGL required?
Minimum version of OpenGL required, I think.
I would guess.
Oh, my God.
3.3.
I mean, even OpenGL 4.0 something is like 10 or, I don't know,
how many years old, but certainly any system should work.
But I think 3.3 should be enough.
Yeah.
How does this compare to using a full-screen real-time
compositor, such as page,
compared with a lightweight terminal emulator?
How does it compare to a lightweight compositor?
That's a good question.
I have no idea.
I think it's more lightweight.
It's much more simpler.
It's certainly not a desktop environment,
so it's nothing like Western.
You can, at some point, easily make the jump to that
if you keep pushing, but no, no, it's not.
It's just one OpenGL context.
You run your terminal screen into that, and that's it.
There you go.
Thank you.
So two questions of first-of-end remos.
Do you implement all the remos by yourself,
or it can be waited until the editors, for example,
some of the console editors handle mouse events?
No.
It's right now in process of KMSCon.
Sorry.
The question is if the mouse handling could be, for example,
handled or actually be read in an editor,
like Emacs, I guess, or Vim or something.
No.
It's not like it's a KMSCon version of GPM.
That would be the next step.
But that would be a nice patch.
I'm sorry.
Yeah.
Okay.
The second question is, so you are handling all the rotation
inside your GAO.
How do you talk about handling rotation for the KMS properties?
Suggestion how to handle the rotation through KMS properties.
I haven't considered that.
I can look into that and then see if that makes sense,
or if I can do it or change my patch.
Sure.
Yeah.
Okay.
Anything else?
Well, if, yeah.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.
Thank you.