Okay. Hello, everyone. I hope this room is not empty. It's hard to see from here. Okay.
It seems like it's not. So do we have slides? Okay. So let's move to the table of contents.
So good morning. Today we are going to talk about accessibility of Linux desktop environments.
First, we will go through a little bit of theory. Unfortunately, it's needed, I believe,
for the later part of the presentation. And then we will try to show you some live demos,
kind of hands-on experience. And we will talk about what's the state of the accessibility,
how we can improve it. And then in the end, there will be also something we tried to create
to help developers improve the accessibility. So let's move to the introduction, please.
So just a few words about us. My name is Wojciech Polaciek. I'm working at Red Hat in a security compliance team.
I am blind. I'm a Linux user. I use mainly Arch Linux and Fedora. And I'm very slowly but steadily developing,
trying to develop a special Linux distribution called Tboitux. It's aimed at blind and visually impaired users.
Now I'm handing over to Lukasz. So, hi, everyone. I'm so glad to be here. My name is Lukasz, director.
I'm also blind Red Hat, but working in the desktop team for a few years, but with Linux for 10 years.
And I'm basically fixing accessibility issues whenever I find them. So mostly in GNOME upstream,
or sometimes GNOME isn't enough, so we need to fix stuff in GTK and sometimes even lower.
And let's continue with the presentation. So I'm handing the word back to Lukasz.
Okay, thank you. So next slide, please. Very shortly, just to talk what is this accessibility actually about.
To put it shortly, it's basically a property of a system which signifies that it can be used by a user with some kind of disability.
But of course, this is a very large scope, so on this talk we will reduce the scope to computer systems and to blind users.
So basically when I try to put it short, the accessible Linux desktop environment is such environment which can be effectively used by a blind user.
That means that we can get the information from the environment and we can reasonably interact with the environment.
We can get a work done, basically. Next slide, please.
So why do we actually need to care about the accessibility? I will try to give you some reasons, hopefully.
And mainly about Linux accessibility. So Linux is basically the only free of charge alternative to Windows and macOS desktop operating systems,
which offers comparable level of accessibility. I think this is very important, especially in cases where, let's say, the funding might be low for the systems.
Why actually blind users would like to use computers? That's also a question which would be answered.
So basically computer for blind user is a gateway to a lot of information to internet. It's also a very efficient communication tool.
It can be also very often used as a work tool. I hope we are quite good examples of that because we are basically developing on Linux every day.
To move forward, I think that the accessibility is also important because I believe that diversity and inclusion in the open source ecosystem is important.
And one more thing, I saw several stands which are dealing with the free open source on mobile devices.
And as Linux can run on such some of these devices, I think this might be actually a future. So we might have alternative to Android and iOS in Linux.
And I would like to see it being accessible. So let's move forward.
On the state of current desktop accessibility in Linux distributions, I have good news and bad news. I will start with the bad news.
Let's say that the initial hard law which is there for especially newbie Linux users is quite high.
The reason is that most of users who are going to start using Linux are switching or previously using Mac OS or Windows.
We've been running workshops in Czech Republic where we were basically trying to introduce Linux to users.
And we very often encountered a situation when we started the desktop environment and everything was fine.
And then we showed them that they were trying to use some elements of the environment like for example they were trying to find out what is the percentage of the battery in the laptop or what is the network they are connected to.
And this was not accessible. They couldn't get this information and they were like so surprised because this is just no discussion about it in other operating systems. It's just there.
This might be a bit problematic especially for new users. The good news is that when you sort these things out, you are actually doing pretty well compared to other desktop, to other operating systems.
There is a lot of things you can do with Linux and it's still improving.
So I think that the biggest hurdle is to get the things going and get the things set up correctly so that it doesn't introduce any additional learning curve or any additional obstacles for newbie users.
Let's move forward. I think the next slide should be about some basic terminology. Just two terms so that we don't spend a lot of time here.
First term is a screen reader. Screen reader is a piece of software which is essential for blind users of any operating system.
And it's basically software which kind of creates a bridge between the graphical user interface and the user.
So the screen reader gives feedback to user about what's going on on the screen. So what button for example is under focus.
Focus means that basically by pressing space bar or enter you will interact with the control which is under focus.
For example the screen reader tells you what's the current line in a text editor. It can read for example the alternative descriptions of images and various other stuff.
But it also helps us interacting with the interface efficiently. So for example it can help us move around web pages by for example jumping over headings and it can for example display the list of links to us so that we can go through it very quickly and various other stuff.
So this is screen reader and please do not confuse it with the speech synthesizer which is the second term and the last term on this slide.
And this piece of software is also important because it's actually the mouthpiece of the screen reader. It's just a software which takes some text as an input and it produces speech as an output.
And these two pieces of software are independent on each other basically. So that's just to set things straight.
Next slide please. This is quite important and we will talk about it and we will show it in the following demos how to make these in a desktop environment accessible.
So the first point is crucial. On the previous slide we talked about a screen reader and a speech synthesizer and together with some other software and configuration it's something we can call accessibility stack basically.
If this is not present in distribution basically you can skip all the other points on this slide because it doesn't make any sense.
You can have perfectly accessible desktop environment but if you don't have the screen reader present from the start basically there is like this is a showstopper.
The second point is very related to the first one because you can have the screen reader present. You can have everything perfectly configured but when you don't know how to start it then like you are in trouble again.
So this is usually accomplished through some standardized keyboard shortcut or something but there should be a way and there should be a documented way how to start it.
Then there are like general or generic desktop interface elements which should be made accessible like icons, panels, as I mentioned on the previous slide.
For example the icons in the panels these are often problematic. Then also the configuration facilities of the desktop environment these are also sometimes problematic and it provides quite bad user experience because as I said this is like standard.
Then another point is about having reasonable keyboard shortcuts because keyboard is a primary input device used by blind users.
So I think we should be able to like switch among applications, open the menus in some predictable way, in documented way, switch around virtual desktops and other stuff.
Then one point which I mentioned separately is login screen because this is also quite often problematic.
To have it fully accessible that means that after you enable the accessibility the screen reader should be started already on the login screen and the user should be able to use this screen to navigate it and to login.
Then there is a documentation part because as each desktop environment is a little bit different we often or we encountered a situation where it looked like it's quite accessible but we actually didn't know how to use that.
Then there are the apps provided by the desktop environment like these essential apps like clock, calendar, file manager which often go with the desktop environment.
And last but not least are third party apps like they are mentioned here just mainly for the sake of completeness because I think it's a little bit out of scope but I would not be happy if we forgot about these apps.
These apps I mean web browsers or word processors or whatever.
So next slide please.
I often get these kind of questions like so how about creating a custom distribution for blind users?
Yeah that would be so awesome because you can configure it however we want and like if something does not work as expected like you know we just like we hack it around somewhere, we modify the configuration and it will be working so great.
And I mean yeah like the idea is good and I believe that the thought behind it is like good positive.
But I think this approach has several problems either like I might sound now weird because at the beginning I said that I'm creating my own windows distribution but I will explain it later.
So basically yeah you can create your own windows distribution and during my 14 years with Linux I saw like three or four of them and each of them died in two years basically.
Because there is quite significant bunch of people maintaining Linux distribution and these specialized news distros are usually maintained by one or two volunteers and that means that sooner or later they will run out of time or their nerves or something and because they have to keep with upstream and stuff like that.
So this I think does not really work and then I think that rather than hacking around things which should work in the first place we should try to ensure that like they really work.
To explain the thing with Wojtuk's I am basically creating the distribution based on Fedora with the hope that after I create for example I package some software or I create some configuration it might be in the end merged into some official Fedora repose.
And so my vision is that in some years this distribution will not be needed because everything will be available in Fedora.
So I was speaking for some time already so I think it's time to hand it over to Lukasz who will tell you something about our real examples and also stuff we prepared to make the process of.
Creating accessible stuff environments easier.
So unfortunately as Wojtuk already said the situation in the distribution landscape isn't ever happy and sunny and so on.
And we didn't make up these things.
So what we did we actually went through the most popular distribution so basically Fedora and it spins and then Wojtuk and it spins as well because we wanted to test more things and we tried to boot up their live images.
And we looked around what happens and what we found one of three things actually happens.
The most happy path was that everything was working and you could enable the screen reader using the shortcut and it actually talked.
I'm not now talking about the accessibility of the environment itself.
We have some issues basically everywhere but in comparison to the accessibility stack not being there it's a minor issue.
The second group of distributions was quite interesting.
There actually was the accessibility stack and everything but the accessibility shortcut wasn't working.
So you pressed it and nothing happened and you didn't know why.
Of course if you knew the next way you could turn the screen reader on in a different way but no new people know that and you don't want to describe them that they have to press shortcuts type of comment and press the return key and so on.
Because they would say OK I will just stay where I am now.
I will put up with all the marketing and stuff and we're not bothered with this system because it doesn't work.
So we definitely don't want this situation either.
But then there was the last group where there was no accessibility stack at the live image.
So there was no screen reader no speech no nothing.
So of course you couldn't even install this thing or do anything.
And it's quite sad because as the video will show it's not so hard to make this thing available.
Basically there was a package missing and everything else would be downloaded through dependencies.
So I don't know it probably was only a mission on the part of the maintainers probably.
So it should be a pretty easy fix.
And of course there will be some size increase of the live image but it will not be something huge.
So I don't think this will be relevant for them.
Next slide please.
So before we look at the demos I have to say something first.
We of course don't use all the environments.
So it may happen that we actually don't know how to use them correctly because as White has said the documentation on the accessibility part isn't always up to date or is a found double.
So we could make a mistake.
So if someone has some more information just say us them to us and we'll be grateful.
And yeah we are using some names of keyboard keys and especially we have the super key which is usually mapped to the windows keys on most keyboards.
And we managed to use both terms for the same key in the video.
So no it's not anarchy but only a different name for one key in all the videos.
And I think we are ready to show the first demonstration the happy one.
So let's go.
Of course.
Oh yeah before we go we had some limitations in the presentation system so we will have to play the audio of the video through a speaker.
So the experience may not be in the recording and probably may not be either either here but we will try and you can always watch the videos at home if you really want.
And that's everything.
And let's go for the first one.
This video will show the work.
And the show is done.
The reference name is the number.
And if the number is played after the old system boots up you have to just press the keyboard and it's shown in the monitor specifically.
Out of the super plus S.
And if you do it on the right moment because you of course don't know when the right moment is the speed starts and you can do basically everything you want.
So what we actually saw we saw normal the home boot of course the boot sequence to what shortened on the video because it would be boring.
And then we saw that on sometime after the machine started up the GNOME environment booted up but the visually impaired person wasn't aware of that because there was no start up sound or something like that.
It would be quite handy to have something like this.
But if the person was right and timed it correctly he could activate the screen reader by the accessibility shortcut and it actually started talking.
So yeah that basically everything was working as best as it can in these times.
And let's ask move to the second demonstration.
We will show which will show a little bit in a certain state of things where we have all the components of the accessibility stack button.
Not the accessibility shortcut.
So let's play the second video.
This video will show another case of an accessibility distribution but when the accessibility shortcut doesn't work this will be demonstrated on a Fedora main 39 live image.
If we try to access the shortcut here.
We get nothing.
So we have to use another way.
So we press out plus F2.
Get to the command screen and type orca and confirm this thing with the entry key.
Screen reader on desktop icon view.
And so far surprise it worked.
So it was just the shortcut missing.
For example, move around the desktop icon.
So here we saw made a scope environment with missing accessibility shortcut and of course no startup sound as well.
But if you used a very complicated way for newbies but a way nonetheless the screen reader start.
As you may have heard there is a career issue in the speech output.
The pitch shouldn't change so often but that's another probably some Fedora configuration issue or something like that.
We didn't make it to the root cause of this thing.
But yeah, the thing actually talks.
But in the last video you will see what happens if you have no accessibility stack out at all.
So let's go.
This video will show the most difficult case.
Namely the case when there is no accessibility technologies on the live image.
This will be shown on the cinema Fedora spin, the 39 version to be precise.
So the first time of all the Fedora spins we actually got some sound when the boot is done.
So that's definitely nice of them.
But if we try to press the accessibility shortcut out past Windows plus S.
We actually get an error message.
Saying that there's no ORCA at all.
So what we will do?
Well, we will open the ground terminal using the outfdue shortcut.
I think it's name.
And then typing the command to install ORCA all the way.
Of course we know nothing about the commands progress and we have to basically hope that we did everything correctly.
Maybe we can infer something from the disc activity or so but that's everything we got.
Is it done or not?
Well, who knows the fact not me.
So let's try.
Let's start with the accessibility shortcut again.
And it does nothing.
But it actually may mean actually nothing as well.
So let us try to out pass F2 method.
And as you can see, to install was a success.
We got everything we need for speech including some voice, the screen reader, the middleman and everything.
So yeah, we can actually try to use this system.
At least we can like read the terminal.
There we go.
It's a blank tab left out.
Window, like user app.
Blank, speech, dash, dispatcher, dash, error, dot, one, one, dot, five.
Left control, window text.
But of course, that's not everything.
To make this system fully accessible, we would need to fix all the cinema accessibility issues and that's the story for another time.
So here we saw something which would completely block Linux newbie because he wouldn't have an idea that he can even open the terminal.
He probably wouldn't even know what a terminal is.
And if he knew that, he would have no idea about the easy way how to install Linux packages because in most cases he wouldn't know about packages, package managers and everything needed for this quite hard way how to install this.
So this was needed to the live environment.
So this basically was quite big luck because I actually know a lot of these commands and tricks.
So I could use some of them and I managed to do the Orca install.
But no, it's not something I would expect from a normal user at all.
So this shouldn't definitely happen.
It should never happen to say it concretely.
So let's go to the next slide.
But enough talking about live media because very often a developer comes and asks,
yeah, I got here some issue report about accessibility, but I have no idea how to do anything about it.
Wow, he may even ask what actually is.
It happened to me as well sometimes.
So we wanted to do something about it.
So if we look at the next slide, we saw that we needed an answer for this issue.
And the answer was a small project that had with maybe four or six people called Linux as developer accessibility guide.
This thing should teach the developers some very important things.
First, it should teach them what accessibility is and what impairments he can actually encounter in the development career.
Because of course the world isn't full of visual impairments.
We have other impairments as well and we want diversity and all the things.
So we want to talk about as many as these impairments as possible.
Then because the usage of computer for these people is kind of different.
They are for example not using the most at all.
We talk a little about how they use the computer.
And then we talk about which accessibility features of the top environment they use, how they use them and why.
Because if the developer tries to find something on their own, they may find accessibility resources.
They are there, they are plenty, but they explain only evaporated stuff.
Of course there are some terms which are common, but the underlying ways how to accomplish the tasks are quite different.
And to add to the issues, you either find some high level descriptions of the accessibility issues which doesn't help you at all.
Because you don't know how to make your application accessible.
Or you find for example the GTK reference which lists every accessibility role in GTK.
But if you don't know when to use which of them, what accessibility roles are, what accessibility states and so on, the documentation is less as well.
So another part of the developer accessibility guide is part where we try to describe how to make most controls accessible like buttons, check boxes, sliders.
And then when we came to it, things like text controls, but text controls are one of the most complicated things to make accessible.
So we didn't write this chapter yet.
Well, until a few days ago we would have to say that you either use the GTK widgets or you have nothing because the accessibility interfaces couldn't be used by the developers itself.
And as far as the guide goes now, we have only stuff for GTK4 and sometimes some comparisons with GTK3.
But we want of course more.
We want descriptions for QT or we will see what else.
But we wouldn't be so successful without thanking our two colleagues from Red Hat, namely Anna Horser and Eshwin Kumar because they help with writing these widget descriptions and so on.
And on the next slide, you can see that the work is publicly accessible on GitLab, of course.
You can see the exact URL there as well.
So if you want and we would like to, if you want to help us, let's basically do it and we are looking forward for maintaining Q there.
And for the end, I'm just, Vojta just to go forward and finish the thing.
Okay. Hello, hello. Does it work? No. Yeah. Okay.
So for the conclusion, I would like to just summarize a few of these points.
As you have probably guessed, we are a small group of people based in Red Hat and we basically share these points which are on the slide.
So we think that accessibility should be integral part of the development.
It's ideal case it should be like a first class citizen which is considered from the beginning.
The second point is that we prefer to cooperate with upstream rather than, as I said, writing our own hacks and creating our own distros because I believe when we for example find some problem in the development,
in one particular distro environment and we fix it or the developers of the environment fix it, it will get fixed in every distro.
So I think that's just how it should be done.
We also know that it's not an easy process.
It will have its problems.
For example, the user base of blind and visually impaired users is currently low.
It's basically a closed circle because you have not many users.
That means there are not many issues reported to developers.
That means developers don't fix it and if it's not working, you will not attract many blind users.
It goes around.
So that's something we will have to improve.
And the last part is probably if you care about accessibility, if you'd like to make your application accessible, look at the developer guide or contribute to the guide or you can even contact us.
I think the next slides lists our contacts.
There is an email.
There is a matrix ID.
There is a GitHub handle.
And so that's everything from our slides for today.
Thank you for your attention and we would be happy to answer some of your questions if you have any.
So any questions?
Hello. Thank you very much.
So one of the criticisms is a bit more technical in nature.
One of the criticisms that is leveled against Wayland as an environment is it's very tricky to get working with accessibility software.
Is that something that you've come across or is it a solved problem and that's just out of date?
Look, this sounds like a question for you.
We have mine.
Ah, okay.
I'll give you this one.
Okay. So Wayland, while the basic accessibility interactions are the same because the accessibility interactions are the same because the accessibility interactions are the same because the accessibility interactions are the same because the accessibility interactions are the same.
Another phase is a separate device API.
But, yeah, wayland changes things a lot because the users sometimes want to, for example, actually use screen reader specific comments and the screen reader needs to get the keys and on Wayland it can't.
can't or it can't now but there are of course some experiments to change things. There are
issues with mouse related things because the Visual Impart wants to use sometimes in a
very, very broken application needs to do a mouse click from the keyboard and avoid
the stuff and it's done very, very differently. And of course you need to get the coordinates
for the widget as well for these mouse tricks and this is also completely different. So
yeah, violent changes things, a lot, a lot changes things but it doesn't change the
overall accessibility landscape at all. So you can use a lot of experience from the
X days but you have to do some things differently to say.
So first of all thank you so much for the talk. Before today I never really stopped to think
about what it means to use a computer if you don't see the screen.
Thank you so much for the talk. For today I never really stopped to think about what it's
about when you're using a computer without seeing the screen. So it was very informative
for me. It seems that a lot of the work that needs to be done is about essentially translating
a user interface, a graphical user interface for example, into speech so that somebody
can hear about what is being displayed. I was wondering if it's any easier to translate
a CLI interface instead. So basically it seems to me that a non-graphical interface would
maybe be easier for a blind person to use. Can you share something about that?
Hello, hello, I will answer. So there are pros and cons to this approach. It's true
that when it's just a pure text interface it's of course easier because it's just
text that are no animations, no things which are kind of custom but it's not a very good
approach. The thing is that in the end, even though you have a CLI interface, you have
to access it somehow. You have to view it somehow basically. And most users will in
the end again use some desktop terminal emulator. And also then there is a thing that as long
as it's like really CLI, that you write some input, you get some output, then you review
the output, decide what to do and do something next. That's quite easy but when you get to
the more interactive ways, for example, when you, I don't know, if I want to use the midnight
command area I would say, then actually it starts to be very tricky and in this case
it's probably easier to have a well-working and well-accessible graphical user interface
application than to think about how to make a midnight commander emit the correct events
and stuff like that so that it can be used from the command line. And one more thing
I want to say that basically when talking about graphical user interface, the most obvious
way is how to fix this is just to add, for example, correct labels to controls or assign
correct accessibility roles to controls. It's not like rocket science usually. The problem
is that it's not well known and not well tested and so then the experience is sometimes poor
and we are in the close circle I talked about. No users, no accessibility, no attracting
of users, no users. Hello, thank you for the speech. I'm just wondering how do you advocate
for integrating accessibility in the web because I feel like as a UX designer I spend a lot
of my time being like, hey, we need to make this accessible and then you have, well, we
don't have a surplus of developers working on a project and it's like, hey, good job,
but we need to make spend more time making this accessible and I don't often feel like
I met with a wall like it functions. Why should we do this? Do you have any tips for preaching
accessibility to developers? So basically advertising accessibility to developers,
question, how to do that? Did I understand it correctly? Yeah. That's the question.
It's tricky because I think the best example would be to find someone who really wants
to use the application and like I think here with the current Linux user base of blind
users it's probably mostly about stories because like again as I said before, no users, no
feedback, no accessibility, no users. I currently do not have any tips in my head. Lucas, do
you have anything? I don't. It's very tricky because while accessibility is usually not
somewhere near the top of the priority list for the development, I don't have so many
tips because it's the circle because you don't want to recommend to your friends something
which you know that they will basically hate every day because they find some unlabeled
buttons but if you don't fix these buttons you can't recommend it never so. It's tricky.
Basically we tried going to FOSDEM to fix this. I don't know if it helps. And yeah,
like then of course what might also help is that there are standards for accessibility
as I said for the web but also standards, there are standards which are derived from
these web standards and they are more focused on desktop and they are in US based standards
there are also U based standards and so if there is a motivation that you would like
to fulfill these standards then this is probably a way, might be but depends.
Thank you for the talk. You said at some point that one stopper is having some accessibility
related packages missing from the distribution repository. I was wondering if there is some
kind of list of usual accessibility packages that one would like to see in repository that
we as package maintainer could verify that we actually package those things in our distribution
repository somewhere. So do you know if such list exists somewhere?
So I can answer that. Two answers. First answer is that I'm sorry maybe there was a misunderstanding
but the problem is that the packages are in the distribution repositories basically but
they are not present on the default life image of the system. But in reality it means like
they never existed for us because as Lukash showed you would need to install them manually
and you cannot do that basically. So this is probably not the problem but if it is a
problem do we have this section in the accessibility guide which lists the packages?
They should definitely be there but I don't think they are actually there yet.
This is a very good item for us. But it's really just a couple of packages basically.
It's an Orca screen reader as Lukash showed. This is the only desktop screen reader for
Linux. It's easy. And there might be some dependencies. It probably doesn't make sense
to say it here but just let us know and we will definitely add it to the Linux accessibility
guide. Thank you for this.
Hi. I have one comment and then two questions quickly. So first one you've moved me. I want
to now try to go home and install Linux from scratch and see how this feels because that's
interesting to me and I don't think I was ever aware of how hard this could be. Second
are you the first question I guess I should say is are you involved are you aware of any
work or connection to commercial interests? Have you worked with Bob Davis for example
at Red Hat or like Rail Workstation to like kind of see if there's I ask is like I think
the commercial driver could like drive more investment. So that's the first question.
Second one and feel free to kick me if this is dumb but is there any work going on with
large language models to like prompt and like answer instead of using labeling so that like
if you put your mouse over something it would just say what does this see and then the large
language model would tell you from scratch like if that's a dumb question please kick
me but I'm curious.
Can you just repeat the second question if there is a way how to like using a large language
model and LLM to ask what it sees you know and describe it.
Okay this is interesting I never thought about it. So first question yes we are in touch.
We are in touch with Bob and we are this is one of the way we also try to improve the
accessibility basically. Yeah so we try to find the cause or a case for Rail Workstation
but as Rail is derived from Fedora we try to fix it basically upstream and then then
take it from Fedora to Rail so that we can leverage this accessibility because I believe
it can help us. And the second question this is actually interesting I never thought about
that using language models but again I feel that it looks like an interesting idea but
I think that in the end it's again just a hack.
It's like we do not invent anything new here yeah we creating the accessibility accessible
user interface was is possible for a very very long time and it doesn't take much effort
it's just that no one is focusing on that. So I think this could be used but in some
very rare and custom cases I would say.
One more last question. Okay we are out of time.
Yes for me I have a multi-factorial disability in one sense and the most painful for me is
the activating power sitting of some function reducing the inserted the first one historical
so the key insert key I want this activate this and the other one are the moving the screen
horizontally vertically it's power sitting is how awful this kind of thing also reducing
the screen to one line and something like that.
Something I don't like it and I don't need it so for me it was very accessible more
accessible without this kind of thing when I work on my computer.
Okay I'm sorry was it like a question or was it more like a comment?
It's not heard very well here.
Was it more like a question or was it more like a comment?
The question is not clear so it was about some functions to disable so what is for me
it is there are problems of some short keys with the mouse with also the pointer and so
on the path and so when I see the screen reducing to reduced to one line or something else the
screen moving horizontally vertically so I did this it prayers in some way so for me
it's difficult to make my job with this kind of things so that's clear I suppose.
I have also reduced vision but that's not not bling.
Yeah I'm not sure if I know the answer to this probably not.
We can probably meet later after the talk so it is up.
So we are at time thank you again for the presentation.
We can move any Q&A out into the hallway while we get our next speaker up onto the stage.