And so, now we have two presenters, Benjamin Dekoshny and Morgan Reschenberger, so Benjamin
is a member of the Mozilla performance engineering team and Morgan is a senior software engineer
working on platform accessibility and Morgan if you want to repeat your name probably with
my pronunciation. No, you got it, you got it. Morgan Reschenberger, that's me. Yeah, we're
going to talk to you about an accessibility project called Cash the World and the way
that we're monitoring and measuring performance. So, I'm Benjamin, I am on the performance
team and I'm going to be talking about the collaboration from the performance side. And
I'm Morgan, I'm on the accessibility team. I'm going to talk about the accessibility
side. We put the matrix rooms for both of our teams here, so if you have topic-related
questions after this, you can follow up there. So, here's the agenda. We're going to just
talk a little bit about accessibility in Firefox. Morgan is going to go through intro
to the rendering and accessibility architecture and some of the changes that happened with
Cash the World. I'm going to talk a little bit about how we're measuring performance
and some of those questions and current problems. We're going to go through our future work
plans and then we're going to open it up for questions. So, the first thing is scoping
context for accessibility in Firefox. The goal is, of course, a faster accessibility
engine and more performant web use for users, all users and especially users using accessible
technologies. We also want to try to create a performance testing infrastructure that
will be able to prove these things and test the more we change our internal infrastructures
we want to be able to make sure that we can catch problems. We also wanted to establish
some accessibility metrics and we want to work in public with public dashboards that
show the kind of performance that we're getting. We want to improve our documentation. We want
to improve the debug experience. And as such, we're going to talk a little bit later about
the profile markers that Nazim talked about earlier, but specifically the accessibility
problems, and we want to set up infrastructure for collaboration. So, scope on this is we're
going to be talking about screen readers pretty much only, and we're not going to be talking
about any of these other accessibility technologies like screen magnification, contrast modes,
on-screen keyboards, subtitles, any of that. That's all deferred till later in this work.
So context for Firefox and accessible technologies is not great from the free software perspective.
Almost all our users are on Windows, and then you have a very small sliver of Mac and Linux,
and Linux is like under a percent. We just have to know where we are, and that's where
we are. In general, 5.5 percent of all Firefox page loads for the month of January had some
accessible technology built in, and that's not evenly distributed across the OSs. We
see a much higher use on Windows, and Linux isn't bad, orca, yay. And then Mac is far
below that. But for the most part, if we were talking about who is touching this work and
who do we have to care about, it's these Windows users.
And then here, just for a little bit more context about, like, in that 5.5 percent of page loads
that use accessible technologies, like, what accessible technologies are they using? They're
using mostly screen magnifiers, which is the black line, and then the purple line is speech
rec in general, and then underneath that is NVDA, which is the Windows screen reader.
So those are the top three that we really have to care about.
Morgan?
And so before we get into all the details about the performance work, I want to give you some
background on how rendering works in web browsers and how it translates to the accessibility
architecture that we're going to be talking about today.
So the general job of a web browser is to convert HTML and CSS written by web authors
into visual navigable content, right? And we do this through a rendering engine in Firefox.
This is called Gekko. It has five different phases and stages that produce artifacts that
are used in the following phases and stages. So first we parse the HTML document. This
creates the DOM or document object model, which is a hierarchical view of the web page.
Then we look at the CSS and figure out the style information for each node, what visual
changes we need to make when we render. Then we do layout, which computes positional and
size information for each of these nodes. It also constructs an artifact with that information
called the frame tree, which becomes useful later. And then we do painting and compositing
and rendering, which is the visual part of rendering.
But this process is all extremely visual, right? And what if you do not navigate the
web visually? What if you navigate it with technology like a screen reader, which turns
visual content into audio? What do you do then? And how does a screen reader figure out what
it should be telling you? Well, that's the job of the accessibility
engine. So like we have a rendering engine, we also have an accessibility engine in Firefox.
It doesn't have a fun name. So if you can come up with a fun name, you should let me
know on Matrix. But what it does is it takes in those artifacts we talked about before,
the DOM, the frame tree, style structs, et cetera, and it marshals them into a new kind
of tree, which we call the accessibility tree, or I like to call it the accessibility tree
because that's more fun. But it takes all of those and computes excessively relevant
information. So this is stuff like semantic role, name, the kinds of actions you can perform
on an element, things like that. This is not necessarily one-to-one, like there is not
a single accessible for every node in the DOM tree or a single accessible for every frame
in the frame tree. We care about different things, which is why we have to build a new
structure. And building the structure happens in the content process. We have one accessibility
tree per web page. So let's take a look at how these queries
happen from an assistive technology standpoint. So at the bottom here, I've got a couple different
kinds of assistive technologies. These are ones that Benjamin mentioned on that graph
from before. So we have screen readers, voice control, window managers, et cetera. These
clients or ATs make requests to Firefox for web content information. So if you are navigating
with a screen reader, the screen reader needs to ask what node is focused and what should
I say about it to the end user. The way that those requests happen are through platform
specific APIs, but they all hit the parent process in Firefox. The assistive technologies
are separate applications. So they're communicating with Firefox through the parent process. Each
web page lives in one or more other processes, one or more content processes, and is not
reachable by the assistive technology directly. So we can't inject the screen reader into
web content for a lot of reasons, security being one of them. All these calls go through
the parent process. And there are some problems with this architecture
that motivate what we're going to talk about next. So let's get into it. Like I said, computation
of the relevant properties that the assistive technologies are requesting, that all happens
using the accessibility tree in the content process. The result gets sent to the parent
process from content via IPC, inter-process communication. This is slow and it's also
synchronous. So if a call gets blocked or is taking a really long time in content, you
can't do anything. The parent process just hangs. And because the parent process includes
all of the browser UI as well, it just looks like Firefox is not responding, which isn't
great. So what can we do about that? Well, our solution
is this project we call Cache the World, which introduces a cache in the parent process that
keeps track of snippets of content information that we need to compute and respond to those
API calls. So we're trying to offload as much work as we can from content into parent. And
this cache gets updated asynchronously based on content mutations. So we no longer have
this problem of synchronous blocking IPC. Cool. So now I'm back and I'm going to talk
a little bit about, like, how do we see if this stuff is working? So the first thing
we did is actually not at all metric or measurement based, but it was more about helping debug
in the profiler. So one of my great colleagues, Michael Kamala, added some accessibility markers
in the profiler to kind of, like, get us an idea of, like, what's going on, where? You
can see the specific calls here. And then I'm going to show you what it looks like kind
of in the profiler. So the red circle is where we start to drop into some of the accessibility
calls. So watch this space because we're going to be adding more markers here. The second
thing we had to do is really come up with, like, how do we test accessibility and what's
going on here? There's a huge amount of screen reader. There's just, like, a whole bunch
of different screen readers, and they're all different, and each OS has a different strategy
for dealing with this. So we have, like, a huge complex testing matrix here. In addition,
we had to, like, in terms of testing, we had to, like, run a large number of variations
to kind of verify our results. We have five different variations starting with the baseline,
and then we kind of, like, have caches on and off with the accessibility implicitly
on by just plugging in screen reader, and also with accessibility forced on with preferences.
So we have a really large matrix of five on our task here, and then we were looking for
specific problematic web content that would really trigger kind of the worst case scenarios
here. And they are, in general, the worst case web content for this are really large
static web pages. So what do we do? We added three specific sites. Actually, I think we
have, like, five sites. But in general, it's like Wikipedia World War II is a great test
page for testing accessibility. We have some search box links because we're Firefox engineers,
and then what WG HTML specs. So these kind of, like, really large static pages, which
is not necessarily how a lot of the web is built right now. But these are, like, specific
problem points that we wanted to be aware of and address. And then comes the question
of, like, well, what are we measuring? What's important? And we have, like, three general
choices here. We have, like, W3C, navigation timing, kind of page load metrics, like OOG
performance metrics, that segment browser page load into distinct phases, DNS redirects,
DOM parsing, and then, like, content-ready pages loaded. We usually traditionally use
visual metrics, but because of the nature of this, nope, can't do that. And then we
have some kind of internal benchmarks that are not really publicly accessible where we
just try to look at specific code flows and time and measure. And, like, that's really
showing the most promise, frankly, and what we're going to be using more of in the future.
And so what we have, we're trying to work in public, and we have some public dashboards
for this work, which are at the end here. Whoops. Sorry. So this is, like, some preliminary
results. This is a graph a little hard to understand, and I'm sorry about that. We have the blue
baseline performance. We have these dotted lines with the caches turned off. And then
we have what the caches turned on. And so we're seeing, like, yeah, not great performance
for these static web pages right now, at least on Linux. I think that actually varies on
Windows. But we're seeing some wins and some more even performance on things like IMD web
pages, which aren't, like, these pathological test cases. So in general, what we're going
to be doing is we're going to be trying to align the profile markers that were put in
to performance metrics using our internal tools at first. And we're just going to try
to start measuring, like, the actual cache creation time. And we also want to start paying
attention to not just straight, classic page load, but we want to start thinking about
page reload, tab switching. And one of the other leads on this project, JNET, has a great
blog post about those kind of, like, anecdotal performance measurements. We definitely want
accessibility first metrics. And we don't, we would like to get away from generic page
load, tab metrics on this. We have a public dashboard, work in progress. It will continue
to evolve as this work evolves. And then really quickly, future work.
Yeah, so the accessibility team at Mozilla is responsible for a lot more than just the
accessibility engine. We're also responsible for high contrast mode, zoom, Firefox front
end usability and accessibility. So we've got a lot of projects apart from this that
we're working on. But our main goal for this half is to shift cache to release. We're currently
in beta and we have a lot of promising results. So we're really optimistic about getting this
to all of our users. We're also planning on working on optimizations based on the performance
work that you're seeing here. We have a couple of optimizations in mind. Like, we know we
can improve on cache granularity. But this work will inform the kind of work that we're
doing next. And then the performance team is going to really try to get these Windows
results in since we know it's so important. At the same time, we want to make sure that
Linux performance doesn't degrade. Also, we would like to kind of like put this into standard
continuous integration test infrastructure. Kind of tune our markers, make sure we're
measuring what we think we're measuring. And then things that we deem successful in a wide
variety of web content, we want to try to push out to public telemetry so that we can
actually measure much larger environments and users. And then, of course, all of the
internal collaborations inside of Bazilla with Perftools and ETL and DevOps to try and
make all the magic happen. We have some questions. If we have time for questions, we have time
for questions. We have time for questions. And if you have other thoughts, you can email
us or, you know, Twitter. Are there any questions? All right. So complete. Yeah.
We actually, on the slide deck, but not in our presentation, we did have some additional
resources and notes for people who are trying to work with accessibility, maybe new to it,
and things that, here are some resources for you to use. Again, Jamie's blog post, really
I'm going to really hype that again. Please read it. Morgan is going to put a video up
that has to be done because there is some internal stuff that can't be shown. But she
has a great walk-through about how to debug CSS for accessibility. And then I have a
web page on color and contrast for accessibility and how you can compute colors that work for
a wide variety of people. And also I want to shamelessly plug that you can contribute
to Firefox. And if you are interested in working on platform-specific bugs or front-end bugs
or whatever, accessibility is a great place to get involved because we span a lot of components
and we could always use your help. So if you are interested, we have an accessibility
room on matrix at the Mizzilla domain and you should reach out and we are there. So.
We will take a question. You mentioned it is not safe to embed the screen
redirecting to the web page because of security concerns. But now we are cashing, you are
providing a little bit more information to this pattern process. Are there any security
considerations you have to look at or address doing this work?
We are paying attention to the kind of information that we are cashing. We don't want to give
any private user information away. Largely, the information we are cashing is already
represented in the parent process in some form. But the way that we compute things is
different than how DOM or layout or other parts of the browser compute them. We are cashing
really, really granular information as well. So, yeah, we are not currently concerned about
security risk but that is a consideration.
Maybe you already said, do you have performance tests with accessibility enabled right now?
Yeah, that's what that website is. Oh, sorry. The question was do we have
performance testing for accessibility? Yes, we are starting to do that.
Is it just a matter of enabling accessibility and running exactly the same tests or are
you doing something different for accessibility?
Yeah, so the question is, what is the method there? You can contact me offline if we are
running close. But we are using a standard framework for performance testing called browser
time, which is open source. And, yes, what we are doing is we have OS specific handlers
that basically start screen readers before we start running that and then stop at when
we are done. So it is just RAI straight style on that, yeah. And then porting that to Windows
too.
One of the difficulties with that approach that we are running into is that we are most
interested in perceived performance. So we want to know how does the user feel about
this? Like, is it perceivably faster? And that is really hard to do because screen readers
are difficult to automate from that perspective. Speech rate is extremely variable. You can
do key presses and stuff, but it is really hard to get the kinds of measurements we want.
So we are aware that the performance testing we are doing right now is a number and it
is something that we can track consistently, but it isn't entirely what we would like to
be.
And there are different strategies on the Windows screen readers about having to have the full
page ready before we actually start in with the speech. And that is like configurable
and that is not the default setting for on Linux, for instance. So Orca, I think, is
actually pretty smart about this. And they can do partial reads and start the speech
earlier. So we are not getting quite.
We have a comment on that.
Oh, sure.
There is a question.
Oh, here it is. Note that the caching of the parent moves information into a process that
is not exposed to web content.
There is nothing before that.
It is not appearing here. Maybe. Yes, here.
Oh, can you talk about how the cache is populated and invalidated?
Oh, sure. How much time do we have?
Two minutes.
Okay.
Go.
So the cache is populated from content. So it is a push-based cache. We aren't invalidating
from parent because we can't observe content mutations from parent effectively. Each content
process is responsible for monitoring their own mutations and pushing or invalidating stuff
in the parent process as needed.
We have an initial cache push that... Oh, no, sorry. On page load, we collect a bunch
of information and push it always so there isn't any sort of mutation that we're responding
to there. That is one of our big performance concerns is the initial cache push varies
by page size or scales by page size, and that's really costly.
But...
That's why you put all those big tests in there.
Yes.
So from initial cache push onward, we're responding to mutations in content from content.
Yeah.
Are there any other questions?
Oh, yeah.
Go into the limit.
On the web app side, what may impact negatively the performance of the accessibility?
Like how could you design web content such that it's optimal for accessibility?
That's a great question, and we'll come back at you later with an answer.
Yeah, we're still kind of early in phase on this, but I feel it would be a great idea
to do some kind of web content help to get people to know the performance choices they're
making for accessibility.
Yeah.
Oh, yeah. Could we come up with some guidelines for performance learning and general guidelines
for how to do performance accessibility? Request submitted. Thank you.
So thank you very much. We are done.
Thank you.