Hello, hello, everyone. Welcome to 4th stand 2024. And this agenda speaker is Will Thompson.
And welcome him. Yeah.
Hello, can you hear me? Great. So, so, how about this? Is that better? We'll see how
we go. Cool. Hi, everyone. Thanks for coming today. I've seen a lot of really great talks
in this room over the years. It's a real privilege to be on this side of the auditorium
for the first time. So, a little bit about me. I'm an engineer at Endless West Foundation,
where I've been for seven or so years. And I've been working on GNOME and GNOME adjacent
stuff for longer than that. And today I want to talk about why it's useful for free software
projects to collect usage data. I want to talk about how this can be done in a privacy
respecting fashion. I'll talk about the Endless OS system for this as an example of maybe
an existence proof. I'm not necessarily suggesting that other projects should take what we've
built and use it directly, though, of course, you can. But I hope to encourage other free
software projects on the desktop or other ways to consider adopting similar techniques
so we can better understand how the software is used. I mentioned Endless, what's that?
So I work for Endless OS Foundation. We are a nonprofit organization. Our vision is simple.
The whole world is empowered. And access to the digital tools of the modern world is a
prerequisite for being empowered. So we strive to ensure access to these tools and create
opportunities for underserved and under-resourced communities around the world. We do a lot
of things which are not Endless OS, even though Endless OS is in our name. But it's Endless
OS I'll be referring to today. So I'll talk briefly about what Endless OS is and what it's for.
In brief, it's an immutable Linux desktop distro. Visually it's known with some modest
customizations to suit our target users. The groups we work with typically have little
to no previous computing experience, but they probably have user smartphone. You can download
Endless OS from our websites and you can in some parts of the world buy it to be installed in
OEM systems. But we as an organization are more focused on, okay, we are more focused on working
with other nonprofits and with companies aligned with our mission to bring computing to underserved
communities. So this might be partnering with another foundation to set up a computer lab in a
disconnected rural village or we might work with microfinance organizations to make computers
affordable to low-income families and so on. And in these contexts, there's often limited or
intimate and internet connectivity. So part of the point of Endless OS is we pre-install lots of
apps and lots of offline learning resources and we make sure the whole system is fully usable
offline. So what do I mean when I say the word metrics? I'm going to use the word telemetry,
metrics, analytics, usage data and so on interchangeably. Sorry if there are technical
nuances of those words. But I'm referring to the concept of end user software, so that software
that runs on a device in your hands, collecting data about how it is used and then periodically
sending this to its developers. You might be saying that sounds a lot like spying. Please
hear me out. I'm not talking about that. The other part of the title was privacy respecting. So
you might be skeptical because when people talk about usage data, they're often talking about
slurping up all kinds of personal data about each user, building profiles of each individual
and then you sell it to advertisers, which is so the easiest way to explain what I mean is
privacy respecting means the opposite of that. But the easiest privacy respecting thing to do
is to do nothing. You don't collect any usage data. You don't have to write any code. You don't
have to think about the ethical or legal issues with the data collection because you're not doing
it. So maybe for a lot of projects, that's fine. And you might ask why? Why would you do this?
Well, software is not made in a vacuum. Normally you're trying to help some group of people do
something they couldn't do before. And so in order to build good software, it's useful to know how
is your software being used. Is it being used at all? What hardware is it being used on? Which
features are used? Which features are not used? And so on. And if we have this information,
we can make informed decisions about how to build the software rather than basing it on
assumption and guesswork and vision alone. The other strand to this is a lot of people are
developing free software at work. I work for a non-profit and I would like us to continue to do
the work that we do, to advance our mission and also to contribute to the open source comments.
And part of doing that is to demonstrate that the work that we're doing has the impact that we
are trying to have on the world. And the organizations we work with have similar
needs. They need to justify to themselves and to their own sponsors that it's worth putting their
time and resources into working with us. So having quantitative data helps to support the impact
we're making. And you might say, okay, that's fine, but why don't you just ask your users,
run some interviews, do some surveys, some usability testing and so on. Wouldn't that be ideal?
And yes, of course, there's no substitute for actually talking to the humans who are using
our software. It's quite rare, particularly in free software projects, to have the resources to
scale that. And for some things, users are not consciously aware of the ways they're using
their software, the software they're using. There are also limits to what you can learn from a
half hour or one hour testing session as opposed to usage over time as part of doing your day-to-day
work or life. It's very useful to find volunteer testers from the community. You can learn very
interesting things from that. Those groups tend to also be quite self-selecting. So this will
sue the results towards people who have a higher motivation to tell you what they would like you
to do with your software. So ideally, you want both, I think. I think you want to talk to end users
and explain the why behind what you can find in data that you have. And in the other direction,
having data about how the software is used can drive the kinds of questions that you want to ask
your end users. And essentially, every website online store, app, and mainstream OS provides
something like this. I'm not arguing that we should do something just because everyone else does it.
And hearing that a big tech company does something might often be a reason to do the opposite thing.
But there are non-evil reasons to want to do this. And I think it's reasonable to assume that
the people who are developing software free or non-free typically want it to be good and useful.
And other projects have similar requirements and constraints to what I've just discussed.
So even with more resources, you can't constantly interview your users. And we're often at a
disadvantage compared to commercially backed software. The big ones are in people and time
and money. All of these things are, of course, related. And I think that rejecting the idea of
collecting users' data outright just creates more unnecessary disadvantages for ourselves.
We should want to have the information that we need to focus the limited time and resources that
we do have. And we have the opportunity to use the structure and the transparency of free software
projects to do something that's actually better than the status quo in the wider industry.
We want to respect our users and preserve their privacy while still being able to make better
decisions and make our software serve them better. The kind of axiomatic thing here is we do not want
to collect personal data. We don't want to track individuals. We don't want to sell that data or
worse have it stolen through some database hack. We don't want to serve targeted advertising and so
on. Of course, handling personal data comes with legal responsibilities as well, which if you can
just not collect personal data, it's much better for everybody. So if you want to hold a word in
mind, think tally not surveillance. An analogy to Cicassidy, who's here with his phone, is think
about a library. So near me, our local library is run by volunteers. And you might imagine that one
day you go to the library and there's someone at the door holding one of these little tally clickers.
And for each person that goes through the door, they click it once. And this helps them to get
some kind of measure of how well used the library is. Maybe they can collect this similar tally on
different days of the week or at different times. And this can help them decide how they staff the
library, advocate for more funding from the local government and so on. The other end of the scale
is if you imagine someone kind of following you around in the library and they're going to look
over your shoulders, say, okay, you've gone to the computer book section. You've gone to the children's
book section. You probably have a child. Okay, watching what you're reading. Obviously, this is
not hyperbole, but this is really not what we're talking about here. So sometimes you can get this
kind of tally information from some kind of service that you control. So FlatHub is the de facto
standard flatback repo. It has, we recently announced that it's reached one million users. So how do
we measure this? Well, it's measured by a proxy. There is a runtime which most users of Flatpack
we claim have at least one app installed which uses this runtime. And due to the way that Flatpack
downloads updates, you can tell the difference between an update and a fresh install. So when a
new point release of that runtime is made, you simply count how many downloads there are of
updates for that runtime in a given period of time, say, a week after you've released the
runtime. And this gives you a pretty reasonable lower bound on how many installations of that
runtime there must be. And there was no identifier needed. We didn't need to look at IP addresses
or machine ideas or anything, just having some knowledge of the ecosystem and how the Flatpack
client behaves. And there are other places where this idea is used. So in Fedora, there's this
thing called count me. EndlessOS has something similar. So with DNF is the package system for
Fedora. And it has to periodically update the list of packages that are available. And so the
approach here is that in one random request per week, and these requests would be happening anyway,
an extra parameter is added, count me. And then it has a value which refers to how long it's been
since you first installed your system, which gives you some indication of what retention is
like of the system. Then from the user agent, it's possible to infer what the distro version is
and what variation of Fedora it is and architecture and so on. And it's a clever idea to piggyback
on the meta link request. And again, let's users be counted without personal data because there's
a fixed frequency. So they published the aggregate data, which I've doctored a little bit to fit
on the slide. So here, again, there's the fixed frequency, which meant that no identifier was
needed. And there are also these kind of statically determined segments of the user base, which
doesn't identify any individual. It identifies a massive group of systems. So the three main ideas
for doing something else here is that we want to generalize this approach to finer grain data.
But it's data that we wouldn't otherwise have because we don't get anything as a side effect of
stuff that's happening entirely on your local device. In the library analogy,
they do have this information, I suppose, of which books are people borrowing. It doesn't
matter who's borrowing them. It's just in general terms what's popular. So the three ideas here,
the first one I've mentioned already is sending on fixed frequencies. If you send information or
rather record it on a daily basis, this means that you can be sure when you look at the data that if
two different events on the same day appear, they must have come from two different systems.
But you don't have to identify which particular system they came from and you can't tell
which events were the same from week one, which systems they were. In the other axis, we're not
interested in individual events. We're interested in patterns of usage data. You generally want to
be able to compare those patterns between different groups of your users. Maybe it's for software
version, maybe it's by local, maybe it's by hardware. It depends on what you're trying to learn.
But these are determined ahead of time. They're static and they are common across a large group
of users or devices, rather, I should say. The third piece is to do some kind of data, which is
instantaneous or which you're collecting on a timer. This is easy, but for some things, it's
kind of continuous data, which this doesn't work for. For example, app usage. You might want to
understand which apps are used the most in terms of time. This is something where you might, on a
given day, open and close an app several times. You need to do some kind of client-side aggregation
to turn this continuous value into a single data point on a fixed frequency that you can record
by itself. So the end-of-the-sometrics system, you'll be shocked to hear, works as I've just
described. It breaks down into a few components, which I'll go kind of following the duration of
the arrows in this diagram. We'll talk about what happens on the end-user's device, then how that's
transmitted to a server, and then what happens once the data reaches the server. So for local
event recording, we have a daemon which runs system-wide. It's a D-Bus service, and applications on
the system use a D-Bus API to talk to the daemon to record when certain events happen locally.
So some of these components are just regular system components that are doing things that they just
do. So our updater, for example, you can see the red box in the bottom left is recording an event
when an update has failed. There's also one extra daemon, this metrics instrumentation thing,
which is for capturing just general stuff about the system, CPU information, disk usage, and so on.
We actually also have a mediocre crash reporting system using this mechanism. It's not ideal,
but it's better than nothing. And as we'll see, it works for a system which is intermittently
connected. So each of these events has some kind of payload associated with it. So we'll zoom in on
the red event from the updater. This is when an update fails, we capture some information here.
We capture the time at which the update failed. We capture the OS version that it occurred on.
We have this UUID. Now, this is not specific to this event that happened for this one machine.
This ID is the same for all updater failures. It identifies the category of event that's
occurred. And then we have a payload, which in this case is just a human readable and localized error
message. And that's kind of gross. We have some nasty pattern matching to untranslate the string
in some cases and take out the values that vary just to narrow this down. We transmit this the raw
event because it was the only practical thing to do given the way error handling works in the
updater. But it's still very useful. This is, from this we can determine, this is the most common
reason updates fail if the disk is full. The updater runs in the backgrounds, so it's unlikely
that people will be actively checking it. So it's useful for us to know, are there fixable errors
that we can sort out somehow. I also talked about app usage. We've patched an MShell on NSOS to
record how long particular apps are used for. And so this one is one which gets aggregated in the way
I just described earlier, where you coalesce this continuous variable into you slice it by day and
by month. Here I'm showing by day. And it's actually the metrics demon which does this. The shell
tells the demon start recording event with this UUID and this payload. And then sometime later,
when you close the app, it says stop. And the demon takes care of coalescing multiple instances
of that into one in any given time period and slicing it if it runs over midnight or over the end of
a month. Okay, so now we've got a load of events buffered in this demon. We have an on memory,
in memory, an on disk buffer with a size limit. So we just delete odd stuff if we run out of space.
And then if and when you're online, the demon periodically reports these to our server and
then deletes the local copy of the event. This is an HTTP request. You might be saying,
I've said there's no device unnotified. Yes, there's an IP address. We'll come to that. That's
an artifact of the internet. And this this upload contains as many events that will be
cashed as we can fit in a single request. Plus a timestamp. Actually, there's more than one
timestamp. There's a clever algorithm to correct for incorrect clocks and a channel. What's a channel?
So this is the kind of static segmentation that I referred to earlier. And on end of the
S we have just a couple of things here. The lesson flags for is it a standalone install,
a dual boots or a live system? Interesting. But the main thing is this image identifier.
And so this is an artifact of how we build and distribute and the source. When you install
in this OS, you're taking a disk image, which has been pre built with a load of apps in it,
and you install that by just DDing it directly to your disk. You image the disk with the same image.
And so we have custom customized these in various dimensions. There's this product idea,
which is how this came to end up on a computer. Was it a download version? Was it an OEM partnership?
Or was it another organization we're working with? Or is it a custom built image that someone
has built using the tools we provide? There's some other stuff about the original OS branch
that was installed and the hardware architecture. And then this is personality, which again is
an artifact of the way the OS works. If you're pre installing lots of learning resources,
you want them to be in the language that the user speaks. So we have different variations for
different decals. And we have a basic one which doesn't contain all of the massive reference
apps. And when we work with partners or in particular projects, we often make a customized
version for that. And that identifier ends up in this personality field. So if you go to the
website today, or in fact at any point since the third of January, and you download the French
version, you will get this image. It has this OS product, which is what we refer to the download
version. Some attributes about the branches, the time stamp of when that image was built,
and the personality. And so any system installed having chosen French will end up with exactly
the same identifier ever since the start of the year. So this is what's on my laptop. And I happen
to know that there are only other two users of this one, and one of them is over there. That's a
unique case because we built this specially for a bunch of laptops in the UK endless team, and we
never publish this image. And that's an edge case. In general, the same OS image is used by many
different systems. So we have submitted a batch of events together with the channel to the server.
What happens? Well, first of all, we discard the IP address. We don't want that. The
HTTP, the endpoint adds a yet more time stamps to this bundle of events and puts it in a readers
queue. Now, something totally separate, which has no idea where this bundle of events came from,
pulls from the readers queue, and it splits the events apart and stashes them into a SQL database.
There's one table in that database for each category of events. So I talked earlier about the
daily app usage event. So this table has a field for the day. It has a field for the app, and it
has a field for the duration. So in this example, of course, in the real database, there'd be many
more rows. But just by way of example, you can see there were two different GNOME terminal events
on the 30th of January. So we do know that there are two different systems. We don't know if the
Chromium user on the same day was either one of those two users or a third user. The next day,
there's an event for GNOME terminal, two and a half hours usage. We don't know if that was any of
those two or three users we've already talked about, or a fourth user. We also have this aggregation
by calendar month, which has higher latency, but it tends to be less noisy. And these tables are not
linked to a device identifier. They're linked to the channel that was associated with the event.
So this has this image identifier which is shared between many systems. And so we can't match up
which different events came from the same system. We can't even identify which different instances
of the same event came from the same system. Of course, there's an element of trust in this,
like the server could be behaving not in the way I described. The best answer we have for this is
that we're not doing that, and the server is all open source. So you can go and take a look at what
it is, what's on our GitHub, is what we run. And the system is on by default because we've designed
it to be privacy respecting. When you first install endless OS, like many GNOME systems, you get
an initial setup wizard, which takes you through some steps to set up your system. This is actually
from the development branch. It looks a little different in the released version. There's a
toggle for enabling or disabling this feature. The toggle is enabled by default, but nothing will
be submitted until the user setting up their system has gone past this page and continued to the end
of initial setup. If you set the switch to off, then nothing will be captured, anything that's
already been buffered but not submitted will be deleted. Of course, you can control this later.
Once the events have been submitted to our server, there's no way for us to delete the events for
a particular system because we don't know which events came from which system. And defaults are
very powerful. The overwhelming majority of systems leave the default enabled. You might say,
well, of course they would. Everyone likes defaults, right? The point of this is to get more
representative data about a large body of systems. The system relates no personal data, it's designed
not to be invasive. Being on by default keeps us honest about that. We really have to be sure that
we're not collecting anything questionable. And some people, you can see some number here, may
prefer that we don't do this. Of course we allow that, but we don't force someone to make a choice.
Decision fatigue is real, particularly during first boots. We've seen that people get scared
off by the number of questions that are asked. What's a keyboard layout? So adding more questions
which people don't have the context to answer is not necessarily helpful. I acknowledge that not
everyone agrees. There are other opinions. This is what we do for now. So what if we learnt?
Some people may have read a blog post that I wrote six months ago with some examples of what we
learnt. So for those who have read it, everything here is new. Parental controls. So some time ago
we developed a feature in NSOS to allow parents to disable access to certain apps which are
installed on the system to control whether their child using the system can install new apps and
to set age rating thresholds on those. As part of integrating this into GNOME, which is now
upstream, this screenshot is from GNOME OS, and we added this to the initial setup flow. So it's
to be more easily discoverable. When you create a new user as well as choosing their name and the
username, you can tick a box which is a little out of focus. The box at the bottom says set up
parental controls for this user. It's unticked by the form that people tick it. If you tick it,
two things happen. One thing is that this three things. The user you create is a standard user,
not an administrator. A separate administrator user is created with a separate password. And then
on the very next page you're offered the option to choose which parental controls you want to apply
to this child. Now in this screenshot, if you sprint at it, no controls have actually been
applied. The default is that you have to actively choose which things do you want to
enable to to restrict. Do you want to restrict access to web browsers? Do you want to turn off
certain apps? Do you want to set an age limit on which apps people can install from GNOME software?
And we instrumented this and a large minority just left the defaults. So 40-something percent of
people who chose parental controls didn't actually enable anything. That doesn't tell us why they
didn't do that. I mean you can come up with some good theories, but it tells us that there's research
to do in this area and it can help us to guide what we do next with this feature. A tool. So GNOME
40 introduced a tool that's offered when you first log in and that's whether you've previously used
an older version of GNOME on the system or this is a fresh installation. NDSOS 5 was the first
release to include GNOME 40 and it looked, as I showed you earlier, very GNOME-y which is rather
different to what the previous versions of NDSOS looked like. So we inherited this tool. When you
first log in you get this prompt and if you choose to take the tool you get a tool which just briefly
walks you through how to use the desktop. I was curious whether people actually take it so I
added a very quick patch to instrument this. This isn't really a show me the code kind of talk,
but just for an example this is what you need on the client side. It's legible. So the top line is
we've just defined a constant for the UUID, we just generate an ID, and then you have the two lines
where you create the payload which is a Boolean which is true if they chose to take the tool
and false otherwise. And then we call this method on the event recorder class to record the event.
That's all you need on the client side. This is a small C library around a small D-Bus API
and there's geo object introspection around it so you can access it from JavaScript
and Python and all other things. Then the server, this is using SQL Alchemy as the ORM.
You define a table like this which has some keys that have a name of the table, the same UUID.
Again this is for all events in this category. The payload and how to turn the payload into an
instance or into a row of this table. It's a little annoying that you have to do database
migrations to add or remove events on the server. It has the downside of having the data in these
nice structured tables but there's an upside in that we can generate the documentation which is on
read the docs of which events the server understands. So the results are in. We captured this bit of
information from 35,000 systems and across those 35,000 systems about 19% chose to take the tool.
My assumption was that more people who are upgrading would take the tool than new installs
because if you're upgrading you're surprised or this looks a bit different what's going on.
Actually it's the reverse. At the top row we see users who are fresh installations and 32% of
those users took the tool out of 5,000 total in the period we sampled this. Whereas for upgrade
and list OS 4 it was just 15% who chose to take the tool out of a total of 29,000.
This is just a snapshot because now that we've answered the question we've deleted this data.
We've erased the data from the database. We add the UID to a list which gets discarded as soon as
it's received from old plants. We've also updated the OS to remove the three lines of JavaScript
I've shown you so we no longer collect this data on up-to-date systems and we discard it if we
receive it from old ones. This is the part where I talk about all the things that are
subpar about this system and what we might do in future. The big one is it's actually really
annoying to have the data split out in this way. All the app users are atomized and we can't answer
questions like does someone who uses app X also use app Y? Is there any correlation between groups
of apps that people use? We could of course submit one event which contains all of the apps that
a given user uses but that starts maybe that's a bit too fingerprinty. It would be nice to find
some way to answer questions like that without implicitly fingerprinting users. It's also hard
in general to slice this in new dimensions that you haven't already chosen to slice by. One question
might be whether parentally controlled accounts behave differently in some ways to
accounts that do not have parental controls enabled. The parental controls flag is not part of the
channel so we can't see for any other event whether it came from a parentally controlled user or not.
This is all just a consequence of what you choose to slice the data by. I think the trade-off is
worth it but I need to acknowledge that it is annoying to not have it identify. There's also
some kind of indeterminate upload latency. The problem here is how do you know when you have
basically all of the data for the last time period? It's particularly bad for monthly events.
Today is the third of February. Let's say I left my desktop at home. I switched it off on the 31st
of January. We can't submit any data for January until February has started because otherwise
we might have to add a bit more to the tally after the fact and you can't do that in the survey.
Now my computer at home is switched off while I'm here. I'll switch it back on on Monday.
That's the fifth. That's a five-day lag. Is that typical? Maybe we can look at the
timestamps when we receive the events but we can't do that because we don't store the received
timestamp for each event because if we stored that we could figure out which events came together.
You can probably imagine ways to solve this by reducing the precision of timestamps
and I think that's true in general. There are some cases where we have more precise
timestamps than we might like largely for historical reasons. There are some complications
if you can't assume that the local clock is accurate. Of course NTP exists but many endless
OS systems are used mostly offline and it's also quite common we found for the real-time clock
battery to have run flat. So it's not that unusual for people's laptops to have a totally incorrect
time until they connect to the internet and then when they go offline and run out of power it goes
back to sometime in the past. There's a lot of research into how to randomize the data that's
submitted. There's randomized responses, differential privacy. I'm sure there are people here who know
more about this than me. We haven't really explored this but the basic intuition is that
you add noise to the data you record. Suppose you're recording a coin flip, maybe the parental
controls one as an example. In 50% of cases you just always say true and then in the other 50%
of cases you submit the true flag. That of course changes the results you get but once you aggregate
it you know that of the 100 responses you get you expect to see 50 truths just from the coin flip
and so then you can look at the rest of the batch of events to figure out the true ratio without
actually having to know if any of the data points which is true is really true. This might be a
way to allow collecting more interesting facts without getting into personal data. There are
lots more questions we might like to ask about the software we ship. There's questions like are most,
desktop Linux systems, single user or do you have multiple different Unix users on the system.
What are the common monitor configurations? How common is it to have an external monitor most of
the time? Do people change this around? Do people have their screens arranged horizontally or
vertically or in a cool circle shape? Do people use workspaces? How do they use them? Which GNOME
shell extensions are in use? I could go on for an hour, I won't do this. I think this data would be
much more interesting if we had comparable data from other GNOME distributions. I'm using GNOME
as an example just because that's what we ship, insert project name here. Every distro reaches a
different group of people. Those groups will have different behavior. For example, I would
claim that the typical Fedora user is probably quite different geographically, perhaps economically,
perhaps in terms of technical skills, the typical NSOS user. If we have a common structure of data
that was shared between all users of a given project, we can compare how the same upstream
software is used in different contexts. Other organizations who do this kind of telemetry
have public dashboards of the aggregate data. I showed you Fedora's published data from their
repo servers. Mozilla has this great Firefox public data report, which gives you sort of daily and
active users, monthly active users, version statistics, locale, top add-ons, and this is all,
you can slice this by country as well as looking at it globally. Steam has this very
interesting hardware survey. They've made a different choice. This is opt-in with a pop-up
dialogue and still anonymous. It's very interesting. The median gamer is probably quite a different
user to the median desktop Linux user. Kind of, you know, a little tongue-in-cheek, haha, only
serious. In December, Spotify publishes this thing where you can open your Spotify app and it
tells you in like really garishly bad images if you are like one of the top 100 listeners to
some artist. And you see a lot of people remarking when they do this that this is kind of creepy,
they have all this data. It's very cynically free marketing for Spotify. Now, of course, that's true.
It is free marketing for Spotify. Other streaming services are available. But it's also fun and
sociable. I've had conversations off the back of this that I wouldn't have had otherwise. And maybe
we can have free marketing too. But we could do this differently. The central entity doesn't know
anything about any individual, but we could potentially publish percentile distributions.
And then on the local device, you could fetch this and determine, oh, right, you are actually in the
top 5% of the ability. Maybe this is a bad idea. I don't know. Anyway, just to wrap up, I guess,
from a starting point, I hope to have made the case that telemetry doesn't have to be creepy.
There are ways that you can gather data about how your software is used without being invasive and
building profiles of your users. And in an industry where I think not enough thought is given to
this, I think we in the free software community can lead by example. We can build something that is
better and allows us to improve our software while showing that a better way is possible to the
broader industry. And if we do that, we can make decisions based on the combination of data and
vision. The two work together to make something that's really great. Tomorrow morning, there's a
telemetry buff in Room 121 and AW1. I hope to see other interested parties there and for people
to tell me all the prior art that we didn't know about. That would be great. Hope to see you there.
Otherwise, that's all I've got. There's some various links. If you follow me on Masterdome, don't
expect too much discussion of this, but you're welcome to come. My blog has an older write-up on
the same topic, which has some more and some less details. And the source code is on GitHub under
the endless M organization. The name of the server and the event recorder is the service that
buffers and submits the events. EndsOS.org is the place to go for more information about the
EndsOS foundation and our work. Thank you.
Hey, hey, hey. Does anyone have some questions? Please raise your hand.
Oh, okay.
I was wondering, you showed us that 10% opted out of sharing
metrics, but how do you know that?
So, in case the question didn't come across the PA, I think if I'm right, the question is, I said
that 10% of people opted out. How can I know this? So, I mentioned that we have a system similar to
Fedora's count me system. So, it sends a daily ping with a retention counter with no other
identifying information, plus there's a Boolean, which says, is the full-fat metric system on or not?
Okay. Thanks.
Does anyone have some problem? Oh, I see you.
Thank you.
Hello. Hello. So, your talk has mainly been focused on how to get metrics.
Sorry, I can't quite hear the question. Sorry. I couldn't quite hear what you were saying. Sorry.
Is this coming through? Yeah, okay. So, your talk was mainly focused on
anonymous metrics effectively, making them as unidentifiable as possible.
And you did say that one of your problems is if you wanted to aggregate,
if you wanted to sort of correlate these metrics to kind of figure out, okay, if person X uses
this app, do they also use the other one? Have you given any thought internally on how you might
do this in a way which wouldn't impact privacy? You mentioned fingerprinting, would it be one
concern? Have you elaborated on that at all? I didn't touch all of that, but I think you're
saying that I mentioned that we would be interested in knowing. This is probably focused on an
anonymous system, and so this is one of the reasons we can't answer the question,
who uses both app A and B? And you'd like, I think if I understand the question right,
it's do we have any ideas for how we could do this? Effectively, yes. Okay, there's a few ways
you could do this, right? One idea that we haven't explored, but I think would be interesting, is to
layer onto this an opt-in system. So you could prompt people to be part of a time-limited study,
and you could temporarily add something extra to this channel, which identifies them specifically
for a fixed period of time. Then we'd turn it off on the client side, analyze it on the server,
and then delete it. Then I think, so you think you can, it's easier to add more stuff to the
channel than to remove it. And the other way to do this would be to look at some of these differential
privacy techniques, and then submit a single event containing aggregate app usage for all apps on
the system in any given week, let's say, but add artificial noise to that. So with some probability,
change the numbers, replace the names of the apps, remove apps from it in a more systematic way than
just shuffle it around. And there are techniques you can use to add noise while keeping the
distribution of data the same. We haven't had an opportunity to go into that, but I think that's
probably, in the general case, the way to
address the points. Maybe there are other ideas. I'd love to hear more. Thanks. Any questions?
If you have any questions, you can raise your hand.
Hello. We still have 10 minutes to ask a question. 10 minutes left. Any questions? You can raise your
hand. Okay. Okay. Climb into the speaker. Thank you very much.