Hello. So this is our talk about the spot nuts. It's a
Techist Tinkering project. So first who we are. I am Pingu. I am 14 years old. I'm a
member of the Techist community. I began hacking like four years ago or something like that.
I'm interested in Python, home automation and stuff and obviously Penguins. And I also work
at the Alexis project. And my name is Snick or Dominic if you like longer names, three to two.
I am more or less the founder of the Techist community about which Penguins will say a few words
right after my introduction. And here I'm working in the intersection between education and free
software. It means I'm showing young people what free software is, what the values around free
software are and also helping develop and promote free software for education institutions.
And in my day job I mostly spend my time as a trainer for Linux administration,
PostgreSQL, Rust and Python related topics. Yes, we mentioned Techist. It's a community
based in Germany. Our goal is to create a comprehensive technical word for and with children
and like to empower young people to question things and hack and build stuff like this project
or the Alexis project. So here you can see where we were. This is an Alexis meeting.
This was in I think at the Frostconn, the second largest
conference in Germany. This here at the left side is our summer camp, name taking sun, where
the trials come and then learn something like I think they are soldering things together and
then programming it. So now what is an offline finding system? It's basically you attach something
to something like a small tech, you attach it to your backpack, then you lose it and then you open
some app on your smartphone or on your laptop and then you can find it or search it or don't find it.
And the more technical offline finding thing. So the tech sent a signal via Bluetooth because
it's offline. So there isn't a connection between the tech and the internet. Then an app like a helper
app on your phone gets this Bluetooth signal and then says, hey, I found this tech there. And then I
as the owner can go on my smartphone, search for the tech and then my phone search in the
database for the tech. So how we got into offline finding. I'm very steady. So my scooter, like my
scooter to drive in the city got stolen and then I had a Samsung smart tech like an offline
finding tech attached to it. And then we drove to the approximate location and then with the feature
that we can send a signal to the tech and the tech response I'm here, we could see where the tech
was basically so what where signal was. And then we did three literation. So we went from multiple
sites to it. And then there was a signal at one point and then we got a scooter back. And also
there's our sketchy chef there. And he always loses stuff and wants to get it back or find it. So
offline finding basically has three components. But the tracking tokens, the small devices that
you attach to the things that you want to find, they aren't connected to the internet because
then it wouldn't be offline and sort of use like some like and then there are the smartphones or
some small helper devices. They get the signal from the tech and then send it to the internet. And
then there's obviously a server where the messages like I'm here and there is a pack are sent to
and then I can get them back from there. So there are obviously some challenges. Some are
privacy related like a stranger must not abuse the beacon for tracking over the long term. And
they should not identify the owners because then I could know where the stuff of some people is. And
the back end, the server couldn't identify the owners either because then I as the owner of the
server could identify the owners. And yeah. But some are also technical like the encryption
without knowing the receiver because then I can identify the owner then Bluetooth because of
the range and yeah because of Bluetooth. And then because of Bluetooth also the energy efficiency.
Yeah, because at one point we tried out in ESP. How long would it last? And I think we did it with
Shah 256 hashing and like lasted for a couple of hours. Because it's small and I think a couple of
hours aren't enough for checking device. Yeah, design overview.
All right. Thank you. Yeah. So after we somehow got snubbed by this by this topic around offline
finding how this works, of course we wanted to try how far we can get building such a system.
Of course, somewhat motivated by our grumpy, sorry, I mean, a sketchy, sketchy chef who asked,
hey, is there some system like this based on open hardware, open source? I'm not so very excited
about Apple controlling where I lose and find rediscover my stuff. So first, what we first
it was we looked at how the Samsung smart tech system worked, which is the sort of tech that
Pingu had attached to the scooter. And we found out that it sends these strange beacons of some
sort using Bluetooth low energy. I will come back to that in a minute. And in the course of time,
while we looked at how this works, we've it's more or less became obvious that actually this sort
of system is an enter and encrypted mailbox system, because there is an owner device and this has a
public key and yeah, what you can do with a public key, you can receive some sort of messages.
And there are helper devices that can see these beacons and more or less just send any sort of
message to the helper device. So if I lose something as the owner and let's say Pingu wants to help
me find it, then they walk around in the city and their smartphone receives the beacon signal and
now they somehow need to get the information back to me, telling me where they saw my beacon. And
that's where these texts come in and they are as probably as dumb as you can imagine, they just
send out a public key and yeah, so all the information you need to somehow get the location
sent back to me. It's a macro incident that these messages carry location information. We could
just as well put anything in there if any of you are into this sort of systems. Apple had a few
vulnerabilities discovered in their implementation. One of the most interesting ones in the recent
weeks was that people actually used the beacons themselves to transport key logger information
out of otherwise air-gapped environments. I think using your favorite search engine or the search
engine you distrust, least will bring some really interesting information up about this. So what we
really want to build is a mailbox system and some sort of key management system because that's the
really interesting part as far as I'm concerned, how we solve these privacy issues and some of the
technical issues with cryptography. So this is the big picture. If this works I can zoom around in
this a bit and now it shows that I should have used the headset. Can I do it with one hand? Yes,
I can. So here's the big picture and what you can see here is all the red circles are showing
secret keys that I use in the system, the green circles are showing public keys that I use in the
system. Let's get a short overview of how this works. So we have the owner device and we give the
owner device a sort of main key. This identifies the owner device and the easiest thing we could do
now is we could make this Bluetooth beacon and simply copy the public key of the owner onto that
beacon and attach it to some bag or scooter or some flash squirrel or whatever you don't want to
lose. So at this point we more or less are done with the mailbox part and with the encryption part
but we got into all the privacy troubles because what you now can do is you can follow the tech
around. It always broadcasts the same public key information. You can just walk around the city
and always rediscover where one person is moving and make a nice motion profile of this person.
Also you could discover several tokens that are linked to the same owner device and get the
information that all these tokens belong to the same owner. These are two of the most inherent
privacy issues that you obviously don't want to make when designing such a system. So the next
thing we do is we derive using hash based key derivation some keys or one key pair for each
token so that we can unlink the tokens from each other. And the rest of the system in case I think
many of you will have heard about this term a ratchet algorithm and the rest of the system more
or less is very close to what for example the signal messenger does with the scriptography. We
transfer this this key pair this device key pair to the tech and now we do one key derivation every
let's say 15 minutes at least that's what Apple does. And the interesting part here because I never
worked with cryptography on this level before is that now we can derive new key pairs on the
tech and it will send out another elliptic curve public key every 15 minutes. So we fix the privacy
issue of following someone around. Now you can follow someone for 15 minutes and after 15 minutes
you see another beacon and you cannot distinguish whether this is the same tech which rotated its
key pair or some other tech of another person. Yeah that's more or less the main secret of the
system and then if I find the tech I can send a message to the public key it is currently
broadcasting and there are some other things mixed in here but I don't want to go into too much
detail about this part right now. And the second secret is that when I try to retrieve my location
information that all the messages that other send to me I just ask the server for all the
information sent to all the public keys I know my tech will have generated within the time frame.
And this request can also be encrypted because we also use another set of keys so that the server
can also not find out that all these keys are linked to my device. They should have zero
knowledge about the ownership relation between the techs and the owners.
Okay our experiments are implemented in Rust. We have split it into the spot nuts crates.
Hazel OS is what is supposed to be running on the techs and the helper device in Rust based
mobile app and in case you happen to need or happen to find the time to review an implementation
of signals, X-Ed SDS implementation in Rust. We also factored out this crate so you can tell us
what obvious mistakes we made in the cryptography there if you like. And the JG crates are a
general implementation of this message of this mailbox system which can be used for the offline
finding system but actually for anything that is supposed to carry public key information to
someone and allow them to anonymously send back some sort of information. So what we have? We
have this implementation of this general JG key exchange and Maywork system with a library
usable as an alpha version and a small server implementation that actually does not care
whether it is used for offline finding or whatever other purpose. And we have an experimental
version of Hazel OS for ESP32 with the limitation that Pingu already mentioned that we get the ESP32
development board to run for something like five hours. So how long did you take to get your
scooter back? Did you manage to do it in five hours? I don't think so. Okay you have to be
quicker next time when you switch from some. Best thing so we can either fix the technical issue
or you can start a running career so whichever is easier. Okay so next things we want to do is
we want to find a decent microcontroller. I happened to give a Rust training last week and
one attendee told me this ESP32 this is nothing to do with microcontrollers. This is a toy. Get a
more hardcore microcontroller and I think this is what we will try. And for Hazel OS to this we
need to build an experimental companion app. Maybe design a nice PCB so it don't have to attach a
breadboard with a development board to your scooter or stuffed squirrel or whatever. And maybe we
can find others interested in open offline findings standard because Google and Apple and
Microsoft and you name it are working on something like this but of course it's not so very openly
developed. Spotnuts is a tinkering. Thank you for the talk. The question is how do you allow the
helper device to send the message to the owner device and at the exactly same time don't allow
some stranger to track the owner. Somehow at the feeling that at least one of my slides went
missing when refactoring the slide deck. There's back an infrastructure. One thing I mentioned is
JGD which is just a small mailbox server. It just has two API endpoints. One receives messages. It
does not care what these messages contain. They are just JSON encoded encrypted messages to the
public key we saw and the owner devices they just ask hey do you happen to have received any message
for this public key I think I might have had. So the thing here is you can actually even in the
Apple ecosystem you can ask the server for all messages you like. You can just send public keys
there and they will give you the information about all messages that were sent encrypted to this
public key. The nice thing is so you can download the whole database from Apple servers as well. The
nice thing is you can do anything with it because obviously you also need the second half of the
key pair. If you don't have it you get a nice bunch of random data. Over here. Hello. It's here. Over
here. Would it make sense to make this key rotation time period not fixed at 15 minutes because if I
was following a tag I could time the key rotation based on the period and then know that it was
rotated at the exact 15 minutes. Yes. Bit of silly question but have you considered Linux mobile
support for the helper device? Can you repeat the question please? Have you considered supporting Linux
mobile phones? Supporting mobile phones to carry the... Is it a part? That's running Linux instead of
Android or iOS. It's supposed to be a web application which will need web Bluetooth support in more
browsers than Google Chrome but actually there's this Rust library and it should be easy to use it in any
sort of app that you like on any platform. That's great. Thank you. Thank you again. Thank you.