[00:00.000 --> 00:09.680] All right, thanks everyone for coming so early. [00:09.680 --> 00:12.440] I admire your acudos for getting here so early. [00:12.440 --> 00:16.400] I'm very impressed to give up this early in the morning. [00:16.400 --> 00:18.160] I'm David Llewellyn-Jones. [00:18.160 --> 00:22.120] I work for a company called YOLLA O in Finland. [00:22.120 --> 00:26.280] We make an operating system called Selfish OS for mobile [00:26.280 --> 00:28.720] phones like this one. [00:28.720 --> 00:31.040] And today, I'm going to talk to you [00:31.040 --> 00:34.960] about COVID exposure notification. [00:34.960 --> 00:36.640] It's actually a project that was done. [00:36.640 --> 00:39.200] It was a personal project rather than something [00:39.200 --> 00:40.600] related to the company. [00:40.600 --> 00:42.720] But it was something that relates to Selfish OS. [00:42.720 --> 00:44.680] So hopefully, it all comes together [00:44.680 --> 00:49.280] in some kind of sensible way. [00:49.280 --> 00:53.840] OK, so exposure notification. [00:53.840 --> 00:55.600] It's almost certainly something that you've all [00:55.600 --> 00:57.840] come across individually. [00:57.840 --> 00:59.720] It essentially is the apps that people [00:59.720 --> 01:02.520] ran on their phones that did contact tracing. [01:02.520 --> 01:03.200] They pinged. [01:03.200 --> 01:06.880] They would give you an alert if you had been in contact [01:06.880 --> 01:11.520] with someone who had a COVID diagnosis. [01:11.520 --> 01:13.720] And I guess they were a big deal during COVID. [01:13.720 --> 01:17.480] I guess probably they're less of a big deal now. [01:17.480 --> 01:21.080] Although, I mean, I track the numbers on my phone [01:21.080 --> 01:22.360] while I'm at FOSDEM, and there's still [01:22.360 --> 01:23.920] a lot of people at FOSDEM using them. [01:23.920 --> 01:26.600] I can tell that. [01:26.600 --> 01:30.000] And the idea about this talk is that I [01:30.000 --> 01:35.000] want to talk about a very particular implementation of it. [01:35.000 --> 01:37.600] And that's the Google Apple exposure notification [01:37.600 --> 01:44.360] protocol, which was released, I guess, in April 2020. [01:44.360 --> 01:46.040] And it was an implementation that [01:46.040 --> 01:49.800] went out to Android devices and iPhones [01:49.800 --> 01:52.520] in order to basically provide the services needed [01:52.520 --> 01:57.120] in order to allow countries or medical organizations [01:57.120 --> 01:59.120] to build apps on top of it in order [01:59.120 --> 02:02.120] to perform this contact tracing process. [02:02.120 --> 02:03.400] And initially, when it was released, [02:03.400 --> 02:07.560] it was called, as you can see here, the Contact Tracing API. [02:07.560 --> 02:09.720] Later on, I guess that name was a little bit ambiguous, [02:09.720 --> 02:10.600] a little bit confusing. [02:10.600 --> 02:13.040] They changed it to the Google and Apple Exposure [02:13.040 --> 02:14.920] Notification API. [02:14.920 --> 02:16.880] I guess when people think about contact tracing, [02:16.880 --> 02:18.440] they think about maybe people phoning them up [02:18.440 --> 02:22.000] and asking them who they've been close to. [02:22.000 --> 02:24.160] But it is all the same thing. [02:24.160 --> 02:27.120] And this was the original specification. [02:27.120 --> 02:31.160] Like I say, the original one was released in April 2020. [02:31.160 --> 02:34.680] And it's made up of these three documents. [02:34.680 --> 02:36.280] It was initially this was version 1.1. [02:36.280 --> 02:39.000] I think they're up to version 1.4 or later now. [02:39.000 --> 02:41.040] There's been several versions that have come out since. [02:41.040 --> 02:44.520] So they've refined the protocol. [02:44.520 --> 02:47.880] And I guess it's worth putting it in a little bit of context. [02:47.880 --> 02:53.720] Back then, when this was released, there was no vaccine. [02:53.720 --> 02:56.200] So at that point in time, people thought [02:56.200 --> 02:59.080] that the World Health Organization predicted [02:59.080 --> 03:01.360] that vaccine was about 16 months away, [03:01.360 --> 03:03.320] but no one was entirely sure. [03:03.320 --> 03:06.240] In practice, the first vaccine was actually approved [03:06.240 --> 03:08.280] about nine months after this. [03:08.280 --> 03:10.920] But you can imagine that at this point in time, [03:10.920 --> 03:13.280] people were very worried about COVID. [03:13.280 --> 03:15.760] They were concerned about how it was going to be dealt with. [03:15.760 --> 03:20.440] And this was seen as kind of like a kind of one really [03:20.440 --> 03:23.360] quite exciting way to address the problem, [03:23.360 --> 03:25.960] given that there were so few tools at that time [03:25.960 --> 03:28.600] to deal with things. [03:28.600 --> 03:30.760] So let me just talk about these three documents. [03:30.760 --> 03:33.240] The first one on the left here is the Bluetooth specification. [03:33.240 --> 03:34.680] So it's made up of three parts. [03:34.680 --> 03:37.240] The Bluetooth specification, I'll [03:37.240 --> 03:39.480] talk about how it works in practice in a second or two. [03:39.480 --> 03:43.320] But it's sending beacons using Bluetooth from your phone. [03:43.320 --> 03:45.720] So the Bluetooth specification tells you essentially [03:45.720 --> 03:51.440] what how to pack the data into a Bluetooth beacon. [03:51.440 --> 03:55.840] And also other details like, for example, for privacy reasons, [03:55.840 --> 03:58.920] the information in the beacon changes periodically. [03:58.920 --> 04:01.040] And you have to change the MAC address of your phone [04:01.040 --> 04:04.880] in sequence with that in order to avoid privacy problems [04:04.880 --> 04:05.800] coming up. [04:05.800 --> 04:06.680] So that's that part. [04:06.680 --> 04:08.040] There's a cryptography specification, [04:08.040 --> 04:09.960] because the data that is sent in the beacons [04:09.960 --> 04:17.560] is cryptographically, I guess, hidden for privacy reasons. [04:17.560 --> 04:19.680] And then finally, there's the framework documentation, [04:19.680 --> 04:21.480] which is the API, which is really describing [04:21.480 --> 04:27.240] how organizations can make use of these tools. [04:27.240 --> 04:29.160] And I guess another important point about it [04:29.160 --> 04:32.520] is that there are two parts to the process. [04:32.520 --> 04:36.920] So Google and Apple released this API for all phones, [04:36.920 --> 04:38.240] but that's actually only half of it. [04:38.240 --> 04:39.920] That sends out the beacons. [04:39.920 --> 04:42.160] But as those beacons go around, eventually, [04:42.160 --> 04:45.480] you have to do something with that data that's being sent. [04:45.480 --> 04:47.000] And for that, you needed an app that [04:47.000 --> 04:50.760] was going to be written by a government organization, [04:50.760 --> 04:52.600] a government level organization. [04:52.600 --> 04:54.960] Like it could be a health organization. [04:54.960 --> 04:57.680] And Google and Apple had nothing to do with that. [04:57.680 --> 05:00.360] They said, you've got to write that part. [05:00.360 --> 05:03.240] We'll just give you this piece here. [05:03.240 --> 05:05.960] And the project that I'm essentially talking about [05:05.960 --> 05:08.960] is about a re-implementation of this for Salesforce. [05:08.960 --> 05:11.080] And so, of course, we had to also deal with those two parts [05:11.080 --> 05:12.000] separately. [05:12.000 --> 05:13.880] So this is only one part of it. [05:13.880 --> 05:15.680] All right, so one of the nice things about this protocol [05:15.680 --> 05:18.560] is it's actually heavily privacy-focused. [05:18.560 --> 05:20.040] So you can imagine also at that time, [05:20.040 --> 05:22.160] there was a lot of concern about the fact that you're [05:22.160 --> 05:24.080] sending beacons out on your mobile phone. [05:24.080 --> 05:25.920] There are privacy concerns about that. [05:25.920 --> 05:28.920] And the protocol tries to address that in a very careful way. [05:28.920 --> 05:32.320] And that's actually rather a nice feature about it. [05:32.320 --> 05:34.280] All right, so when it was released, [05:34.280 --> 05:37.280] this is actually a slide from Google and Apple. [05:37.280 --> 05:39.720] I did not draw these pictures. [05:39.720 --> 05:41.720] And this basically describes how it works. [05:41.720 --> 05:48.600] So very briefly, Alice and Bob are wandering around the world. [05:48.600 --> 05:51.160] They have the app installed on their device, [05:51.160 --> 05:52.440] sending out beacons. [05:52.440 --> 05:55.360] When they get into close contact with each other, [05:55.360 --> 05:57.360] their phones are communicating via Bluetooth. [05:57.360 --> 05:58.640] Well, they're not exactly communicating. [05:58.640 --> 05:59.920] They're listening for these beacons. [05:59.920 --> 06:02.680] And they're storing the beacons that they hear. [06:02.680 --> 06:04.440] OK, so that's step one. [06:04.440 --> 06:06.040] So when they get close to each other, [06:06.040 --> 06:09.280] Bob's beacons get stored on Alice's phone. [06:09.280 --> 06:13.440] Later on, Bob turns out to have COVID. [06:13.440 --> 06:16.720] And he gets a positive diagnosis. [06:16.720 --> 06:19.320] And he basically goes to his doctor. [06:19.320 --> 06:21.760] He gets an official diagnosis. [06:21.760 --> 06:25.160] And he puts that information into the app. [06:25.160 --> 06:29.640] And he voluntarily uploads data about his beacons [06:29.640 --> 06:32.680] to some cloud server that's run by a government or health [06:32.680 --> 06:34.200] authority. [06:34.200 --> 06:37.000] And he uploads 14 days worth of his beacons. [06:37.000 --> 06:38.760] So in practice, these beacons are actually [06:38.760 --> 06:41.800] generated using a cryptographic process. [06:41.800 --> 06:44.680] So you don't have to store on your phone 14 days worth. [06:44.680 --> 06:49.560] You just store essentially the seed that generates them. [06:49.560 --> 06:50.840] But they change frequently. [06:50.840 --> 06:53.360] So they change between every 10 and 20 minutes. [06:53.360 --> 06:56.240] It's actually a random interval somewhere between those two. [06:56.240 --> 06:59.040] In order to prevent people from tracking you, [06:59.040 --> 06:59.960] they are unlinkable. [06:59.960 --> 07:02.080] So when they change, you can't link one from the other. [07:02.080 --> 07:05.240] And that means that there's a limit to how far you can track [07:05.240 --> 07:06.240] people. [07:06.240 --> 07:10.240] But Bob can regenerate those keys to upload to the server. [07:10.240 --> 07:13.440] Alice just gets random, essentially random numbers. [07:13.440 --> 07:16.400] She can't tell what those numbers are, essentially, [07:16.400 --> 07:17.680] or what they mean. [07:17.680 --> 07:20.320] So then later on, so I guess the next day. [07:20.320 --> 07:21.360] So this happens later. [07:21.360 --> 07:24.040] But then the next day or later on that day, [07:24.040 --> 07:25.680] the keys get uploaded to the server. [07:25.680 --> 07:27.760] And the server then dumps all of the keys [07:27.760 --> 07:31.320] that it receives back to every single device out there. [07:31.320 --> 07:33.560] It downloads these keys to all of their phones, [07:33.560 --> 07:34.280] to everyone's phones. [07:34.280 --> 07:36.360] So it downloads them to Alice's phone. [07:36.360 --> 07:40.240] Alice then, Alice's phone then matches them up [07:40.240 --> 07:42.640] with the keys that she captured along the way. [07:42.640 --> 07:45.520] And she then knows that she was in contact with someone who [07:45.520 --> 07:47.920] has a positive diagnosis for COVID. [07:47.920 --> 07:49.640] She gets an alert on her app. [07:49.640 --> 07:51.520] And then her life is held for two weeks. [07:51.520 --> 07:53.360] That's basically the process. [07:53.360 --> 07:56.480] She has to isolate and go through all of the steps [07:56.480 --> 07:58.560] that you have to go through. [07:58.560 --> 08:00.480] We don't know whether Alice has COVID or not. [08:00.480 --> 08:01.560] The idea is she might have. [08:08.920 --> 08:11.160] So at the time, it was a kind of a big deal, like I said, [08:11.160 --> 08:13.000] because it was seen as a kind of a white horse [08:13.000 --> 08:14.760] to get us out of the problem of COVID. [08:14.760 --> 08:16.120] At that time, everyone was focused [08:16.120 --> 08:18.680] on this thing called the R number, [08:18.680 --> 08:21.440] where the idea being that the R number was the number of people [08:21.440 --> 08:24.160] you essentially go on to reinfect after you have COVID. [08:24.160 --> 08:25.760] And the objective was to keep that down [08:25.760 --> 08:27.400] if the R number was above one. [08:27.400 --> 08:29.640] There was an exponential growth in the number of people [08:29.640 --> 08:30.320] that would have COVID. [08:30.320 --> 08:33.320] If it was below one, then it would be reducing numbers. [08:33.320 --> 08:35.200] And the aim of all of this process [08:35.200 --> 08:37.600] was to keep people isolated if they might have COVID [08:37.600 --> 08:40.760] in order to reduce that R number and prevent it [08:40.760 --> 08:43.520] from spreading. [08:43.520 --> 08:45.720] Google and Apple came out with this protocol [08:45.720 --> 08:47.560] because they immediately identified [08:47.560 --> 08:49.920] that there were going to be serious risks involved [08:49.920 --> 08:53.120] in tracking people with their mobile phone to do this stuff. [08:53.120 --> 08:58.240] So they knew, for example, that there [08:58.240 --> 09:00.000] were privacy risks associated with it. [09:00.000 --> 09:02.160] They knew that if they let governments go ahead and just [09:02.160 --> 09:03.880] do it, they would just collect everyone's data [09:03.880 --> 09:04.680] was the concern. [09:04.680 --> 09:05.640] Or some of them might do. [09:05.640 --> 09:08.000] Maybe not all of them, but some of them might. [09:08.000 --> 09:10.920] They also knew that sending out Bluetooth beacons. [09:10.920 --> 09:11.960] Sorry, you have a question. [09:11.960 --> 09:12.460] Sorry. [09:12.460 --> 09:16.720] If Alice receives keys from infected people, [09:16.720 --> 09:19.880] can she not match that with the relative time [09:19.880 --> 09:22.320] she got the ring detected? [09:22.320 --> 09:25.920] So she knows that the key is the one [09:25.920 --> 09:28.680] that she found two days ago, and two days ago was the time [09:28.680 --> 09:30.160] she met up. [09:30.160 --> 09:32.800] So is there a problem? [09:32.800 --> 09:34.560] So I guess I should make clear. [09:34.560 --> 09:37.920] So I mean, there are privacy controls in this, [09:37.920 --> 09:41.040] but they're not complete, right? [09:41.040 --> 09:42.880] And the idea is that essentially, at the point [09:42.880 --> 09:45.560] when you upload your keys, you've got your diagnosis, [09:45.560 --> 09:49.920] that's essentially when some of your privacy is lost. [09:49.920 --> 09:52.720] So but you voluntarily choose to do that. [09:52.720 --> 09:54.320] That doesn't happen automatically. [09:54.320 --> 09:55.400] Does that make sense? [09:55.400 --> 09:56.640] OK. [09:56.640 --> 09:59.000] But it's a good point. [09:59.000 --> 10:00.400] So yes, but Google and Apple were [10:00.400 --> 10:02.280] concerned about the privacy implications [10:02.280 --> 10:04.760] because it reflects badly on their systems, right, [10:04.760 --> 10:07.640] if their phones are doing this stuff. [10:07.640 --> 10:09.640] Also, the power consumption is a problem, [10:09.640 --> 10:12.840] so they didn't want government organizations to deploy apps [10:12.840 --> 10:15.760] that would have problems in terms of battery on their phone. [10:15.760 --> 10:17.160] And they also wanted it to be effective, [10:17.160 --> 10:19.640] because if it's not effective, that also reflects badly on them. [10:19.640 --> 10:21.320] So they identified that there were all these risks [10:21.320 --> 10:23.040] that they wanted to avoid. [10:23.040 --> 10:25.920] And even at that time, the APIs that Google and Apple [10:25.920 --> 10:28.560] provide prevented people from doing this kind of stuff [10:28.560 --> 10:29.920] for precisely those reasons. [10:29.920 --> 10:31.480] Not, I mean, it was nothing to do with COVID. [10:31.480 --> 10:32.960] I mean, they've always just prevented you [10:32.960 --> 10:34.720] from sending out beacons and collecting beacons [10:34.720 --> 10:37.440] like this in a sensible way. [10:37.440 --> 10:39.360] Primarily for power consumption reasons, right? [10:39.360 --> 10:41.120] They don't want people draining the battery [10:41.120 --> 10:44.720] of the phone to do that. [10:44.720 --> 10:50.120] So my concern as a YOLO employee was that, well, [10:50.120 --> 10:52.240] Google and Apple have released this stuff. [10:52.240 --> 10:54.320] It's probably going to get quite popular. [10:54.320 --> 10:56.120] And I didn't want selfish users who [10:56.120 --> 10:58.080] are using an alternative operating system, which [10:58.080 --> 11:00.680] is not Android or iOS, to essentially get [11:00.680 --> 11:03.080] sidelined by this process and be left in a situation where [11:03.080 --> 11:07.080] they can't basically participate in this scheme. [11:07.080 --> 11:10.640] So the objective then was to create an open source [11:10.640 --> 11:13.680] implementation that could be used on other sorts of phones. [11:13.680 --> 11:14.960] So this is Selfish. [11:14.960 --> 11:18.920] Selfish OS is a Linux distribution [11:18.920 --> 11:23.880] that has a cute UI layer on top of it. [11:23.880 --> 11:27.720] And it is sold by, well, it's released by YOLO. [11:27.720 --> 11:30.520] And then there is also a paid-for license [11:30.520 --> 11:35.160] that you can get for it to get additions on top of that. [11:35.160 --> 11:37.320] So yeah, the idea was to write an application that essentially [11:37.320 --> 11:38.960] mimicked that functionality. [11:38.960 --> 11:46.320] And so this was the app that myself and Oscar Rosler, [11:46.320 --> 11:47.640] I don't know if Oscar is here. [11:47.640 --> 11:49.560] Is Oscar in there? [11:49.560 --> 11:50.120] I guess not. [11:50.120 --> 11:51.040] He said he might turn out. [11:51.040 --> 11:53.120] But I guess he's not here at the moment. [11:53.120 --> 11:57.760] So we developed this together as a kind of a personal project. [11:57.760 --> 11:59.800] And you can see here's the process. [11:59.800 --> 12:01.760] This was like the first time that we actually [12:01.760 --> 12:04.240] got Beacons appearing on the device. [12:04.240 --> 12:06.280] So that was a kind of an exciting moment. [12:06.280 --> 12:08.680] And there are, I guess, like several steps. [12:08.680 --> 12:09.840] There's the collecting Beacons. [12:09.840 --> 12:11.240] There's sending Beacons. [12:11.240 --> 12:13.040] And there's also uploading your data [12:13.040 --> 12:15.200] if you get a positive diagnosis. [12:15.200 --> 12:17.120] So we implemented all those parts. [12:17.120 --> 12:19.400] And we tried to implement it in the same way [12:19.400 --> 12:21.480] that the Google and Apple did in terms of structure. [12:21.480 --> 12:24.000] So you have a service underneath that sends out these Beacons. [12:24.000 --> 12:27.360] And then we had a separate app, which was the equivalent [12:27.360 --> 12:30.480] to the app that had to be developed by governments. [12:30.480 --> 12:31.960] And in theory, other people could [12:31.960 --> 12:34.200] have developed other apps on top of the service. [12:34.200 --> 12:35.760] I don't think that ever happened. [12:35.760 --> 12:37.600] We only implemented one version. [12:37.600 --> 12:40.840] And the version we implemented was the COVID WARN app, [12:40.840 --> 12:42.840] which was Germany's implementation. [12:42.840 --> 12:46.160] And the reason we did that was because it was both very early [12:46.160 --> 12:48.600] in the process in terms of being deployed. [12:48.600 --> 12:50.160] But it was also incredibly open. [12:50.160 --> 12:51.680] It was an open source deployment. [12:51.680 --> 12:53.840] And they were really committed to doing it as an open source [12:53.840 --> 12:57.280] product, I mean, an open source project. [12:57.280 --> 12:59.640] Not just an open source chunk of code, [12:59.640 --> 13:04.000] but the development model was also supposed to be open. [13:04.000 --> 13:06.520] It was actually developed by SAP and Deutsche Telekom. [13:06.520 --> 13:08.600] I think my understanding is that SAP did all the code [13:08.600 --> 13:09.400] development. [13:09.400 --> 13:12.480] Deutsche Telekom provided the infrastructure for it, [13:12.480 --> 13:16.800] the back-end service infrastructure. [13:16.800 --> 13:19.880] So like I say, we did this independently of YOLA. [13:19.880 --> 13:21.760] For those interested, it was Qt-based. [13:21.760 --> 13:24.440] It used Bluezy for the Bluetooth stack [13:24.440 --> 13:28.240] and OpenSSL for the crypto stuff. [13:28.240 --> 13:31.760] And this was the timeline, essentially, [13:31.760 --> 13:32.800] that we worked to. [13:32.800 --> 13:35.240] Or we didn't work to that we ended up with, I guess. [13:35.240 --> 13:37.920] So on the 10th of April, that's when the Google and Apple [13:37.920 --> 13:39.800] specs first came out. [13:39.800 --> 13:42.000] We released our first version implementation [13:42.000 --> 13:43.520] of those specs on the 19th. [13:43.520 --> 13:46.560] So that was quite soon afterwards. [13:46.560 --> 13:48.360] Then the COVID WARN app from Germany [13:48.360 --> 13:50.040] came out on the 16th of June. [13:50.040 --> 13:53.440] And then on the 30th of June, so quite a few weeks later, [13:53.440 --> 13:57.880] but we got our first app that had equivalent functionality. [13:57.880 --> 14:01.800] So we were pretty happy with that. [14:01.800 --> 14:05.120] We thought that could be considered a success. [14:05.120 --> 14:08.120] But it was only possible because the specifications [14:08.120 --> 14:10.440] that Google and Apple rolled out were very good. [14:10.440 --> 14:12.840] There were clear specifications that you could re-implement. [14:12.840 --> 14:18.680] And the COVID WARN app team also did a lot of good work [14:18.680 --> 14:21.040] in releasing source code very early. [14:21.040 --> 14:22.600] They had a huge amount of documentation [14:22.600 --> 14:24.840] that was all on GitHub. [14:24.840 --> 14:33.320] They even had, I guess, a CCC evaluation [14:33.320 --> 14:35.560] of the project in terms of its privacy. [14:35.560 --> 14:38.000] So they were really quite committed to being open [14:38.000 --> 14:40.600] with their stuff. [14:40.600 --> 14:47.120] In practice, what we found was, so they released some reference [14:47.120 --> 14:49.920] versions of the servers. [14:49.920 --> 14:51.800] And we found that really helpful. [14:51.800 --> 14:52.800] That was really important. [14:52.800 --> 14:55.680] They also released the source code for their app [14:55.680 --> 14:57.400] that we re-implemented. [14:57.400 --> 14:59.840] We actually ultimately found that that was not so useful. [14:59.840 --> 15:01.560] It was useful for some edge cases, [15:01.560 --> 15:03.720] but really it was a server deployment that was really [15:03.720 --> 15:06.440] helpful for us. [15:06.440 --> 15:11.560] So I guess the real reason that I wanted to talk today [15:11.560 --> 15:14.800] was to talk about our experience working with these companies [15:14.800 --> 15:18.680] as small developers, as independent developers. [15:18.680 --> 15:21.560] And I guess it was very much as you might expect in terms [15:21.560 --> 15:22.760] of Google and Apple. [15:22.760 --> 15:24.840] They produced, I mean, they're phenomenal at producing [15:24.840 --> 15:26.320] good specifications and good APIs. [15:26.320 --> 15:27.160] That's what they do. [15:27.160 --> 15:28.000] That's their bread and butter. [15:28.000 --> 15:29.720] They're absolutely flummole at it. [15:29.720 --> 15:31.840] But zero engagement. [15:31.840 --> 15:34.640] So we tried to contact them and talk to them about their stuff [15:34.640 --> 15:38.360] to try and get information or to get help. [15:38.360 --> 15:39.560] Nothing at all. [15:39.560 --> 15:41.160] Which is, I guess, not surprising, right? [15:41.160 --> 15:42.440] They're a massive company. [15:42.440 --> 15:44.120] There's no reason why particularly they would. [15:47.600 --> 15:49.960] They promised test data to test stuff against, [15:49.960 --> 15:51.200] and that never materialized. [15:51.200 --> 15:55.000] So there were also some bumps in the road for us. [15:55.000 --> 15:57.400] And although they did release the source code, [15:57.400 --> 16:00.600] they released the source code for a reference app [16:00.600 --> 16:02.760] quite early, but they didn't release the source code [16:02.760 --> 16:04.960] for the underlying system, the Bluetooth system, [16:04.960 --> 16:10.920] until somewhat later, until I guess that was late in July. [16:10.920 --> 16:13.840] So for us, that was already too late. [16:13.840 --> 16:17.040] So working with them, like I say, there were good points, [16:17.040 --> 16:19.160] but they were not really having an open source development [16:19.160 --> 16:21.640] model. [16:21.640 --> 16:24.520] For SAP, working with the SAP code [16:24.520 --> 16:25.520] was a little bit different. [16:25.520 --> 16:29.880] So this is essentially the SAP implementation [16:29.880 --> 16:33.400] that they developed on top of the Google Apple notification [16:33.400 --> 16:34.120] protocol. [16:34.120 --> 16:36.960] So this is the app, and this is the server [16:36.960 --> 16:39.440] back-end on the left-hand side. [16:39.440 --> 16:43.240] And the main parts of it are this verification server, [16:43.240 --> 16:45.480] and from our point of view, and the Corona WARN app server. [16:45.480 --> 16:48.160] So essentially what they do, you have to have a download [16:48.160 --> 16:49.720] server to download all of these keys that [16:49.720 --> 16:53.400] are going to Alice's phone and to everybody's phone. [16:53.400 --> 16:55.960] That happened in the Corona WARN app server. [16:55.960 --> 16:57.360] You have to have a verification server [16:57.360 --> 17:00.360] so that when you get a positive diagnosis, [17:00.360 --> 17:04.360] you can upload it in a secure way with your identity, [17:04.360 --> 17:06.640] essentially proving that, not with your identity, [17:06.640 --> 17:08.520] but proving that you've got a positive diagnosis [17:08.520 --> 17:10.120] from your doctor. [17:10.120 --> 17:13.000] And then you need an upload server to upload your keys to. [17:13.000 --> 17:15.120] And that happened also in the Corona WARN app server. [17:15.120 --> 17:17.760] So in practice, this was actually two servers, not one. [17:17.760 --> 17:19.800] So those were the two bits we were interesting in, [17:19.800 --> 17:23.080] and we had to deploy them to test against. [17:23.080 --> 17:24.840] We couldn't deploy the whole lot. [17:24.840 --> 17:26.840] That was going to be impossible. [17:26.840 --> 17:29.800] That was not really a plausible thing to do. [17:29.800 --> 17:31.440] But they did have a reference implementation [17:31.440 --> 17:32.640] of these two servers. [17:32.640 --> 17:35.160] And so we tried to deploy, we tried to deploy, well, [17:35.160 --> 17:38.560] so we did end up deploying those to an AWS server [17:38.560 --> 17:40.640] and testing our code against those. [17:40.640 --> 17:43.280] And what we found was that the download server implementation [17:43.280 --> 17:44.400] was a little bit broken. [17:44.400 --> 17:47.840] We had to fix it, but it was otherwise pretty good. [17:47.840 --> 17:50.480] There were just some glitches. [17:50.480 --> 17:53.680] Whereas the verification server and the upload server [17:53.680 --> 17:54.680] were pretty minimal. [17:54.680 --> 17:57.520] So we actually essentially had to re-implement parts of those [17:57.520 --> 18:01.480] based on the specification in order to do our testing. [18:01.480 --> 18:03.160] But they did release a lot of code with this stuff. [18:03.160 --> 18:05.240] And the fact that they had these reference servers [18:05.240 --> 18:08.520] was absolutely key in allowing us to re-implement [18:08.520 --> 18:11.560] the protocol. [18:11.560 --> 18:16.360] So our overall experience was that they [18:16.360 --> 18:18.320] produced lots of code and lots of documentation, [18:18.320 --> 18:20.040] which was really good. [18:20.040 --> 18:24.720] They were one of the earliest teams [18:24.720 --> 18:27.280] to get this stuff out into the wild, [18:27.280 --> 18:28.920] which was in terms of source code, which [18:28.920 --> 18:31.400] was also phenomenally good. [18:31.400 --> 18:33.920] They worked through GitHub issues. [18:33.920 --> 18:36.720] So when we found problems, we would submit pull requests [18:36.720 --> 18:39.640] to try and fix the code. [18:39.640 --> 18:41.520] And it was clear that they were trying [18:41.520 --> 18:43.360] to engage with that process. [18:43.360 --> 18:45.840] But in practice, they were just taking way too long. [18:45.840 --> 18:48.640] So by the time, so for example, we [18:48.640 --> 18:53.440] would fix something in one of the reference servers. [18:53.440 --> 18:57.880] And people would be using our PR code to deploy elsewhere. [18:57.880 --> 18:59.960] But by the time they got to actually thinking [18:59.960 --> 19:02.440] about integrating it, it was already too stale. [19:02.440 --> 19:04.720] It was no longer useful. [19:04.720 --> 19:06.160] So they were clearly trying to do that. [19:06.160 --> 19:11.000] But I guess it was just a question of time. [19:11.000 --> 19:12.680] So as a consequence, some of the code [19:12.680 --> 19:13.960] was left broken for quite a long time, [19:13.960 --> 19:15.880] even though there were fixes that they could have just [19:15.880 --> 19:18.680] integrated. [19:18.680 --> 19:20.240] Some of the reference implementations [19:20.240 --> 19:22.440] differed quite significantly from reality [19:22.440 --> 19:24.560] in terms of the servers. [19:24.560 --> 19:26.600] But there was something to work with, [19:26.600 --> 19:28.960] and that was really helpful. [19:28.960 --> 19:30.480] And so our overall impression was [19:30.480 --> 19:32.280] that they had this real commitment to openness. [19:32.280 --> 19:34.320] They were genuinely committed to doing this well. [19:34.320 --> 19:36.560] They had an open development process. [19:36.560 --> 19:38.600] But the team were just a bit overwhelmed [19:38.600 --> 19:40.320] with the amount of effort involved in doing it. [19:40.320 --> 19:43.400] That was the impression we got from the outside. [19:43.400 --> 19:45.400] So we were very impressed by this. [19:45.400 --> 19:48.040] But obviously, there's obviously scope [19:48.040 --> 19:52.520] for learning there in terms of how to do it better. [19:52.520 --> 19:56.680] So my overall reflections on how things went [19:56.680 --> 20:01.960] were that at the time, I think again, [20:01.960 --> 20:03.240] you have to put this into context, [20:03.240 --> 20:06.600] there was huge pressure on governments all over the world [20:06.600 --> 20:09.560] to get this stuff deployed. [20:09.560 --> 20:12.720] So the first deployment was in South Korea, [20:12.720 --> 20:15.000] was I think the COVID 100 meter app. [20:15.000 --> 20:16.200] And that went out really early. [20:16.200 --> 20:17.200] And as soon as that went out, [20:17.200 --> 20:21.880] and that used GPS positioning, GNSS, [20:21.880 --> 20:25.320] so that it uploaded people's locations to the cloud, [20:25.320 --> 20:29.680] I mean, essentially zero privacy in that situation. [20:29.680 --> 20:31.960] But as soon as that was out, all other governments, [20:31.960 --> 20:34.480] there was a lot of pressure for them to do something similar. [20:34.480 --> 20:37.120] There was a huge rush to deploy this stuff. [20:37.120 --> 20:38.280] And so there was a lot of pressure, [20:38.280 --> 20:41.480] and there was a lot of competing requirements. [20:41.480 --> 20:44.240] The requirements for openness was just one requirement. [20:44.240 --> 20:47.120] There was also requirements for efficacy, [20:47.120 --> 20:51.200] requirements for speed, requirements in terms of privacy. [20:51.200 --> 20:52.520] All of these things combined [20:52.520 --> 20:54.840] are actually quite challenging to get right. [20:56.560 --> 21:01.360] And both Google and Apple and the CWA team, the SAP team, [21:01.360 --> 21:03.160] the impression I got was they really got that. [21:03.160 --> 21:05.040] They really understood these competing impressions, [21:05.040 --> 21:07.480] but they also understood the need for openness. [21:07.480 --> 21:10.040] They understood that in order for people, [21:10.040 --> 21:11.520] not just for them to do it well, [21:11.520 --> 21:13.960] but for people to trust the apps that they were running, [21:13.960 --> 21:15.560] they also needed to be open about it, [21:15.560 --> 21:17.280] and that's one of the reasons they did it. [21:17.280 --> 21:20.760] And in that sense, I think they were pretty successful. [21:20.760 --> 21:22.480] Like I say, there was scope for improvement. [21:22.480 --> 21:26.520] With, you know, there was, as I said, with Google and Apple, [21:26.520 --> 21:29.000] there was zero inward flow of information. [21:29.000 --> 21:31.360] With SAP, there was, and so with both teams, [21:31.360 --> 21:32.800] there was a lot of outward flow of information [21:32.800 --> 21:34.160] that was excellent. [21:34.160 --> 21:36.160] With SAP, there was some inward flow, [21:36.160 --> 21:39.480] but not as much as perhaps we as independent developers [21:39.480 --> 21:40.480] would have liked. [21:40.480 --> 21:44.400] And I guess more generally, what we kind of felt was [21:44.400 --> 21:47.720] that there was this commitment to openness, [21:47.720 --> 21:50.200] but not necessarily a full appreciation [21:50.200 --> 21:53.240] of how much effort was involved in that process, [21:53.240 --> 21:55.080] in getting that process to work really well [21:55.080 --> 21:58.280] and effectively for the companies involved. [21:58.280 --> 22:01.320] So, for example, and I guess I should say [22:01.320 --> 22:03.640] that YOLA also works with a lot of open source code, [22:03.640 --> 22:05.800] so we also have an open development process [22:05.800 --> 22:06.800] for a lot of our stuff. [22:06.800 --> 22:08.320] And we have the same challenges, right? [22:08.320 --> 22:13.560] So, people provide PRs for us to integrate into the system, [22:13.560 --> 22:15.760] and knowing what is good to integrate [22:15.760 --> 22:18.120] and what is bad to integrate takes a lot of time and effort, [22:18.120 --> 22:19.280] right? [22:19.280 --> 22:22.240] It sounds like you're getting free implementation, [22:22.240 --> 22:23.480] which I guess in some sense you are, [22:23.480 --> 22:26.360] but there's still a lot of effort involved in that process. [22:26.360 --> 22:28.160] And like I said, I guess with this process [22:28.160 --> 22:30.720] that we as independent developers experienced [22:30.720 --> 22:33.960] from the outside, it felt like that perhaps [22:33.960 --> 22:37.400] these other competing things were impacting on the effort [22:37.400 --> 22:41.480] that they could put into that process. [22:41.480 --> 22:44.320] So yeah, so overall, given that these organizations had [22:44.320 --> 22:47.120] absolutely no duty towards us as independent developers, [22:47.120 --> 22:50.720] we were kind of very happy that we [22:50.720 --> 22:52.280] could work with the things that they'd given us [22:52.280 --> 22:54.280] and that they were actually high quality. [22:54.280 --> 22:55.840] So we're very pleased with how it went, [22:55.840 --> 22:57.600] but like I say, we felt that there was definitely [22:57.600 --> 23:00.040] some room for improvement. [23:00.040 --> 23:04.320] OK, so that's my summary of how things went. [23:04.320 --> 23:05.840] Thanks for taking the time to listen. [23:05.840 --> 23:08.200] There's some links in case you're interested in more. [23:08.200 --> 23:11.480] We are actually on the Linux on mobile to stand in Building [23:11.480 --> 23:13.480] K up the top of the stairs, so if you [23:13.480 --> 23:15.160] want to talk more about it later, [23:15.160 --> 23:18.480] please come and visit us and we'd be happy to do that. [23:18.480 --> 23:19.480] Thanks very much. [23:19.480 --> 23:28.480] Is there time for questions? [23:28.480 --> 23:30.640] Yeah, there's five minutes for questions. [23:30.640 --> 23:33.080] We don't have a microphone. [23:33.080 --> 23:36.080] Yeah, you speak up and have to repeat the questions. [23:36.080 --> 23:36.680] OK. [23:36.680 --> 23:38.000] Especially for the live stream. [23:38.000 --> 23:39.000] Right, understood. [23:41.640 --> 23:44.360] What's your experience with the, like, [23:44.360 --> 23:46.720] they used rootin for the proximity, [23:46.720 --> 23:49.280] but there was some discussion like either you [23:49.280 --> 23:52.760] would get too much notification, contact notification, [23:52.760 --> 23:56.040] because it was too wide, but it would get too little, [23:56.040 --> 23:58.320] because it was too restricted. [23:58.320 --> 24:01.280] Right, so your question is, what's [24:01.280 --> 24:03.720] my experience with the Bluetooth parts, [24:03.720 --> 24:06.320] the Bluetooth notifications? [24:06.320 --> 24:08.360] Because as you were saying, at the time, [24:08.360 --> 24:10.200] there was a lot of discussion about whether it [24:10.200 --> 24:13.480] was going to give enough accuracy in terms of proximity [24:13.480 --> 24:14.320] detection. [24:14.320 --> 24:18.560] So the key thing with the COVID notifications, of course, [24:18.560 --> 24:22.040] is that there was this two meter, [24:22.040 --> 24:24.080] there was always this, like, or some distance [24:24.080 --> 24:26.800] that you were supposed to stay away from people. [24:26.800 --> 24:29.360] And so in terms of the Bluetooth, [24:29.360 --> 24:31.120] you wanted to try and mimic that process. [24:31.120 --> 24:32.560] You wanted to accurately distinguish [24:32.560 --> 24:35.200] that someone was within two meters or some distance [24:35.200 --> 24:36.880] or outside that distance. [24:36.880 --> 24:40.280] That's essentially what you're talking about in terms of proximity. [24:40.280 --> 24:42.200] So it's interesting, actually. [24:42.200 --> 24:45.400] So when we started out the process, [24:45.400 --> 24:49.680] the protocol didn't specify greatly how to tackle that. [24:49.680 --> 24:51.600] So what happened, what actually Google and Apple did, [24:51.600 --> 24:54.440] is they set a bunch of risk parameters [24:54.440 --> 24:56.720] in their implementation that you could tweak, [24:56.720 --> 24:58.080] that you could change. [24:58.080 --> 24:59.840] And they came from the download server. [24:59.840 --> 25:02.960] They were set by the government organizations, [25:02.960 --> 25:04.560] not by Google and Apple. [25:04.560 --> 25:08.840] And they basically said, they gave weightings. [25:08.840 --> 25:11.960] How much risk do you associate with a power level [25:11.960 --> 25:13.320] of this for the Bluetooth? [25:13.320 --> 25:15.600] And how much risk association do you [25:15.600 --> 25:18.560] put with how long that association went on for? [25:18.560 --> 25:20.240] Like, is it five minutes or 10 minutes [25:20.240 --> 25:22.960] that they're within range of this distance? [25:22.960 --> 25:24.960] So you could tweak those parameters. [25:24.960 --> 25:28.920] And later on in the process, Google actually [25:28.920 --> 25:31.160] released a lot of information that [25:31.160 --> 25:35.920] provided essentially weightings for some of the parameters [25:35.920 --> 25:38.520] for almost every phone on the market. [25:38.520 --> 25:40.360] So they had a process where they would basically [25:40.360 --> 25:43.640] put two phones on stands next to each other [25:43.640 --> 25:46.920] and measure the power consumption that came through. [25:46.920 --> 25:49.960] And then based on which phone was communicating [25:49.960 --> 25:52.360] with which other phone, they would then [25:52.360 --> 25:53.920] tweak the parameters to try and make [25:53.920 --> 25:57.280] that proximity more effective. [25:57.280 --> 25:59.320] So we use those parameters. [25:59.320 --> 26:02.640] But I have to say, we were not in a position [26:02.640 --> 26:05.080] to check whether or not they were working effectively. [26:05.080 --> 26:08.440] Because what's happening behind the scene is quite opaque. [26:08.440 --> 26:10.880] We're sending data up to a server. [26:10.880 --> 26:12.440] And the whole point is that you then [26:12.440 --> 26:14.560] know nothing about what happens after that [26:14.560 --> 26:16.480] because of the privacy aspects. [26:16.480 --> 26:18.400] So it's actually very hard for us [26:18.400 --> 26:22.320] to judge whether or not it was working well or working badly. [26:22.320 --> 26:25.120] But it was clear that there was a lot of work going on [26:25.120 --> 26:28.200] to get it to work well in the background. [26:28.200 --> 26:30.720] My more general experience with Bluetooth [26:30.720 --> 26:32.960] is it's not great at this stuff. [26:32.960 --> 26:35.800] If there's a person between you and another person, [26:35.800 --> 26:39.160] then water is a big problem for signal propagation. [26:39.160 --> 26:42.760] So stuff that's in between causes a lot of dampening [26:42.760 --> 26:44.040] of the signal. [26:44.040 --> 26:47.520] So it's quite tough with the impression I got. [26:47.520 --> 26:49.240] But I'm not an expert in that, so. [26:49.240 --> 26:50.240] I'm neither. [26:50.240 --> 26:52.240] OK. [26:52.240 --> 26:53.740] OK. [26:53.740 --> 26:57.720] I'm curious, a lot of governments [26:57.720 --> 26:59.720] implement solutions over proprietary solutions. [26:59.720 --> 27:02.000] Or do you have an implemented open source solution? [27:02.000 --> 27:04.800] So I'm just going to come a bit closer so I can hear from you. [27:04.800 --> 27:07.960] A few governments that implemented open source solutions, [27:07.960 --> 27:08.600] but not many. [27:08.600 --> 27:11.360] Most have been implemented proprietary solutions. [27:11.360 --> 27:12.720] But I don't know of any government [27:12.720 --> 27:14.800] that re-implemented another government's open source [27:14.800 --> 27:15.640] product. [27:15.640 --> 27:17.800] Was there any collaboration between governments you're [27:17.800 --> 27:20.840] aware of, or even between projects of government? [27:20.840 --> 27:23.840] You see that one of the projects that have never been [27:23.840 --> 27:27.560] between you and that physical failure in the resource [27:27.560 --> 27:28.840] can't repeat the process. [27:28.840 --> 27:30.840] And share it with the government. [27:30.840 --> 27:33.760] And it's distributed to governments of limited value. [27:33.760 --> 27:34.260] Right. [27:34.260 --> 27:34.760] OK. [27:34.760 --> 27:37.080] So the question was, was there any evidence [27:37.080 --> 27:38.960] that all of this open source release of code [27:38.960 --> 27:41.760] resulted in reuse of code by other governments? [27:41.760 --> 27:44.120] That's essentially your question, right? [27:44.120 --> 27:45.200] And I think there was. [27:45.200 --> 27:47.120] So I can't think of specific instances. [27:47.120 --> 27:49.200] But my understanding was that people [27:49.200 --> 27:51.360] did take the COVID WARN app code and use it [27:51.360 --> 27:52.360] in their implementations. [27:52.360 --> 27:54.520] Or perhaps more generally, I think [27:54.520 --> 27:57.200] people definitely took the Google reference implementation [27:57.200 --> 27:59.680] and used that in their code, which was open source. [27:59.680 --> 28:04.840] And of course, was a yes, sorry, which [28:04.840 --> 28:08.280] I guess was an example of open source being reused. [28:08.280 --> 28:13.320] Perhaps more specifically, with the COVID WARN app, [28:13.320 --> 28:15.400] because the Google and Apple notification protocol [28:15.400 --> 28:18.320] was the foundation for it, it was used really quite widely [28:18.320 --> 28:19.000] across Europe. [28:19.000 --> 28:21.680] Not all countries used it, but a lot of countries did. [28:21.680 --> 28:23.280] And at this point in time, I think [28:23.280 --> 28:27.920] there's 11 countries that all use the same back end. [28:27.920 --> 28:30.200] So they're all uploading their keys to servers. [28:30.200 --> 28:34.640] So now if you use the COVID WARN app, or contract, [28:34.640 --> 28:37.960] our version of it, you actually get keys from countries [28:37.960 --> 28:39.680] across Europe, half of Europe, essentially. [28:39.680 --> 28:42.120] It's quite a large area. [28:42.120 --> 28:45.880] So I mean, you could argue that's more a consequence [28:45.880 --> 28:49.800] of having a common API than it is open source. [28:49.800 --> 28:55.160] But there definitely was benefit in sharing of information, [28:55.160 --> 28:56.080] sharing of code. [28:56.080 --> 28:57.040] It did happen, I think. [28:57.040 --> 28:57.560] It did happen. [28:57.560 --> 29:03.120] But perhaps it's not as clear that it happened as it might [29:03.120 --> 29:04.720] otherwise seem. [29:04.720 --> 29:16.880] All right, I'm being told my time's up, so.