[00:00.000 --> 00:09.280] So this is more of an entertaining talk, let's say. [00:09.280 --> 00:16.040] It's a lot less technical, probably a lot weirder, I would say. [00:16.040 --> 00:24.280] And hopefully, I will go fast enough so that you can have all the crazy idea questions [00:24.280 --> 00:25.780] at the end. [00:25.780 --> 00:31.400] Even a little demo, I have some hardware with me, if you didn't try VR, at least I think [00:31.400 --> 00:34.200] today you should try this, if you feel like it. [00:34.200 --> 00:37.780] So that will be, like, say, just there where it's fresh. [00:37.780 --> 00:47.480] So I will, so those are my slides, it's a jiz and frail, yes, why not. [00:47.480 --> 00:58.140] I will start by saying that I gave a couple of workshops to kids, actually, on discovering [00:58.140 --> 01:04.860] VR, and that was pretty nice. [01:04.860 --> 01:09.660] I thought initially I would give workshops and give, you know, a roller coaster. [01:09.660 --> 01:14.820] Some kids asked me about murdering clowns on top of a roller coaster. [01:14.820 --> 01:20.660] I did not prepare this, I admit, but that was, yeah, interesting, different inputs. [01:20.660 --> 01:26.980] I think it's a really interesting way also to consider teaching, computer science, programming, [01:26.980 --> 01:31.420] because I was saying there are my Pine Watch, like there is a computer everywhere. [01:31.420 --> 01:33.020] That's just a computer. [01:33.020 --> 01:36.780] That's why it might sound weird, it might be a little bit outside of people's comfort [01:36.780 --> 01:42.700] zone, but in the end, you can program a computer, you can program that. [01:42.700 --> 01:46.780] So I will introduce myself a bit first. [01:46.780 --> 01:49.860] So my name is Fabien, Fabien Benetou, Utopia. [01:49.860 --> 01:57.060] I work mostly at the European Parliament as a WebXR consultant, so I just do VR and AR [01:57.060 --> 02:07.300] on the web, that's it, and I do it as a prototypist, submitting all the tips you had or the suggestion [02:07.300 --> 02:11.340] you had at the beginning of the day on the TypeScript, on quality code. [02:11.340 --> 02:13.540] I don't know any of that. [02:13.540 --> 02:18.740] As I was saying before earlier, chatting with somebody at the entrance, if my code runs [02:18.740 --> 02:20.740] during the duration of the demo, I'm happy. [02:20.740 --> 02:28.380] The next day, yeah, it's a bonus, but yeah, that's the length mostly. [02:28.380 --> 02:33.860] And I gave a couple of talks here at first, them on connecting exotic hardware, like the [02:33.860 --> 02:39.460] Watch or IoT, everything, of course, using JavaScript. [02:39.460 --> 02:44.180] And this one will be on how to do this on a plane in a VR headset. [02:44.180 --> 02:51.140] So my motivation behind this is I have a bunch of notes, I have a wiki, that's the wiki, [02:51.140 --> 02:57.980] and I have a, you can still hear me, the 2D visualization of it, but it feels flat, like [02:57.980 --> 03:02.220] I don't want to have my notes in my model scheme or I don't want to have my notes behind [03:02.220 --> 03:08.260] the thin screen, I want them everywhere, so I can play with it, like I organize the space. [03:08.260 --> 03:14.460] So for this, I already had a couple of versions, I used Mozilla Hubs, so a little show of hands, [03:14.460 --> 03:16.940] how many of you tried VR at all? [03:16.940 --> 03:23.500] Okay, I would say two-third, half-two-third, how many of you tried Mozilla Hubs? [03:23.500 --> 03:27.180] Okay, five, six. [03:27.180 --> 03:30.620] So it's a social VR experiment. [03:30.620 --> 03:36.580] It's a pretty good solution to be honest, works quite well, and you can do this, meaning [03:36.580 --> 03:42.540] you can be in of your headset with your hands moving, you can be on a laptop, on a mobile [03:42.540 --> 03:49.900] phone, and you can see this shared virtual space, it's using 3GS for the 3D environment, [03:49.900 --> 03:54.500] and you can have PDF, you can have your webcam, you can take screenshots, you can honestly [03:54.500 --> 04:03.580] do pretty much most of what one would want to do, so I did quite a few hacks or modifications [04:03.580 --> 04:04.580] of it. [04:04.580 --> 04:12.700] It's obviously open source, that's the interesting part, but it's not enough, so you can do a [04:12.700 --> 04:17.140] bunch of hacks or explain a little bit how, for example, I turn on enough the lights in [04:17.140 --> 04:23.180] my office while I'm in VR, director kits on how to record with traveling, a bunch of [04:23.180 --> 04:31.980] different things, to the point that I did my little toolkit on how to customize, yeah, [04:31.980 --> 04:34.860] that's not going to work here. [04:34.860 --> 04:41.700] So you will have the link on the page, so imagine a bunch of functions related to hubs [04:41.700 --> 04:44.100] to do all this. [04:44.100 --> 04:48.940] So I really think basically here the point is hubs is a good solution, it works quite [04:48.940 --> 04:52.860] well if you want to do just social VR. [04:52.860 --> 04:58.860] The thing is first of all, it doesn't work offline, and I started after the end of the [04:58.860 --> 05:05.100] pandemic to start to fly again, to go see family, and I thought that's one of the best [05:05.100 --> 05:09.820] use case for VR, like I don't like being on a plane, I don't know if you do, but I think [05:09.820 --> 05:14.580] if you're the pilot, it's fine, but otherwise as a passenger, it sucks. [05:14.580 --> 05:18.460] And one of the beauty of those things is you put the headset on and you're basically somewhere [05:18.460 --> 05:19.460] else. [05:19.460 --> 05:25.020] But I don't want to be in VR on a plane and do a roller coaster or something, I just [05:25.020 --> 05:31.220] want to code, I want to be there and be able to do something that I have agency over, I [05:31.220 --> 05:35.940] want to build stuff, I want to create things, and I don't want to be like this. [05:35.940 --> 05:38.540] I want to feel like I'm somewhere else. [05:38.540 --> 05:46.940] Then I started to build on that, including, let's say, managing the history in my browser. [05:46.940 --> 05:51.500] So I would take a snapshot of the different pages that I visited, be able to organize [05:51.500 --> 05:52.500] them. [05:52.500 --> 05:58.180] As you might be able to see, this doesn't look at all like Mozilla Hubs. [05:58.180 --> 06:02.900] So Mozilla Hubs is amazing, but it's a huge stack, actually, and it's a stack that depends [06:02.900 --> 06:08.220] on AWS, which I obviously don't have on my plane or the plane I'm flying with. [06:08.220 --> 06:13.060] So then I started to rebuild it from scratch, basically. [06:13.060 --> 06:20.380] One warning, also, when you saw to hear programming in VR, I had a discussion with somebody just [06:20.380 --> 06:31.460] yesterday about it, it might be productive, I would say, at some point, but that's not [06:31.460 --> 06:32.460] my point. [06:32.460 --> 06:36.700] I don't find that interesting, like I have a really nice desk at home, it moves and all, [06:36.700 --> 06:42.220] and I have my ergonomic keyboard, and I can move my 4K screen, I like it, but I don't [06:42.220 --> 06:43.220] have that on the plane. [06:43.220 --> 06:48.540] So the point is being able to move in space when I have the space, and if I don't, to [06:48.540 --> 06:52.500] be able to use the space around me in a really compact manner. [06:52.500 --> 06:56.740] So the point is, this whole argument is like, don't look at this presentation thinking, [06:56.740 --> 06:58.700] is it to replace the way I work today? [06:58.700 --> 06:59.700] That's not the point. [06:59.700 --> 07:03.260] It might be, if you're into it, but it's more exploring what could be interesting in [07:03.260 --> 07:04.760] terms of interfaces. [07:04.760 --> 07:08.700] So yeah, word per minute, if you're starting to sit next to me with your laptop and we [07:08.700 --> 07:12.340] do a competition of how efficient you are, you're going to win. [07:12.340 --> 07:21.020] I mean, I hope for you, but that's not the point. [07:21.020 --> 07:26.260] And I was able, actually, on the plane to make friends, because the person next to me [07:26.260 --> 07:33.260] was like, what is this guy doing with his keyboard on the tray, plus there was no USB-C [07:33.260 --> 07:39.340] on it, so I had to plug the adapter, put it on top of the device, and then put, it was [07:39.340 --> 07:41.100] not wireless back then. [07:41.100 --> 07:44.420] So now I'm pretty happy I just received this little thing. [07:44.420 --> 07:52.220] So it's a Bluetooth keyboard, and it's mechanical one that's way more efficient. [07:52.220 --> 07:59.380] Back then I didn't have this, so yeah, that was a good excuse to meet someone and chat [07:59.380 --> 08:03.500] about what does it actually mean to program in VR. [08:03.500 --> 08:07.180] The next step being actually tinkering with it. [08:07.180 --> 08:17.500] So I went from having the history of my browsing behavior, let's say, to moving content and [08:17.500 --> 08:22.860] moving content by just pinching basically a page. [08:22.860 --> 08:26.820] So that's a render of a page, that's not a web page itself. [08:26.820 --> 08:32.100] What hopefully, I don't know if you can see all the way there, but basically with my right [08:32.100 --> 08:38.060] hand I'm able to move things around, and with my left hand I'm able to execute code. [08:38.060 --> 08:42.340] So that's, let's say the main trick, if there was one thing you need to think about that [08:42.340 --> 08:49.260] presentation is about moving things around freely or naturally, but also you will see [08:49.260 --> 08:52.380] the, so I move things. [08:52.380 --> 08:58.540] The red sphere there, and then if I pinch on that piece of code, the sphere there is [08:58.540 --> 08:59.540] blue. [08:59.540 --> 09:03.580] So I'm just moving things around, but it starts to manipulate text, which takes this code [09:03.580 --> 09:10.660] and then code can change the environment you're inside of. [09:10.660 --> 09:23.820] And the one trick to do this, I will hide it away, but you can guess. [09:23.820 --> 09:26.620] How to execute code in JavaScript? [09:26.620 --> 09:32.460] The one thing you're not supposed to use, evil, because it's evil, I think that's the [09:32.460 --> 09:34.060] one use case when you can use it. [09:34.060 --> 09:38.060] When your string of text is actually code and you're not running a bank or something [09:38.060 --> 09:44.780] like this, like this is my code running on my headset, I don't have anything that I won't [09:44.780 --> 09:49.780] accept to break, let's say, but even then it's executing locally. [09:49.780 --> 09:54.220] So that's the one reason, in my opinion, to use it. [09:54.220 --> 09:58.460] So the idea behind it is like, oh, I can manipulate text. [09:58.460 --> 09:59.780] That text can be code. [09:59.780 --> 10:05.660] I can change that text directly while I'm in the headset, either pinching it like single [10:05.660 --> 10:11.260] letters or with a keyboard to be more efficient, and then I can execute it, which is going [10:11.260 --> 10:18.340] to change the whole environment I'm inside of. [10:18.340 --> 10:26.780] So the trick for it was basically what I like to think of the escape valve, which is to [10:26.780 --> 10:31.900] say that, yeah, I could redesign an entire programming language, I could make it a programming [10:31.900 --> 10:38.780] language for VR, but I can also rely on one I already know, and that already has an engine [10:38.780 --> 10:40.180] in it, so namely JavaScript. [10:40.180 --> 10:42.500] So that's a little, let's say, trick. [10:42.500 --> 10:49.260] So I'll show a couple of examples, but that was an excuse not just to manipulate text [10:49.260 --> 10:53.060] and this code, but then to start to make my own programming language. [10:53.060 --> 10:57.060] So how many of us here have made their own programming language? [10:57.060 --> 11:03.060] Yeah, I see, there are very few hands that are like this, most hands are like that. [11:03.060 --> 11:04.060] I'm like that too. [11:04.060 --> 11:09.300] I don't think, honestly, I feel a little bit, maybe not ashamed, but I'm not proud of it [11:09.300 --> 11:15.900] either, what I'm trying to convey is the endeavor, trying to do it is super interesting. [11:15.900 --> 11:19.540] I don't think I've learned as much recently as trying to do this. [11:19.540 --> 11:24.900] I also recommend, if you check the slides or the presentation after, don't learn my [11:24.900 --> 11:25.900] programming language. [11:25.900 --> 11:27.220] I don't think it's a good one. [11:27.220 --> 11:32.420] It's a very interesting one for me by building it, by seeing the limitations of my understanding [11:32.420 --> 11:35.300] of what a programming language is. [11:35.300 --> 11:41.060] But so this, why we recommend this course specifically, it's because he's not going [11:41.060 --> 11:46.220] to, let's say, let's do an alternative to C, which for most of us would be too far and [11:46.220 --> 11:47.220] not practical. [11:47.220 --> 11:53.020] He's basically saying, well, JavaScript is a language, the browser will notice an environment [11:53.020 --> 11:57.820] you can already work with, you can transpile, you can use Babel, and that's enough, let's [11:57.820 --> 12:01.860] say, to get the foundation of what a programming language is. [12:01.860 --> 12:05.420] So I really recommend this kind of course, if you're considering programming, making [12:05.420 --> 12:06.420] your own programming language. [12:06.420 --> 12:12.860] And also, I don't know how many of you are familiar with this book. [12:12.860 --> 12:28.100] Yeah, I don't think I can go back easy either. [12:28.100 --> 12:33.540] So yeah, if you ever try to learn, I think, the foundation of programming and programming [12:33.540 --> 12:39.100] languages, usually you often do it with Lisp, which works well because it's so compact that [12:39.100 --> 12:44.260] you can keep it all in your mind, basically. [12:44.260 --> 12:48.700] But again, most of us are not familiar with Lisp, and even if we are, then it's not what [12:48.700 --> 12:50.420] we work with on a daily basis. [12:50.420 --> 12:56.060] So, SICP, but the JavaScript version has been out for a couple of years now, so if you want [12:56.060 --> 12:59.940] to learn, it's at the foundation of a programming language. [12:59.940 --> 13:05.740] While being familiar with the language itself, I really recommend it, it's quite taking, [13:05.740 --> 13:15.980] but I think it's really, it's a good investment. [13:15.980 --> 13:20.780] So then I'll show a couple of the features of the environment, but I won't show all [13:20.780 --> 13:23.300] of them, go too deep in there. [13:23.300 --> 13:27.340] But then what's interesting is you don't have just one line of code, but then you start [13:27.340 --> 13:32.860] to have groups of lines of code or history, the stack of what you executed. [13:32.860 --> 13:39.780] Of course, because your NVR code is not everything, so for example, some of the code is about [13:39.780 --> 13:41.500] loading a 3D model. [13:41.500 --> 13:46.540] So if you pinch it again, you execute that line of code, you get a 3D model out of it, [13:46.540 --> 13:49.900] and then you can do it a couple of more times. [13:49.900 --> 13:54.900] You get, I see that the contrast is not high there, but you get the history of what you [13:54.900 --> 13:58.580] just executed, then you can save it and execute it again. [13:58.580 --> 14:05.620] So you start to have the same, let's say primitives you're used to. [14:05.620 --> 14:19.820] And then yeah, you can do shortcuts, you can visualize the groupings. [14:19.820 --> 14:27.020] So for the people who twisted their hand instead of raising it, this is also where I'm at in [14:27.020 --> 14:32.220] the sense that I don't actually know if it's a fair thing to call this a programming language. [14:32.220 --> 14:35.820] Is it just like a bunch of utils, a bunch of shortcuts? [14:35.820 --> 14:36.820] I'm not sure. [14:36.820 --> 14:41.660] Again, what I find interesting in that process of making a programming language is, yeah, [14:41.660 --> 14:42.660] it's up to you. [14:42.660 --> 14:46.300] It's up to what you find interesting, it's up to what you find efficient. [14:46.300 --> 14:52.900] So yeah, I start, try to document it, stack back a bit, but yeah, all what it prompts [14:52.900 --> 14:59.340] me to think of is really, really interesting, really valuable. [14:59.340 --> 15:05.300] And also, if you don't go down that route, there is a subreddit for it, it's a subreddit [15:05.300 --> 15:10.540] for pretty much anything, but yeah, for R slash programming languages, where you can [15:10.540 --> 15:12.220] ask simple questions like this. [15:12.220 --> 15:27.620] Like what is the most interesting aspect of it? [15:27.620 --> 15:34.300] So of course, I had to omit that, so I display my issues from my repository in my VR environment [15:34.300 --> 15:37.780] so that I can move them like as if I was on a wall. [15:37.780 --> 15:43.820] I can display 3D models from, let's say a library of 3D models, this way I can organize [15:43.820 --> 15:49.380] it like tiles and then start to make a 3D environment to work out in off. [15:49.380 --> 15:54.140] I can do honestly this part, the graphs, I mostly did because a friend asked me during [15:54.140 --> 15:58.940] lunch, I was like, this is annoying, everybody find graphs sexy, it is looking good, but [15:58.940 --> 16:01.700] I'm not sure what's the point, so just okay, I'll just make a graph. [16:01.700 --> 16:08.700] So those are actually the pages of the wiki where I'm taking the notes out of the display [16:08.700 --> 16:12.340] there and they're all manipulable. [16:12.340 --> 16:16.300] Taking screenshots also in VR, I think that's a great way to document if you're doing a [16:16.300 --> 16:20.660] process, of course you can have the code out of it, you can save it back to your wiki, [16:20.660 --> 16:26.100] for example, but the movement, how things are going to be organized and how you want [16:26.100 --> 16:31.060] to, let's say, organize the tiles to make the 3D world, you want to tell a story to [16:31.060 --> 16:36.820] someone who might not have, no one to use a headset, then just taking a screenshot, capturing [16:36.820 --> 16:49.260] it and then sending it elsewhere, it's a pretty efficient way to do it. [16:49.260 --> 16:54.500] That was just this morning, so there's the same thing of managing issues, displaying [16:54.500 --> 16:59.940] the issues from the repository live, I can again grab them, move them in places, but [16:59.940 --> 17:06.140] using Swagger, so if you're not familiar with Swagger, it's a meta API, it's an API to manage [17:06.140 --> 17:15.060] all APIs, so once you have access to, when you load using Swagger, I can load from GitHub, [17:15.060 --> 17:20.740] whatever is going to have a Swagger specification, and that means making them as, for example, [17:20.740 --> 17:23.540] manipulable blocks. [17:23.540 --> 17:28.020] It's federated, so it's social, but if you have your own server, then you can connect [17:28.020 --> 17:33.700] to it, and we can be friends, we can exchange code also, so I can move the code in my event, [17:33.700 --> 17:37.700] and you can receive it and execute it. [17:37.700 --> 17:44.100] One port that is kind of funny is snapping, so those are blocks. [17:44.100 --> 17:48.700] You cannot hear it, so you'll have to trust me, but when the blocks are close enough, [17:48.700 --> 17:51.260] you hear them snap. [17:51.260 --> 17:58.500] The point of those blocks, or blocks rather, or not just blocks, is you can attach code [17:58.500 --> 18:02.780] on it, so then you can manipulate them literally like a legal block. [18:02.780 --> 18:07.180] That's again the part of the interactions that become interesting, yes, you can have [18:07.180 --> 18:11.940] a keyboard, and the keyboard works well, but if you just do the keyboard, yeah, you could [18:11.940 --> 18:14.980] just use a bigger screen, if you're always not that interesting for it, but if you can [18:14.980 --> 18:21.260] have both, programming with a keyboard, or naturally manipulating, or simply, let's say, [18:21.260 --> 18:25.220] directly manipulating with your hand, I think that becomes again interesting. [18:25.220 --> 18:28.780] So again, you need to imagine those are not just blocks, or blocks of code, it's called, [18:28.780 --> 18:32.780] let's say, for swagger, so that you can get the information from an API or execute through [18:32.780 --> 18:40.780] that API. [18:40.780 --> 18:48.580] So why also do I do all this with the blocks, is because I think that's a great way to learn. [18:48.580 --> 18:54.340] So you can imagine this, the blocks for running code that is abstract, let's say list things, [18:54.340 --> 18:59.180] but you can use that for chemistry, let's say it's a C, and it's a C, or it's an H, [18:59.180 --> 19:03.380] H, and an O, and you bring them together, they snap when it makes a molecule. [19:03.380 --> 19:10.740] I think the same kind of things apply to pretty much any field of study, so physics, chemistry, [19:10.740 --> 19:15.500] sociology, like you have a set of atoms and a set of rules, if you can, then, to have [19:15.500 --> 19:22.540] a certain organization specialized together, then you can learn this field, hopefully, [19:22.540 --> 19:28.180] in a more embodied way. [19:28.180 --> 19:46.100] Nope, I guess that's going to be it. [19:46.100 --> 20:06.420] I have a Twitter archive, because it's not working properly, but not for recent things. [20:06.420 --> 20:24.900] Well, okay. [20:24.900 --> 20:28.380] So that's, it's fine. [20:28.380 --> 20:30.100] All this, I think it's pretty interesting. [20:30.100 --> 20:34.180] The one thing I'm even more awkward with, rather than calling the thing I've done in [20:34.180 --> 20:35.980] my programming language, is this. [20:35.980 --> 20:39.780] So that headset is an Oculus Quest or a Meta Quest. [20:39.780 --> 20:45.860] I don't have a Meta account, I don't think it's, I don't think it's a, yeah, I don't [20:45.860 --> 20:51.540] think surveillance capitalism is a good business model, let's say, as a society, but it works [20:51.540 --> 20:52.540] really well. [20:52.540 --> 20:53.540] So that's pretty awkward. [20:53.540 --> 21:00.540] I really don't think it's such a good thing, and I called it an adversarial dependency, [21:00.540 --> 21:04.700] because it helps me to build more, explore more, but on something I don't want to rely [21:04.700 --> 21:07.300] on, because it's not aligned with my set of values. [21:07.300 --> 21:13.820] So I still show all this, because, yeah, you won't see the photo, but there are a couple [21:13.820 --> 21:19.780] of headsets, like the links, that should be there in a couple of weeks. [21:19.780 --> 21:25.500] The point is, don't associate VR with just Meta or Facebook. [21:25.500 --> 21:28.740] It is a convenient one, but there are alternatives coming out. [21:28.740 --> 21:32.740] There is also a similar VR. [21:32.740 --> 21:34.180] It's not the only way to do VR. [21:34.180 --> 21:38.340] So adversarial dependency, in the sense that, yes, you can rely on it temporarily, as long [21:38.340 --> 21:42.780] as you can have it just as a replaceable block, and as long as you can swap it whenever [21:42.780 --> 21:48.740] you get something that's more than with your value, that is much better. [21:48.740 --> 21:49.740] But that's it, yeah. [21:49.740 --> 21:53.980] The whole point of this is building your own scaffolding, being a programming language, [21:53.980 --> 22:00.500] being a tool chain, using VR, because I find that needs to be the most interesting interface. [22:00.500 --> 22:02.940] But yeah, that would be my recommendation for you. [22:02.940 --> 22:07.780] Just make your interface, make your scaffolding, using VR or not. [22:07.780 --> 22:12.100] And if you want to try the VR one, I'll be around with it. [22:12.100 --> 22:15.100] Thank you. [22:15.100 --> 22:28.340] I think we have time for a couple of questions. [22:28.340 --> 22:47.500] Do you have any questions? [22:47.500 --> 22:55.580] I really like the VR experience in the plane, but have you thought about the incidents that [22:55.580 --> 22:59.420] may occur, like hitting my neighbors, punching him? [22:59.420 --> 23:00.420] No. [23:00.420 --> 23:01.420] Thank you. [23:01.420 --> 23:02.420] Yeah. [23:02.420 --> 23:09.980] So, I don't recommend punching anybody, but you're not good, so you define the interactions [23:09.980 --> 23:10.980] however you want. [23:10.980 --> 23:15.700] So basically, if you put things, let's say, to the object, either a block of code or a [23:15.700 --> 23:21.060] hat, whatever, a meter away from you, you're going to punch your neighbor. [23:21.060 --> 23:26.460] But because now you have access to the code in there, you can just say, oh, I bring, let's [23:26.460 --> 23:32.380] say, everything within a 50 centimeters radius, and yeah, you can just do it on the spot. [23:32.380 --> 23:38.380] Again, it's using WebXR, so it's VR and AO on the web, meaning there is no, like, unity [23:38.380 --> 23:40.820] build, there is nothing there. [23:40.820 --> 23:43.460] Everything is done directly in the headset. [23:43.460 --> 23:48.060] You can put your Node.js server, or even your Python server, whatever you want to run. [23:48.060 --> 23:53.260] So it means you can refresh the page, and then you have your new content, the new way [23:53.260 --> 23:58.580] to handle it, so either you do it without even having to reload, but you can fix it [23:58.580 --> 24:18.780] on the fly, let's see.