So this is more of an entertaining talk, let's say.
It's a lot less technical, probably a lot weirder, I would say.
And hopefully, I will go fast enough so that you can have all the crazy idea questions
at the end.
Even a little demo, I have some hardware with me, if you didn't try VR, at least I think
today you should try this, if you feel like it.
So that will be, like, say, just there where it's fresh.
So I will, so those are my slides, it's a jiz and frail, yes, why not.
I will start by saying that I gave a couple of workshops to kids, actually, on discovering
VR, and that was pretty nice.
I thought initially I would give workshops and give, you know, a roller coaster.
Some kids asked me about murdering clowns on top of a roller coaster.
I did not prepare this, I admit, but that was, yeah, interesting, different inputs.
I think it's a really interesting way also to consider teaching, computer science, programming,
because I was saying there are my Pine Watch, like there is a computer everywhere.
That's just a computer.
That's why it might sound weird, it might be a little bit outside of people's comfort
zone, but in the end, you can program a computer, you can program that.
So I will introduce myself a bit first.
So my name is Fabien, Fabien Benetou, Utopia.
I work mostly at the European Parliament as a WebXR consultant, so I just do VR and AR
on the web, that's it, and I do it as a prototypist, submitting all the tips you had or the suggestion
you had at the beginning of the day on the TypeScript, on quality code.
I don't know any of that.
As I was saying before earlier, chatting with somebody at the entrance, if my code runs
during the duration of the demo, I'm happy.
The next day, yeah, it's a bonus, but yeah, that's the length mostly.
And I gave a couple of talks here at first, them on connecting exotic hardware, like the
Watch or IoT, everything, of course, using JavaScript.
And this one will be on how to do this on a plane in a VR headset.
So my motivation behind this is I have a bunch of notes, I have a wiki, that's the wiki,
and I have a, you can still hear me, the 2D visualization of it, but it feels flat, like
I don't want to have my notes in my model scheme or I don't want to have my notes behind
the thin screen, I want them everywhere, so I can play with it, like I organize the space.
So for this, I already had a couple of versions, I used Mozilla Hubs, so a little show of hands,
how many of you tried VR at all?
Okay, I would say two-third, half-two-third, how many of you tried Mozilla Hubs?
Okay, five, six.
So it's a social VR experiment.
It's a pretty good solution to be honest, works quite well, and you can do this, meaning
you can be in of your headset with your hands moving, you can be on a laptop, on a mobile
phone, and you can see this shared virtual space, it's using 3GS for the 3D environment,
and you can have PDF, you can have your webcam, you can take screenshots, you can honestly
do pretty much most of what one would want to do, so I did quite a few hacks or modifications
of it.
It's obviously open source, that's the interesting part, but it's not enough, so you can do a
bunch of hacks or explain a little bit how, for example, I turn on enough the lights in
my office while I'm in VR, director kits on how to record with traveling, a bunch of
different things, to the point that I did my little toolkit on how to customize, yeah,
that's not going to work here.
So you will have the link on the page, so imagine a bunch of functions related to hubs
to do all this.
So I really think basically here the point is hubs is a good solution, it works quite
well if you want to do just social VR.
The thing is first of all, it doesn't work offline, and I started after the end of the
pandemic to start to fly again, to go see family, and I thought that's one of the best
use case for VR, like I don't like being on a plane, I don't know if you do, but I think
if you're the pilot, it's fine, but otherwise as a passenger, it sucks.
And one of the beauty of those things is you put the headset on and you're basically somewhere
else.
But I don't want to be in VR on a plane and do a roller coaster or something, I just
want to code, I want to be there and be able to do something that I have agency over, I
want to build stuff, I want to create things, and I don't want to be like this.
I want to feel like I'm somewhere else.
Then I started to build on that, including, let's say, managing the history in my browser.
So I would take a snapshot of the different pages that I visited, be able to organize
them.
As you might be able to see, this doesn't look at all like Mozilla Hubs.
So Mozilla Hubs is amazing, but it's a huge stack, actually, and it's a stack that depends
on AWS, which I obviously don't have on my plane or the plane I'm flying with.
So then I started to rebuild it from scratch, basically.
One warning, also, when you saw to hear programming in VR, I had a discussion with somebody just
yesterday about it, it might be productive, I would say, at some point, but that's not
my point.
I don't find that interesting, like I have a really nice desk at home, it moves and all,
and I have my ergonomic keyboard, and I can move my 4K screen, I like it, but I don't
have that on the plane.
So the point is being able to move in space when I have the space, and if I don't, to
be able to use the space around me in a really compact manner.
So the point is, this whole argument is like, don't look at this presentation thinking,
is it to replace the way I work today?
That's not the point.
It might be, if you're into it, but it's more exploring what could be interesting in
terms of interfaces.
So yeah, word per minute, if you're starting to sit next to me with your laptop and we
do a competition of how efficient you are, you're going to win.
I mean, I hope for you, but that's not the point.
And I was able, actually, on the plane to make friends, because the person next to me
was like, what is this guy doing with his keyboard on the tray, plus there was no USB-C
on it, so I had to plug the adapter, put it on top of the device, and then put, it was
not wireless back then.
So now I'm pretty happy I just received this little thing.
So it's a Bluetooth keyboard, and it's mechanical one that's way more efficient.
Back then I didn't have this, so yeah, that was a good excuse to meet someone and chat
about what does it actually mean to program in VR.
The next step being actually tinkering with it.
So I went from having the history of my browsing behavior, let's say, to moving content and
moving content by just pinching basically a page.
So that's a render of a page, that's not a web page itself.
What hopefully, I don't know if you can see all the way there, but basically with my right
hand I'm able to move things around, and with my left hand I'm able to execute code.
So that's, let's say the main trick, if there was one thing you need to think about that
presentation is about moving things around freely or naturally, but also you will see
the, so I move things.
The red sphere there, and then if I pinch on that piece of code, the sphere there is
blue.
So I'm just moving things around, but it starts to manipulate text, which takes this code
and then code can change the environment you're inside of.
And the one trick to do this, I will hide it away, but you can guess.
How to execute code in JavaScript?
The one thing you're not supposed to use, evil, because it's evil, I think that's the
one use case when you can use it.
When your string of text is actually code and you're not running a bank or something
like this, like this is my code running on my headset, I don't have anything that I won't
accept to break, let's say, but even then it's executing locally.
So that's the one reason, in my opinion, to use it.
So the idea behind it is like, oh, I can manipulate text.
That text can be code.
I can change that text directly while I'm in the headset, either pinching it like single
letters or with a keyboard to be more efficient, and then I can execute it, which is going
to change the whole environment I'm inside of.
So the trick for it was basically what I like to think of the escape valve, which is to
say that, yeah, I could redesign an entire programming language, I could make it a programming
language for VR, but I can also rely on one I already know, and that already has an engine
in it, so namely JavaScript.
So that's a little, let's say, trick.
So I'll show a couple of examples, but that was an excuse not just to manipulate text
and this code, but then to start to make my own programming language.
So how many of us here have made their own programming language?
Yeah, I see, there are very few hands that are like this, most hands are like that.
I'm like that too.
I don't think, honestly, I feel a little bit, maybe not ashamed, but I'm not proud of it
either, what I'm trying to convey is the endeavor, trying to do it is super interesting.
I don't think I've learned as much recently as trying to do this.
I also recommend, if you check the slides or the presentation after, don't learn my
programming language.
I don't think it's a good one.
It's a very interesting one for me by building it, by seeing the limitations of my understanding
of what a programming language is.
But so this, why we recommend this course specifically, it's because he's not going
to, let's say, let's do an alternative to C, which for most of us would be too far and
not practical.
He's basically saying, well, JavaScript is a language, the browser will notice an environment
you can already work with, you can transpile, you can use Babel, and that's enough, let's
say, to get the foundation of what a programming language is.
So I really recommend this kind of course, if you're considering programming, making
your own programming language.
And also, I don't know how many of you are familiar with this book.
Yeah, I don't think I can go back easy either.
So yeah, if you ever try to learn, I think, the foundation of programming and programming
languages, usually you often do it with Lisp, which works well because it's so compact that
you can keep it all in your mind, basically.
But again, most of us are not familiar with Lisp, and even if we are, then it's not what
we work with on a daily basis.
So, SICP, but the JavaScript version has been out for a couple of years now, so if you want
to learn, it's at the foundation of a programming language.
While being familiar with the language itself, I really recommend it, it's quite taking,
but I think it's really, it's a good investment.
So then I'll show a couple of the features of the environment, but I won't show all
of them, go too deep in there.
But then what's interesting is you don't have just one line of code, but then you start
to have groups of lines of code or history, the stack of what you executed.
Of course, because your NVR code is not everything, so for example, some of the code is about
loading a 3D model.
So if you pinch it again, you execute that line of code, you get a 3D model out of it,
and then you can do it a couple of more times.
You get, I see that the contrast is not high there, but you get the history of what you
just executed, then you can save it and execute it again.
So you start to have the same, let's say primitives you're used to.
And then yeah, you can do shortcuts, you can visualize the groupings.
So for the people who twisted their hand instead of raising it, this is also where I'm at in
the sense that I don't actually know if it's a fair thing to call this a programming language.
Is it just like a bunch of utils, a bunch of shortcuts?
I'm not sure.
Again, what I find interesting in that process of making a programming language is, yeah,
it's up to you.
It's up to what you find interesting, it's up to what you find efficient.
So yeah, I start, try to document it, stack back a bit, but yeah, all what it prompts
me to think of is really, really interesting, really valuable.
And also, if you don't go down that route, there is a subreddit for it, it's a subreddit
for pretty much anything, but yeah, for R slash programming languages, where you can
ask simple questions like this.
Like what is the most interesting aspect of it?
So of course, I had to omit that, so I display my issues from my repository in my VR environment
so that I can move them like as if I was on a wall.
I can display 3D models from, let's say a library of 3D models, this way I can organize
it like tiles and then start to make a 3D environment to work out in off.
I can do honestly this part, the graphs, I mostly did because a friend asked me during
lunch, I was like, this is annoying, everybody find graphs sexy, it is looking good, but
I'm not sure what's the point, so just okay, I'll just make a graph.
So those are actually the pages of the wiki where I'm taking the notes out of the display
there and they're all manipulable.
Taking screenshots also in VR, I think that's a great way to document if you're doing a
process, of course you can have the code out of it, you can save it back to your wiki,
for example, but the movement, how things are going to be organized and how you want
to, let's say, organize the tiles to make the 3D world, you want to tell a story to
someone who might not have, no one to use a headset, then just taking a screenshot, capturing
it and then sending it elsewhere, it's a pretty efficient way to do it.
That was just this morning, so there's the same thing of managing issues, displaying
the issues from the repository live, I can again grab them, move them in places, but
using Swagger, so if you're not familiar with Swagger, it's a meta API, it's an API to manage
all APIs, so once you have access to, when you load using Swagger, I can load from GitHub,
whatever is going to have a Swagger specification, and that means making them as, for example,
manipulable blocks.
It's federated, so it's social, but if you have your own server, then you can connect
to it, and we can be friends, we can exchange code also, so I can move the code in my event,
and you can receive it and execute it.
One port that is kind of funny is snapping, so those are blocks.
You cannot hear it, so you'll have to trust me, but when the blocks are close enough,
you hear them snap.
The point of those blocks, or blocks rather, or not just blocks, is you can attach code
on it, so then you can manipulate them literally like a legal block.
That's again the part of the interactions that become interesting, yes, you can have
a keyboard, and the keyboard works well, but if you just do the keyboard, yeah, you could
just use a bigger screen, if you're always not that interesting for it, but if you can
have both, programming with a keyboard, or naturally manipulating, or simply, let's say,
directly manipulating with your hand, I think that becomes again interesting.
So again, you need to imagine those are not just blocks, or blocks of code, it's called,
let's say, for swagger, so that you can get the information from an API or execute through
that API.
So why also do I do all this with the blocks, is because I think that's a great way to learn.
So you can imagine this, the blocks for running code that is abstract, let's say list things,
but you can use that for chemistry, let's say it's a C, and it's a C, or it's an H,
H, and an O, and you bring them together, they snap when it makes a molecule.
I think the same kind of things apply to pretty much any field of study, so physics, chemistry,
sociology, like you have a set of atoms and a set of rules, if you can, then, to have
a certain organization specialized together, then you can learn this field, hopefully,
in a more embodied way.
Nope, I guess that's going to be it.
I have a Twitter archive, because it's not working properly, but not for recent things.
Well, okay.
So that's, it's fine.
All this, I think it's pretty interesting.
The one thing I'm even more awkward with, rather than calling the thing I've done in
my programming language, is this.
So that headset is an Oculus Quest or a Meta Quest.
I don't have a Meta account, I don't think it's, I don't think it's a, yeah, I don't
think surveillance capitalism is a good business model, let's say, as a society, but it works
really well.
So that's pretty awkward.
I really don't think it's such a good thing, and I called it an adversarial dependency,
because it helps me to build more, explore more, but on something I don't want to rely
on, because it's not aligned with my set of values.
So I still show all this, because, yeah, you won't see the photo, but there are a couple
of headsets, like the links, that should be there in a couple of weeks.
The point is, don't associate VR with just Meta or Facebook.
It is a convenient one, but there are alternatives coming out.
There is also a similar VR.
It's not the only way to do VR.
So adversarial dependency, in the sense that, yes, you can rely on it temporarily, as long
as you can have it just as a replaceable block, and as long as you can swap it whenever
you get something that's more than with your value, that is much better.
But that's it, yeah.
The whole point of this is building your own scaffolding, being a programming language,
being a tool chain, using VR, because I find that needs to be the most interesting interface.
But yeah, that would be my recommendation for you.
Just make your interface, make your scaffolding, using VR or not.
And if you want to try the VR one, I'll be around with it.
Thank you.
I think we have time for a couple of questions.
Do you have any questions?
I really like the VR experience in the plane, but have you thought about the incidents that
may occur, like hitting my neighbors, punching him?
No.
Thank you.
Yeah.
So, I don't recommend punching anybody, but you're not good, so you define the interactions
however you want.
So basically, if you put things, let's say, to the object, either a block of code or a
hat, whatever, a meter away from you, you're going to punch your neighbor.
But because now you have access to the code in there, you can just say, oh, I bring, let's
say, everything within a 50 centimeters radius, and yeah, you can just do it on the spot.
Again, it's using WebXR, so it's VR and AO on the web, meaning there is no, like, unity
build, there is nothing there.
Everything is done directly in the headset.
You can put your Node.js server, or even your Python server, whatever you want to run.
So it means you can refresh the page, and then you have your new content, the new way
to handle it, so either you do it without even having to reload, but you can fix it
on the fly, let's see.