[00:00.000 --> 00:10.000]  Hello everyone and welcome to the talk controlling the web with a PS5 controller.
[00:10.000 --> 00:16.000]  Now before I jump into the talk there are a few things I wanted to share with you all
[00:16.000 --> 00:22.000]  because I gave this talk at a small meeting and I got two very important questions.
[00:22.000 --> 00:32.000]  You are coming in from the community. So just wanted to admit that yes I'm going to PS5 flash it.
[00:32.000 --> 00:35.000]  Please don't ask me how much I paid for it.
[00:35.000 --> 00:40.000]  And the second is I am not a gamer. I know it's very, I heard it in the very first statement
[00:40.000 --> 00:46.000]  but the last time I played a game on my PS5 console was maybe six months back.
[00:46.000 --> 00:51.000]  So I'm sure that I am not a gamer and I wouldn't be able to answer any gaming questions.
[00:51.000 --> 00:58.000]  So who am I? Well, I am Harsheela Kewal and I work as a network advocate at Contentful.
[00:58.000 --> 01:04.000]  If you haven't heard about Contentful, of course you will content platform with an API for architecture.
[01:04.000 --> 01:09.000]  I'm not going to talk more about that but I'm also a Mozilla representative.
[01:09.000 --> 01:12.000]  So here at 4GEN I am volunteering with Mozilla.
[01:12.000 --> 01:19.000]  So if you want to catch up with me, talk about open internet security, privacy and everything around what Mozilla is doing.
[01:19.000 --> 01:21.000]  You can find me at the Mozilla stack.
[01:21.000 --> 01:29.000]  And I like to think about myself as an experimentalist because I do a lot of different kind of experiments.
[01:29.000 --> 01:33.000]  Build a lot of city projects and share my learnings with the community.
[01:33.000 --> 01:35.000]  This is what I'm going to do today as well.
[01:35.000 --> 01:40.000]  And if you want to know a bit more about it and find me on the internet, you can head on to that lecture.
[01:40.000 --> 01:44.000]  Alright, so I think I'm going to talk today about something like this.
[01:44.000 --> 01:49.000]  I'm going to start with an introduction, talk a bit about APIs and WebHID.
[01:49.000 --> 01:56.000]  Because then look into the implementation of it and understand how you can implement WebHID into your applications.
[01:56.000 --> 02:05.000]  And with that I'm going to try to do some life coding if life coding gods are kind of unmean.
[02:05.000 --> 02:10.000]  And then I'll show you a little demo that I created and walk you through the code.
[02:10.000 --> 02:18.000]  Now before I talk about Web APIs, can I get a show of hands for the folks who already know what an API is?
[02:18.000 --> 02:22.000]  Alright, so the majority of you know what an API is, that is wonderful.
[02:22.000 --> 02:27.000]  But for the folks who are still not sure what an API is, here's a very quick recap.
[02:27.000 --> 02:34.000]  API stands for Application Programming Interface and it allows two different programs to communicate with each other.
[02:34.000 --> 02:39.000]  And Web APIs basically allow programs to communicate on the web.
[02:39.000 --> 02:44.000]  So it can be clients interacting with the server, server interacting with other servers and stuff like that.
[02:44.000 --> 02:49.000]  Now these Web APIs can be categorized into two categories.
[02:49.000 --> 02:52.000]  The first is the third party APIs.
[02:52.000 --> 02:57.000]  Now very important to note, these APIs have not been in the browser.
[02:57.000 --> 03:10.000]  What these APIs allow you to do is interact with a web show, which is external web shows in any data browser, maybe?
[03:10.000 --> 03:15.000]  So for an example, you want to create something for YouTube, right?
[03:15.000 --> 03:18.000]  You can use the YouTube API to create a bot.
[03:18.000 --> 03:22.000]  You might use Telegram API if you're using something on Contentful.
[03:22.000 --> 03:24.000]  You're going to use the Contentful API.
[03:24.000 --> 03:29.000]  Now in comparison with this, there is a browser API which is built in the browser.
[03:29.000 --> 03:34.000]  And this API allows you to extend the capabilities of the browser.
[03:34.000 --> 03:41.000]  For example, we have a Web Audio API, which allows you to interact with the audio that you have in your application.
[03:41.000 --> 03:47.000]  There is a Web Storage API, which allows you to interact with the local file system of the client's machine.
[03:47.000 --> 03:52.000]  And then there is the Web HID API, of which we are going to look in today.
[03:52.000 --> 04:02.000]  Now before I go on talking more about Web HID API, one thing that is very important to note over here is it's still experimental.
[04:02.000 --> 04:04.000]  Not all the browsers support it.
[04:04.000 --> 04:14.000]  So if you are creating applications which you want to use in production, please make sure that you have a check if the browser is supporting that or not.
[04:14.000 --> 04:24.000]  Now Web HID APIs allow you to create applications which can interact with new interface devices or HID each other.
[04:24.000 --> 04:36.000]  So these devices basically are bi-directional communication devices, which means that they can take input from the device and can also provide the output to the user.
[04:36.000 --> 04:40.000]  So in my case, I have a PS5 controller.
[04:40.000 --> 04:45.000]  So whatever if I click on this, nothing's going to happen right now, but it's basically taking an input.
[04:45.000 --> 04:54.000]  And if it is vibrating in the NVIDIA stage, then basically I am getting this output from the application.
[04:54.000 --> 05:06.000]  Now prior to the HID, the device could only unilaterally define protocols which were mainly designed for keyboards and hardwares.
[05:06.000 --> 05:18.000]  So hardware innovation either had to overload data in the existing protocols or they would have to create non-standard hardware that specified drives.
[05:18.000 --> 05:24.000]  Which kind of created a lot of complexity if someone wanted to create an application of that.
[05:24.000 --> 05:32.000]  The HID basically kind of reduces that overhead and allows you to interact with devices.
[05:32.000 --> 05:42.000]  Accordingly, their HID protocol was defined for USB devices, but now it has been implemented for other protocols such as Bluetooth.
[05:43.000 --> 05:52.000]  So now talking about the implementation part of it, it is a part of the navigator API of the browser.
[05:52.000 --> 05:58.000]  So this is the code secret of how you can connect your device to your application.
[05:58.000 --> 06:01.000]  Now as I mentioned earlier, it's still experimental.
[06:01.000 --> 06:04.000]  So whenever you are implementing the code, just add a check.
[06:04.000 --> 06:09.000]  Just add a list condition to see if HID is available in the browser or not.
[06:09.000 --> 06:18.000]  And with this device function, you need to compulsorily pass on the filter parameter which takes the vendor ID and the product ID.
[06:18.000 --> 06:25.000]  So each device has a unique product ID and the vendor ID is basically the ID for that manufacturer.
[06:25.000 --> 06:38.000]  And if you want to find out what is the vendor ID and the product ID for the device that you want to control, I have added a link in the slides so you can go ahead and get the information from that list.
[06:38.000 --> 06:46.000]  Now similar to request devices, you might have a situation where you just want to not include particular devices.
[06:46.000 --> 06:48.000]  You want to exclude particular devices.
[06:48.000 --> 06:52.000]  You don't want your users to use specific devices for your application.
[06:52.000 --> 06:55.000]  In that case, you can use the exclusion filter.
[06:55.000 --> 06:57.000]  Now this is an optional parameter.
[06:57.000 --> 07:00.000]  And again, this takes the vendor ID and the product ID.
[07:00.000 --> 07:04.000]  Again, this is unique to that particular device.
[07:04.000 --> 07:10.000]  So let's try to write the code for this.
[07:10.000 --> 07:16.000]  So over here, I have a very simple application which is not running.
[07:16.000 --> 07:19.000]  Let me quickly start the show.
[07:19.000 --> 07:23.000]  It has basically just two buttons right now, connect and then devices.
[07:23.000 --> 07:28.000]  If I click on connect, it's been defined because the device is still not yet connected.
[07:28.000 --> 07:32.000]  And then device is not going to do anything again.
[07:32.000 --> 07:40.000]  So over here, I have the connect function.
[07:40.000 --> 07:49.000]  I already have my fingers in here because it makes it easy to work with it.
[07:49.000 --> 07:53.000]  What I'm going to do is I'm going to use...
[07:53.000 --> 08:00.000]  Now again, one thing I forgot to mention is by default, this is an asynchronous API.
[08:00.000 --> 08:12.000]  I can simply use a bit and then call the function.
[08:12.000 --> 08:16.000]  I'll switch very much and even I don't get...
[08:16.000 --> 08:21.000]  IntelliSense does not give me or help me out over here.
[08:21.000 --> 08:27.000]  Okay.
[08:27.000 --> 08:40.000]  Now what this will do is this will give me an array of devices that the user might connect with the application.
[08:40.000 --> 08:44.000]  Now what I'm interested in is just the one device that I am connecting.
[08:44.000 --> 08:49.000]  Over here, so I'm going to set this up.
[08:49.000 --> 08:54.000]  Device is coming from devices.
[08:54.000 --> 08:58.000]  If I save this, if I come back to the browser, I'm going to do a quick refresh.
[08:58.000 --> 09:03.000]  And if I now connect, I get the option to connect to which device I want to connect.
[09:03.000 --> 09:09.000]  Right now it's just because I have set the ID for the wireless controller that I have right now.
[09:09.000 --> 09:15.000]  I can click on this, click on connect, and now you see me get some functions in here.
[09:15.000 --> 09:20.000]  So my device is now connected with the application.
[09:20.000 --> 09:25.000]  Now you might also want to get the list of devices that are already connected to the application.
[09:25.000 --> 09:29.000]  And you can do this using the get devices function.
[09:29.000 --> 09:35.000]  This will again return the array of all the devices that is connected.
[09:35.000 --> 09:42.000]  Now it returns an interface which is called the HD interface which contains all these properties.
[09:42.000 --> 09:50.000]  The first is the collection which returns an array of report which is again report is a format of data for each of the devices.
[09:50.000 --> 09:54.000]  Then it contains an all input report event.
[09:54.000 --> 09:58.000]  So this event gets triggered wherever we interact with the device.
[09:58.000 --> 10:04.000]  And there is an open flag which will tell you if the device has an open connection or not.
[10:04.000 --> 10:07.000]  So by default the connection is always closed.
[10:07.000 --> 10:13.000]  We have to use an open function to open the connection between the device and the application.
[10:13.000 --> 10:19.000]  And then you get the correct ID, the product name, and the vendor ID of the device.
[10:19.000 --> 10:29.000]  Now as I mentioned we need to use the open connection event or the open connection function to open that connection.
[10:29.000 --> 10:33.000]  Otherwise we won't be able to interact with our device anymore.
[10:33.000 --> 10:49.000]  So I'm going to come back over here and I'm in here and I'm just going to do a wait device.
[10:49.000 --> 10:52.000]  You're open.
[10:52.000 --> 11:00.000]  So just a quick note over here.
[11:00.000 --> 11:05.000]  And here right now the open is false because the connection is not open.
[11:05.000 --> 11:07.000]  I'm going to quickly refresh this.
[11:07.000 --> 11:09.000]  Connect it again.
[11:09.000 --> 11:16.000]  And now we see the connection is open and now I can interact with this.
[11:16.000 --> 11:22.000]  Now that my connection is open I still need to invent an event list now.
[11:22.000 --> 11:30.000]  But before we go into that I just want to quickly talk about the format of the data that is exchanged between the device and the application.
[11:30.000 --> 11:34.000]  And as I mentioned earlier with the HIG we already called it reports.
[11:34.000 --> 11:42.000]  And there are basically three types of reports, the input report, which is basically the connection that is coming in from the device to your application.
[11:42.000 --> 11:49.000]  The next is the output report which is the information of the binary data that is coming in from the application to your device.
[11:49.000 --> 11:56.000]  And then there is a feature report which is a bi-directional report.
[11:56.000 --> 12:05.000]  Now whenever you want to listen to the interaction that the user is making there is an input report in my list now that you can hook it up.
[12:05.000 --> 12:10.000]  Or you can also use the un-input report function to do that.
[12:10.000 --> 12:13.000]  So let's take a quick look at the full process.
[12:13.000 --> 12:22.000]  So over here now that my device is connected, I am going to...
[12:32.000 --> 12:41.000]  Again this can be an hc2nus function. I am passing the parameter in here.
[12:41.000 --> 12:45.000]  And this contains three basic information. You can get the data.
[12:45.000 --> 12:49.000]  So this is going to be in binary. You can get the information of the device.
[12:49.000 --> 12:52.000]  And you can get the report ID as well.
[12:52.000 --> 12:57.000]  So almost all the HIG devices have a report ID in their unit.
[12:57.000 --> 13:03.000]  So if it's an input report, if it's an output, if it's a feature report, each of those have a unique ID.
[13:03.000 --> 13:10.000]  If it does not have, you can simply replace the user as the report ID.
[13:10.000 --> 13:16.000]  Let's show the information in here.
[13:16.000 --> 13:21.000]  Now I am going to try this again. I don't have much to look into in the memory for this.
[13:21.000 --> 13:27.000]  For demo which I should have done before but now...
[13:27.000 --> 13:32.000]  I am going to connect it over here. Now you see, I get this thing of data coming in.
[13:32.000 --> 13:36.000]  If I click on this, it's going to log more things.
[13:36.000 --> 13:40.000]  It's going to be logging all the information or the data that is coming in.
[13:40.000 --> 13:43.000]  If I click on one of these, it will say it's an e-buckle.
[13:43.000 --> 13:49.000]  And you can see the information is in binary port.
[13:49.000 --> 13:55.000]  Now there are other events that you might be interested in like the connect event.
[13:55.000 --> 13:59.000]  So whenever the device is connected to the application,
[13:59.000 --> 14:03.000]  you can use the connect event or trigger any events that you want to make.
[14:03.000 --> 14:08.000]  For example, you might want to show the user the device that gets connected.
[14:08.000 --> 14:12.000]  So you can use that connect event disk over here.
[14:12.000 --> 14:14.000]  Similar to connect, there is a disk connect.
[14:14.000 --> 14:18.000]  Maybe I use a file-stake disk connect event.
[14:18.000 --> 14:23.000]  So you can use this to listen to that and you can show whether the device is disconnected.
[14:23.000 --> 14:27.000]  Do you want to connect again as a path and you can have that logic going in there.
[14:27.000 --> 14:33.000]  And remember, we had explicitly opened the connection between the device and the application.
[14:33.000 --> 14:37.000]  Now you might want to, after the user has interacted with the application,
[14:37.000 --> 14:40.000]  they have done with the application, you might want to close that connection.
[14:40.000 --> 14:43.000]  So you can use the close function in here.
[14:43.000 --> 14:48.000]  But the close function will still let the connection exist.
[14:48.000 --> 14:56.000]  I just want to close the connection in a way that the access of information is not going to happen,
[14:56.000 --> 15:00.000]  but the device is still going to be connected to the application.
[15:00.000 --> 15:04.000]  So if you want to remove access, you can use the forget function,
[15:04.000 --> 15:10.000]  which will basically tell the device to close the connection and remove the access.
[15:10.000 --> 15:14.000]  So if the user wants to connect again, they will have to connect to the application
[15:14.000 --> 15:17.000]  and go through that whole process again.
[15:17.000 --> 15:24.000]  Now as I mentioned very early on, this is an experimental API.
[15:24.000 --> 15:30.000]  So it's only available in Chromium-enabled browsers and then Opera as well.
[15:30.000 --> 15:35.000]  There are a lot of security issues that the community is discussing with the community
[15:35.000 --> 15:40.000]  is trying to talk about and that is why a lot of people have still not implemented that.
[15:40.000 --> 15:46.000]  Hopefully when we come up with better security and privacy features
[15:46.000 --> 15:52.000]  for this particular API, we might get support for all the other browsers as well.
[15:52.000 --> 15:57.000]  Now as I mentioned, I have a demo created, a very simple demo.
[15:57.000 --> 16:00.000]  If you have a PS5 controller with you and if you want to try it out,
[16:00.000 --> 16:06.000]  you can go to the website and you can try it out.
[16:06.000 --> 16:09.000]  Here is a quick demo.
[16:09.000 --> 16:15.000]  So I'm going to click on connect and I'm going to connect my controller in here
[16:15.000 --> 16:19.000]  and I have a very little nice demo where I have Mario with me.
[16:19.000 --> 16:25.000]  So what I am doing is I'm really interacting with Mario with my PS5 controller
[16:25.000 --> 16:30.000]  so it can move back and forth, it can go up and down.
[16:30.000 --> 16:37.000]  And one thing that I kind of added for fun was lasers.
[16:37.000 --> 16:43.000]  It's very handy to me but I enjoy the process of creating this and learning more about WebHID
[16:43.000 --> 16:48.000]  and I'm just going to quickly walk through the code of it.
[16:48.000 --> 16:51.000]  I'm actually implementing WebHID.
[16:51.000 --> 16:54.000]  So over here I have few stage defined.
[16:54.000 --> 16:59.000]  So I have the left stick, the left axis or the left thumb stick, the right stick,
[16:59.000 --> 17:01.000]  and then there are buttons.
[17:01.000 --> 17:04.000]  I haven't configured all the buttons for iOS
[17:04.000 --> 17:08.000]  because it didn't make sense to add everything in here.
[17:08.000 --> 17:13.000]  And then as I mentioned, make sure to add a check in the browser's report,
[17:13.000 --> 17:16.000]  as I did a lot, that's what I'm doing here.
[17:16.000 --> 17:21.000]  If it's a post, I am connecting it to my device.
[17:21.000 --> 17:26.000]  After it gets connected, I'm just doing some changes in the UI
[17:26.000 --> 17:30.000]  and then I am listening to the event business.
[17:30.000 --> 17:37.000]  Now I have added a report ID check just to make sure that I am getting the correct input from my device.
[17:37.000 --> 17:48.000]  And then I am seeing if the data at unit U8A is normalizing it,
[17:48.000 --> 17:53.000]  which corresponds to the left stick x-axis,
[17:53.000 --> 17:58.000]  and I'm normalizing it to get the value between minus 1 to 1 inclusion.
[17:58.000 --> 18:03.000]  And if that happens, I have another function that takes care of the change of the position
[18:03.000 --> 18:06.000]  and doing the same with the left stick y-axis,
[18:06.000 --> 18:10.000]  and then I am doing similar for my buttons.
[18:10.000 --> 18:13.000]  Now for buttons, I have configured for all of this,
[18:13.000 --> 18:16.000]  but right now it's just implemented for the shortcut button,
[18:16.000 --> 18:18.000]  which is getting the values in Boolean.
[18:18.000 --> 18:22.000]  If I click on that, it is true, and that stuff happens in the UI.
[18:22.000 --> 18:25.000]  Now this is the update position function.
[18:25.000 --> 18:31.000]  So if I come over here, it checks if the left stick equals 1,
[18:31.000 --> 18:33.000]  minus 1 is going to move forward.
[18:33.000 --> 18:37.000]  If it equals minus 1, it's going to move backwards.
[18:37.000 --> 18:40.000]  If the left stick y-axis is where it's going to go up,
[18:40.000 --> 18:43.000]  similarly if it's minus 1, it's going to go down.
[18:43.000 --> 18:45.000]  And for the circle, if the circle is true,
[18:45.000 --> 18:58.000]  I am busy being adding new elements and trying to give it an independent effect of the value of shooting a laser.
[18:58.000 --> 19:02.000]  So that was the very small demo that I created.
[19:02.000 --> 19:06.000]  There are other applications that you can try out.
[19:06.000 --> 19:11.000]  Algometer Space is one of the most interesting applications that I came across.
[19:11.000 --> 19:17.000]  It basically allows you to connect your Algometer machine to a web application,
[19:17.000 --> 19:22.000]  and you can kind of give me fire experience when you are using that Algometer machine.
[19:22.000 --> 19:26.000]  The source code is publicly available, so you can go and check it out.
[19:26.000 --> 19:33.000]  There is another project that's called ReMap, which allows you to customize your keyboard key mappings.
[19:33.000 --> 19:34.000]  Again, this is using web.
[19:34.000 --> 19:41.000]  I'll show you another hold, which is again open source, so you can go and check and take a look at the code.
[19:41.000 --> 19:47.000]  Now then, these were a couple of resources that I found helpful while I was...
[19:56.000 --> 19:58.000]  Thank you.
[20:26.000 --> 20:29.000]  You
[20:56.000 --> 20:59.000]  You
[21:26.000 --> 21:29.000]  You
[21:56.000 --> 21:59.000]  You
[22:26.000 --> 22:29.000]  You
[22:56.000 --> 22:58.000]  You
[23:26.000 --> 23:28.000]  You
[23:56.000 --> 23:58.000]  You
[24:26.000 --> 24:28.000]  You