[00:00.000 --> 00:23.120]  Yeah, so I'm going to do two small presentation because those are short talks and I didn't
[00:23.120 --> 00:25.560]  want to take too much time today.
[00:25.560 --> 00:32.400]  So we're going to speak about FFMPEG and mostly FFMPEG6.0, and then I will speak about
[00:32.400 --> 00:39.200]  a new project called VLC.js, but it's a lie, it's not really VLC.js.
[00:39.200 --> 00:42.600]  So who am I?
[00:42.600 --> 00:47.560]  My name is Jean-Retier, some of you know me, some don't, so I'm president of Vidoran,
[00:47.560 --> 00:54.840]  I've been working on VLC for, okay, I'm close to 40, so 17 years.
[00:54.840 --> 01:00.320]  I've been involved in X264, which is a Vidoran project, David, which is a AV1 decoder and
[01:00.320 --> 01:08.200]  lately a bit on FFMPEG, mostly on the community management, which is a funny topic.
[01:08.200 --> 01:12.440]  I shouldn't be the one presenting this presentation, but the people who should do this presentation
[01:12.440 --> 01:17.720]  are maybe in this room and don't want to present, so that's why I'm presenting.
[01:17.720 --> 01:21.560]  Jokes aside, like if you look at the first time, open media room, like there is almost
[01:21.560 --> 01:24.520]  no FFMPEG talk, which is completely insane.
[01:24.520 --> 01:29.760]  VLCR is better, thanks to Kiran and Tourémy, but it's ridiculous, like if you look also
[01:29.760 --> 01:34.080]  in the archives, look in the archives, there's almost no FFMPEG, general FFMPEG talk.
[01:34.080 --> 01:35.080]  What?
[01:35.080 --> 01:38.000]  Everything in the multimedia in the open source world and outside of the open source world
[01:38.000 --> 01:39.760]  is actually based on FFMPEG.
[01:39.760 --> 01:44.480]  And when I mean everything, I mean everything you see online, and most of those, like you
[01:44.480 --> 01:48.960]  go to those big trade shows and they are all amazing cloud encoding, so on, and it's just
[01:48.960 --> 01:52.400]  like a very nice whopper to FFMPEG.
[01:52.400 --> 01:57.400]  But of course, when I say FFMPEG, please understand, this is FFMPEG plus LBX264 plus
[01:57.400 --> 02:06.160]  LBX665 plus LibVPX plus David plus all the other libraries that I forget.
[02:06.160 --> 02:14.420]  And even on our voici, Mademoiselle, you know those French fashion thing that we have which
[02:14.420 --> 02:18.560]  is called Hacker News, even on Hacker News, like when there is a release of FFMPEG, it's
[02:18.560 --> 02:20.400]  not even the top Hacker News, right?
[02:20.400 --> 02:22.760]  So that means that we are doing something wrong,
[02:22.760 --> 02:25.400]  which means we don't communicate enough on FFMPEG.
[02:25.400 --> 02:27.280]  So here I am.
[02:27.280 --> 02:29.840]  So the community is healthy.
[02:29.840 --> 02:33.720]  We've had some fights in the past, to be honest.
[02:33.720 --> 02:35.640]  The folks are long gone.
[02:35.640 --> 02:37.680]  And most of the people working now on FFMPEG
[02:37.680 --> 02:41.080]  also put lots of new people who are not there
[02:41.080 --> 02:43.440]  at the folk time, but also people from both folks
[02:43.440 --> 02:44.880]  are still working on FFMPEG.
[02:44.880 --> 02:47.480]  That's pretty cool, especially since we've not
[02:47.480 --> 02:49.280]  seen that many open source community being
[02:49.280 --> 02:51.440]  able to work together after those kind of events.
[02:54.280 --> 02:56.680]  So here I'm just going to speak just quickly
[02:56.680 --> 03:00.640]  about FFMPEG 5.0, which was around almost exactly
[03:00.640 --> 03:02.120]  one year ago.
[03:02.120 --> 03:04.800]  It was very important because we
[03:04.800 --> 03:07.000]  tried to match the new release schedules that I'm
[03:07.000 --> 03:08.240]  going to talk about.
[03:08.240 --> 03:13.320]  But it was probably the biggest API breakage ever
[03:13.320 --> 03:14.920]  on FFMPEG.
[03:14.920 --> 03:17.440]  I think just a train of commerce removing deprecation
[03:17.440 --> 03:20.200]  samples was around 130 commits.
[03:20.200 --> 03:21.480]  And the diff was huge.
[03:21.480 --> 03:25.240]  So some APIs were there deprecated to 2013
[03:25.240 --> 03:27.480]  and were removed in 5.0.
[03:27.480 --> 03:29.720]  So this is probably going to impact a lot of you
[03:29.720 --> 03:32.720]  because a lot of distribution are still on 4.4.
[03:32.720 --> 03:36.040]  But 5.0 is a big change of APIs.
[03:36.040 --> 03:39.120]  And mostly one big thing is that it's
[03:39.120 --> 03:41.760]  one API to decode both audio and video,
[03:41.760 --> 03:47.160]  and not AV codec, video, decode 4, 5, 6, and so on.
[03:47.160 --> 03:48.760]  All those APIs.
[03:48.760 --> 03:51.040]  It's not doing subtitles yet, but I
[03:51.040 --> 03:53.000]  was promised that someone will do it this year.
[03:53.000 --> 03:54.960]  Where is Anton?
[03:54.960 --> 03:57.480]  Yeah.
[03:57.480 --> 04:00.880]  And yeah, we did a lot of new things.
[04:00.880 --> 04:04.480]  AV frame-based API in sw scale, new bit fields,
[04:04.480 --> 04:07.720]  streaming filters, a lot of things
[04:07.720 --> 04:11.320]  to clean AV format and AV codec.
[04:11.320 --> 04:13.640]  It's disentangling those two libraries,
[04:13.640 --> 04:15.800]  working on the decoder contacts, et cetera.
[04:15.800 --> 04:20.160]  You should look at the release notes on that.
[04:20.160 --> 04:24.120]  There are some people who are doing amazing work, mostly
[04:24.120 --> 04:26.000]  Andreas and James, who are basically
[04:26.000 --> 04:28.320]  removing all the craft on FFMPEG.
[04:28.320 --> 04:31.880]  So one day, the whole FFMPEG will be thread safe.
[04:31.880 --> 04:33.440]  We believe that, right?
[04:33.440 --> 04:36.360]  And AV codec, any of the format will be completely split.
[04:36.360 --> 04:37.200]  Yeah, OK, maybe not.
[04:37.200 --> 04:39.680]  But there is a lot of work to be done,
[04:39.680 --> 04:42.480]  and that's very important for security reasons.
[04:42.480 --> 04:46.080]  Michael, who's still probably the oldest FFMPEG contributor,
[04:46.080 --> 04:49.560]  is still fuzzing FFMPEG every day.
[04:49.560 --> 04:51.400]  Slice thread setting is W scale.
[04:51.400 --> 04:55.320]  IMF digmixing, which is good because so many professionals
[04:55.320 --> 04:57.520]  are using IMEF format, and they usually
[04:57.520 --> 05:00.880]  do weird things on FFMPEG, or above FFMPEG,
[05:00.880 --> 05:02.400]  and then we have to deal with their shit,
[05:02.400 --> 05:03.600]  because it's wrongly marked.
[05:03.600 --> 05:07.200]  So now we're actually getting that directly into FFMPEG.
[05:07.200 --> 05:11.040]  Dolby vision, I'm not sure exactly which part of the vision,
[05:11.040 --> 05:13.840]  because there is, as many of you know, a five or six profile.
[05:13.840 --> 05:16.400]  But I think at least profile five were there.
[05:16.400 --> 05:19.080]  And of course, a lot of things, and one of the cool things
[05:19.080 --> 05:21.600]  was the integration of LiPlaCibo, which
[05:21.600 --> 05:26.280]  used to be the MPV video filtering framework, mostly
[05:26.280 --> 05:30.160]  GPU accelerated, that is now into FFMPEG.
[05:30.160 --> 05:34.680]  And you can use that without GPU, easily with emulation.
[05:34.680 --> 05:39.080]  So the old APIs, like you know the old APIs,
[05:39.080 --> 05:42.840]  and now what's interesting is that it's more async-based,
[05:42.840 --> 05:48.000]  and so you don't need to do those horrible weight.
[05:48.000 --> 05:53.120]  5.1, so that was like six months ago in July.
[05:53.120 --> 05:55.440]  This one is important for you, because it's an LTS.
[05:55.440 --> 05:59.680]  5.0 is not LTS, so we're going to try to make that,
[05:59.680 --> 06:03.520]  to fix at least the security bugs for a couple of years.
[06:03.520 --> 06:07.040]  And most of the things that were added were a lot of features,
[06:07.040 --> 06:08.880]  but one of the major APIs that was merged
[06:08.880 --> 06:13.240]  was the change of the audio channel layout API,
[06:13.240 --> 06:15.840]  which was supposed to come in 5.0, but we missed,
[06:15.840 --> 06:18.000]  and then we said, well, it's going to take too much time,
[06:18.000 --> 06:22.080]  so we did that with 5.1.
[06:22.080 --> 06:24.720]  A lot of optimization on ARM in that release,
[06:24.720 --> 06:26.960]  mostly on HVC decoding, a lot of things
[06:26.960 --> 06:29.960]  on everyone decoding in hardware,
[06:29.960 --> 06:32.600]  because there is still 25 different APIs
[06:32.600 --> 06:34.240]  to do hardware acceleration.
[06:34.240 --> 06:35.880]  But soon there will be a new one that
[06:35.880 --> 06:38.560]  is going to replace all of them, which is Vulkan video decoding,
[06:38.560 --> 06:41.920]  and we'll have a 14 standards.
[06:41.920 --> 06:47.600]  JPEG Excel decoding, and a lot of things on SVTV1.
[06:47.600 --> 06:51.040]  So yeah, the channel layout API was developed in 2013,
[06:51.040 --> 06:52.000]  I think, by Vittorio.
[06:52.000 --> 06:53.480]  I'm not sure he's around.
[06:53.480 --> 06:54.760]  Yeah, Vittorio.
[06:54.760 --> 06:58.960]  That was done during the fork, and it was quite complex.
[06:58.960 --> 07:01.760]  But this is good, because it's ready to do what we called,
[07:01.760 --> 07:03.800]  well, well, marketing calls NGA, which
[07:03.800 --> 07:05.160]  is next generation audio.
[07:05.160 --> 07:07.960]  What marketing also calls Dolby Atmos,
[07:07.960 --> 07:10.320]  those kind of object-based audio,
[07:10.320 --> 07:14.360]  and the new channel layout API allows
[07:14.360 --> 07:18.000]  to be a lot more flexible to custom layouts and weird things
[07:18.000 --> 07:20.160]  without us having to do everything directly inside
[07:20.160 --> 07:20.920]  the FFMPEG.
[07:20.920 --> 07:23.560]  So, and I'm still not starting about my main topic, which
[07:23.560 --> 07:25.800]  is FFMPEG 6.0.
[07:25.800 --> 07:27.360]  I hope when I was submitting the call
[07:27.360 --> 07:32.200]  that this would have been tagged, and that's important.
[07:32.200 --> 07:35.640]  I think this is even bigger in terms of a number of commits,
[07:35.640 --> 07:40.080]  and mostly in terms of contributors, because in the last six
[07:40.080 --> 07:42.760]  months, there have been around 191 contributors.
[07:42.760 --> 07:47.400]  That's huge, and that's a lot bigger than the previous release.
[07:47.400 --> 07:49.280]  What is important?
[07:49.280 --> 07:54.360]  There is not that many important API breakage and changes,
[07:54.360 --> 07:56.120]  but there is new APIs.
[07:56.120 --> 07:58.120]  And also, it's a major bump, so we
[07:58.120 --> 08:00.840]  are going to remove more things that were deprecated
[08:00.840 --> 08:02.080]  in the last few years.
[08:02.080 --> 08:04.920]  And there was two new APIs, so that we
[08:04.920 --> 08:06.760]  didn't remove them in 5.0, but we're
[08:06.760 --> 08:09.160]  going to remove that soon?
[08:09.160 --> 08:11.800]  Soon.
[08:11.800 --> 08:18.480]  One of the major changes is one of the most difficult thing
[08:18.480 --> 08:22.360]  that we've seen is multishralling the FFMPEG CLI.
[08:22.360 --> 08:28.000]  So all those big guys are at YouTube and Vimeo and Facebook,
[08:28.000 --> 08:33.280]  and all those providers of FFMPEG nice UIs
[08:33.280 --> 08:35.240]  are basically one of the things they complained about
[08:35.240 --> 08:37.080]  is the lack of multishralling and FFMPEG.
[08:37.080 --> 08:40.600]  So they invent a lot of weird frameworks to do that,
[08:40.600 --> 08:44.600]  so there is a lot of work to do that directly inside FFMPEG.
[08:44.600 --> 08:47.720]  It's going to go on for the whole year, I think,
[08:47.720 --> 08:50.560]  for all 2023, but that means that a lot of things
[08:50.560 --> 08:52.600]  will be better for you to use.
[08:52.600 --> 08:54.160]  And of course, when you do that, you
[08:54.160 --> 08:57.080]  need to actually care about thread safety and cleanups,
[08:57.080 --> 08:58.560]  so that's a lot of cleanups.
[08:58.560 --> 09:01.400]  What was done for 5.0 was that the mercs are now
[09:01.400 --> 09:04.680]  in their own threads, there will be more things.
[09:04.680 --> 09:07.440]  There is now a risk 5 optimization, or at least
[09:07.440 --> 09:11.600]  the framework to do that, inside FFMPEG.
[09:11.600 --> 09:13.040]  One of the things that is important
[09:13.040 --> 09:16.400]  is that you've probably seen that all the big guys building
[09:16.400 --> 09:18.800]  GPUs have now shipped AV1 encoders.
[09:18.800 --> 09:22.800]  So in 6.0, we've got Intel, N, Nvidia, and AMD.
[09:22.800 --> 09:25.720]  So you can actually encode AV1 in hardware,
[09:25.720 --> 09:27.160]  and that's actually very fast.
[09:27.160 --> 09:32.000]  You can reach 30 FPS in 1080p without any problem with those
[09:32.000 --> 09:35.480]  cards, and it's actually decent quality.
[09:35.480 --> 09:37.680]  It's not as good as the SVT AV1, of course,
[09:37.680 --> 09:40.160]  but it's pretty good.
[09:40.160 --> 09:45.920]  There was a lot of work on the FFT code by Lynn.
[09:45.920 --> 09:48.520]  She's over there, she can tell you about that.
[09:48.520 --> 09:52.360]  And I think it's like, I don't know how much faster it is,
[09:52.360 --> 09:54.800]  but it's a lot faster, so all the audio codecs
[09:54.800 --> 09:57.280]  and all the audio filters that require to use the FFT
[09:57.280 --> 10:01.560]  and sometimes is better than the external FFT libraries
[10:01.560 --> 10:03.080]  that everyone is using.
[10:03.080 --> 10:06.720]  New API for restricted frame for encoders,
[10:06.720 --> 10:09.400]  API breakage for deprecation.
[10:09.400 --> 10:12.920]  We have of course what I hate, lots of new YUV format
[10:12.920 --> 10:14.640]  and pixel format, because there is always
[10:14.640 --> 10:16.080]  a good reason to add them.
[10:16.080 --> 10:19.640]  And when I'm downstream as VLT, I hate that, but any.
[10:19.640 --> 10:25.800]  Lots of things on channel layouts and H.274, mostly
[10:25.800 --> 10:30.160]  about external filters.
[10:30.160 --> 10:31.920]  One of the big parts on those features
[10:31.920 --> 10:33.400]  is everything related to hardware.
[10:33.400 --> 10:35.760]  So I said about everyone hardware recording,
[10:35.760 --> 10:39.120]  a lot of pixel formats, especially for hardware.
[10:39.120 --> 10:40.720]  There is finally the Android media
[10:40.720 --> 10:45.360]  codec using directly NDK, and not with a Java crop that is
[10:45.360 --> 10:48.440]  directly integrated into FFMPEG.
[10:48.440 --> 10:52.080]  I think that requires API Android 23,
[10:52.080 --> 10:54.000]  but I'm not exactly sure.
[10:54.000 --> 10:56.520]  And we also have the encoding and not just the decoding,
[10:56.520 --> 10:59.040]  but also based on media codec.
[10:59.040 --> 11:01.440]  We have the Intel folks have done a lot of things
[11:01.440 --> 11:06.920]  to have a 10-bit, 12-bit, 42444 VP9 decoding directly
[11:06.920 --> 11:07.840]  inside FFMPEG.
[11:07.840 --> 11:11.040]  That's one of the reasons why we have new pixel formats.
[11:11.040 --> 11:13.080]  In terms of actual features, there
[11:13.080 --> 11:17.120]  is of course lots of new codecs, lots of new filters.
[11:17.120 --> 11:19.960]  The ones I prefer are the FTR, which
[11:19.960 --> 11:24.320]  is a annoying company who doesn't want us to reverse engine
[11:24.320 --> 11:25.280]  is that.
[11:25.280 --> 11:30.560]  Bonk, APAC, there is a SIM SSIM 360 filter,
[11:30.560 --> 11:34.920]  and some very cool bitstream filter for the DTS to PTS one.
[11:34.920 --> 11:35.560]  Look at that one.
[11:35.560 --> 11:37.520]  It's quite useful.
[11:37.520 --> 11:38.280]  Yeah.
[11:38.280 --> 11:41.440]  So FFMPEG CLI multi-threading, as I said.
[11:41.440 --> 11:43.520]  This is partly done in 6.0.
[11:43.520 --> 11:47.680]  It will be continued on 6.1 and 7.0.
[11:47.680 --> 11:50.840]  It is difficult, and it's long.
[11:50.840 --> 11:55.400]  But this is going to improve all your lives, or at least,
[11:55.400 --> 11:59.120]  especially if you want to do a multiple HLS dash,
[11:59.120 --> 12:00.920]  multiple transcode, multiple resolution,
[12:00.920 --> 12:05.800]  and do that directly without using third parties.
[12:05.800 --> 12:09.280]  FFMPEG releases, this is a slide I took exactly
[12:09.280 --> 12:10.520]  from a previous talk.
[12:10.520 --> 12:14.240]  And we never talked about that during first time,
[12:14.240 --> 12:15.960]  so that's why I'm talking about it.
[12:15.960 --> 12:20.840]  The problem that was like FFMPEG releases were kind of,
[12:20.840 --> 12:22.320]  well, before there was not.
[12:22.320 --> 12:26.080]  So we all took the good show on, and hope it was great.
[12:26.080 --> 12:28.680]  And then we were seeing what Mplay was doing,
[12:28.680 --> 12:29.640]  then VLC was copying.
[12:29.640 --> 12:31.680]  And well, if Mplay on VLC agreed,
[12:31.680 --> 12:33.000]  then everyone was using that.
[12:33.000 --> 12:35.720]  Then we started having releases done by Michael,
[12:35.720 --> 12:37.960]  and sometimes they were not very predictable.
[12:37.960 --> 12:42.160]  So one of the idea is to start to come to a more predictable
[12:42.160 --> 12:47.200]  fashion, which is one major API break and API break
[12:47.200 --> 12:50.800]  every year around December, January, so we're in February,
[12:50.800 --> 12:51.880]  and we fuck this year.
[12:51.880 --> 12:53.800]  But that's the idea.
[12:53.800 --> 12:57.280]  So one major where we allow API and API breakage.
[12:57.280 --> 12:59.480]  We remove APIs.
[12:59.480 --> 13:02.160]  When it's deprecated, it must be there for two years
[13:02.160 --> 13:03.080]  before we move that.
[13:03.080 --> 13:07.640]  But we will bump the SO numbers.
[13:07.640 --> 13:11.080]  And then one or two releases during the year,
[13:11.080 --> 13:14.480]  depending on security and what we need, so 5.0, 5.1.
[13:14.480 --> 13:17.280]  And every two years, one of the.1 will be LTS,
[13:17.280 --> 13:20.480]  and we'll continue that for two or three years.
[13:20.480 --> 13:22.920]  So the plan was to do FFMPEG Cs.0 in January.
[13:22.920 --> 13:24.960]  I hope it's going to come next week.
[13:24.960 --> 13:25.440]  We'll see.
[13:28.760 --> 13:32.040]  Yeah, this was not on schedule, so I'm adding a shorter talk
[13:32.040 --> 13:35.080]  in the middle of my two talks.
[13:35.080 --> 13:38.440]  Zavid 1.0 was released last year.
[13:38.440 --> 13:41.320]  It is insane.
[13:41.320 --> 13:46.560]  200,000 lines of handwritten assembly,
[13:46.560 --> 13:49.360]  I don't think there is any open source project that I've had.
[13:49.360 --> 13:51.800]  I'm not sure there is even a non-open source project that
[13:51.800 --> 13:54.520]  has that much assembly.
[13:54.520 --> 13:57.000]  And yes, handwritten assembly is faster
[13:57.000 --> 13:59.640]  than using whatever version of whatever compiler
[13:59.640 --> 14:02.000]  and activating whatever amazing feature that is going
[14:02.000 --> 14:04.200]  to auto vectorize something.
[14:04.200 --> 14:07.800]  We still do five, eight, 16 times faster than C,
[14:07.800 --> 14:10.120]  so don't bring that up.
[14:10.120 --> 14:14.360]  It is insane, yet it's necessary.
[14:14.360 --> 14:16.160]  So when you decode everyone, so everyone
[14:16.160 --> 14:21.640]  is now in all your iOS devices, all your Android devices,
[14:21.640 --> 14:23.880]  all your applications that they code everyone.
[14:23.880 --> 14:26.240]  It's on macOS, it's on Windows, it's of course in Chrome,
[14:26.240 --> 14:29.240]  it's of course in VLCMPV and all the other things.
[14:29.240 --> 14:32.080]  So it's literally everywhere.
[14:32.080 --> 14:36.160]  A lot of work was done in the David 1.0 about frame-threading.
[14:36.160 --> 14:39.240]  Like there is lots of, please see the talks from Ronaldo
[14:39.240 --> 14:40.480]  for a few years ago.
[14:40.480 --> 14:40.840]  Wow.
[14:40.840 --> 14:42.760]  OK, thank you.
[14:42.760 --> 14:46.240]  About the different spreading models that are inside David,
[14:46.240 --> 14:50.640]  and David 1.0 has everything in a simpler way.
[14:50.640 --> 14:55.600]  We are going to, it's extremely fast, very fast.
[14:55.600 --> 15:00.720]  David 1.0, 1.1 releases is coming soon, soonish, a lot
[15:00.720 --> 15:02.560]  of fixing, especially because there
[15:02.560 --> 15:05.400]  were a lot of conformance tests that we were not passing.
[15:05.400 --> 15:07.600]  And for some reason, they got out.
[15:07.600 --> 15:09.560]  And there is, of course, lots of new assembly,
[15:09.560 --> 15:11.360]  especially for AVX 512 and Neon.
[15:14.040 --> 15:15.320]  Cool.
[15:15.320 --> 15:16.800]  We're going to speak about, well,
[15:16.800 --> 15:22.600]  I'm going to do a demo, which is vlc.js, which is actually
[15:22.600 --> 15:24.920]  not in JS.
[15:24.920 --> 15:27.600]  So what are we talking about?
[15:27.600 --> 15:33.600]  Yeah.
[15:33.600 --> 15:36.960]  So this is Chrome.
[15:36.960 --> 15:39.840]  And this is why I'm on macOS and not my usual Linux
[15:39.840 --> 15:41.920]  for the people who wonder.
[15:41.920 --> 15:45.880]  This is vlc.n, ffmpeg, and all the dependencies
[15:45.880 --> 15:47.600]  compiled to WebAssembly.
[15:47.600 --> 15:50.760]  And what you cannot see, but this
[15:50.760 --> 15:54.000]  is doing hardware decodings through WebCodecs, right?
[15:54.000 --> 15:56.040]  So what happens here is that what you're seeing
[15:56.040 --> 15:59.440]  is that it's actually decoded on the hardware through WebCodecs.
[15:59.440 --> 16:01.360]  And then you take the output frame
[16:01.360 --> 16:05.320]  directly into WebGL, and, well, OpenGL ES2, which
[16:05.320 --> 16:08.760]  is compiled to WebGL, and display that.
[16:08.760 --> 16:12.880]  And this is a 4K H264 MP4, blah, blah, blah.
[16:12.880 --> 16:13.720]  OK.
[16:13.720 --> 16:14.920]  That's boring, JB.
[16:14.920 --> 16:17.200]  I can do 4K H264 everywhere.
[16:17.200 --> 16:18.360]  Sure.
[16:18.360 --> 16:19.040]  Sure you can.
[16:19.040 --> 16:22.800]  So let's do something a bit more complex.
[16:22.800 --> 16:25.840]  So this is the same, probably a Divx,
[16:25.840 --> 16:28.040]  except it's MKV.
[16:28.040 --> 16:30.000]  The MKV part is done in Wasm, right?
[16:30.000 --> 16:31.600]  It's a normal VLC demuxer.
[16:31.600 --> 16:33.440]  There is no JavaScript involved, right?
[16:33.440 --> 16:37.920]  I'm not demuxing MKV and remixing as MP4, like HLDS.js.
[16:37.920 --> 16:39.760]  It has, of course, chapter support,
[16:39.760 --> 16:43.120]  because, well, what's the use of that?
[16:43.120 --> 16:48.080]  But also, if I found my mouse again, no worries.
[16:48.080 --> 16:52.600]  Yeah, you have also chapter subtitles,
[16:52.600 --> 16:54.080]  which are not WebVTT, right?
[16:54.080 --> 16:56.680]  Normal DVB subtitles.
[16:56.680 --> 16:58.880]  OK, so that's not too amazing, right?
[16:58.880 --> 17:00.520]  So let's do something more complex.
[17:04.240 --> 17:05.600]  OK.
[17:05.600 --> 17:12.400]  4K VP9 in software decoding directly inside the web browser.
[17:12.400 --> 17:14.360]  OK, that's pretty much better, right?
[17:14.360 --> 17:17.360]  WebM on macOS, right?
[17:17.360 --> 17:19.400]  So, well, yeah, but professional.
[17:19.400 --> 17:24.000]  They use, like, actual format, like MP4.js.
[17:24.000 --> 17:25.520]  Let's do.
[17:25.520 --> 17:28.080]  So that's something that is ATSC over the air, right?
[17:28.080 --> 17:31.280]  So that's htvc83ts, right?
[17:31.280 --> 17:32.920]  All the stack that is not in the web browser.
[17:32.920 --> 17:37.040]  It's decoded and displayed directly into your web browser.
[17:37.040 --> 17:40.120]  And that's where you realize that the US TV is really dumb.
[17:43.000 --> 17:48.600]  OK, but, OK, so that was hardware accelerated or not,
[17:48.600 --> 17:50.720]  because that's why it's TVC.
[17:50.720 --> 17:52.280]  As you can guess, right?
[17:52.280 --> 17:55.960]  I can either force web codec or AV codec.
[17:55.960 --> 17:59.200]  So now I'm going to force software decoding.
[17:59.200 --> 18:01.280]  And I'm going to show you something a bit more complex,
[18:01.280 --> 18:07.120]  which is this is Korean TV, which is interlaced.
[18:07.120 --> 18:10.640]  And the interlacing is happening as a WebGL shader
[18:10.640 --> 18:12.040]  directly inside your web browser, right?
[18:12.040 --> 18:15.360]  So we're decoding 8264 interlaced.
[18:15.360 --> 18:16.840]  Of course, we cannot do that by hardware,
[18:16.840 --> 18:20.240]  because, of course, API doesn't support interlaced codec.
[18:20.240 --> 18:22.960]  So we decode full in software.
[18:22.960 --> 18:26.640]  And then we display directly and do sharpening
[18:26.640 --> 18:30.840]  and the interlacing of those very old multicast formats
[18:30.840 --> 18:32.840]  without any change.
[18:32.840 --> 18:34.680]  OK, and I guess I got no.
[18:34.680 --> 18:36.040]  Yeah, I got one minute more.
[18:36.040 --> 18:39.840]  So this is DNECHD.
[18:39.840 --> 18:43.320]  Of course, the output is not 420, but it's 420, 422.
[18:43.320 --> 18:47.400]  And that's actually interlaced and decoded as MXF directly.
[18:47.400 --> 18:49.240]  All those professional formats, you
[18:49.240 --> 18:51.480]  can play that directly inside the web browser.
[18:51.480 --> 18:56.680]  So of course, if we can do 422 and down sampling for 420
[18:56.680 --> 19:03.640]  in software, well, on the GPU, I can also do 444.
[19:03.640 --> 19:08.000]  So this is YUV444, 10-bit, right?
[19:08.000 --> 19:09.440]  Of course, BBB, right?
[19:09.440 --> 19:11.560]  But things that are absolutely not
[19:11.560 --> 19:15.360]  possible with any type of APIs.
[19:15.360 --> 19:19.120]  I probably also can show you that we can do other filters.
[19:19.120 --> 19:25.760]  And this is a 360 movie that we have with the support on VLC.
[19:25.760 --> 19:30.720]  And of course, all the mapping from Tetidal to Equial Drunk
[19:30.720 --> 19:33.760]  Duo is done on the GPU.
[19:33.760 --> 19:36.000]  Of course, that means that everything
[19:36.000 --> 19:40.520]  that we do with LiPlasibo in theory should get in.
[19:40.520 --> 19:42.960]  And I'm out of time.
[19:42.960 --> 19:43.960]  Thank you.
[19:43.960 --> 19:53.960]  Thank you.
[19:53.960 --> 19:55.440]  Do we have any questions from the room?
[19:55.440 --> 19:56.680]  Yeah?
[19:56.680 --> 19:59.480]  So an eight question.
[19:59.480 --> 20:03.440]  So you said you have 200,000 lines of specialized code.
[20:03.440 --> 20:09.080]  So perhaps there is no slowdown when you flip or rotate
[20:09.080 --> 20:12.960]  the image or do such transforms because you
[20:12.960 --> 20:14.680]  have a specialized version for that.
[20:14.680 --> 20:15.520]  Is it so?
[20:15.520 --> 20:17.240]  You mean on David?
[20:17.240 --> 20:18.240]  Or here?
[20:18.240 --> 20:18.720]  Oh, sorry.
[20:18.720 --> 20:20.480]  I cannot differentiate.
[20:20.480 --> 20:23.800]  So on David, it's really the decoding part.
[20:23.800 --> 20:25.640]  David is 200,000 lines of assembly
[20:25.640 --> 20:26.880]  just to do the decoding.
[20:26.880 --> 20:29.120]  It's around 35,000 lines per architecture.
[20:29.120 --> 20:30.560]  And we do lots of architectures.
[20:30.560 --> 20:33.360]  And then they give you a decoding surface.
[20:33.360 --> 20:37.120]  And then we use it in VLC, in PV, in FFMPag, and we do things.
[20:37.120 --> 20:39.840]  And here, we would compile it directly inside WebAssembly,
[20:39.840 --> 20:41.120]  run that in WebBrowser.
[20:41.120 --> 20:45.280]  And all the transformations are done then on WebGL.
[20:45.280 --> 20:46.600]  So that doesn't change anything.
[20:46.600 --> 20:51.920]  Just to know whether there is a slowdown of any amount
[20:51.920 --> 20:55.240]  just because of those common transforms, you say?
[20:55.240 --> 20:58.320]  No, that shouldn't be.
[20:58.320 --> 21:00.640]  On the dark question?
[21:00.640 --> 21:04.640]  Can you compile assembly to WebAssembly assembly?
[21:04.640 --> 21:06.920]  Like, could you compile David in the WebAssembly?
[21:06.920 --> 21:12.200]  Could you handle the WebAssembly?
[21:12.200 --> 21:14.720]  So one of the things that we tried with Lynn, again,
[21:14.720 --> 21:18.280]  was what we call an assembly transpiler,
[21:18.280 --> 21:21.440]  where you take actually the ARM handwritten assembly
[21:21.440 --> 21:25.480]  and try to transpire that to webassembly.scmd, right?
[21:25.480 --> 21:28.360]  So that you could use neon and do exactly the opposite of what
[21:28.360 --> 21:30.520]  the WebBrowser are actually doing,
[21:30.520 --> 21:35.600]  where they take the wasm assembly
[21:35.600 --> 21:39.640]  and compile that just in time for neon and so on.
[21:39.640 --> 21:43.480]  It's a very insane project that I had the idea a few years ago.
[21:43.480 --> 21:45.400]  It's really not sure whether we are going
[21:45.400 --> 21:48.800]  to be able to do that because you're transpiling assembly.
[21:48.800 --> 21:51.000]  Like, what the fuck are you talking about?
[21:51.000 --> 21:53.040]  But yes, I think that's the right solution
[21:53.040 --> 21:55.960]  instead of rewriting again all the assembly from FFMPag
[21:55.960 --> 21:57.720]  and David again.
[21:57.720 --> 22:01.280]  If you have time, please come and help us.
[22:01.280 --> 22:03.240]  I might actually find also some money for that,
[22:03.240 --> 22:05.720]  if people care.
[22:05.720 --> 22:09.760]  Please ask questions.
[22:09.760 --> 22:12.720]  I don't eat people.
[22:12.720 --> 22:13.080]  Yes?
[22:13.080 --> 22:16.400]  Last time I checked, compiling straight into WebAssembly,
[22:16.400 --> 22:18.920]  everything that was thread, posix,
[22:18.920 --> 22:21.080]  everything was pretty not yet finalized.
[22:21.080 --> 22:23.160]  Like, what is the process for compiling?
[22:23.160 --> 22:24.720]  It works fine.
[22:24.720 --> 22:28.520]  But that's also why you see that I'm on macOS, right?
[22:28.520 --> 22:30.760]  And I'm on Chrome and not displaying my usual Firefox
[22:30.760 --> 22:32.760]  and Linux because in order to have threads,
[22:32.760 --> 22:36.640]  you need to have what we call shared array objects,
[22:36.640 --> 22:39.280]  which is basically common things.
[22:39.280 --> 22:42.200]  And because in the web world, what they call web work,
[22:42.200 --> 22:45.000]  particularly serialization and deserialization to move data,
[22:45.000 --> 22:50.360]  now this is almost done, works everywhere, mostly on Chrome.
[22:50.360 --> 22:51.840]  Now works on Safari.
[22:51.840 --> 22:54.840]  It works on Firefox, but it's buggy.
[22:54.840 --> 22:56.600]  And also one of the things that is annoying
[22:56.600 --> 22:57.880]  is the off-screen canvas because you
[22:57.880 --> 23:00.240]  want to be able to read directly in the back buffer
[23:00.240 --> 23:03.200]  before displaying it that doesn't work anywhere correctly.
[23:03.200 --> 23:06.480]  And finally, the hardware decoder only works in Chrome
[23:06.480 --> 23:07.920]  for now.
[23:07.920 --> 23:11.080]  But maybe Firefox will do something, won't it?
[23:11.080 --> 23:11.680]  Oh, sorry.
[23:11.680 --> 23:14.200]  What's the concept of the payload that the pages
[23:14.200 --> 23:16.880]  to download to get that?
[23:16.880 --> 23:20.000]  25 megawatts?
[23:20.000 --> 23:21.920]  So the idea is that we're going to, like,
[23:21.920 --> 23:24.200]  VLC is a module approach.
[23:24.200 --> 23:28.120]  So there is a very small core and 400, 500, 600 now,
[23:28.120 --> 23:29.480]  maybe, modules.
[23:29.480 --> 23:33.960]  And so what I want is to actually be able to load.
[23:33.960 --> 23:36.080]  And that's almost working, in theory,
[23:36.080 --> 23:37.360]  so you can load a shadow object.
[23:37.360 --> 23:40.080]  So you would only stream the core,
[23:40.080 --> 23:43.840]  and then the core will know which one to go and download.
[23:43.840 --> 23:44.840]  Yes, Steve?
[23:44.840 --> 23:50.600]  You mentioned that there were big patches for FNPEG
[23:50.600 --> 23:53.560]  to organize the code.
[23:53.560 --> 23:58.360]  It's easier to read emails, so when you pre-clap them.
[23:58.360 --> 23:59.640]  I'm not answering that question.
[23:59.640 --> 24:01.800]  Thank you.
[24:01.800 --> 24:04.680]  So the question was, when is FNPEG community
[24:04.680 --> 24:09.000]  moving to a sane thing, which is GitLab?
[24:09.000 --> 24:10.080]  Not my shorts, right?
[24:10.080 --> 24:11.760]  Like, you know my opinion, right?
[24:11.760 --> 24:16.640]  Videolan, VLC, all the iOS, Android ports, X264, and so on.
[24:16.640 --> 24:18.920]  Even David is on GitLab and would like it.
[24:18.920 --> 24:21.560]  The FNPEG community prefers to stay on email.
[24:21.560 --> 24:24.200]  So I think it's a mistake because we are
[24:24.200 --> 24:26.080]  losing too many patches because it's
[24:26.080 --> 24:29.800]  very difficult to, but that's a community opinion,
[24:29.800 --> 24:32.480]  and the community is a majority.
[24:32.480 --> 24:33.680]  Last question.
[24:33.680 --> 24:37.000]  So if you render it in OpenGL or in the VLCJS,
[24:37.000 --> 24:38.840]  you're bypassing the media engine, right?
[24:38.840 --> 24:41.440]  So how do you do the audio video sync?
[24:41.440 --> 24:43.120]  Well, of course we are.
[24:43.120 --> 24:47.120]  So the answer is, how are you doing audio video synchronization?
[24:47.120 --> 24:48.000]  Like VLC, right?
[24:48.000 --> 24:49.560]  Like the core of VLC.
[24:49.560 --> 24:52.000]  VLC was done by this guy and other guys
[24:52.000 --> 24:54.680]  to actually do live TV, right?
[24:54.680 --> 24:56.600]  So the core of VLC is a clock, and the clock
[24:56.600 --> 25:01.440]  is basically working on doing synchronization audio and video
[25:01.440 --> 25:03.960]  and resampling the audio when it's too late and so on.
[25:03.960 --> 25:06.000]  So here we are doing exactly that, right?
[25:06.000 --> 25:08.280]  So we output audio with Web Audio.
[25:08.280 --> 25:09.240]  Well, no.
[25:09.240 --> 25:11.400]  A small part of Web Audio called audio worklets.
[25:11.400 --> 25:13.880]  And so we know how much is actually being played back,
[25:13.880 --> 25:16.200]  and then we control the V out, which is basically
[25:16.200 --> 25:18.240]  the core of VLC to synchronize audio and video.
[25:18.240 --> 25:19.320]  And we're using that there.
[25:19.320 --> 25:22.640]  But I'm not using any type of media source extension
[25:22.640 --> 25:24.840]  or any other open media, blah, blah, blah.
[25:24.840 --> 25:27.960]  We are really like, it's mostly a video game.
[25:27.960 --> 25:52.960]  OK.