All right, good morning.
Again, I'm Sergio Amarata.
I'm a board member on the RISC forum,
and an active member of the RISC committee that
writes specs for RISC.
And I'm also the maintainer of the Librisque Open Source
project.
So today, we'll be talking about how RISC can support end
to end live streaming with packet recovery.
But in particular, I will explain
how we can support this in a broadcast scenario, meaning
streaming to as many users as your bandwidth can support.
So we'll cover the topic in two sections.
First, we'll provide a roadmap or an update
on the RISC specification and the Librisque project itself.
And then we'll go to a practical application
and show you how you can do live streaming in a large scale
with the open source tools provided.
So on to part one, the development roadmap.
The last time I gave an update at FOSDEM regarding RISC
was February 2020, a few days before the pandemic shutdown.
Now, four years later, we will explore what happens
instead.
I guess if I have waited one more year,
I could have blamed the Thanos snap for the delay.
So let's do a brief recap of the beginning of the protocol.
In 2017, the VSS forum created the RISC activity group
for the purpose of creating a unified interoperable protocol
for transmission of IP data over loss in networks.
The requirements were that it needed
to be based on the UDP protocol, and it needed
to include negative acknowledgment retransmission
requests.
So one year later, after a successful multi-vendor
interop event, the simple profile specification
was published.
You can see that in the bottom.
The RISC activity group then proceeded
to add multiplexing and encryption capabilities
and publish the main profile specification in early 2020.
It was at that time that the Libris Library Open Source Project
first was published.
And you can refer to my talk I gave back then,
where I go into detail explanation
of what the simple profile does and the main profiles
and the differences, et cetera.
So as you can see on this slide, the RISC activity group
has been quite busy adding features to the protocol
to accommodate all possible use cases
during the last four years.
What started with the simple profile, the first release,
as the desire to add packet recovery to an RTP stream
with an MPEG-TS payload, has now turned into a reach protocol
that will work with any payload and which includes
multiplexing, encryption, and authentication.
So Libris, the open source project,
currently supports a simple profile and main profile.
And we're working on adding support for the advanced profile.
So in addition to the core specifications for the protocol,
the RISC activity group has also published a series
of recommendation or best practice documents.
These are documents that extend the protocol into specific applications,
into specific niches, and you need to actually consider that
in the library that is compatible with this specification.
So the library Libris, when applicable,
has been made compatible with these recommendations,
the code synchronization, the relay, et cetera, et cetera.
So enough history about the protocol and the specification documents,
those are all publicly available.
They're not behind any payload.
The VSF documents can be downloaded, the PDFs,
and you can look at the specs and all the recommendations.
Let's talk about the Libris open source project itself, right?
In case you are not familiar with what RISC is,
we can define it with just one simple sentence, like you see up there.
It's a new protocol for transmission of IP data
across lossing networks using UDP with NAT-based retransmissions.
Before getting into anything else,
I'd like to clarify the three most common misconceptions
people have about the RISC protocol.
And this has come about in talks and conversations.
People tell me, oh, well, RISC is only for MPEC-TS, false.
Advanced profile includes support for any payload
with clearly identified payload types in the header now.
There's even an registration with support binary payloads, et cetera.
And misconception number two, you need a large buffer
and therefore latency is large, a second or more, right?
In order for you to use RISC, false.
You really need two to six times the round trip time,
the RTT, between the two endpoints
you're trying to send the data through.
So the shorter the RTT, the total buffer required will also be shorter
and you can talk about 10 milliseconds, 20 millisecond total latency.
It's just depending on what network you're deploying it in.
In addition, and this is a major misconception
on that second point, RISC also supports real-time data channels
with no added latency, with lossy channels
that you can have APIs going back and forth in real-time
and send data that cannot wait for these buffers.
Misconception number three, you can only use RISC
for transmitting in one direction, right?
You send data over there, this is packet recovery, you're done.
That's false.
The protocol allows for bidirectional transmission
both with and without packet recovery on both directions.
The limitations are usually introduced
by the implementer of the protocol.
The specifications are broad enough
so that each implementer has the freedom
to add or remove features at will.
So let's talk about the Libris Development Roadmap.
How do we determine where to go next, right?
So we divide it into three categories.
The first one is we want to improve the reach of the library.
And by improving the reach, we mean improving the adoption
of the library by client applications
so that users can go ahead and have it available on every device.
Libris, of course, adds support for all the different specs
like I showed before and all the recommendations
so that all these reach features that make the protocol
have more use cases are available immediately
under the Libris library.
The second is distribution, right?
We make sure that our library compiles on every platform
so that it can easily be adopted by anybody
and that it makes it when possible
into open source applications like FFMPEG, Libris, OBS, etc.
As a matter of fact, when running it within the video LAN servers,
it compiles in all 21 different architectures
that are predefined in their CI.
So we're pretty confident that if somebody wants to use it, they can.
In the distribution aspect, we also have it on the major distros now
available in Debian, OpenBSD, etc.
And the third aspect of how to determine what the roadmap is
is we do timely enhancements and timely bug fixes
very quickly when they come about.
So on the feature set, I think that the most important addition recently
that allows the application to be used into this broadcast market
like one too many, the media service scenario
is the EAPSRP6A authentication protocol.
It was introduced in 2022 and what it allows you to do
is instead of the normal model where you have a pre-shared key
that you have to share among two endpoints
which makes it very insecure because if that communication of that key
for the encryption gets compromised, your entire network is compromised now.
This protocol allows you to do a username and password,
a unique username and password for each of the connected clients
and part of the protocol, doing that username and password exchange
which is different for them, includes the negotiation
and the exchange of the pre-shared key.
So there's no risk anymore of that pre-shared key to ever be compromised.
Other features that allow the broader adoption
is that we're working on a one-way satellite application,
we're working on multicast discovery and a few other things.
So third aspect, the distribution.
Many FOSS projects already have Libreps compiled by default
or have it as an option.
If you know of additional projects or if you know,
please drop me a line, I'd like to keep a database
of which projects are actually already included in it, if possible.
Ritz is also a part of my own day-to-day operations
which gives us the advantage of finding the bugs
before they are found in the wild and we fix them very quickly.
OK.
So performance enhancements over the last few years,
we now have the ability to automatically configure
based on the network conditions.
The Libreps library does an RTT with a new packet
that was released, the Echo packet, 10 times per second.
What that does is it lets us measure, with a UDP,
you know, ping, not a regular ping, the network conditions
between the two endpoints.
We know the inter-packet spacing, the variance, mid and max,
we know the latency, we know all these things
and with those values, with those parameters,
and if you look at the default configuration,
the library will auto-adjust its buffer
to the perfect buffer for that link
without you having to guess or know anything about the network.
It will also adjust the initial buffer,
the reordering buffer based on your jitter on the network.
Your inter-packet spacing jitter, gaps in maximum jitter,
and make sure your reorder buffer is at least that much.
We've added, you know, because we've done these
very large improvements, we've realized we need better metrics,
so we've added support for Prometheus
and other things straight out of the library,
so that you can actually grab that and, you know,
plug it into third-party tools
and immediately create your dashboard
that gives you the proper visibility in the connections.
And, you know, last release was just a couple of months ago.
So the top priorities for 2024 for the development roadmap
is we want to add support for DTLS encryption and authentication.
We want to fully add support for the new advanced profile
that adds, you know, the new header ID with the special payloads.
And we want to try to see if we can get back support
of the library into VLC 3.0.
So the goal of the original project, like we mentioned before,
was an interoperable standard for this type of transmission.
There were, you know, half a dozen or a dozen different methods
or there still are of doing UDP with packet recovery,
each vendor specific, et cetera, et cetera.
Our goal was to create an interoperable standard
with multiple implementations,
and I think we've achieved that at this point,
at least at the higher broadcast level
and tier one, tier two companies
and a lot of the open source projects that support REST.
They all talk to each other,
even if it's not the same implementation.
So now to part two, right?
Let's look at REST as a live streaming platform, right?
And particularly we want to look at a model that does N2S.
How do you use REST and Libris in particular
to do an N2N streaming chain,
like the one we're doing here, for example,
or, you know, any one-to-many scenario, right?
Lots of viewers.
So let's diagram, you know, a simple scenario here.
We have three components,
sources, the sender, which is a REST device,
and many receivers on the bottom,
and the box here on the bottom, you know,
symbolizes a single one of those receivers.
So we see the logos up there for FFMPEG, BLC,
and Open Broadcast Studio.
That could be also G-streamer, any source,
any encoder, it doesn't matter.
Somebody that has the ability to generate
compressed or uncompressed video stream, right?
Well, we need a binary stream of some kind
pushed to the library.
Libris in particular doesn't care about what the payload is.
You can push anything in the payload,
we'll deliver that to the other side,
even though the spec for simple profile and main profile
say that you're transmitting MPEG-TS,
the library doesn't look at the payload
or restrict it in any way.
Okay.
So the source is sending a UDP,
or RTP media stream into the input.
We buffer it so that we have it available
for retransmission, and the minute the buffer is full,
we start listening on, we put the sender
in what we call listening mode.
It opens a UDP port and start listening
for receivers that want that stream, right?
So the minute our receiver wants to connect to us,
then the handshake happens.
I'm obviously oversimplifying the process
of the handshake that all happens.
The SRP68 protocol is quite complex.
It would take a talk just to go through the details
of that handshake and everything that happens.
So this is only symbolic.
The handshake happens.
The username is sent to us,
and we check for that username within our database
of username and passwords.
It's not really a data-major password,
but a password hashes to keep everything safe.
If the authentication succeeds,
then we send as part of the SRP68 protocol
the pre-shared key so that the receiver
can decrypt the data now.
Once the data is decrypted, that's it.
We have an end-to-end transmission from source
to hundreds of destinations
with just the risk protocol in between.
So with proper planning and setting everything up correctly,
you can have a 300 millisecond glass-to-glass,
one to hundreds of listeners.
You need a good network.
Like I said, the latency is more dependent
upon the RTT between the endpoints than anything else.
I mentioned 300 milliseconds
because in our large-scale deployments,
we've done this anywhere within the U.S.
with 300 milliseconds glass-to-glass.
When you have to expand it and have users that are
across the ocean or with crappy networks or Wi-Fi,
the latency will auto-adjust.
The protocol will auto-adjust.
For those players, suddenly they get 500 milliseconds.
We notice as a rule of thumb that somebody in Wi-Fi
gets a penalty of another 200 milliseconds automatically.
So how do you do this from a practical point of view?
The LibreSploracle includes some command-line utilities
that allows you to send, receive, and relay.
The RISC2RISC is the one...
If you want to do a relay application one-to-many,
this is the ideal scenario.
You can also do it with a RISC sender, to be honest,
but the RISC2RISC is effective because it acts as a relay,
doesn't encrypt or decrypt, doesn't do anything,
but receives data and sends data both in the RISC format.
You can put this in a CDN, your data sender anywhere,
and you configure in the RISC2RISC a listener
with authentication, and then you put your stream from anywhere,
your source, like from here, to that endpoint.
Then you configure the other end,
the one that's going to send to the older viewers
with a database of user-oriented passwords,
and now you have the full authentication.
It adds no additional latency in that process.
It's only the latency that you decide to put as far as buffering.
As far as quality and quantity,
the sweet spot seems to be between 3 and 5 megabits per second,
resolution 720p or 1080p, whatever code you're using
gives you better or less quality,
and that seems to traverse all the different VPNs,
corporate networks, et cetera, without any issues.
Quantity, the RISC2RISC can handle 100 simultaneous connections,
and the number seems low,
but because of the threading model
and the fact that it has to do retransmissions,
after that the retransmissions get compromised.
The way you scale is that you can instantiate multiple instances
of the same RISC2RISC application within the same machine,
and in our case, we have 1500 simultaneous viewers
going off of this type of transmissions 24-7.
So the RISC password utility is also a command line utility
available on the project that allows you to create
the username and password combination hashes,
just like the HD password file in Apache has a similar format,
that's why we created it this way.
You run the utility, put a username and password
and that outputs this username with a hash,
and then you append that to a file,
and then the sender can grab that file
and use it as an authentication database.
In the case you want to scale that to a much higher level,
you integrate directly with the library
and you use the library callbacks to do the authentication
yourself against your own databases,
and you can scale that to thousands of users.
The command line sender is a typical scenario
of what I put in the diagram,
what I was using in the diagram,
you put the input any type of UDP stream,
output you encrypt it, and then the output URL,
if you look at RISC, is in your column, column,
you add 127, you add the add,
just like you do typically for FFMpeg or VLC
or that type of stuff, when you want to listen instead of send,
and it creates a listening on that port,
and that's all you would need to do to create a sender
and use the sender as a really, as well, just for one stream.
On the receiver side, you want a player, for example,
that you can put the username and password, right?
You put the RISC in FFMpeg as the input,
RISC, column, forward, slash, forward, slash, et cetera,
or VLC, or any one of your choice.
In our case, we did a custom VLC application
inside of Raspberry Pi where we were doing this 1500
at the same time.
There were Raspberry Pi's running VLC 3.0 inside
with a lib-RISC implementation inside.
The transmission of the secret in this case,
which is a password for the username and password,
should be handled in the same way you share passwords now
for any account outside the scope of the protocol,
and that's it.
Then it becomes very simple to create a large-scale network
with this.
So the summary is the key feature for this
is this new type of authentication
that makes the secure implementation on a large scale,
and it gives you better latency, lower latency,
then the equivalent HLS or dash,
with a security model that's built into the protocol.
It's no longer the browser or the DRM inside the browser,
everything.
It's the protocol handles the entire DRM.
So we have a really solid roadmap for the future.
We were looking for additional contributors
and people that want to help adding the next set of features.
We're looking for open-source projects
that want to implement the library.
We'll help you put it in.
And that's it.
Thank you very much.
Thank you.
Okay, the question is, what if you're pushing your stream
to Africa with a really bad connection?
What is the acceptable packet loss?
I'm not sure what you mean by acceptable packet loss.
To me, zero is an acceptable packet loss,
and the protocol is capable of achieving zero
if you give it enough buffer.
You give it a second buffer,
and the round trip is 200 milliseconds,
and you will get zero packet loss.
We've done tests and we've done transmissions between Australia.
I was just two weeks ago doing a demo,
a transmission from Australia to Madrid.
16 cameras at 10 megabits per second each
were being transmitted in real-time using RISC,
and they were being used in Madrid
for a production of the event.
And the transmission didn't have a single packet loss,
and it was all done across open internet.
We used one second buffer there
because the connections were relatively good,
but if you go and, you know,
if your transmission is really bad,
just increase your latency, and the protocol will recover.
We have part of our CI integration process
tests that add 50% and even 75% packet loss.
And you see spikes in bandwidth,
but we recover every single packet
if you give it enough buffer.
Does it support simultaneous build rates?
Yes, we support multiplexing.
In all this example, I've done just one UDP input.
You can configure the library and the command lines
to ingest multiple UDP inputs,
give it a different ID,
and then on the other side, you can demultiplex them.
I assume that's what you mean by maybe having
different build rates within the same stream.
The camera, like, sends it on the fly
according to network to the combination?
Correct, yes.
And one of the specifications that you saw
on the recommendations was called source adaptation.
It was written precisely to accommodate that scenario.
What is the best case, best use,
or the best practice recommendation
on how to do source adaptation?
Reduce the build rate, adjust the build rate
based on network conditions.
It's all documented in a part of a spec as well.
So for non-MPEC-PS payloads,
as you mentioned,
is there already a mechanism like a composite trail
to basically define the mapping of different payloads?
Absolutely.
For advanced profile, there's a GitHub repository
that has the mappings already.
We have a dozen or two dozen of them.
I'm one of the administrators of the repository.
All you need to do is go in and, you know,
put an MR for whatever binary payload you want to define.
All right, thank you.
I have another question here.
Is it also possible to multiplex and demultiplex subtitles?
Yes.
The protocol itself doesn't care what you put in.
We consider each of them as a binary payload of some sort.
You're the one that determines what the format of that payload is.
And you have this pipe.
You put multiple UDP streams.
One of them is going to be your VTT payload
or closed caption or whatever you want to put in
with whatever format you want.
We don't define or control the format of what you put in.
We do to decide on multiplex and mulling.
We give you the capability to give them IDs
so that in the other side you can map those IDs
to different outputs when it comes out.
Thank you.
But it means that you don't do any timing, right?
In between the different streams.
That's all user-side.
Well, no.
When you give us...
The question is,
that means that you don't do any timing or synchronization.
On the contrary,
because we are taking care of the multiplexing,
when we ingest all the different UDP streams,
the timing is guaranteed.
The minute we receive that UDP stream,
we actually, in the library, the implementation that we did,
we grab the timestamp at the network card.
This stream came in at this time,
and then we reproduce that exact timing on the other end.
We reproduce the spacing, the pacing,
and the latency.
We make it fixed, so that is not variable.
That means that when you multiplex many things
in the same tunnel,
you're guaranteed they're in sync on the other side,
or at least as they were when they came in.
We're starting the use cases of the protocols
to the more for...
kind of the current adoption on endpoint devices,
mobile devices, browsers.
Okay, the question is,
the use cases of the protocol,
what is it towards more,
point-to-point devices,
point-to-multipoint, browsers, etc.
This is the last question of our time.
The original idea was to just do point-to-point transmissions.
That was the original scope
when we created the first version of the spec.
That has changed.
We achieved that, and now we went beyond that.
Now we want to tackle the distribution.
We want to tackle the one-to-many, the media servers.
We have actually a project going on with Miss Server
to add a lot of this functionality and the scalability
as part of the project itself,
so that we have at least one media server
that already supports that in a very scalable way,
where it becomes very simple for an application
like VLC, or VFF Play, or Gstreamer
to hook up to this media server
and start the playback immediately using the Pshuoroko.
Thank you very much.