Okay, I'll start because the 10 minutes apply to me as well, even though I wear this nice
blue shirt.
So please sit down and I'll start right away.
So I'll be talking about social audio applications that you may want to re-implement with Janus
if you want.
Quick slides about me.
Nobody cares.
But what is social audio?
It's basically whenever you have something that is primarily audio and not strictly video
as part of their formal communication.
So whether it is messages or podcasts or virtual audio rooms or stuff like that, you may have
heard about stuff like Clubhouse, Twitter Spaces, Reddit Talk, they are all examples
of social audio.
So people talking with each other, maybe they take turns and then they broadcast to a very
large audience.
And of course it does seem like a very good fit for WebRTC, especially for the real-time
kind of participation.
And you didn't hear about me because I don't know if there is any secrets about that, but
actually Twitter Spaces uses Janus for the live part and then they distribute it somehow
else.
And how do they usually work?
So as I said, they are typically live conversations, so we have a limited number of people that
talk to each other, exchange ideas, they take turns, so it's not always the same people
talking for two hours like a podcast, for instance.
And then you may have possibly thousands of attendees, like for instance any time Elon
Musk speaks in a Twitter space, there's a million of people listening, let's say things
like that.
And there are of course different challenges to tackle because for the live conversation
part it needs of course to be real-time because it needs to be something that happens as fast
as possible.
For the distribution to the audience you may want, a bit of latency may be okay and this
is why for instance they take advantage of CDNs or stuff like that most of the times.
But of course there's a problem that of course the more latency you have for the audience,
if somebody from the audience needs to come into a conversation there may be a bit of
latency there.
And so that's something that needs to be taken into account.
And so you may want to use WebRTC for everything but there's scalability issues at play there.
And so I wanted to check whether or not Janus, which is the WebRTC server that I work on
for a living, could be used for the job.
And I came up with a few potential ideas and one of those may be relying on the AudioBridge
plugin.
The AudioBridge is basically an audio mixer that lives within Janus.
So you have multiple people connected to the AudioBridge plugin.
They create a single pair connection that the AudioBridge mixes all the audio streams
so that you send one stream, you receive one stream that contains the audio of everybody
involved except you.
Which is really nice because for instance it's easy to bring C-Pend points in if you
want using the plain RTP functionality.
You can play jingles, for instance you have your own show, your own context that you want
to play something in there or maybe a snippet from another conversation.
If you do stereo mixing which is support you can use spatial positioning of participants
to make it easier to understand for people.
And of course this takes care of the live conversation but we want to make it available
to other people as well so to a wider audience.
And so what you can do is take advantage of RTP for Worders which is basically an easy
way by which the AudioBridge plugin sends a plain RTP stream towards an address that
you specify containing the mix that is being mixed there.
And the nice feature in the AudioBridge plugin is that you can also tag participants so
that you may say don't send me a mix of all participants but only the ones that I tag
in a specific group.
For instance this one may be a technician so those two need to hear the technician who
gives tips but all the attendees only need to hear those two.
That's basically the main idea.
And of course whatever happens in here is basically handling a mixed stream so there
may be a script here that sends these mix to IceCast to make a very simple example
or to YouTube Live for Audio or to whatever platform you want to use as a CDN for distributing
the Audio if it's not WebRTC.
If you want to use WebRTC you can use something like this.
So you have your active participants connected to the AudioBridge they are talking to each
other.
You RTP forward to the streaming plugin which is the plugin in Janus that takes care of
broadcasting RTP to a wider audience.
And then the streaming plugin is what distributes the Audio which is the greatest advantage
that you don't have to perform specific mixing for these participants.
They are already receiving a mixed stream.
All people connected to the AudioBridge instead have a dedicated context for mixing because
they need to receive everybody except them so it's not the same Audio for all of them.
And whenever you want somebody from the listeners to join in the conversation they mute the
streaming parts, they join the AudioBridge temporarily, they become active participants
that everybody else can listen to because they are now mixed in the AudioBridge.
And of course for scalability purposes you can just RTP forward to multiple streaming
plugin instances on multiple different instances of Janus how you distribute it is entirely
up to you.
You can use a tree based distribution wherever you want and you can also take advantage maybe
of Multicast because of course if it's just a plain RTP stream that you are forwarding
if you forward it on a Multicast group then multiple Janus instances can all pull from
that Multicast group that same mixed Audio and can distribute it more efficiently.
And one other value is that using this approach if you want you can also do something like
interpreter services.
You have two different AudioBridge rooms for different rooms, you have the speaker join
the room of their language and you have an interpreter on the other room and then you
distribute those two streams separately and then you allow the audience to listen maybe
to the English channel or the French channel and depending on the language you will speak
in you will hear the translator or the actual speaker on either one.
So which makes little sense for an actual social Audio application if we want it's
maybe more for a conversational scenario but it's still a good side effect of that.
If instead you don't want to mix in Janus for a few reasons because you don't want to
terminate Audio there, mixing is more intensive or whatever you may want to use the SFU approach
instead which means that participants in the conversation now need to establish maybe one
single peer connection not necessarily more than one but they are exchanging multiple
Audio streams.
So they are sending their own and they are receiving as many as there are other participants
in the room and you can still externalize this conversation via RTP for Worders as before
but now Audio is not mixed so you have different Audio streams for each of the participants
there.
Each participant in the conversation each of them is sending one and receiving two and
you have a separate component that is receiving the three different Audio streams from the
different participants and so if you want to distribute something via regular CDN that
requires a single Audio stream to distribute and so that component receiving RTP for Worders
needs to act a bit like a mixer acting live basically.
And once this happens so once you have a mix there everything is pretty much as the example
as I made before you have a mixed stream you can distribute it via CDN or via Janus as
we've said before if you don't want to mix for the attendee as well you want something
closer to a regular webinar or something like this you can still do that but then you have
to take you have to use that approach that I was talking about of wording to the streaming
plugin for each of the different participants and so something like you have the presenters
that you're contributing Audio to the video room this becomes an Audio broadcast for that
specific presenter in the streaming plugin and people listen to that participant over
there you can again involve multiple streaming plugin instances if needed so that you can
widen the audience if you want but again if you have multiple participants speaking
you have to do the same for each of them because otherwise of course since Audio is not mixed
you would only listen to one single participant which means that the audience need to create
subscriptions for more than one participant at any given time and of course you have to
make this dynamic in case there's presenters that come and go basically which is what is
expected in a social audio kind of application which means that it's probably easier to do
something like this where you still do some sort of you keep the audio conversation using
an SFU for WebRTC participant because it gives a better audio quality between each them maybe
but then for distributing the conversation it's okay to mix it and so even mix it for
WebRTC usage so that you distribute a single audio stream instead which makes sense but
again if you want to do that that works for instance this is what we do for our virtual
event platform for meetings so that definitely works anyway and again you can also do this
sort of multicast distribution if you want to take advantage of a wider distribution
of the media and if I spoke too fast which is very likely I did write a blog post about
this which goes a bit more in detail and explains things a bit more precisely than I did right
now and I think I managed to stay on time and these are some references so you can find
me on mastodon mainly I'm still on Twitter but who knows for how long and that's the
blog post that I was mentioning before so that's all thank you
okay there's time maybe for one or two questions if anybody is curious so I don't know if you
have any not specifically in the audio bridge but this is something that you can enforce
at the application level if you want so for instance you may decide that some users always
need to be there and some some use so for instance you may have the concept of the actual
presenters and panellists that come and go for instance this is more of an application
level context than the mixing context as far as mixing is concerned you you just know yeah
exactly so any other question or can we move to so okay then okay thank you very much
for that one question