How is it possible?
About interactive live streaming in a very worse.
How is this possible or is it possible?
To me, I'm Enrico and I'm interested in interactive live streams.
Sorry.
So, now it's better. I'll take it so.
Sorry.
Here are my contactless and I worked for different companies
and even most likely in a conference system topics.
And now we're talking about lessons.
And in the 30 versus is quite interesting situation.
When you're in a 30 versus for example when you're in Macedon,
you read in post.
The interesting point here is the post came to you.
Means you have an app in Macedon or inclined and you don't care who posted the post
on which instance the post itself is cloned from instance to instance
through the further worst.
Means you get a copy or a clone of this post.
This is a quite interesting concept.
So the instance in the background communicating to each other.
How is he doing this?
Of course with activity part, we had to talk right before this.
So I will not go deep in it but the main idea of activity part is like
you have an inbox and an outbox.
And everyone in the 30 versus in terms of activity part is an actor.
The users are actors, the servers are actors.
And on the end you can send to every actor in the 30 versus a message or a post.
And that's the way how it works.
So activity we describe the things like in activities, it's like activity part,
like subscription, follow and so far.
And the other topic is content.
It's all described in JSON.
And how I said, the instances in the background communicating to each other
and the content is flowing through the 30 versus.
Activity part and live streams.
They are in the 30 versus already implementation of activity part like
OWNcast or PeerTube are the main famous.
But the thing is we want a little bit more.
I mean you have in OWNcast and PeerTube live streams but not interactable.
It is not possible.
It means without leaving your PeerTube instance or leaving your OWNcast instance
you cannot interact with another stream or another instance.
It's not possible.
Yeah.
That leads to a problem.
It's called scaling in the 30 versus.
That means on the end more or less the...
More or less every instance provider in the 30 versus responsible for himself,
you have to scale by your own.
You have the possibility of course with hardware
where you make an HLS CDN on top of this or this object storage.
Those are the common ways how you can increase the amount of users that can watch you.
But on the end you stay alone more or less.
PeerTube try to solve this problem with PeerTube mail loader.
It's quite awesome.
Sometimes you see it.
You're watching a video and then you see that other people are watching you as well.
This means PeerTube Peer exchanging the chunks of HLS files.
We are bit torrent and over-verb.
It means you make a real PeerTube Peer connection to the other viewers.
I put it on the top because this is the most common way in the 30 versus to share live streams.
There are other ways as well, but most likely the basement PeerTube here in the browser.
There's another way, it's web torrent in the background.
Of course they can clone...
Even PeerTube can clone videos from one server to another server.
This is possible.
And the new concept is remote runners.
This is quite awesome.
You can scale PeerTube with a remote runner.
It means you can run other services that do the transcoding for you.
Quite often it's re-expensive.
This is the possibilities you have to scale your application or your instance.
Oncast has a quite interesting feature.
Oncast has a general concept.
Oncast is you have a server and you only stream for yourself like this.
But they have a dashboard.
On the dashboard you can see every live stream in this time.
But this dashboard is nothing else than an HTML page.
They are linked to the live server.
It means it's like a list of links.
It's not really scaled because when you're watching there a stream, you're watching it from the server as well.
This is the current state of it.
But what we have now, we have ActivityPub.
It is possible to share the information there as a live stream.
This already worked as PeerTube as well.
There's a live stream but you cannot share the stream itself.
And what we want is we want to share a live stream.
So in the live stream you want to have it interactive.
Means an interactive live stream is a little bit more as if you have a stream with a track like a video and audio.
No.
We want to have it, you have a stream with a track and the tracks inside of the streams can change.
You added new tracks, you added removed tracks, you enabled tracks, you disabled tracks
and the tracks coming from different sources, different instances.
When we can reach this, then we have interactive live streams in the Furryverse.
It's not only that you share a stream, a static stream, it's a little bit more.
This is what you want to achieve.
It's like a conference in the Furryverse.
And we already talked today about it.
There's a protocol, it's called WIP and WAP.
Of course we need a real-time protocol.
It's clear we need WIP and WAP.
It's a real-time protocol, it's a moment.
And on the other side, there's another interesting approach, WIP and WAP.
In short words, what is WIP and WAP?
You make an HTTP request to a server and receive a WAP-ATC resource.
That's it.
No complicated signaling, only an HTTP request.
It's a little bit like an activity path.
You make a request and you get a resource back.
This is written there.
For the first one, you make a request to offer a resource.
Hey, I have a resource here you can have.
And for the second one, you make a request, you subscribe to the resource.
This is only the different.
This is the main idea.
When you have this, here's a little bit more in detail,
you can ignore this one, the eyes, only this two are important.
You offer something with an HTTP, of course, and you get something back.
And then you have all what you need for the resource.
Finish.
And then you have such kind of architecture.
You can do something like this.
A, you are sent off a resource like a client.
You offer this to an endpoint.
And the endpoint offers to the next endpoint.
This is for WIP and for WAP, turn around as well.
It's like you can make an, you can establish like a pipe.
Yeah, sounds, it's really great.
And then you can do this, you can clone streams.
Because when you clone streams, only you send a request to an endpoint.
Give me this and send this to another endpoint and clone this to another site.
That's it.
However, there's a problem.
WIP and WAP is static.
You cannot update the resource.
When you one time have offer and the resource as a miss a request,
you get an STP and you cannot update the STP anymore.
It is static.
Means you will receive a track, all the tracks that insert in the stream
and nothing more, no way.
Means you have a static resource.
It's cool for a live streaming, but we want interactive live streaming
where the resource are changed.
This is quite important.
So we want a little bit more dynamically inside WIP and WAP.
This is not enough for us.
And our trick sources is two things, two important things.
A little bit smaller things, but the two main ideas behind us is like this.
When you subscribe an egress endpoint and receiving a resource,
you have to subscribe as well a channel.
It is so opposite.
You get a channel as well.
Because you need a channel to get the information that the egress resource,
the receiving a resource is updated.
This is the first thing, what you need.
Without is not possible.
Normally you do it in a conference system.
Perhaps you do it with a signaling server, your resources update,
you get a new STP.
But we only want rest.
We have no WebSocket server.
You need established an extra resource like a channel to receive this information.
The second point is you have to annotate the tracks.
You have to know what this track is.
For example, this is the main track or it's a guest track.
And here, Schick is using the STDP attribute media title.
It's not used normally, some people are using it,
but it's there for title of the track, for example.
Here it's used for some meta information.
For example, it's the track that you received as muted first,
but the track is the main track or another track.
And the rest is activity problem.
You rely on the things.
Yeah, Schick itself is an instance written in Go, based on PyM.
It came with the JavaScript SDK.
You get in front end, it's a web component, not an iframe.
You get in web component.
And this SDK is implemented in PeerTube plugin.
Because Schick itself can nothing, only makes this exchanging.
And it looks like this.
You have a PeerTube instance on the left side,
and you have a PeerTube instance on the right side.
You are here starting your stream,
and you want invite people on the form another instance.
This PeerTube instance has a possibility to a Schick instance,
and this is a complete other Schick instance.
They are not related to each other.
And this user is on his, and with this Schick instance,
and with this protocol and background,
he can exchange and communicate with each other,
like a conference, but this is a stream.
And then on this side, he is the owner of this,
he is in streaming this one.
It's then transcoded in RTMP,
because from RTMP then in HLS.
At the moment, I have not the direct HLS transcoding.
But theoretically, you can, from verbiage,
directly in HLS transcoding, but it's not implemented yet.
Yeah, and let us look how it's looked.
I think I have, yeah.
For this one.
Yeah.
So, I have here the two PeerTube instances.
I make it like this, and so like this.
It depends on the time I already created a live stream,
but you can do it directly now, because we have more time.
Sorry.
When you're looking, I'm not sure how familiar you with PeerTube.
Here, inside of PeerTube, I have the chick plug-in.
This is this one, and you can configure the chick plug-in,
and you have here, this one is relating to the chick server.
It's called stream.chick, means he knows this one.
Yeah.
Here's an ASESC, okay.
Theoretically, you can use this.
This is, okay.
And the other one, let me see.
Yeah, this is the other one.
Yes, as well.
That plug-in.
But he is related to forstem.chick, is another chick instance.
It's a complete different.
They are in different servers.
Yeah, they are complete separate from each other.
See, this PeerTube instance,
follow this PeerTube.
You see?
Means this one get all live videos from the other one, cloned.
And, of course, this one has his own chick is following this instance.
The communication between chick and the PeerTube is over activity pub.
So when this chick, when the PeerTube instance get a new live,
the chick get it as well as copy over activity pub.
That's the idea behind it.
The implementation is stored,
steal from owncast is exactly the same,
because owncast has a cool implementation for it.
Yeah.
That's a good idea, owncast and PeerTube together.
I only want to mention.
So, and what we can do now is,
we can create a live stream right now.
It's like this.
I hope I have time.
Yeah, I have time.
Make it permanent, makes no difference.
Yeah.
One interesting point when you create a live stream,
it should be short as possible.
PeerTube can nine second delay something like this.
Nine, fifteen seconds, something like this is the shortest what PeerTube arrives.
I mean, when we're talking about interactive,
it's definitely not take 30 seconds or 60 seconds.
It's too much.
Okay.
So what we can do as well is,
let us invite the other guy from the other instance.
What you have to know is the activity pub ID from this guy.
Yeah, this one.
Now we create a live stream.
I hope so.
No, we don't create a live stream.
I have to update the live stream.
Sorry.
My mistake.
So now we have a live stream.
Here's online.
And in the back,
I have to take this one because I'm not figure out how I can find this live stream
than on the other side.
Maybe someone will explain.
Now activity pub has synced to both.
So we have the live stream as well on the other instance.
So when I have this one, I'm logged in as user one to three.
I can assess now here.
I'm now in.
Now I'm in the web component.
It's a web component rendered in peer tube from the plug-in.
It's not an iframe.
And I can do this here as well.
So now two guys in two different stream, but they are not connected at the moment.
First, they have to join.
He's joining.
And he's joining as guest.
Takes a while.
So let me see.
So now we can do it.
And of course we want the other guy is seeing something.
So now the internet is a little bit slow.
Sorry about this.
Now they're both on different check instance, different SFOs.
And the SFOs communicating with them and established with only rest endpoint.
And the information like mute and unmute what you need.
And exchanged like, sorry.
Like the channel that for the web egress component is established.
And even when I, let me come back.
And even I can do this one.
Sorry.
No, I can't.
Sorry, the connection is bad.
So you see the other side.
Now I have the track mixed.
So I can even mix the live stream.
And then all is working fine.
Theoretical wise, and my internet goes not down.
I can online goes as well.
I can go live with this.
Let me see that he can see this live as well.
One moment.
I think it's here.
Yeah, it was here.
Somewhere here.
This one should be.
Yeah, now we are live as well.
Okay, sorry the internet is not so good.
Yeah, that's it.
And so we have established a clone stream between two instances in the first bus.
That's it.
Yeah.
Yeah, question.
I'm curious.
I've worked a little bit with Activity Pub, but not Super Induct.
I'm curious if there's like a, is there a live stream post type in Activity Pub,
such that like other implementations like a master.on server or something could play this live stream,
or does it look like just a link to a live stream?
How does that go this way?
The question is, is there an Activity Pub attribute or something like inside, right?
I'm not sure.
You have the content type of video inside, and you have as well the annotations that it's a live video or not.
This came from PeerTube itself alone.
So, inside of the JSON is only the host server inside.
It means when you share this JSON to another PeerTube instance, you get a description like who is the owner,
which actor is the owner of this live stream, and where is the home server,
the home instance for this live stream.
This is all what we have inside.
And then, Schick annotates this with extra attributes like who is the guest,
and this has the host server at Schick instance.
Because you can only follow with Schick another instance when your own instance has as well a Schick instance.
When you not have a Schick instance, the button to join, you have to go then to the other instance.
This is the main. This is the mechanisms behind it.
I think, what's the question?
Yeah.
Okay.
Yeah.
This only works when both instances implement in Schick instance.
And this is supposed to work as well for own cast, because it makes no difference.
Only the front is needed for own cast.
And this is the main idea behind it, that you have a way to scale your streams in the background with extensions.
Yeah, based on activity.
Perhaps an interesting point.
It's like a little bit controversial.
You can use such kind of technology for, I will not say advertisement, but for recommendations.
When you have a live streams, often you have the problem you want inform other people that you have as well live streams.
Other people didn't know about you.
And here you have something like a pool where you can add streams and then you can chat doing the live streams.
Because in a back, a live streams and an active live streams, nothing else as that you have different kind of sources from different kind of furry growth instances.
And such kind of things are then possible.
Okay.
Okay.
Yeah.
You mentioned that you're using data channels to change information about back of this.
What exactly is set up the data channel?
Renegeration, the STP.
I have the egress endpoint.
I mean, the receiving end point needs a data channel from the offer of the resource.
The question was what came through the channel, the STP.
The STP and the mute event as well.
Yeah.
This is coming soon.
Yeah.
What's the reason for the delay so much lately?
Here in this one, I think also what's the reason for the delay in the latest thing?
First, the network here, I guess.
Second one, no, most likely the network.
I have this one here.
One moment.
When you have this one, I hope I'll be online still.
I'm not sure.
This delay, what you have here, this is more bigger.
This came from the transcoding form.
VapRTC to RGMP.
That is at the moment not optimized.
This is the reason for this delay where you have such kind of, yeah.
But the rest, I think it's the network.
I guess.
So it's not VapRTC to VapRTC.
It's converted somewhere?
It's like this.
You have a VapRTC to VapRTC converted.
Which one you mean between the server or between the?
On the right-hand side, the video is quite delayed.
Yeah.
Where did the left?
Yeah.
This one.
Yeah, there's a big delay at the moment.
Yeah.
Yeah.
Now, the thing is, in this case, you have three VapRTC connections now.
One is from the client.
Maybe I can show you this here in the slides.
Sorry.
You have three connections.
One to your chic instance.
One from the chic instance to this one and one to this one.
It's like a pipe.
And I guess this was this quite fast because they are in the same location.
But I guess this one makes a trouble at the moment.
I guess.
Yeah.
Some other question?
Yeah.
I missed part of the presentation, sorry about that.
As far as I understood, you are using Weep and Web as a way to get those two to communicate with each other.
So, as I was saying before, in the last year, the view of what specification basically forces you to create an offer for that as well.
So it makes changing Weep and Web impossible within the specification.
Are you using the old mode where you were expecting an offer to do something?
How are you dealing with this synchronization where you have to wait for an offer and stuff like this?
Yeah.
I try to repeat the question.
Weep and Web, I think, have two options.
First, you send an offer and get an answer back.
And second, the second option is you say, hey, I want an offer from you.
Then you get an offer and you send the answer back.
What is the difference between this one?
For the first, you need only one request.
It's like, give me one post request.
You send an offer and get an answer back inside the post request.
For the second option, you send first a post request, get an offer, and send again a post, a patch.
I think it's a patch afterwards.
Yeah, something like this.
I implemented the second one because I implemented it in June and I think now is a new version out where they are supposed only one request.
Yeah.
For Web, for Weep in one, for Weep, I only need one request.
Yeah, that's right.
But because we are not here, I not use Weep and Web how it's supposed to be because I need to dynamically, so I established Web at the C Channel as well.
So that is additional.
Okay.
Yeah.
Yeah, if no questions anymore, then thank you for watching.
Thank you.
Quite interesting.
Yeah, because you're talking about this problem already.
I wrote a long post because I liked the old mode.
I liked the way that we are doing things.
Federation is possible thanks to the mode.
Just leave a couple of minutes to sit down.
Yeah.
Yeah.
Yeah.