I need to support me for the in-depth technical details because he is more proficient than
I am in these areas.
This is a very high level overview of the project.
We are not going to go deep into the detail, but if you have questions, let go deeper,
just ask them in the end.
If you want to have product side view or customer view, you can always use the official contact
channels and you will be answered there.
So a little bit of background about OpenTalk.
There is a company behind OpenTalk.
It was founded in 2021 in the middle of the pandemic by a group, so a group is doing since
more than 30 years I think already consulting and training for Linux and mail operations
hosting and it is also the provider of the well-known mailbox operator MailboxOrg.
And the OpenTalk company right now has around 20 employees right now, so it is increasing
slowly but steady.
So who are we?
I am Wolfgang.
I joined OpenTalk roughly one and a half years ago and became the backend team tech
expert, so more or less the technical lead in July this year, or last year already.
I have a master's degree in embedded systems design, but I am much more on the software
side than on the hardware side.
I am doing Rust since 2015 and I am still in the honeymoon phase and from all the languages
I have done, this is the longest honeymoon phase I ever had.
And I am the co-founder and organizer of a Linux user group and you can find me on the
FEDIVERS.
So Stefan.
Yeah.
I guess I have been like two and something years with OpenTalk now and I am mainly on
the media team which is our thing for all the real-time stuff, audio, video, recording,
streaming, webRTC.
It is kind of in between front and then back end.
And yeah, I also have been in university before, long time doing parallel programming, operating
system stuff and also some real-time things and software defined radio.
So if you are interested in that, just talk to me later on I guess.
Okay.
Some information about the project in general.
So the project is written, or the front end is written in TypeScript, the back end is
written in Rust.
It is free software under the copy left EUPL 1.2 license.
You can find technical documentation online under this domain docs.opentalk.eu.
There is also a FEDIVERS account called OpenTalk Meeting.
You will find it by that.
And there is a Matrix channel as well, hosted on matrix.org.
This is, yeah, the Matrix channel is where some of the devs are hanging around and answering
technical questions but it is not an official support channel in that regard.
Okay.
So the user interface, this is what the video conferencing software looks like.
So it is roughly similar to what you know from other programs.
It was important to make a nice design that looks good and is, yeah, comfortable to use.
We also have what we call the dashboard.
This is where you can create meetings.
You can add start and end date.
You can add meeting series and you can also get an email or maybe that's on the next slide.
You get an email when you are invited to a meeting or when a meeting is canceled and
also the creator of the meeting gets the invite so they can put it into their own calendar.
Okay.
So short list of the features.
We have a lobby with a mic and camera check so you can check that everything is working.
We have some interesting moderation tools, one of them being the coffee break which we
will show in the next slide, a timer so you can assign tasks to people and say, okay,
you have 10 minutes for this and if you want then report when you are ready and when everybody
is ready the timer ends or when the timeout is approached.
Meeting participants, we have a poll feature and breakout rooms.
Screen share, yeah, that's well known for conferencing software.
One important information here is that multiple people can share the screen at the same time
which comes in handy for peer programming.
Yeah, you have the speaker view where you always see the large picture of the speaker
of the person who is currently speaking.
You can call in from the mobile or landline phones via SIP and we have integrations for
a shared editor which in this case is Etherpad and a whiteboard which is SpaceTech currently.
Yeah, I already said the invitations end.
Right now we are in the course of finishing recording and streaming so you can record
the meetings and you can as well live stream it and the idea is to also allow streaming
it to multiple platforms at the same time so you can have YouTube Twitch and on-cast
stream at the same time if you want.
If you are interested in that, talk to Daniel over there, he did one of the work.
Yeah.
Okay, so here you see a screenshot of the coffee break, that's what it looks like.
Everybody gets this full screen as soon as the coffee break is started but you can go
back into the conference anytime you like.
So for chit chatting up front before everybody is back, just like in real life.
And this is another nice feature we have, we call it the shared folder feature.
So in the dashboard when you create the meeting you can enable this shared folder switch.
It must be configured for this OpenTalk instance but then the system will create a shared folder,
it will create a folder on a next cloud instance.
This is the part that needs to be configured.
It will create two shares to this folder, one of them being read write and the other
one being read only.
And the moderators of the conference receive the read write link so they can put their
material into this folder up front while all the other people have access to this either
by clicking on the link in the invitation mail or by opening it through the icon during
the conference.
Okay, so this is a more technical part, I'll give the word to Stefan here.
So that's what it looks like from a rough perspective of the developer or the administrator
of the system.
So it's not just one big service but we tried as much as we can to use existing components.
So what we built mainly is the dark or the dark colored parts and the other services
are more or less what you get just from the different projects.
So we use Yarnos and RabbitMQ for communication and Yarnos as media gateway but we manage
all the video rooms our own using our controller backend and as said there is a web front and
written in TypeScript and React here but it's kind of symmetric to what happens on the other
side with the, I like to call them back end clients for streaming call and all that stuff.
They just have another way of starting the whole process but actually they do the RTC
and signaling just as the front end would do and by now they also have a way to authenticate
services against key cloak via service authentication so that's also, we can see that later, where
you can extend, that would be a way to extend our system in that part.
It's meant to be scalable so you can have multiple instances and they just share their data
where Redis and so forth is session stuff and for the persistence data like which rooms
do we have, what users are in which rooms invited, that stuff that would be stored in
the normal relational database and we'll do a lot of integration stuff on that OpenID Connect
key cloak side with other like user systems or databases what people tend to have already on
site. Okay so this is a sneak peek of Rust code, it's currently not ready yet but we have
approaching this. We are right now working on extracting the protocol data types into a
separate library which was not the case when I started working with OpenTalk and the idea is to
publish the client library to crates.io which is the default publication platform for Rust code
and yeah it should be as easy as this, I mean the authentication is usually a little more
involved than these two lines but you basically connect to the instance and can do things with
the client so this is now the web API for managing appointments and so on so here we create an
event with a title that we set and then we invite a user with the email address elizetexample.com
and the role of a regular user you could do the same for a moderator as well so the idea is to
allow automation and integration in a very easy and approachable way if you're familiar with
Rust code. This is also what we will be using for the recorder which connects to the meeting
session for the call-in via a landline or telephone and for other maybe future services.
So yeah talking about these kind of services that's actually the flow you have there, you build your
new backend service which will act like a client to the conference, it first needs to
authenticate and get some access token however you set it up and then you usually just go to the
backend and say hi that's me and that's the token I got so I'm authenticated and I would like to
join this room over there which has this ID and then you essentially and by that you open a
web socket where all the signaling happens and you see like the publications of media streams
so the backend will just announce when new users arrive and will also announce what screen share
and camera stream they have so you can then start the webRTC connection with the Janus and on that
signaling channel you just exchange STP and other credentials to get the right stream set up and
here you would like in our case we usually use GStreamer as a media stack here which is then
that up to get all the streams and for instance do video mixing and when you're done with your
recording so somebody tells you on the signaling okay stop now recording you will just upload the
file of the video it produced to the controller again which puts it into a F3 storage which is
also currently we use for development purposes we use Minio but you can use whatever F3 you would
like and there it also becomes available on the dashboard then which also would work with other
artifacts so like whiteboard or yeah meeting minutes would be the same thing just another
document format right and what I missed out is the other way is when you don't initiate
the action yourself there's also the RabbitMQ service where you can just attach and listen for
the controller to call you and say hey your service you should do something like start a
recording for instance and then just start the signaling session right that's that's basically it
yeah okay that's also your part yeah so we talked or we've seen a lot of components which are open
source and which we integrate there's also been as we are a company that also been other
yeah companies and software developers which we integrated with so I guess that's one of the
main things and themes that we and other people have yeah projects and try to integrate with
each other and there is like UCS and integration where they basically have their key cloak and
they use a management part and we just connect there and there is Innova phone which does mainly
zip and has some also some platform and we try to integrate there also wire my D connect and
made some adjustments to our zip stack so that we are compatible with them and yeah so it goes on
like MGM is like we just started I guess they they talked about how we could do like components
where you just would have the video but it's like in the starting phase and not just the whole front
end and yeah as much people many people use it right now and this has been a high demand we did
outlook plugin but there's also been some talk I know for Sunderbird plugin but it's just not
yet yet on the way I guess and so yeah maybe just if you have some some questions or need or want to
do something on your own just talk to us and we'd be happy to try to tell you what's going on and
to support it as far as we can okay yeah yeah that's it more or less
so we try to keep it short so if there's some specific questions and details yeah I'm gonna just go ahead
you haven't mentioned entry and encryption yet at all and I know that did see has already some
support of entry and encryption and also matrix is now getting into the real-time communication
business and I was wondering what is your strategy here yeah I can say a word I guess it's it's not
so easy is the starting point the thing is if you want to do end to end encryption you basically
don't trust the back end that's that's a deal and we're talking about a web application right now
which is like a problem because in the first place you would load your application from the
server you don't want to trust so we are looking into how we can ensure that you can really maintain
the integrity of all your personal keys and all that stuff and that's pretty hard to do in a
browser environment and yeah of course we could encrypt media connections but that's just half of
the deal so yeah basically we're in the process that's also a goal for certain projects we're working
in but it's not yet a thing I can say okay that's that's a route we we're gonna take right now and
here are the details so we didn't put it on the slides yet so if there are question on then
topic yeah maybe we can have a have a discussion in detail later on or maybe if you have specific
needs and that direction also let me know I'm interested in what do you consider that are
still like very important features or properties which is not yet in any open source video
conferencing solutions and which you are working on which you also don't have yet but you're working
on what is what are kind of still important pieces to come so yeah as mentioned this is a whole
streaming and recording a part which will right now in is one of the main things so we can support
like bigger conferences with a with a feeling of being in a room so for now we're just doing the
the low levels or finishing the low level streaming part and the first UI part to enable
streaming but we're thinking a lot about how to integrate like the to have a mode where you have a
stage and an audience and the stage would be like a normal web conf and web RTC conference and the
audience would get to see the live stream and get a chat interface but it would all happen in our
user interface that's something to come I guess but we have no time frame for that right now and
the other part we are from the project side in is all the telephony part like zip and 8 3 2 3 I
guess which is the old video conferencing standard on telephony nets yeah I guess there's much more
but there was another question so I reckon an organization was 100 people but once in a while
we host conferences for a few thousand and now I wonder should we then have a very large
a Janos media gateway just for this one event per month or is there a way to scale easily down and
up the resources because I've heard of Federation of Media servers in the matrix context and I think
this is a very interesting concept when organizations have joint conferences so yeah we we also thought
about that hard and long and there is like a limit on if you don't cascade Janos instance
there's a limit on how many subscribers can be for a single publisher so the speaker in the room
and that's for for our experience in the yeah say three to four hundred depending on how you
configure load balancing and all the stuff and instead of doing cascading and all that we are
right now looking more into the streaming direction then into have it and having it cascading and real
time all because usually the audience will not interact heavily and you would have to invest a
lot into getting all of the people like fast in there it might be a thing and we are looking also
into the matrix how matrix does it with underlying they use live kit as far as I know
but yeah we are exploring the other direction was having it on streaming and getting people in and
out of the room or you know so into the web RTC conference or back into the stream view that would
be my take on that because then you can just have a have it more resource efficient like have a
small meeting which is easily manageable and also have a streaming set up which can easily scale
lots of people thank you so the question is is there a support for island audio as in in a
large meeting where two people can talk to each other alongside with the orator without
interfering with the others this has been on the road map for quite some time already the idea is to
lower the main room audio volume and have a private talk with a subgroup of the conference but it has
not been implemented yet I guess it's already we already have an specification for it but not
the time to build it yet