Okay, so we are going to start the next course.
The next course is about optimizing your co-applications for integration, learnings from integrating
of the project with NextCloud.
Thank you.
Hello, everyone.
My name is Wieland Lindenthal.
I'm one of the lucky co-founders of OpenProject, and today I'm here because I'm also the managing
engineer of the OpenProject NextCloud integration.
Just to get the first things of the most important messages out right now, I assume that here
are more core developers of open source software, collaboration software, and also product measures
here in the room.
For you, please take home this message, prioritize integrations on your roadmap.
Right now is a great moment where the public sector, not only the public sector, but many
people ask for integrations that we integrate our open source software together to provide
an alternative to the Office 365, to the Google Cloud, to Google applications, and so on.
Right now, people ask us that we join together forces, so together that we are stronger and
become stronger in each single application so that we provide an alternative to the
big tags.
Okay, so integrations are really important.
And with that, every single application, if we integrate, every single application doesn't
need to do what the other application does, so we can focus on our own stuff and get better
in our own silo, on our own puzzle piece of the big picture.
Another bad news, building integrations is unnecessarily hard.
The good news is, you always encounter the same problems again and again.
So that's kind of predictable what is going to fail when you do an integration.
So my call out here is, please drink your own champagne and participate as core developers
in the integration projects that integrate into your system, because then you see the
issues that people face when integrating.
Because integrations are amazing, right?
Just imagine what the web would be without a link or a browser would be without extension
or a mobile phone without application.
It would be boring, right?
And it's just unpredictable what integrations come out if you just let people do.
And people have great ideas.
So the one example of such a great idea was the public sector, the city of Cologne and
the University of Duisburg-Essen approached us with the idea of, hey, you had open project,
we like you really, really much for organizing work, organizing tasks, organizing our teams.
And we also use NextCloud for organizing our files to share the files with our people.
And we always can find the same problem again and again, where are the files for our tasks?
So when you do some work, you do a real job.
You usually need some files as input to work on the task.
And then at the end, you need to deliver some files and put them somewhere, and you need
to communicate someone where the files are.
So why not tracking that at the same entity, which is the ticket, we call it a work package,
and solve that problem and have some real integrity between the two systems, like work
package and real ID linking to the files in NextCloud.
So for the last year, roughly eight people were working on this deep integration, a lot
of people, because it's pretty hard.
But anyway, so what we got is what you can see here is like an open project with a work
package.
And you can see there's some tasks, like second fermentation dough, you want to do this, you
want to have a nice pizza.
And on the right hand side, you have the files tab, where you see all the files that are
related.
And now it's not only that you have attachments, which were there already not open project
before.
You can also add multiple NextCloud instances to your open project and say, OK, these files
should be stored here, with that contributor, these files should be stored there with this
client, with this subcontractor.
So you can have multiple NextCloud instances connected to OpenProject.
And you will always see which files are relevant for finishing a task and where you should
put the files when you're ready and done.
And you hand over the task to the next one, the next design, let's say, and you follow
a workflow process with an open project.
And on the other side, or NextCloud, when you're on a file or you're on a folder, you
can see where these files or folders are relevant and which tasks, which business processes
they come up.
So you can see here on the right hand side, we have this OpenProject tab.
They can see, OK, this file is relevant in two places and two work packages.
You can quickly see what status they are in so that the problem of where the files for
my task are finally solved.
Now I want to get a little bit deeper into the architecture of that thing because we
want to talk about problems that come with integrations.
So it's a very, very high-level architecture picture.
As you can see, there's an OpenProject server, there's an NextCloud server, there's an OpenProject
front-end that runs in the browser and the same browser that runs the NextCloud front-end,
same different browser windows.
And for this integration, we extended the OpenProject core so it's not a plugin, it's
right in the OpenProject core, which extends the OpenProject front-end.
And on the NextCloud side, we put everything into a NextCloud app, which is basically a
bundle of code that you execute within NextCloud, or you could call it a plugin or extension.
So that plugin extends the NextCloud server and also the NextCloud front-end.
What we chose for the communication between the two systems is that we want that every
user, because users have permissions on the one system and users have permissions on the
other system, we wanted to make sure that these things match up, OK, so that an OpenProject
user that is looking for a file can only see files, can only select files to link them
that he or she has access to.
But on the same time, we wanted to make sure that all linked files are visible to all users
because someone says, like, this is the relevant file, maybe you don't have access, but you
should know that these files exist.
So we have these references by file ID from OpenProject to NextCloud.
And the other way around, of course, as well, when you're in NextCloud and you saw these
files linked, the work packages linked, you also should only link those work packages
that you have access to.
So work packages are organized in projects, you're a member of a project or not, so depending
on this, you have access or not.
So we need to have some one-on-one matching between the users.
So we use OAuth for this one-on-one matching.
So users are always authenticated and don't need to re-authenticate every time they log
in.
OK.
But there's one thing on this architecture you can already spot, maybe, that it smells
like trouble.
That is the arrow in the middle that goes from the OpenProject front directly to NextCloud
because that is a request, a cross-site request, in the browser between two domains.
And here's where trouble starts.
We'll come to this later.
So clipping.
So yeah, from this work, I just want to come up with a couple of things that lessons learned
and I would just point them out for you to think and digest a little bit what it means
to your project.
So of course, OK, if you want to have integrations, of course, provide an API, that's obvious.
NextCloud and OpenProject does that pretty well, I think.
We have docs and examples.
But then the tiny things start, like, OK, you have an API.
You think it's cool and safe.
But in the example of this direct access that I set on the architecture image, this is about
direct file upload and downloads.
So this is something that is kind of obvious, that you don't want to send the data through
the OpenProject server, like, let's say if you want to go buy a movie, and then to NextCloud,
you want to upload this directly or download this directly to the OpenProject front-end.
So it turns out the API that NextCloud provides for this thing is working-ish, but not, you
know, in a way that it did not work, OK?
It didn't work, but it was really unfortunate because they had everything in place, but
I bet they never had the scenario in their own company to use that in a cross-site domain
environment.
And so they didn't come across, like, twice a course problems, you know, your course security
on the browser that prevents you to do stuff that you shouldn't do in the browsers, like,
browser security features.
And if you just have non-browser clients, these things don't show up.
If you just have a NextCloud client on your desktop, these things don't show up.
They are not browser context.
So my lessons learned is here, also for us OpenProject, we are not saving these fields
neither.
Make sure that if you provide an API, also test it in a browser setting, right, and not
just write it and hope it works.
You have these course preflights.
You need to check if the course preflight itself doesn't fail, right, there can be issues.
OK.
Another one, a tiny one, but I need to say this, you know, we as OpenProject, we want
to link files.
And these files we don't reference by path because the path is different for every user
or can be different for every user.
Because if you get a file shared with you and you save it in your NextCloud with a different
file name, the path is completely different and it doesn't work for the other user.
They say, another user, there's an OpenProject project.
So we need to reference the file.
But there's no API for getting files, thanks, for getting files by ID, especially not if
you want to know what is the original file name that the owner of the file gave to the
file or if you want to know if the file was already deleted, if it was trashed, it was
in the trash bin.
Information like this, we want to ask them frequently in order to update our knowledge
about the file on OpenProject.
It wasn't there.
Probably never needed on the NextCloud side.
So that's cool.
They don't need it.
But, yeah, kind of obvious.
There's main objects of NextCloud and we need them, we need the information.
And, yeah, this problem became visible here.
If you want to build, allow, if you want to allow integrations into your system while
creating a plug-in interface is a great idea.
OpenProject has a plug-in interface.
You can extend OpenProject code as you like.
You need to be with server admin in order to make it run.
NextCloud is much ahead in that sense.
They have the app store.
You just can click from a catalog and as a, not as a server administrator, but as an
application administrator who doesn't like to talk to server administrators.
You can simply download, download, install extensions to NextCloud.
It's in the core of NextCloud and that's really, really cool.
It's amazing.
And I'm jealous I would love to have this OpenProject.
Yeah.
What about the source, you know, you have four of them.
It's all PHP, you know.
Actually, no.
The app store, I think, is Ruby.
Is it?
Yeah.
Okay.
Tomorrow it's done.
No.
It could also be Python.
I just know something.
Okay.
Thank you.
I don't know how much you care about whether it's either of these.
Okay.
So you have this plug-in interface built in your application and that the typical things
that every person that is implementing this interface comes across.
For example, if you want to have cross-site fetching, cross-site calls, you need to allow
in that application, extend the content security policy for cross-site requests.
There's something, for example, that we discovered that OpenProject did not allow or make easy
in the first place.
So we had to extend that.
It's a typical one.
It only, you know, in the theory everything is working and then you run it in your browser
and in the very last moment of implementing all the features, you see it's not working
because of content security policy.
Okay.
This is a nice one.
So when you're like an application administrator in OpenProject and you're also an application
administrator in NextCloud and you want to set up the connection, the integration, the
first thing that you do is you put in the URL of the other system, right?
And you want to know if you put, or you should get some feedback if the, actually, if the
URL is valid.
Let's say you're next, you want to know that OpenProject is actually responding.
OpenProject is responding in the necessary version.
We don't have that.
We don't expose that information.
You want to know if the setup is complete.
There's more about the other way.
Like, let's say from OpenProject you want to connect to NextCloud.
Of course, we also want to know if our extension is installed.
So it would be great if you have an API endpoint that gives you information and saying, nah,
it's not installed, so the validation of the URL is failing.
There's something that is very, very useful for the administrators to get a relevant
feedback for why things is not working.
Because if you just put in a URL, you don't have any idea why it's not working.
You need to get feedback.
And this is something I think every core developer should think of as providing as a super nice
candy for developers.
Another one that is, that's a topic.
So we're working a lot with OAuth.
And setting up OAuth, I don't know if you know what, it has basically a lot of features
like a secret, an ID, it has the redirect URL, and so on and so on.
And if you are a host or a tester, you would like to script that stuff.
So you would like to have an API to set this stuff up.
And it's great if you provide an API for setting up OAuth.
Also, if you have an API, you can do stuff like having one button and some setting and
say, disconnect the integration all over the place.
Delete it.
I want to start again.
I screwed up.
I don't know what is going on.
I screwed up.
So for this, you need an API so that you can do this.
And if you had actually deleted an OAuth connection, then please also make sure that the tokens
are deleted.
So that was something that we discovered with NextCloud, that the tokens were still around
for quite some time.
And the good thing is it came up and we could provide tickets and this thing could get solved.
That's also another one, nice one.
Just imagine you do the OAuth connection, so there's one user in NextCloud and connects
as the user in an open project and that user happens to be admin in an open project.
Suddenly the access as an admin comes to NextCloud.
So there's a lot of powers.
You can create projects.
You can delete users.
And maybe for the integration, that is not needed and vice versa.
The open project might not need complete administrator rights on NextCloud.
So it's like opening the surface for attacks or the opening for problems to vulnerabilities
by a lot.
And I think open project and NextCloud, we could do better in having scoped OAuth access
control so that you don't allow creating or deleting projects or that you don't allow
deleting users and so on.
So limiting just for the user scope actions.
We can do better, we'll come.
Also what is nice, what I really enjoyed about working with NextCloud is that they provide
a design system.
They have frontend components that allowed us to have more consistent UI that look and
feel like the NextCloud and also increase the development speed for us.
At OpenProject, we also are working on a design system and UI components.
So if you want to have a talk, listen to a talk about this, Parimal, where are you tomorrow?
Same time?
Yeah.
Okay.
Have fun.
Yeah.
Then also what is really handy for developers that develop integrations for your system
is if you have containerized development environments already at hand, so it's easier
to check out your code with the latest supported versions.
For example, NextCloud has some multiple versions you need to support, you need to check, you
want to find out if it's still working.
And of course, the current main branch to see what is actually changing in the main
branch is my code still live or it's already dying.
So it's really nice to have, for example, Docker compose files that help you setting
up and lower the barrier to get started in developing an integration for your application.
Yeah.
Also as a bonus on this, if you can provide examples how to set up your CI quickly so
that you don't need to fiddle around with that, if you just have good standard practices,
how to have CI running with that core image and with the plug-in, it would be also very,
very handy for developers to have.
I think we could sneak into other integrations and we could see how they did it in NextCloud.
So it wasn't such a big trouble, but if you have a good example, that is really, really
helpful.
So with all these small spots of lessons learned that we have had in the last year, I just
want to send you home, let it sink in, and just remember, integrated, we're stronger.
Thank you.
Thank you, we have six minutes for questions.
So the question is why we need to deal with multiple NextCloud versions.
I think NextCloud has multiple versions in production, like supported, so it's like I
think 23, 24, 25, maybe 22 still, I'm not sure, and they have different code, but we
just provide one, currently, we just provide one piece of code that is the NextCloud app,
our NextCloud app, and we don't want to provide three different ones.
So we want to make sure that in that app, all versions are supported.
It's different for OpenProject.
OpenProject always has just one running, one up-to-date version, so you'd always need
to update to the latest version unless you want to get into trouble.
So this is why you need multiple versions.
More questions?
Okay.
Thank you.
Ah, Ben.
I want to add on to this that I think declarative configuration is always better than scripting,
so if you can, if it's possible, the application, for example, provides OAuth configuration statically
instead of via the script, I think that's always better because it allows you to basically
do status-deploy all your applications, including the whole OAuth context.
Yeah, so the comment was just, if you can provide it as a, let's say, a YAML file or
something like this, a declaration, it's better than to have an API.
I'm not so sure about OAuth in that moment because then you would need to have a center
when you set up both systems at the same time, you wouldn't need to have a center that decides
on the OAuth client ID and client secret and make it configurable from a center.
Yeah, if these IDs and secrets there get actually generated by the applications itself because
they get updated and maybe your center is not getting updated, maybe you create just
a vulnerability here, right?
So we were discussing this forth and back, but we believe that, you know, next time an
OAuth project that continuously update and maybe some central distribution software might
not get touched in three years and then it's outdated in the algorithm, so it's not secure
anymore or whatever.
Yeah.
So, Trey, yeah, I'm not sure maybe you talked about it or I probably was not listening.
I think you mentioned files as an example for the actual what is integrated between
both systems.
Was the scope broader than that or was it primarily set and was that any discussion
because there were multiple things that could be integrated in the sense, right?
Yeah, I mean, this integration basically focuses on files and folders, right?
Because you want to, it's about integrating digital assets into work packages, right?
So you have these links.
So that's the example I chose.
It would be interesting to have some standards also for API standards.
Why not?
Let's say every system has users.
Why does a user need to be represented different in every system?
Why can't we have a standard on that or the avatar or, you know, it's like there's stuff
of permissions.
Why can't we have permission standard somehow for all open source vendors?
I don't know.
Let's think about it, but I didn't see a reason why it needs to be so different.
Users have been around for so long.
Yeah.
More questions?
How do you mind?
No one asked the technical question.
There was a lovely portion of pizza.
Did you make that yourself?
Yes, that's my pizza.
Oh, lovely pizza.
Pizza Napolitana.
Milan style.
Okay.
Thank you very much.
Thank you.
Thanks.
Thank you.
Bye.
Bye bye.
Bye by with two.
Bye bye.