So good afternoon. My name is Michiel Lekin. I'm a freelance software developer from the
Netherlands. Last year here at FOSDEM, I first announced Mox, a modern, secure, all-in-one
e-mail server. As you may know, running your own mail server has a bit of a reputation
for being hard to do, but what have I told you? Running a modern mail server can be easy.
Alright. So, thank you. So the goal of Mox is to make it really easy to run your own mail
server so that you actually do it, and then you can stay in control of your data, and
you can help keep e-mail decentralized. Now, Mox is a new implementation, entirely new,
and written in Go. Now, that's a lot of work, and you might ask, why would you do that?
Because we have so many open source components that you can just use, and that's true. And
for the past decade, I've used many of those components to good use. But a few years ago,
I had to reinstall my machine, so I got a completely new one, and I just felt a bit reluctant
to install that same software again that I've been using for the past decade. And for two
reasons, at least. One is to see. The language with small mistakes have big consequences.
Don't get me wrong, I like C as well, maybe in the past. And the software written in C
is very high quality, but I wanted a new machine that lasted for another decade, and I see
that C is not really going to be part of that too much at some point. But the bigger problem
is basically the complexity. Over time, as e-mail has grown, new protocols added, new
extensions, new software components have been added as well. So, to make our fully modern
e-mail system, you need many components and make them all work together. I think many
self-hostings, at least, they stop halfway, so they have a semi-modern e-mail system set
up. You can make it easier to get all this configured and set up with a distribution
or a Docker image or something, but you still have all these components working together.
There's many integration points, a bit of friction, some data loss. Sometimes there's
security issues when, you know, message headers are seen as authoritative but added by some
component. So, I think what happened is with all this complexity, some people, you know,
just stopped running their own mail servers anymore because it just too much work, and
they migrated to the cloud, centralizing e-mail, that's not a great development. So, what we
need is an easy to use mail server. So, you need to write a set of features. So, Mox
tries to deliver many features. I'm at four for reading your e-mail, S&TP for sending
receiving e-mail, SPF, DKIM, DMARC for message authentication, because just S&TP is not enough,
but that's also not enough. You need TLS, of course, for encrypting your communications,
but S&TP for sending receiving e-mail is unverified TLS. So, you want MTA, STS, and ordain to,
you know, check that you're talking to the right person, right machine. So, Mox implements
both for incoming and outgoing e-mail. Then there's ECME for, you know, your management
of TLS certificates. You want to make it easy, don't do any manual TLS tumbling. Junk filtering
is part of Mox. So, based on historic messages and their non-junk classifications, Mox will
reject, accept, incoming mail, more about that in a moment. Then internationalization, so
you can have Unicode in your e-mail addresses and your headers, both in your domains with
IDN and your local parts. Autoconfiguration is in their various flavors, all supported
by Mox to make it easy for mail clients to find and write service settings for new accounts.
Then we've got a webmail included in Mox. We'll have a quick look at that in a moment as well.
An admin web interface, so all configuration is in files. We want the full power. You can
use the admin interface to quickly navigate and make some changes like add, remove, an
e-mail address, an account, or a domain. A web service included, it may sound a bit
crazy over the top, but modern e-mail basically requires HTTP stack with MTA, STS, Autoconfig,
Jmap soon. Now it's already part of the deal. What I've noticed is people trying to run
Mox and a web server on the same machine. That's really annoying because configuration
gets complicated. Instead, I just added some web server to Mox for some static file serving
and reverse proxying. That problem is also solved. Permit these metrics, structured logging,
operations become a bit easier. Then the Mox Quickstart. That makes all this stuff easy
to do. Installing Mox, you take a new machine, you've got a domain, you run the Quickstart
and you pass it an e-mail address at your new domain. Mox will generate, so the Quickstart
will generate configuration file, decon keys, etc. Create a new account. We'll print all
the DNS records that you'll copy, paste into your zone file, or you have to manually edit
them in your web interface of the DNS operator. That's not so great. Then Mox also, the Quickstart,
also Linux, generates a system-d unit service file, so you just enable that and start it.
Then you've got a fully working modern e-mail system. All of this is MIT licensed, so you
can do whatever you want basically. Then as developers, a little bit about code. As I
said, it's a new codebase. It's a modern codebase coherent. All of this is in the same style.
It's very self-contained, so few dependencies. That makes it possible to have it in the same
style. It's about 73,000 lines of go and 21,000 lines of tests, mostly unit tests and a bit
of integration tests and some fuzzing tests. There's 11,000 lines of type scripts, very
strict type scripts for webmail and interfaces. The code is cross-referenced with RFCs to make
it very, not easy, but to make it something more maintainable. You can look back and see
why you did certain things. Of course, Mox is written in Go, so it brings a whole bunch
of advantages like memory safety, standalone binaries, completely subtly linked, also includes
a few assets, so it's really just one file that you need. Fast compilation time, great
for developers. Dependency management is pretty much solved in Go. You get reproducible builds
out of the box, and it also works with cross-compilation, which is trivial to use in Go.
Now, there's not much to see about a server, but we have a webmail that I can show you.
It's not pretty, but it looks mostly like a standard email client, I think. Mailboxes,
message list, message view. Let's open up mailing list. There's some threading in there.
You can select multiple. I'm using keyboard shortcuts as well. Mark some unread messages
and mark them read. Then there's HTML support with or without external resources and tracking
pixels. Then there's a little example of Unicode addresses. The search is easy to use. We've
got some quick filters on that side. We could send a message, but I'm sending a message
from another mail client that should be arriving. There it is. Select some text to quote as civilized
people and send a response. That's a webmail. It's not pretty, but it mostly works for my
needs of email, sending and reading. Then I would like to say many things about lots
of features, but I just limit to one thing, spam filtering in Mox. Analysis for incoming
messages is based on historic messages in an account, based on their junk and non-junk
flags. It's always per account. Whatever one account does is not related to what happens
for the incoming message for your own account. Of course, this means in order for this to
work, you need to have the proper flags on all the messages or as many messages as possible.
Email clients don't always help with this, but Mox does help with that because in the
default setup, you get an account where incoming messages or messages moved to the junk mailbox
gets the message flag. If you move something to a archive mailbox, it automatically gets
the non-junk flag. Also, if you move it to the trash mailbox. Also, if you're in the
webmail and you have a message open for five seconds, that's long enough for it not to
be junk, probably, so it also gets the non-junk flag. That means most of the messages in the
store will have these flags set properly. There's a difference in how Mox handles known
senders versus first-time senders. For the known senders, they're recognized from sender
address or just the domain of the sender address. Maybe it's another person at the same company.
Or we look at SPF or Decombe signals in a mail message or we look at the IP address of
the remote server or various subnets of the IP address. If there are recent historic messages
from that same sender, we look at the junk and non-junk classifications of those messages.
If the recent ones were junk, we reject the message and otherwise we accept the message.
But if it's a first-time sender, we don't know enough about that sender. We do, of course,
something else. So, the BEG analysis is also part of Mox. It's essentially a reputation
of words. So, you look at the words in the message, then you look at historic messages
and their words and their junk and non-junk classifications. If there are too many spammy
words in the message, you reject. If there are enough hammy words in a mail message, you
accept. Then you can also configure a DNS blog list in Mox, but it's off by default for
a few reasons. One, often these DNS blog lists are centralized services. We don't want to
rely so much on them. And you would be sending the remote IPs of those you communicate with
to some central party, which is also not great. Then we don't want to break existing email
flows. So, this is also one of the reasons why it's only on first-time senders. So, if
you've been communicating with someone for a long time and suddenly someone puts their
mail server on a blog list, you can keep communicating with them at one break, only if that person
really started spamming you all of a sudden, then you mark a few messages as junk and then
in the future, the mail filter will just adjust. Now, in Mox, being all in one mail server
really helps with this because during the S&TP transaction, all this historic data, these
messages and flags and words are available for analysis. Then a special handling for messages
from mailing lists and forwards because essentially most of the analysis disabled and DMARC policies
are not enforced. Now, what do you do with an incoming junk message once it's classified?
Well, one does not simply deliver to the spam mailbox. That's not friendly for users and
not for senders or recipients and senders, I think, because the sender thinks that the
message has been seen and doesn't get a reply. The recipient may be receiving some message
and doesn't get it, so they wait or they constantly check both the inbox and the spam box. I
think it erodes trust in email. So, I understand that it's done to not give spammer's feedback
about their spam runs, but users should come first. So, instead what Mox does, Mox rejects
the message at the S&TP level while it's coming in with a temporary error code and a very
generic message. So, the generic message means that the spammer doesn't know for sure why
it's being rejected and a temporary error code means or causes the sending server to
try again a few times and at some point tell the original sender, you know, this message
cannot be delivered and then they know they can find another way to communicate. So, you
don't have this problem anymore of lost messages in the spam box. But just like with the spam
mailbox, Mox has kind of the same thing but different. It's the rejects mailbox. So,
anything that's rejected is still stored in this special mailbox. It's a fixed size mailbox.
Old messages automatically removed and so you're waiting for some kind of transactional email.
Maybe you did a sign up to a website. It's not coming in. Then you can check the rejects
mailbox if the message, because maybe the sending website used the infrastructure with
a bad reputation. Then I can just move that message from the rejects mailbox to the inbox,
Margaret is non junk and the next time because of the historic based filtering, next time
those messages from that sender will be accepted. So, the important point is that you don't
have to keep checking the rejects mailbox because the sender knows you didn't get it
and that's different from the spam mailbox. So, this seems to be a graph for me but if
you have ideas on how to improve on this, let me know. Then a bit about the roadmap.
There's still a lot to do in Mox. So, I want to implement a simple HTTP based API for sending
some messages and also receiving some feedback. Just so web apps for example can just with
a simple call make some sense of emails. If you know of any standardized ways of doing
this, let me know. Yeah, okay. But I said simply, really the dumbest thing. But I guess maybe
it can be that simple. Then I want to add calendaring. It's not email but users and myself
included expect it to come with email. I need some more SNTP and IMAP extensions. A JMAP
will be coming at some point. So, so far I focused on IMAP because all my meal clients
were using IMAP and I wanted to have a working meal system. But, you know, I, JMAP will be,
will be coming. I want to encrypt all data at rest. It's not currently done. I want to
be able to have a second Mox as a backup, Mx and a backup instance. In order to do junk
filtering on the second instance, I will need all the data as well or the historic messages.
So I want to synchronize everything to the other one. And then I, you know, the, the,
once all the data is there, you can also use it as a, as a, as a failover machine. So that
will, that will be nice. Forwarding to external addresses not yet done because it's a complicated,
gets complicated quickly. I think modern email is not really set up for that anymore.
So, Mox is a different way of applying rules to incoming messages.
Then lots more on the list. Too much for today. So, final slide. So it's been a year since
I first put out the Mox code. I've gotten quite a lot of feedback. So thanks everyone
who sent in bug reports and made feature requests or sent in patches. Very helpful. Then also
thanks to NLMet. They've been funding continued developer the Mox since August last year.
So that's been instrumental to keeping working and being able to keep working on this. Also
thanks to everyone who wrote all those RFCs about email. They're very, they're excellent
and they match practice quite often. So my call to action today. If you're not doing so
already start running your own mail server, you know, staying in control of your data and
keep email decentralized. And you have many options ready. And now there's just another
one called Mox. So give it a try. Send me an email. It's a great way to communicate.
Thank you.
Thanks.
Oh, I saw you first. You only have three minutes.
First of all, I think it's a quite incredible project for one person. And I was wondering
how many third party libraries do you use and how much the code you write directly to implement
all this?
Yes. So I think the main external library is called Beebolt, which is a fork of the
database layer. So the messages are stored in files. The database layer is a database
thing and it's pretty much a key value store. But anyway, that's the main external dependencies.
There's something for Prometheus. And then there are a few dependencies that I wrote
myself. So those are not really all that external. And otherwise it's mostly the Go standard
library and the extended Go standard library. So very few external things. So yeah, it feels
a bit like a not invented here syndrome. So I want to rewrite everything. But it has
been very instrumental because sometimes I've made sweeping changes and there's no one.
I don't have to make pull requests, try to convince people to do something that suits
my needs so I can do whatever I want. So it's really sped up development, I think.
Fantastic project. I have a quick question regarding database. I don't know if it was
answered right away because I heard the database was whether the data is a sort of a database
if you like or could be changed or whether and whether whatever it is, could we use Unix
normal tools to just go through them? No. No, you cannot use normal Unix tools. What
I don't really want is say having a meal day that someone else also makes changes to because
I have to do lots of work to make sure that I synchronize or chase as well. So I've chosen
a simple approach. So messages are just stored in a file system individually at the moment
and there's one database per account that has the index for all the messages in that
account and that stores also the message flags, etc. So the database is also essential basically
for all the history and all the data. I could talk for a long time about the database library
but.
Okay, quick one. What is your experience with scaling this up? How many does a MOX thing?
I've not tried. It caught me because of the Bayesian per user and we tried that and that
didn't work out.
So I have no idea where the limitations are. So I would like to try to see where it breaks
and I don't know at the moment. So I've only run it small scale and really targeting it
small like self hosting a bit and not the tens of thousands of users or something.
So I see many hands which is great. We have a little more time since we have the switch
of succession. Anyway, when people leave in the meantime maybe be silent so we can use
the time for a few more questions. Let's try how many we can get. I didn't see the order
so forgive me.
Thank you. Do you have any plans for LMTP support?
No. But why would you use it? Why would you need it?
I'm writing a small...
Oh, you need the microphone. You need the microphone.
Sorry. I'm writing a small like mandrel clone and now that they shut down and for that I
need to be able to put an email message into the server.
Yeah, okay.
So maybe a better solution would be to put it in the go code and make like a fork or something.
From what I've seen LMTP is almost like SMTP but just it has this improvement of getting
reply codes per...
It's just simpler. It's lightweight. It's just a dumb-down version for mail drops.
Did I get that right? You reject mails but still deliver them in the rejects mailbox.
Yes.
Whoa.
Wow.
Scary.
Yes. About the reject.
So I think that's basically like grey filtering if you are...
It's basically like grey filtering except that you will continue to reject them or do you do anything special if they come back or if they go...
Yeah, so if they come back with the same message, I deduplicate it based on the message ID or the hash of the entire message if there's no message ID.
But then we'll still be considered rejected.
Yeah, it will still be rejected.
Yeah, yeah.
And does this interact with the junk no junk flag for Thunderbird and other IMAP clients?
Well, so I think there's the flag dollar junk and dollar non junk and as far as I see Thunderbird sets it without the dollar.
So it's not useful.
But it does also interfere, I guess, because Thunderbird does not a client side and it would work but it's kind of duplicate then.
So I disabled it. I disabled the automatic classification on my Thunderbird setup and I just let the server basically do it myself.
So I now don't get a lot of junk. The filtering is okay.
I still get a few pericas sometimes one a day and I just junk it and then it's okay.
Okay. Thank you for your questions. There's still a matrix chat. Mikaela will be around.
Thank you Mikaela.
Thank you.