So we have Guillaume. He's going to talk about the merging process for the Rust compiler.
Okay. Yeah, you can hear me. Perfect. So, hi, everyone. So, I will be talking as he mentioned
about the merge process in the Rust compiler. So, who I am first, Rust language reviewer and contributor.
I'm a member of a few teams. So, I'm in the Rust doc team. Not to be confused with the former documentation team.
Also, Docs.Rust team and DevTools team. So, very documentation oriented. And I'm working at Huawei currently.
So, we will start by taking a scenario. Hold on.
So, when you have made a pull request, you open it. And the first thing that will happen on the pull request will be that
the bot will assign you a reviewer. So, in this case, myself. So, very likely pull request on a Rust doc tool.
And after that, you will have some tags. So, it's waiting on review and it's concerning the Rust doc team,
which helps us to find the right people in case the reviewer assigned isn't available in a week, if I remember correctly.
So, explanation a bit about how the bot is picking the people.
So, we have a repository with the list of all teams and its members, formers and everything.
And the bot basically pick someone from this repository.
And this website, the governance page on the Rust long.org website is generated from it.
So, if you need to contact someone from one of the teams, whatever reason, that's where you go.
So, now the approval itself. So, let's say that the pull request is implemented with no request from the reviewer or anything.
If it has no performance impact for this to have this information, if we have a depth, we have tools automated that allow us to actually check its actually the case.
So, if needed, we just say, hey, Rust bot, can you run a perfect check on this?
We come back to this later and we have a very nice page with some metrics and a lot of steps.
So, another important thing is checking that there is no breaking change.
So, of course, if you are changing something in the STD, for example, or changing how projection works on anything,
then it becomes a lot more complex and the process becomes a lot longer.
Same, we will come back to this later.
So, if it adds a new feature, it's very likely that we will need to be sure at 100% that it's not something that we'll need to change or deprecate or literally just remove at some point
because it happened a few times and it's not great.
And obviously, the CI must pass.
So, that's a lot of small conditions.
So, now about the CI.
So, there are two levels of CI.
The one that you will see directly when you open the pull request.
It's a lot of tests, almost all of them.
But it's only on Linux X64 because, as you may know, we support quite a lot of targets, not as much as GCC yet, but at some point, maybe.
And this checks, for example, if the call is wait-formatted, if you have all the tests passing, and by all the tests, I mean literally all the tests,
so you have all the rest of the tool suite, tool test suite, the compiler error output,
the compiler checks if the code is giving the right result, the assembly, pretty much everything,
and it includes the tools. So, if you made a change in the compiler that breaks a tool, like REST doc, Clippy, or REST FMT,
then we need to be aware of it right away.
Otherwise, we are going to have quite a bite time.
And all that is done directly on the pull request.
So, at the current time, it takes around one hour to run this small subset.
And when the pull request has been approved, we make the full run of all these tests and for all platforms.
And this time, it's run, I think if I remember correctly, it's like on 40 targets or something like that, and it takes roughly around three hours.
We have our own infra for this. We have dedicated the team for that too, the infra team.
And I think it's currently done on AWS to be confirmed.
But in short, nothing can be merged if the CI doesn't pass.
We enforced this, I think it was three or four years ago.
A few things that were merged and were expected to be fixed in very soon coming fixes were quite bad experiences,
and we decided to have a zero-tolerance policy.
It's working quite nice, so currently we keep it.
So now it's a build queue.
When we approve the command with the pull request with the command AddBulls R+, you might have seen it or not.
We have a build queue, and that's where you can see pretty much everything that is happening.
So in the current case, you see the pull request, the first one which is pending.
So it allows you to see what is being tested and eventually how long it remains.
And you can see also everything that is approved and everything.
And it's sorted by priority first, which you can see because I had to make a small screenshot.
And the second thing is how old the pull request is.
We generally have around 20 pull requests at the same time in this build queue.
So to make things faster, we have what we call a roll-up process.
We group a full pull request that we are sure have no performance impact or anything.
And we say, okay, make a roll-up of five pull requests.
You can see the button, create a roll-up.
So we pick a few pull requests and we click on the button,
and it generates a pull request for us with our account.
And after that, we give it quite a high priority and like that,
we can have a big bunch of pull requests to merge at once.
Very useful.
And that is for the build queue.
So what I explained a bit before, what is tested.
So we have the compile test.
So if your code is supposed to compile or not,
because, for example, we want to ensure certain cases in very weird cases that don't compile
or in other cases compile.
And that's how you can discover things like you can't implement directly on projections.
And if that doesn't speak much to you, it's a good sign.
We have all the unit tests.
Unit tests are mostly for the tools.
But we have a few tests with, like I mentioned, just below the error output.
It's quite important.
So we ensure that the Rust doc and the Rusty errors are looking exactly as you might expect.
If you ever used, and I think a lot of you used already Rust,
you might have appreciated the errors and the output.
Yes, because they are very, very strongly tested.
Currently, just for the UI test, we have around 20,000 tests.
So it's quite monstrous.
And running it takes quite some time.
I think it's, well, at least 10 minutes, something like that.
It's quite heavy.
Maybe you don't know it, but the documentation example are tested, all of them.
You can just test them manually in your code by running cargo test.
The cargo tool will take all the unit tests in your code.
The test folder will run on everything.
And it includes, of course, everything that is in the documentation.
So that allows us to reduce the maintenance burden by being sure that we don't give examples that are not compiling anymore
or completely broken, quite useful.
Once again, it reduces the burden.
And, of course, we have all the tools.
So cargo, RustDoc, Clippy, RustFMT.
So as I mentioned, when you change something on the compiler, sorry,
when you change something in the compiler,
since these tools are using directly the compiler,
they are actually compiler extensions except cargo.
Cargo is just tested to ensure that not a new option is breaking something.
So for the others, they are extensions of the compiler and we need to ensure that no changes is breaking anything
because that would be problematic.
We generate a lot of documentation and we have to ensure that we have no deadlinks.
And, in fact, we do have some of them and we ignore them on purpose.
So sorry for that.
We can't fix them because, funnily enough, in the STD, we re-export stuff that is in the core
and they share the same documentation.
So if you are looking at the documentation in the STD pages,
all the links are working in the core create.
They're not.
So try to use STD as much as possible.
And it's just very basic, but we have quite a lot more.
We mentioned in the previous talk, the inline assembly, it's part of the things.
Something we realized when working on the GCC backend this time is that GCC doesn't allow to specify a syntax
that's thanks to this test suite.
So currently, we can't implement all features and it's going to take quite a long time,
but hopefully at some point, someone motivated will do it.
Don't know.
So on which OS and architectures are tested, everything.
We have target tier policy.
You can go check it on the page just linked below.
But basically tier one, the platforms are the platforms that are fully tested,
implemented, and everything.
So macOS, Linux, and Windows.
And they must pass all the tests and we build them and we ensure that what we have built
and has to be able to be uncomplaced and working on the target and everything.
So strict, very strict restriction.
On the tier two platforms, it's a lot more relaxed.
We just need it to build.
If it works, well, it's good.
If it doesn't, well, too bad.
And for the tier three platforms, it exists.
Yeah, that's good.
So for example, if you want to build on the Nintendo 3DS, you can.
We don't know if it would work, but you can.
And you can see the list of the platforms each tier on the page just below.
Like I mentioned, we have quite a lot and we hope to be able to expand it a bit more
by adding at least the GCC backend at some point.
A lot of work remaining.
So what about releases now?
Because as you might know, we make a release every six weeks.
So it's very fast release cycle.
So when this happens, the build queue is frozen.
We don't allow anything below like a priority of 10,000 to be merged.
It's a completely random number, but generally if you go higher than 10, it's quite important.
So in this case, we freeze everything.
And the only things allowed to be merged are the patches to make actually the stable and beta branches update.
An important thing that isn't noted here is that we don't have the need to freeze for the nightly.
We just say at a given time of every day, okay, this will be the nightly version for today.
Yay, and that's it.
So back to this, the third point.
All relevant information is updated and reared.
And by that, I mean the websites, the documentation, the book, I think, too.
Pretty much everything.
We generate the binaries.
So that's what I mentioned.
That's the things that need to be working for at least tier one polyform.
And of course, we make a blog post.
Generally, the blog post is written not for the current stable release, but we write it at the beta version.
And then depending if we need the backports, for example, we realize that in the current beta version,
something is completely broken and we don't want that.
And it's an easy enough fix.
Either we backport a patch that was merged on the nightly directly onto the beta branch.
Or we say, okay, too bad.
We revert that and we'll do it the next time.
It happened quite a lot.
And it's not uncommon.
Let's just say it's better if it doesn't happen.
It allows us to not have the dot one version coming up like a three days later because we realize that we broke something.
And the blog post is released.
So now a performance.
What I mentioned is that we need to check sometimes the performance.
So we have to speed it now.
So for the performance, we have a lot of benchmarks you can see on the left.
It's generally for the number of instructions that have been written.
It's what we consider the most important metric and most, let's say, stable.
So when you have all green numbers and quite high, oh, yeah, 8%, yeah, that's quite right.
So when you have all green numbers, it's green and everyone is parting.
And if you have all red numbers, either you have a very good reason or it's not going to be merged until you can make them at least black.
And we have, like I said, a lot of metrics like cycles, memory usage, disk usage, because we started to worry about the binary size.
We realized that all the doc attributes were generated in the binaries, which is not great.
So we are going to fix that at some point.
And you can see on the right that, yeah, maybe you can see.
Anyway, just to believe what I say, the results are showed in the nice comment directly on the pull request.
So other cases, when you add a new feature or introduce a breaking change, there are three possibilities.
The mostly non-one is the RFC, request for comments.
It has its own repository.
It takes a lot of time and effort and comments.
It can go really fast, like two days, or it can take indefinite amount of time.
Some examples, some RFCs have been open and still are commented on before the 1.0, so that gives you an idea.
We have the MCP, major compiler changes.
So not too big changes in the compiler.
We find it not to greet how the query system is working.
So let's try this solution, and they discuss mostly design and very technical points.
Interesting, but if you don't know this area, well, it's not very understandable.
And the last one is common to all teams.
So the FCP, the final comment period, it's something that we want.
And we just want to be sure that everyone is on board.
So we ask for an FCP, and once more than half of the members of the team are okay with it, then we approve it, and here we go.
So we, of course, for every poll request that is merged, we check for potential examples.
No, that's before, sorry.
When we make a new feature that potentially changes current behavior,
we look for potential regressions in all the crates ecosystem.
So we make what we call a crater run.
And with this version of your code, we run on all crates, and we generate a nice report.
You can see on the left.
And if you only have flaky stuff, we say, okay, no impact.
So it's good.
We don't care, and that's pretty much it.
And same as for the performance, we have a nice comment explaining everything in short,
which is much more easy to read that the thing on the left, which is actually not good.
And now a little part I like to do every time, tips for potential new contributors.
We have a lot of classified tagged issues with ELZ or E-Monitor or both issues.
Take a look at them.
We try to be as helpful as possible to newcomers.
It's important for us to have new blood.
We have always good surprises with newcomers.
We wrote a receipt of guide, which is not up-to-date at all.
So at least you have a vague idea of what's going on, because I think not many people have an idea.
And you can try also to write compiler plugins or eventually contribute to ClipIt
to see how the compiler higher internal levels work.
About ClipIt, it's really simple to contribute to it, like they have a full guide or anything.
So if you want a big, nice first step, take a look at ClipIt and how it works,
and it gives a very, very nice introduction.
And I am making publicity for myself.
I wrote a small receipt towards Crate, which makes a few things simpler to write plugins and extensions to the compiler.
If you want to write, go ahead.
It's made to be usable as much as possible.
And thank you for listening.
Thank you so much for watching.