I bet it gets started because this one runs is really short on time.
Okay, right, hello, it's still me.
This is an unfortunately short talk.
I'm going to be a bit straight for time, so I'm just going to run straight into it.
So yeah, I work for OpenJDK.
I work for Red Hat OpenJDK.
I've been working on containers since I started at Red Hat, but since I joined the OpenJDK
team, I try to diversify a little bit.
So I started working on backpots about four years ago, but it's always been an issue as
the kind of thing I did around the side.
So I'm perennially a beginner and I still have to do myself a beginner.
Loads of people in the audience here are significantly more experienced with backpots than me, and
I hope if we have questions, they will help pitch in with answers.
Okay, a little bit of preamble then.
I'm not specifically talking about starting OpenJDK development itself, although there
is some significant overlap.
Why do it?
There is a clear commercial reason, which is why so many companies are involved, but
I'd argue there's another good reason is that it's fun.
The kind of problems that you need to solve with backpots are a little bit different to
regular development, so it's quite interesting, I think.
There are a lot of documentation in OpenJDK community.
Some's out of date.
Some's a bit misleading.
These two, URIs in particular, are not.
These are fantastic.
The top one is the quick cheat sheet thing, and the second one is your reference guide.
It would be futile for me to try and rehash them here.
I wouldn't do a better job.
So instead, I'm going to try and focus on a couple of things that come up when you're
just starting, and also some tips and tricks that I've picked up along the way.
The focus really should be that doing backpots is a community thing, and you're joining a
community, and certainly you will require sponsorship, reviews, and approvals.
So it's important to recognize the community aspect of it and to build relationships with
people.
Okay, so the very, very, very first thing you need to do is to sign or be covered by
the Oracle contributor agreement, because, well, that's really, really important.
It's important for legal reasons.
Your organization, if you work for a company that's doing this, then you might be covered
by a sort of umbrella that's been signed, but check.
It's important for legal reasons, but it's also important practically, because a whole
load of automation won't happen for you until that's happened.
So once you've signed that, you've got access to OpenJDK bot that operates on GitHub.
It'll ignore you until that's done, so that's important.
And the very, very first thing you should do is clone JDK, try building it, make sure
it runs, and start forking GitHub repositories.
Building it then, again, this is actually pretty well documented in the source tree.
Building and testing pages in particular.
A couple of quick tips.
If you're building on a laptop, so that's saving up for a desktop, but turn on Ccash,
it makes a huge difference in that sort of environment.
You're going to require a boot JDK, which is a version or two one way or the other of
what you're building.
So you're going to start stashing those around.
I download them from Adoptium, other vendors are available.
I very much encourage that you run tests for backboards.
The tier one test suite requires the Google test framework in the hotspot area, so download
that too.
And you need the job test tool, JTreg, which again, you might require more than one version
of that, but it will be hit cross-up bridge when you come to it.
Finally, unless you're actually working on fixing this, sometimes you will find that
trying to build an older tree with a modern compiler will throw warnings which are promoted
to errors by default.
So if you're not trying to fix that and it's just getting in the way of what you're doing
how to do, you can disable warnings as errors by passing that to the configure script.
I think that even innate you these days, I think that's the way to do it.
So as I said earlier, fork all the things.
So go find the open JDK trees and all the backboards trees, create forks on GitHub,
and then go into the settings and turn on actions for them.
Do that first.
Okay.
To produce this slide, I spent quite a lot of time confused, so I think it's fair to
say this can be confusing.
You have the notion of the open JDK community as the sort of outer layer of abstraction,
and your relationship with that is of starting as a contributor once you sign the OCA.
Do that because you need to do that.
The next thing then, you've got the notion of projects and you've got the notion of groups
and they're distinct.
The updates project, I mean, Dalibor covered this this morning.
The updates project is the one that covers virtually all of the backport's work these
days with the exception of 8U.
Once you've done a couple of backports and found sponsors, you will be able to apply
to become an author in a project, which is very, very useful and gets you an account
in the open JDK infrastructure.
To do that, you email the project lead for the project is Rob McKenna.
For 8U, it's Andrew Haley.
If you look at the commits in the Java trees, you'll see that they all have a very common
format.
The first line of the commit message starts with the bug number and that points into the
Java bug system.
This is very, very useful database of information about bugs.
And when you're doing backports, you'll spend a lot of time tracing things in there.
But the problem is, until you've acquired an open JDK account, you're limited in terms
of how you can interact with it.
You can't write to it, so you can't comment, write labels and things like that, but you're
also limited in terms of what queries you can run.
So thank you, Alexis Shippoliv for the JDK backports monitor project.
This is Java program which basically constructs complicated JIRA queries and performs them
and writes reports in a variety of different formats.
Alexis not only created this, he also hosts an instance of it and publishes reports regularly
to his own web server.
So thank you, Alexis.
If you make heavy use of this, and you work for an institution, I would strongly suggest
you try and run it yourself.
It might iron out some bugs.
And also, if Alexis turns his website off, you will not be in hot water.
So how to find a good backport bug?
Here's an example of a parity report from Alexis' Backport Monitor.
This is showing the first section, towards the top of the document here.
The first section is a list of bugs which have been addressed in Oracle's build of 2103
and have not yet been fixed in OpenJDK updates project.
So signals that you should avoid a bug, as a beginner certainly, is someone already working
on it?
Has someone filed a request for review?
Has someone opened a pull request that addresses the issue already?
Or has a company, I think A for Amazon in this case, flagged a bug as of interest to
them?
So eyeballs are already on these bugs, so to start out, probably look somewhere else.
And then these are signs that a bug might be suitable.
Open-sourced tests that are previously closed, test-only patches.
Those are pretty attractive because the backport projects are happy to accept test-only patches.
It's very low risk to the product.
It increases the quality of testing.
And they're generally quite easy to backport, sometimes trivial.
So that's a good sign.
And finally, it doesn't show up well on a projector.
Other flags then, depending on who you are, what your interests are.
So platform-specific bugs, Windows bugs, Mac bugs, port-specific bugs, IOT64, etc.,
will get less attention than more general bugs.
I mean, almost nobody's doing Mac or Spark ports.
So if that's an area that you can specialise in, that might be something worth looking
for as well.
Okay.
I've got a couple of tips here about Git.
So nowadays, I think all the JDK trees are in Git, or at least all the ones I deal with,
which is nice.
And time invested's perfecting Git, mastering Git, is worthwhile.
There is actually a talk, I think, tomorrow in a different room from somebody who's going
to give some super tips on Git.
So I recommend that if you can fit it in.
So the first stage then, this is a prerequisite for what's coming in my slides, the trees
have grown really big, right?
So if you clone them all, just the object storage for Git has grown to 200 megs-ish
for 8U.
By the time we get to 21U, it's 1.2 gigabytes.
And if you've got all of those cloned, disk space is cheap, but that's network time, etc.
If you stick all of those Git repositories as remotes in a local, one local repository,
then you can deduplicate an awful lot of that storage.
I think it last time I measured it, it's about a gig and a half for all of those.
But that's not the main reason to do it, actually.
I would say the main reason to do that is because it makes comparing objects between
JDK trees really easy.
You've always got access to any object in any of those trees, from any of those work
trees, so you can look at the state of a particular patch as it was in mainline if you're in a
backport's tree, etc.
Without having to ferry patches around or export things.
My favorite Git trick that I learned recently was the notion of Git work trees.
When you make a clone, you by default get one work tree, it's the checkout, right?
So the bit outside of the .git directory.
You might be familiar with repositories which have zero work trees, that's bare repositories,
such as if you were hosting them on a web server.
But you can actually have more than one work tree for a single Git repository.
The syntax is something like that.
It creates a new sibling directory adjacent to your clone, tracking a particular branch
from one of the remotes.
That means you can have a work tree for each of the backport trees with only one common
Git storage.
Again, the reason that's really useful is if you're somewhere over in the 17U directory,
you can still access and compare against objects in any of the other trees, just as if you
were in a, it's essentially as good as being in a real Git clone.
That's really handy.
I blogged about it when I first discovered this, and that's somewhere at the bottom of
the slide there, trying to talk about how it's useful for this stuff.
Path changes then.
So, files move around a lot between JDK versions, especially after 8U.
What we used to have to do is use a corn shell script to basically take a patch and then
mess the paths in the patch around to match your target tree, which was tricky.
What you can do nowadays with the current new workflows is the opposite.
If you're trying to backport a patch to 8U, for example, and the paths are all wrong,
what I suggest you do is prior to trying to backport the patch, move the path files
as they exist in 8U, in your local clone, to the patch that matches the source, the
path from the patch you're trying to backport, which is a little counterintuitive.
Commit that, but then that means when you pick the patch, you don't have any path conflicts,
and you just have all the other problems to deal with instead.
Once that's done, you can then revert the original commit, and the final delta is as
if the commits, the fixups were all in the right place.
You can either squash that yourself, rebase locally, or push it to get, and the gamebook
and JDK bots would do that anyway when the commit was eventually accepted and integrated.
I must have like seconds left, I guess.
Just in passing, I use this all the time.
You can type, I use this as a shorthand to open every file matching a particular regular
expression in my editor in one go, and then I can, for quick substitutions, across a full
tree.
One last one then, a UI thing.
I'm a complete holdout for Wayland, so that was a fascinating talk, but I'm still on X.
When you run some of the tests for client stuff, which we're using AWT robot, and it
moves to the mouse pointer or keyboard input, that can be a little confusing if you've
paged off to do an email while you're waiting for the test to run, but also it can fail
sometimes in esoteric environments.
This is a real example of a backport I was doing last week with five new tests and all
swing ones, four of them passed and one failed for me for who knows why.
The solution in the next environment at least is a tool called Zephyr.
It's just an X server in an X client.
Fire that up, you get a little mini X window, run a terminal inside it, try running the tests
in that.
It isolates all of the mouse and keyboard operations from your main desktop, and in that
circumstance all the tests pass and I have confidence they're okay.
See how I'm closing?
I have to say, as I said, it's fun and it's a community thing, but I have to reiterate
one line from the excellent cheat sheet you were all earlier.
You're quite close to the customer and there are a few safety nets, so be careful.
The reviewers are fairly overburdened.
They do a fantastic job, but you take some time over the patches and make sure that they
update their high quality for everyone.
That's it.
Thank you.
Yeah, have we got any questions?
Hi.
I'll send you a mic.
Oh, okay.
Sorry, just one little thing.
I just came in at the end, you were talking about fixing up patches and stuff.
Yeah.
So when we completely relay out the source code for JDK9, those scripts that were actually
put in the bin directory that actually will transfer patches between the old and new layout,
you may or may not have found them.
I can't remember the name of them either.
Unshuffle patch.
Unshuffle patch.
I don't know whether they're useful, but I just let him mention them.
Well, so Andrew Hughes maintained the 9TW, well, he maintained that script past 9U's
life a little bit to further facilitate that, but I think it might be dead now.
There was somebody over here, yeah.
I'm going to say if there are anything that you can do about it, I can't remember the
name of the either...
Unshuffle patch.
I'm going to say if there are anything that you can do about it.
Yeah.
Thank you very much.
Yeah, I had a question about testing my back port.
How can I make sure that I sufficiently test my back port if, say, for example, I'm porting
a fix that doesn't have a regression test?
Okay.
That's interesting.
So I think first thing is that unless you're really sure you don't need to run the tier
one tests anyway, but flag it with your sponsor or reviewer, because it might be that they
need more test coverage upstream in a newer tree anyway, and that should be back ported with it.
Well, you might want to go and look at the justification that was given for there being
no regression test when it was done in mainline and say and read about questions the reviewers
asked then and what the actual fixer said.
I mean, in theory, any fix that doesn't have a regression test needs to have some keyword
attached, which explains why.
If it's a dot fix, it's pretty obvious.
If it's test itself, it's pretty obvious.
Most other things, there can be like no-reg build.
It's just fixing a build problem, things like that.
So go look for the clues as to what to do in the mainline fix.
You can at least run all the tests from the subcomponent of where that fix was done.
So that's what I usually do.
Because tier one is not usually running all the subcomponents tests there, but if you
do a single chain in one area, then you just run all the tests there.
Yeah, that GitHub actions things runs in a very limited set of tests, very, very limited.
Thank you.