[00:00.000 --> 00:13.200]  Alright. Hello everyone. Thanks for coming. My name is Adam. I work for Red Hat and I
[00:13.200 --> 00:18.440]  do send-off stream for Day Job. I'm the send-off stream lead and I want to talk to you about
[00:18.440 --> 00:25.020]  send-off stream and how we, Red Hat, use it to get work done on RHEL and how you can
[00:25.020 --> 00:31.040]  use it to participate but also build your stuff and we'll see. Okay, so just to set
[00:31.040 --> 00:35.480]  the context, before send-off stream we did something like this, like when we created
[00:35.480 --> 00:41.320]  RHEL, where Enterprise Linux, I keep saying RHEL, we took what's in Fedora, that's where
[00:41.320 --> 00:47.280]  the innovation happens and then we had like a long process to build RHEL out of some
[00:47.280 --> 00:55.080]  of that and when that got out, somewhere later, sent-off stream, sent-off Linux happened
[00:55.080 --> 01:01.200]  and yeah, that was interesting but the problem was that when you find something in that rebuild,
[01:01.200 --> 01:07.360]  you can't really change much because the goal was to be a rebuild of RHEL. So what we did
[01:07.360 --> 01:14.040]  with sent-off stream, we kind of switched it a little bit. Now the process is like there's
[01:14.040 --> 01:19.000]  more steps, so we have Fedora still, there's nothing changing about Fedora, that's still
[01:19.000 --> 01:24.440]  the primary place where innovation happened, that's the upstream, there's something called
[01:24.440 --> 01:31.080]  Fedora ELN, which is like a subset of the content, rebuilt with RHEL configuration and
[01:31.080 --> 01:36.160]  that's the next major RHEL release. So if you go out and look at Fedora ELN, that's
[01:36.160 --> 01:47.200]  what RHEL 10 is sort of right now, then sent-off stream happens, this is where the development
[01:47.200 --> 01:52.400]  of RHEL happens in public and if you talk to Red Haters, they will sort of somehow combine
[01:52.400 --> 01:56.560]  these together, sent-off stream and RHEL because this is really our development space and I'll
[01:56.560 --> 02:01.360]  have more details later, so this is tracking the next minor version of RHEL and this is
[02:01.360 --> 02:06.560]  what you can use, but you can also contribute to it and we'll get into details. So this
[02:06.560 --> 02:11.280]  is basically if you heard Fedora ELN sent-off stream, this is what it is and I have some
[02:11.280 --> 02:18.600]  more details here, this is Fedora, Fedora Rohite sources, this has its own sources and this
[02:18.600 --> 02:22.600]  is the build-build flag and you can also see there's a different amount of packages, it's
[02:22.600 --> 02:28.800]  just like more information there. Okay, so let's talk about how we get work done or how
[02:28.800 --> 02:36.360]  you can get work done in sent-off stream, how that works. So there's this diagram and we're
[02:36.360 --> 02:41.640]  not going to go box after box, no worries, but I just want to demonstrate that you see
[02:41.640 --> 02:50.680]  that we have bugzilla where work tracking happens, there's a merge request coming to the GitLab
[02:50.680 --> 02:55.960]  and then basically everything is synced, so the upper part is the sent-off stream infrastructure,
[02:55.960 --> 03:02.560]  this is RHEL internal infrastructure and as change happens, it gets built in both, it
[03:02.560 --> 03:08.520]  gets tested in both and when that passes test, both get released further to the process and
[03:08.520 --> 03:14.320]  finally it gets into both RHEL and sent-off. There's something called sent-off development
[03:14.320 --> 03:21.680]  compose and production compose, which is basically when this compose is sort of like
[03:21.680 --> 03:29.240]  a repo and ISO and container images, just like a snapshot that you can consume and yeah,
[03:29.240 --> 03:34.880]  there's one that happens after the test, that happens every day and then the verification,
[03:34.880 --> 03:39.480]  this is like an internal process paperwork and stuff and then that goes through the production
[03:39.480 --> 03:47.120]  compose and I can even show you how that happens in the system, so this is what a bugzilla
[03:47.120 --> 03:53.080]  bug looks like, someone was adding, this is like half a year ago, a multipass TCP to RHEL,
[03:53.080 --> 03:59.800]  so they created this, they did the merge request in GitLab, everything was visible publicly
[03:59.800 --> 04:05.240]  and they submitted a build first in sent-off stream, that got through, got built in RHEL
[04:05.240 --> 04:11.400]  as well, if I scroll down there's tags, that was like the multiple steps, the gates pending
[04:11.400 --> 04:18.040]  candidate, that's how you can know where it's in the process and it basically got through
[04:18.040 --> 04:21.040]  that and now if you're using sent-off stream you already have it installed because that's
[04:21.040 --> 04:28.600]  half a year ago for that change, but this is basically the flow how it works.
[04:28.600 --> 04:36.240]  Let's talk about contributions now, for some context I'm starting with RHEL 8, RHEL publicly
[04:36.240 --> 04:41.920]  said that we'll do minor releases of RHEL every six months and major releases of RHEL
[04:41.920 --> 04:53.040]  every three years and this is what we've been sort of doing eight and nine and something
[04:53.040 --> 04:58.920]  called ABI, I got that as a note for myself, with RHEL we make some promises to customers
[04:58.920 --> 05:04.320]  about ABI guarantees and support statements etc, basically whatever you would expect from
[05:04.320 --> 05:10.280]  Enterprise OS, so we don't want to break things for customers in the major version and this
[05:10.280 --> 05:17.440]  will influence what contributions we can take, so the easiest one is bug fixes, if you find
[05:17.440 --> 05:25.200]  a bug and you can fix it, feel free to do so, we'll be very happy to take it, test it
[05:25.200 --> 05:29.880]  and if it doesn't break things for the customers, merge it, get it in and that's the easiest
[05:29.880 --> 05:38.280]  way to contribute, you can also contribute stable updates from upstream and buy stable
[05:38.280 --> 05:45.120]  that gets to the promises, basically as RHEL ages it gets further from the upstream because
[05:45.120 --> 05:51.320]  we need to keep things sort of stable in the ABI way, so we still release updates like
[05:51.320 --> 05:59.080]  every single minor release, but most of them are backwards, so again welcome to contribute
[05:59.080 --> 06:05.960]  updates that are stable, we'll again fix them, test them, build them and get them in, but
[06:05.960 --> 06:14.160]  yeah, and backported features here, this is what I mentioned basically already, I just
[06:14.160 --> 06:21.360]  have a slide for that, okay, what we can take is the ABI non-compatible updates and if you're
[06:21.360 --> 06:25.560]  wondering about details there's the document called RHEL application compatibility guide,
[06:25.560 --> 06:30.080]  you can find it online on the Red Hat's website and it'll explain exactly how it works, but
[06:30.080 --> 06:37.280]  most packages have the ABI stable for the entire 10 years of RHEL life cycle and we
[06:37.280 --> 06:41.400]  take it very seriously at Red Hat because customers build applications and they want
[06:41.400 --> 06:46.200]  them to run forever without changing them, so we don't want to break this for them, so
[06:46.200 --> 06:52.120]  please don't submit things that would break ABI, we would need to politely explain why
[06:52.120 --> 06:59.520]  not and reject it, that's what you can contribute to Fedora ELN for example and we'll get to
[06:59.520 --> 07:00.520]  that.
[07:00.520 --> 07:06.320]  Okay, I have maybe for docs, typos, man pages, there's a thing for customers, if they go
[07:06.320 --> 07:11.120]  to the customer portal and they have a bug with documentation, they can report an issue
[07:11.120 --> 07:16.720]  and get it fixed that way, otherwise we tend to batch them together so they land all at
[07:16.720 --> 07:22.680]  once so maintainers can focus more on actual feature development, back porting and stuff,
[07:22.680 --> 07:32.520]  so these are welcome but they might take longer to get in because of this and this is a detail
[07:32.520 --> 07:39.240]  image of the life cycle, if you want to get your change into a specific minor version
[07:39.240 --> 07:45.320]  of RHEL, we don't have a way in the bugzilla to really communicate it but you can get in
[07:45.320 --> 07:49.800]  touch with the maintainer and you can sort of anticipate, by the way, minor release,
[07:49.800 --> 07:54.880]  this is the dark blue, extended update support, this is the light blue and then update services
[07:54.880 --> 08:00.360]  for SAP, so even like we're done with minor, we still might be supposing it for up to four
[08:00.360 --> 08:07.160]  years and the arrows is like where CentOS stream work happens, so it tracks all of them
[08:07.160 --> 08:14.360]  and just changes make it to the minor releases and yeah, you can sort of anticipate like
[08:14.360 --> 08:19.320]  where it gets but there's no communication like where exactly, so if you really need
[08:19.320 --> 08:24.640]  to, you would need to talk to the maintainer, yeah, we have this for eight and there's also
[08:24.640 --> 08:30.280]  seven, there's like a lot of things going in the background and if you want to contribute,
[08:30.280 --> 08:38.160]  let's talk about how, so you can open bugs in bugzilla, you can test stream, if you find
[08:38.160 --> 08:46.440]  something, you can open bugs and hopefully get it into the next minor release, you can
[08:46.440 --> 08:51.160]  open merge requests in GitLab, create a GitLab account but first please make sure that you
[08:51.160 --> 08:56.160]  have a bug so you start the conversation with the maintainer so they know what's coming
[08:56.160 --> 09:01.760]  and they can also help you with the change and then you can track the change, this is
[09:01.760 --> 09:07.680]  again from the diagram, we have these three tags in Koji which like we used to track the
[09:07.680 --> 09:13.400]  process and you can preview things in the development compose or the production compose
[09:13.400 --> 09:24.560]  base where it gets, you can get the composites on this URL and there's slash production slash
[09:24.560 --> 09:30.680]  development but otherwise they go to the mirror so if you go to CentOS or you will find CentOS
[09:30.680 --> 09:40.320]  stream there, okay, let's have a look at use of CentOS stream, of course you can use it
[09:40.320 --> 09:46.960]  to preview REL test features that are in development, see what's coming before it actually gets
[09:46.960 --> 09:53.840]  to REL, I think one of the interesting part is that you can use it, if you build something
[09:53.840 --> 10:00.240]  on top of REL you can use CentOS stream in your CI to preview how it would work on the
[10:00.240 --> 10:06.720]  future REL so you can get ready for the next minor release and one advantage like compared
[10:06.720 --> 10:10.960]  to a rebuild is that you can, if you find a bug like in CentOS stream you can actually
[10:10.960 --> 10:21.760]  get it fixed for you and get it in REL proper so this is like what we're trying to do there
[10:21.760 --> 10:27.440]  and this is actually one of the most interesting for me so we have special interest groups,
[10:27.440 --> 10:34.080]  there's like the Hyperscale SIG, there's Cloud SIG, there's the K-Mode SIG and they work
[10:34.080 --> 10:38.600]  in the CentOS stream community and they build their own stuff on top of CentOS stream so
[10:38.600 --> 10:44.200]  they have like a stable enterprise platform but again compared to a rebuild they can actually
[10:44.200 --> 10:49.040]  influence what's happening, they can submit changes, unbreaks things for them and get
[10:49.040 --> 10:54.080]  it into REL proper, I know the Hyperscale SIG they're maintaining bunch of stuff before
[10:54.080 --> 10:58.600]  they actually merge it and there's really interesting stuff going on, you're welcome
[10:58.600 --> 11:02.880]  to come in and create your own SIG and use the community build system to build everything
[11:02.880 --> 11:09.280]  and CentOS stream is definitely there, that's the primer build targets.
[11:09.280 --> 11:17.080]  So that was mostly CentOS stream, I have something about CentOS stream 10 and REL 10 as well
[11:17.080 --> 11:22.640]  basically we saw this diagram and with REL 10 we're right here so if you want to contribute
[11:22.640 --> 11:30.720]  towards REL 10, get it in Fedora Rohide which means get in Fedora ELN if it's within the
[11:30.720 --> 11:37.760]  REL package set and at this point you can change APIs, you can do whatever Fedora would
[11:37.760 --> 11:43.800]  normally do through Fedora changes so this is like the most flexible time of contribution
[11:43.800 --> 11:55.720]  to CentOS stream 10 and REL 10 and later when we get to do stream this is from REL 9 but
[11:55.720 --> 12:01.440]  the process is the same we have like Rohide and Fedora ELN, imagine this like Git branches
[12:01.440 --> 12:08.520]  and Fedora Rohide is the rebuild ELN, follows it and we branch CentOS stream from that and
[12:08.520 --> 12:15.640]  then later start doing REL and yeah we call that bootstrap that phase that will be happening
[12:15.640 --> 12:24.200]  somewhere later and yeah that's how it happens so you can get your changes to Fedora ELN
[12:24.200 --> 12:39.280]  right now and that was a different...