That's a very nice soothing start to the talk of just people saying shh.
As some of you may know, I really like to start talks with raising hands.
So put your hand in the air if you use Humbrew.
Lots of people, cool.
Put your hand in the air if you've contributed to Humbrew.
This clump over here will make sense for the next question.
Put your hand in the air if you maintain Humbrew.
Put your hand in the air if you're concerned about what happens if there's a CV during this talk
and no one is able to march a critical PR to fix open SSL.
Because all the maintainers are here.
Yes, good. Thank you.
Yeah, so a little bit of background for you folks.
Let's see if this is working.
There we go.
Oh, sorry. No, this is a Humbrew. We're Mac people here.
Okay, there we go.
So I forgot, Humbrew doesn't actually support this version anymore.
No, back to that one.
Oh, there we go. Okay, that's fine. Humbrew supports this one.
Sorry, the jokes don't get any better from here.
They're only worse.
Hi, I'm Mike McQuade.
This is my almost becoming yearly tradition at this point.
Sort of state of Humbrew talk at FOSDEM.
The distribution's ruined kindly.
Let's me come and do this here, even though Humbrew isn't really a distribution,
but it feels like the least square round peg hold situation at the conference here.
You can find me at various places on the internet
if you want to talk or ask me things during or after or whatever.
I'm currently the CTO of a startup called Workbrew,
which is trying to do some interesting stuff around Humbrew.
I'll talk incredibly briefly about that at the end
with two former GitHub people.
I spent 10 years at GitHub, which I left as a principal engineer last year,
and I'm Humbrew's project leader,
which is something I have to get elected to do every year.
No one has ever run against me, so please, someone do that
and set me free from my life of enslavement to an open source project that I suffer for.
And I've maintained Humbrew for apparently 15 years this year,
which is a little bit worrying.
So I'm going to talk through some stuff we've done in the last year or so.
Some of it may be new to you, some of it will not be.
None of it will be used to any of the maintainers.
I don't know why they're here,
but hopefully they will just laugh at my jokes and stuff like that anyway.
The first major thing, I don't know if any of you noticed
how many of you run Brew Update or noticed updating Humbrew.
Lots of people complain at me about how Humbrew does this automatically
without being prompted.
You can opt out, but please don't.
This should have got, for most people, most of the time, a lot faster than the last year.
And the main reason is that we have stopped using Humbrew's GitHub repositories
as the main data source for Humbrew.
So when Humbrew was first created in 2009,
one of the relatively innovative things it did was to use Git
and put essentially all the data on a GitHub repo
and then instead of building some complex update information system
which is going to pull from some server somewhere that someone would have to host,
it's like, no, we'll just do essentially just run Git fetch in the background.
And Humbrew has kind of had a long-going battle with...
Like a little bit of a battle with GitHub
and more of a battle with the performance characteristics of this.
So Humbrew Core, the main kind of Humbrew repository for all our formula,
for all our packages, has kind of grown and grown over the years.
Like we've had over, I think, 11,000 contributors,
like millions of commits, hundreds of thousands of pull requests at this point.
And as a result, it is very, very, very, very, very, very, very, very slow
to do almost anything related to Git.
And particularly with Git fetch, like a no-op Git fetch was probably at its worst,
taking about 30 seconds just to be like,
no, actually, you don't have any updates or anything required at all.
So when I was lucky enough to be simultaneously working on Humbrew and GitHub,
I added a call to the GitHub API that was there specifically
to try and make Brut update a bit faster.
So you could go to the GitHub API and it could quickly respond like,
hey, don't run Git fetch, you don't need to, it's going to be really slow,
and you don't have any changes anyway.
A few other package managers use that now as well, which makes me happy.
But over the years, lots of people at GitHub have kind of grumbled
about using a Git repo as a CDN that's kind of nicely global.
Globally distributed, and I believe at our peak,
we had a couple of GitHub servers that were essentially dedicated purely
to people fetching from Humbrew Core.
So eventually, after leaving the company, it's kind of weird that it took me
to leave the company to actually make my coworker sloppy.
We, like with a bunch of work from other maintainers,
we kind of moved over to essentially just curling a JSON file off the internet now.
So instead, we have like a 15 meg-ish, I think, compressed file for Humbrew Core,
for Humbrew Cask.
When there's an update, we don't have any sort of clever binary diffing
or anything, unfortunately, so we just download the whole thing again.
But that seems to be a lot faster for most of the people, most of the time.
And we still, optimistically, will be able to make it faster in future.
So in case you didn't know, Humbrew has like a JSON API.
This is basically the kind of the basis of what we're using.
We've had to kind of add some bits and pieces and modify, move things around.
And one of our maintainers here added like nice signing to this and stuff like that
so that we could meet the kind of security requirements,
the performance requirements we wanted for this new API way of downloading.
It's actually, our API is really, really fast because it's posted on GitHub pages.
So if you've had an idea of like statically building your API,
it's incredibly painful in some respects, but also kind of fun in other ways.
But yeah, don't dig too deep on how that's implemented
because it's pretty disgusting.
Another thing, somewhat relatedly, if you have set any of these variables in the past,
like commonly people will set these things because Humbrew was updating too often
and it was too slow and annoyed them,
or shortly after we rolled out the API stuff, a bunch of people opted out
because it was a little bit buggy and stuff like that,
or it also updates too often considering un-setting them for a little bit.
And then if things are still annoying for you, feel free to set them again,
but you might have a better time without these than you used to.
Similarly, if you still have these reports on your disk, you can now un-tap them
and then you will get much more space back
and just generally your updating could be potentially a little bit faster
and happier and all this type of stuff.
The other relatively big thing we did in the last year,
not super exciting for everyone, but our analytics were hosted by Google for a very long time.
We had a lot of people who didn't like us having analytics at all
and I chose to ignore those people because we need them to be able to do our job, unfortunately.
But I guess a concern we did hear again and again from people was like,
hey, we don't mind you having analytics,
but we're a bit concerned with all this data going to Google
and if you look at the analytics docs, you can opt out of certain data collection,
but that's kind of a line on trusting Google to do what they say,
which I kind of do, but I understand not everyone does.
So we've kind of now moved to kind of a nice cloud-hosted like EU instance of inflex DB,
which means that we're gathering essentially the same data we had before,
but we're not kind of tying it to individual users.
We don't have the ability to kind of do stuff like capture IP addresses even if we wanted to
and that makes everything a little bit nicer.
So we've now destroyed all of our existing Google Analytics data
and this means that if you want to know what Hummoo was doing
or what user accounts were like two years ago, tough luck,
but we do have this new analytics system
automatically kind of deletes data after 365 days,
so this should get us a nicer, slightly more privacy-focused approach in future.
And the other thing that has been kind of principal with our analytics is trying to have it.
So if people may not trust us with Gather Analytics,
I understand that like it's a touchy point in the tech industry with privacy and all this stuff nowadays,
but we do try and make all the information we gather public,
so we've got these pages like under formula brudo sh slash analytics,
various pages of the analytics we gather.
We've got a few more things there than we used to be able to have
and you can kind of see the download counts, percentage counts, all this type of stuff.
And basically maintainers don't have access really to any more information than you do.
Like we have a couple, a handful of people can access our InfluxDB console directly,
but like the data in there is in such a kind of messy, horrible format
that no one is querying that directly.
They're all just using the same web pages as you and I might use,
which feels like again, from a privacy perspective,
we're all kind of on the same page, whether you're a user of Hummoo or people maintaining it.
So also, again, another thing to stick your hand in the air for,
who considers Hummoo to be slow?
Yeah, a few people.
Put your hand in the air if you feel like it got faster in the last year.
Mostly just maintainers who made it faster, so...
It's all right, you still count iValue.
So this is a relatively common critique we hear about Hummoo,
is it's slow or why does it upgrade all my things all the times and things like that.
So we are working on this, this is kind of a background, medium priority thing
for us that we kind of considered for quite a while.
So in the last year, hopefully,
brew update, that's mainly got faster from the API stuff we mentioned before.
Hopefully brew upgrades, we've now made it a lot,
in certain cases at least, we can now upgrade
fewer of your dependencies than we used to.
This is a little bit of a hack, but I'm going to talk later on
about how we might be able to make this better going forward.
And then similarly around brew fetch,
some of our maintainers noticed that there was a bunch of work
happening there that didn't need to happen.
So I guess if you do find Hummoo to be a little bit too slow,
then be relatively confident that we do feel your pain
and we are trying to make things faster most of the time.
A really weird performance optimization we decided to do,
considering everything I've said before,
is I don't know if anyone who's not a maintainer ever went
and clicked around on the repo pages on GitHub,
but due to the Git issues I mentioned earlier,
a lot of these pages were time out and stuff like that.
And another thing that Git and GitHub people
who knew a lot about Git have said to us for a while
is due to some complicated Git internal stuff
that I don't really understand,
you have structured the Hummoo repo in pretty much
the worst possible way for Git performance.
Git apparently really does not like having directories
with thousands of files in them,
and we had, I think, a directory with 8,000 files
and it was something like that,
which means you can see it on the GitHub interface
because all these operations list in the directory,
if you did a Git blame or Git log on this directory,
all of those were time out,
which meant increasing amounts of the GitHub user interface
was just not useful for when you were using Hummoo,
and that also contributed to why Git fetch was so slow
Git GC was so slow, like opening PRs,
like the pushes and the pulls and all this stuff involved,
which is like getting really slow
and getting slower and slower and slower.
We were also seeing more instance with GitHub
that GitHub didn't seem to think we're related to this,
but I kind of did.
So we've now like sharded our repos,
so essentially like everything is split into directories
based on name, and because we have quite a lot of libraries,
so Lib gets its own special directory,
it doesn't get bundled in under L,
we've done the same thing for Hummoo Cask as well,
like again, as I say,
GitHub would be wanting us to do this for ages,
but we've finally actually done this now,
and that now means that on these pages,
you can actually finally now see the commit information
and timestamps and all this type of stuff,
and it makes it a bit more useful for people
when it wasn't before.
So a more exciting thing for us is,
we moved to like using Ruby 3.1,
Hummoo, who knew that Hummoo was written in Ruby?
It's this widely known thing, yeah, cool.
And so Hummoo originally I think was on Mac OS,
10.5 I think the first version,
and back then Apple provided like loads of stuff with the OS,
including Ruby, 1.8 or whatever I think it was at the time,
and Hummoo kind of particularly in the early days
tried to use as much stuff from the system as possible
and not pull in its own kind of libraries.
We still try and do that where we can,
but Ruby was an example where Apple said a few years ago
that like, okay, we're kind of deprecating
the system version of Ruby and Python
and I think Perl and stuff like that,
and for Apple kind of deprecating this stuff,
we've sort of been playing chicken and being like,
well, you say it's deprecated,
but you keep upgrading it for us,
so we're gonna just keep using your version
as long as we can, and like eventually kind of went
to some Apple people for the last release
and were like, hey, the Ruby you supply is 2.6,
that's really old, when are we gonna get a new one?
And they were like, did you not read
when we told you it was deprecated?
And we were like, yeah, but, yeah, but please.
And they said, no, this time we mean it.
So like finally we've kind of,
we've always had our own kind of thing
we call portable Ruby, which allowed us a way
to distribute a kind of a Ruby
that you could install anywhere in your system.
So it worked regardless of where your homebrew is,
and it would work on a variety of Mac OS versions
and stuff like that.
And that was now moved to Ruby 3.1,
so now we have a system where essentially everyone
on Mac OS at least, on Linux,
there's some configurations where you don't need this,
but everyone has portable Ruby now
and supplies kind of a nice,
relatively new version of Ruby.
So this is nice for us, it probably has some,
it's had some mild performance increases,
and it lets us use like newer language features,
makes homebrew easier to kind of maintain,
makes it easier for homebrew like Ruby users
to kind of not be used to this kind of ancient version of Ruby,
and then there's stuff like Surabay and Rubikop
and all these other libraries we kind of depend on
that were kind of creeping towards
deprecating Ruby 2.6, or had already done so.
So let's just kind of keep more up to date
and stuff like that as well, which is very nice.
We've also released a official like homebrew Mac OS package.
This is another thing that's been kind of requested
for a long time, people have a love-hate relationship.
I think homebrew was one of the first projects
to do the whole curl this bash script into your terminal,
and then we'll install it that way.
Who has security concerns about that approach?
Almost everyone, good.
We're gonna keep doing it, so yeah.
All right.
But if you don't like that, then you can use this instead.
So this is kind of the more standard installation process
you would expect, where you get a nice installer
and you kind of click through these things
and stuff like that.
And you should end up at the end with essentially
the same stuff, and it prints the same messages for you
and all this type of stuff as the bash installer,
but you can do this through like MDM tools
and things like that.
But as I mentioned earlier,
I've actually been working on a few little bits
which are kind of not strictly homebrew related.
So I've been working on workbrew,
which is this thing where we're building kind of
some close source stuff on top of work,
on top of homebrew to try and kind of find this balance
where there's been a bunch of things
where like the package is an example of one
where people have asked for it over the years,
some people wanted to get involved and built that,
and that's all fine.
Whereas on workbrew, there's been a bunch of stuff
that people have asked for over the years
and I've asked for it, it's homebrew volunteers
and they don't want to do it, say okay, well fine,
we can do some of this stuff for you for money.
So we have our own package here now,
which does a few more things than the homebrew one does
and stuff like that.
Not going to go on about workbrew too much,
but if you are interested, go and have a look at our website
and there's a little demo of like what we're doing
and we're kind of recruiting people
who we want to work with on this stuff.
So get in touch.
But on homebrew stuff,
that's looking forward to the next year.
So we meet together as kind of a homebrew group each year,
so I'm not entirely sure what our roadmap is,
we're going to kind of try and decide some things tomorrow,
maybe as a group, kind of figure out like what we see
as the most important things,
but some ideas kind of I've seen flipping around
and things that I have
and kind of have currently open issues for them
or stuff around like handling conflicts better.
So there's this kind of ability for packages
and homebrew to conflict with each other,
that means you can't have either of them installed,
sorry, you can't have both of them installed
at the same time.
That's kind of a pain in the ass,
it doesn't really work very nicely,
so we're hoping to improve some of that.
There's also kind of inherent conflicts
between CASCs and formulae.
Who feels like they understand
the difference between CASCs and formulae?
Okay, only the homebrew maintainers, great.
So homebrew had this kind of somewhat alternate approach,
like the kind of integrated with homebrew,
but was kind of its own separate ecosystem a few years ago
that kind of merged into homebrew proper a few years ago
called homebrew CASC.
So homebrew, at least in the official kind of repo,
is all about taking open source software.
We build it from source,
we give you binary packages, and then we ship that to you.
Homebrew CASC is a little bit different,
that's for distributing proprietary software
where the upstream package,
well, the upstream supplier of the software
provides the binaries for you,
and then we download that and install it for you.
So for example, Wget might be a formula,
because we can download the sources
and build that from scratch,
or something like Google Chrome, or Zoom,
or whatever would be a CASC.
So there's some cases in which there are CASC and formula
for the same thing, like Docker, for example,
is both an open source project
that kind of, you get some nice binaries,
you can build from source,
but also there's like all the gooey stuff and whatever.
And if you do, if you install the Docker formula
and the Docker CASC at the same time,
things get angry and start shouting at you,
and it doesn't work very nicely.
So that's something that we're probably
gonna try and make better this year.
Another thing is we're continuing to work on our API stuff,
we're trying to make it smaller and faster
and consider ways that we can do that
to again make that updating experience
more pleasant for people to use.
The other, also the API,
as someone who's kind of been consuming
the Humbrew API a lot recently, it's pretty crap.
It was originally kind of created
in the relatively early days of like,
I don't know, 2013 or something like that.
And we've just kind of bolted on bits at this point
where it's got like six arms and three legs
and they're all the wrong shape and it's, yeah, yuck.
So hopefully we can have something that's a little bit nicer
for people who are kind of trying to integrate with Humbrew
to use, release this year as well.
And the stuff I mentioned earlier about upgrades.
So part of the reason Humbrew is often upgrading
everything all the time and people get grumpy
because that's really slow,
is because we don't have a good way
of figuring out what upgrades are needed and when.
So historically we had the kind of conservative approach
of, well, if there's anything else that's new,
that's in your kind of dependency tree,
we will always try and upgrade everything every time
just to be safe.
But then we realized like, well,
you upgrade a ton of stuff all the time
and then that makes people sad and angry on the internet
and all this type of stuff.
So then what I mentioned we did last year
was we basically said, well, we can kind of infer a little bit
from the way the binary packages were built.
The binary package was built with OpenSL 1.1.1
and now we have OpenSL 1.1.2.
We know that this package doesn't need 1.1.2
so we don't have to upgrade it, yada, yada, yada.
But hopefully we actually have like,
there's a lot of the kind of bigger, proper package managers
and distributions have like actual like ABI
which stands for application binary interface,
essentially like what libraries you can link again
and change the versions without breaking things.
They have a lot of tooling around that stuff
that we could kind of adopt and similarly like we can have a way,
even with our existing tooling to kind of make this stuff
a little bit more explicit, which would mean
that we don't need to upgrade as much stuff
as much of the time.
But because we're an OpenSL project,
maybe what we do in the last year
will be something that we haven't thought of yet,
that we think of because someone in this room
has a good idea in a pull request or you file a bug report
and then that makes us think of something that's smart
and then we go and do something in a clever way
or you file a really well written feature request
that then inspires us to do something cool.
So I really encourage you,
even if you've never been involved
in an OpenSource project before,
we're generally, myself excluded, a fairly friendly bunch
and we will all try and help you get involved with Homebrew
and help you along the way,
particularly with something like a pull request,
like if you have an idea and you think you can kind of make it happen
and you can write some code in some sort of form,
even if it's only like 10% of the way to working,
feel free to open a pull request and then just say,
hey, like this is what I tried, this is what I need help with,
and then we can kind of help you along the way.
It's often much easier to talk about the code
than it is to talk about the ideas about the code beforehand.
We're not the type of project where every pull request needs an issue
open to beforehand, like we believe in discussing the code whenever you can
rather than kind of discussing some abstract conception
of what the code might look like when someone decides to write it.
So I think we've got a little bit of time for questions now
and also if you don't feel comfortable asking any questions in this format,
then feel free to ask me anything privately.
I'm on Mastodon and Twitter and you can email me and stuff as well.
And yeah, thank you very much for having me.
APPLAUSE
Are there any questions? Oh, all right.
Just going to ask, where's the...
Oh, the beer costume.
OK, so anyone who was here last year,
I was wearing a head to toe beer costume
because I love my Uber maintainer friends,
but they're not always the most organized bunch.
And someone posted a picture before Fosdame last year saying,
like, here's a beer costume.
Wouldn't it be funny we can make Mike wear this lol?
And I was like, yeah, basically like challenge accepted.
You're not organized enough to make that happen.
And unfortunately they were and I had to wear a beer costume.
There are pictures on the Internet. Don't look for them.
Thankfully they were not organized enough to bring it this year,
so that is why I'm not wearing the beer costume.
And shame on you, sir, for reminding people that it exists.
LAUGHTER
Any more questions?
Awesome. Thank you, Mike.
APPLAUSE
You