Welcome. So to break the flow of presentations a bit, there's now going to be a discussion,
so not just a presentation, but a discussion. But I'm still going to give a short presentation
first to create some context. First, who am I? Okay. I need to speak louder. Okay. No
worries. So I'm an embedded software architect. This discussion is also mostly focused on
the embedded aspects. I'm working on Linux OS integration for as a consultant for dozens
of customers. And I'm also a maintainer of the build route project, which is a team of four
maintainers or five, depending on how you count. And that's actually the context from which I come.
I mean, from which I give this presentation. I don't actually care about S-bombs. It's just
something that needs to be done. And so, yeah, there we go. Because maybe not everybody is
familiar with it. I'll give a quick overview of what an embedded Linux build system is. So
basically, it's taking a lot of sources, some open source sources coming from the internet,
some in-house components coming from various ways that you can get at them. Sometimes these
in-house components are going to be binaries as well. And then the embedded build system takes
all that together with the configuration and produces a number of artifacts. One thing to
note is that the number of artifacts is really small. So we are talking about maybe five files
or something like that. It's not like when you create an operating system that you have all
these files that you need to keep track of. So from my point of view, as a maintainer of such
an embedded Linux build system, the problem is actually quite simple. We know what the inputs
are. We have just a few outputs. And we can just say, okay, these outputs are generated from these
inputs. So that's actually what we do in build routes. We don't have SPDX at the moment. We don't
have anything complicated. We just have a list of packages with the package name in a version,
where it comes from, the source URL. Also, the tarballs themselves and the hashes for checking
the tarball, the patches which are applied to them, the licenses, and the dependencies. So the
build dependencies. So what other packages were used to generate this particular package? And
then the assumption is that all of this together goes into your target image. So there's no
distinction of what is used for what particular output. There's also a list of files for package
which you can kind of use to reconstruct the, to get more fine grains. And then what I think is
the actual thing you want to have is the CVE information. So the two things which I think are
needed are the licenses. And that's part of this top part. And the vulnerabilities, the CVE information.
And so there's a separate tool that extracts that. And it uses CPE IDs to relate our package name
and version to what is in the CVE database. Now when you do this, this is of course not
reproducible because it uses CVE information over a certain time. And new CVEs are created all the
time. So it's something that you have to rerun all the time. So as I said, it's very simple. There's
a lot of things which are missing and where my question is basically, is this something relevant
to work on? So one thing that is missing is external files that you supply yourself. It's
basically all the configuration which I mentioned here, this part. The assumption is that as a
user, you know what these files are. You can inject them yourself. Same with the built root
source. We could make a tarble of the built root source, but we didn't really see the point. Then
we come to more important things, things that's vendor dependencies. So if we are, as bombs are
used for two purposes basically, one purpose is for license information and second purpose is for
vulnerability tracking. Now if you have a vendor dependency in some package, we just see the top
package that vendors it in and not the actual vendor dependency. So we don't have the package
name and version of that. This used to be not much of a problem because not many people were
vending, but now you have go stuff, rust stuff, NPM stuff, which brings in all these dependencies
and they're kind of invisible to us. We also have everything in one file not spread out over
dozens of SPDX files. If it's good, this is bad, I don't know. We don't have a SPDX format. We have
it only at the package level, not at the individual source file level. So our inputs are basically
tarballs, not C files. And as I mentioned before, we don't have mapping of source to target files.
So that brings us to the discussions points. For me, the most important thing to discuss is
what, why are we doing this? So what are the consumers? How is this information going to be
used? Because that kind of determines what should be used as input as well. If you look at, for
instance, the SPDX specification, it doesn't really say whether you have to look at a source file
or you can treat the tarball as a source. It just says, okay, there is a relationship there. It's
not, I'm going to give you the microphone. I'll come up here with you. So it turned around. There
is, sorry, back to the question. You got me confused now. SPDX. Individual source files or
tarballs? Individual source, a tarball is just a file. Exactly. And you can use SPDX at any level.
And so if you just want to look at the packaging level, the package level, and say, hey, it's this
source file, this tarball file, that's fine. That works. You do not need to take it down to the
source file. And it's a minimum set of fields that you just basically, all the concepts you had up
there, you should be able to express right now with what we've got in SPDX today without any
trouble. And so you basically would put a package there with the metadata that you want to keep
at the higher level, and you point maybe to a file that has a hash. Simple enough. Yep. So I see.
Another remark there.
I'm guessing that at some point you would want either a kind of dependency tree, which can
pop up. So you get a full list of more dependencies. And in conjunction with what was said about the
tarball, which contains other files, could this be done in a way that you eventually have in a
flat format, or in a way that can be flattened, all the dependencies at the top level, so you can
parse them as much as you want. So the remark question is, if I understood correctly, so the
next one is basically a hierarchy of dependencies, but you can flatten it to just have input and
output. What I think for an embedded build system, I think it's enough to only have this flat one
without the hierarchy, because the hierarchy is difficult to determine for the embedded build
system. And I don't think it has useful information, unless there is anybody that can say there is
actually useful information in the hierarchy. Yeah, I'm going to try to speak about that a little
bit later. But yeah, there's like a ton of uses for having a structured S-bomb. And if you saw,
for example, the Siemens use case, where they enrich S-bombs, you need to have that structure to
enable to let you know when the enrichment happens, where the extra information is going to be
happening. Also, if you want to compose an S-bomb by taking pieces from another one, like my
friend Ivana here has been doing a really great work on composing S-bombs. So if you want to
compose an S-bomb by taking pieces from one and moving that data to another one, you need to have
that structure. And there are like several use cases that need to, you need, where you need the
structure. But then I wonder what the, I mean, if you compose S-bombs, there is supposedly also a
corresponding composition of the binaries themselves that the S-bombs describe, right?
Because in the end, an S-bomb is a description of a binary. The binaries can be repackaged,
for example. You can have a binary product of a field, and then you seep it, and you ship it,
and so it. So indeed, if you're going to repackage stuff, then this is relevant. I'm surprised
that our use cases were repackaging stuff. So, yeah, there's a question from, from the chat.
What about handling patches? Good question. So what we currently have in Biltrooth is just patches
are one of the sources, and they're, they're described as one of the sources, or, well,
they're included in the, in the tree as one of the sources. Is there anything else to say
about patches? There's also a specific relationship for patches. Yeah, there's a, in SPDX, there's a
specific relationship for patches, because it's a modification rather than an actual source.
What about naming? Yeah, so is it, I think there was one more remark about patches.
Yes, I was the one in the chat. The thing is, if you have a curl, and you have a curl that you
have patches for, it's not the same curl. There could be other vulnerabilities in your distribution
than the original one from Github, or another one from VST. Yeah, so indeed, it's essential that you
track patches rather than, I mean, you have to, to, you definitely have to record that what you
are using is not curl version X, but curl version X plus patches, and then also which those patches are.
Which takes us to the naming problem. Yeah, so if you've got a naming problem, you say OpenSSL, is it OpenSSL?
It's the OpenSSL. It's the OpenSSL, or is it OpenSSL wrapper, and is it OpenSSL someone's patched,
or modified, or built in a particular way because you've got so many options, and we, yes.
So the remark was, if you say this is OpenSSL, or even OpenSSL version X, that doesn't necessarily uniquely
be identified because it can be patched, or it can be built with certain options, so that information
about how it's patched and how it's built has to be recorded as well.
So do you capture that as part of build route now?
Well, so implicitly, yes, but not explicitly. So it's captured because we have the baseline, which is basically
identified by CPE ID, which is the upstream version, let's say, or well the version as published by the
maintainer of the package, and then the patches are recorded separately.
The configuration, as I mentioned before in build route, we don't really record, that's up to the user themselves
to record it, so it's, I mean, that's definitely a room for improvement there.
Yeah, there's a problem with anything that's building from source in this way, is that the name and version are unique.
This is why we have to attach this.
Right, yeah, exactly.
This is why, recently we have hashed.
Yeah, you have to check the hash, and that's why you need to build information, because just because it says
OpenSSL 1.1.2 doesn't mean that the place your security vulnerability was actually even compiled into the code, right?
Yeah, so the remark was that, what was the remark?
The package inversion information when you're talking about.
Yeah, package inversion information is not enough, are not unique, yeah.
So like it's not going to be the same, like it might not even be the same between two people that built the same thing.
Yeah, because of configuration.
Yeah, so.
And the solution for that is hashing.
Yeah, you can hash the outputs.
Hash the outputs, yeah, but then the thing is simply hashing the outputs.
Okay, then you have an identifier of something, but okay, you have, you actually have the output.
You could hash that, but they don't give you any information.
Right, that's what you need to build.
You need actually to build information itself.
We're also, even there, the usefulness is a bit limited because in the end it goes to the CVE database and in CVE database,
you don't have this information anyway.
You don't have it in a, well, you may sometimes have it in an informal way in the description,
but you definitely don't have it in a formal way saying if configured X then, so unless there is also some changes there,
I don't think there's much use to, I mean, it's important to record it for manual analysis,
but since there is on the other side no formal recording of it, I'm not sure if it makes sense to record it formally.
So what we do for the CVEs is we don't, we only put in the, we put the CVE in and then just once, which ones we've patched.
So like, that way if you go look it up, you know, I don't need to worry about this one, but if there's any new ones.
Yeah, that's basically what is done in built-in as well, but not in an SPDX format, just...
The CVE ID is a field external reference you can associate with the package.
Yeah, so the remark is in SPDX, yeah.
In SPDX is the external references and you can associate a CVE with a specific package.
You can also associate a Perl with that same package and if you wanted to put both of them there, you could.
It's flexible there and because of the time scales of vulnerabilities and so forth,
what you want to record at a point in time whether something's patched or not or other things like that,
this is all hopefully able to be done.
The question is, you know, are people having tools that are semantically accessing it right now?
Yeah, well, so one of the things about the consumer's tools is actually we are seeing tools emerge.
In fact, I know of two off the top that are basically consuming S-bombs and matching them to vulnerabilities.
So that takes care of the monitoring over time because there's two different time cycles.
There's what's known at build and then there's what's known in the field over time.
And so the two projects I'm aware of are Daggerboard and the other one is the one that's sitting in the SPDX repo
that looks up vulnerabilities.
You basically feed it in an S-bomb in SPDX and it will go and query the databases for vulnerabilities.
Yeah, SPDX tool.
To OSV.
To OSV, yeah.
And so there are tools out there that are emerging and I think we'll be seeing more and more in the years.
Yeah, maybe as a reaction to that.
So my intuitive reaction is, yeah, but we also have a tool that generates this information already.
I mean, as part of build routes, you can just run that tool again five years later and you get that information.
But there's a cave-out where I think it's actually useful to have the built information formally recorded
and that's basically the same thing as what archaeologists do.
You don't know what techniques are going to emerge later and that can be useful then.
So if you build something now, you should record all the information now that can be recorded.
The other little add to that is that use case is also very important in the high assurance world
where you are being asked to attest exactly which vulnerabilities you know about.
But that's an audit case or a high assurance case and so some places they will want to have that.
Okay, I would like to move on to a different subject which I mentioned here.
That is vendor dependencies because I think that's a, I mean, from the point of an embedded build system,
that's an important thing to solve.
We actually have multiple vendor dependencies.
I'll first give an intro and then, yeah, multiple vendor dependencies.
So we have some vendor dependencies which are directly included in the source code.
For instance, Tomlip is a good example.
Tomlip is a library that is meant to be vineered in.
And so, yeah, people just copy it into their source code.
So that's really difficult to trace.
Then there is Git submodules.
You clone a Git tree.
That is the information you have in the, as a build system.
That's the information you have in your build metadata.
But then if there are submodules referenced from there,
that information is not part of the metadata of the build system.
And then Cargo will go in PM modules, obviously.
And then there are some cases where the build system itself vendors things in.
For instance, in open embedded, you have the kernel meta which is kind of entered in.
And which is not, I mean, I don't know if it's taken into account for the SPDX.
But you need to take special action there.
That's the important thing.
It's not using the normal parts of taking sources.
So, yeah, my question to the audience is how can we deal with these vendor dependencies?
You had a remark.
A question?
An additional question.
Okay, sorry.
I'm going to keep you.
This is actually just a problem beyond Sbom generation.
Because if you're trying to do air gap builds, these are huge problems in general.
But that we've encountered.
So, you know, ideally we could download all of these sources and archive them without,
and then go tell the tool to go pull them from the archive,
which is often difficult if not impossible.
So, yeah, it'd be nice if the tool like cargo and go.
I think cargo is not too bad.
But go and MPM are pretty bad last we checked for being able to do that kind of thing.
And that would make this a lot simpler too.
Yeah, so the thing that I want to add to that is that, so getting the sources is not the difficult part.
So if it's just for licensing, that's doable.
But for supply chain, you want to know provenance.
And that's the hard part.
There is, unless the, I think for all these things,
unless the source, the upstream itself gives some provenance information,
preferably in a formal format,
then there is no, it's really hard for us as a build system to go and look for it.
For cargo and go, it's actually doable to do it on our side,
because it's, you have a log file which gives the exact information,
not in an easily consumable format, but the information is there.
For MPM, it's, yeah, of course it's MPM.
And GITSIP modules also, the information is there, but you don't have version numbers.
You have GIT hashes which are not something that you will be able to check against the CVE database.
And to some extent, that's also the case for cargo and go versions,
because often they're also, they're specified by GIT hash.
And then they're directly included in sources as usual, hopeless.
Any other?
Well, it's just some perspective.
So one thing to keep in mind about those is,
there are those dependencies that get rendered in,
and you need to capture those in the S-Bomb.
You can see them in two ways.
One is from the dependency list angle,
and in that case, just having the version and then name,
and ideally the hashes of those dependencies is enough.
And the other angle is if you're trying to actually inventory all of the files that get rendered in.
So it depends on the use case of the S-Bomb.
You may want to capture one or the other or both.
But the, so just for dependency's sake, it may be, depending on the build system, of course,
it may be the case that you only need the list of dependencies without the actual file information.
Yeah, the thing is the file information is easy to get,
it's the list of dependencies which they record.
I think picking on what Adolfo just said is,
a lot of builds dependencies just say latest, or not explicit.
On the view that people say, I want to keep up to date patch-wise,
because we're told we've got to keep everything patched,
but for making it dependable, it's the worst thing because it's going to change.
So, yeah, I was trying to just get a debate,
but actually that's, I think, is a lot of the things,
and people are generally lazy, I'll just pull in open SSL.
I don't care what version, I'll just pick up the later,
because it'll be version X today, it'll be version Y next week,
and I'm not bothered about that, I don't want to have an admin.
So, how do we handle that?
Because with the growth of ecosystems,
that basically is what people find very convenient.
Yeah, so I don't think we have an answer to that.
I think we can get one last question, the question you had,
and then we have to stop, I think.
And the question would be, how would you handle
vendor dependencies that come as binaries?
Yeah, the question is, how do you handle
vendor dependencies that come as binaries?
I guess you record what you know, that's the perfect answer.
Actually, this is probably the answer in general,
to any question you record what you know.
Actually, you want to record it in as formal way as possible,
because like for this cargo, go dependencies,
actually our source store has the log file,
which has the exact information of the dependencies,
it's just not something that you want to go and crawl through afterwards
to reconstruct it.
Would it be doable to have two phases, first download the sources,
then you are in a container, you cut the internet,
you stop the interfaces, and you build.
So there's more remark to the offline build thing.
So to solve the offline build problem,
what you can do is cut the build in two phases.
The first phase, where you just do downloads,
and then you have a download territory,
which you expose to the second phase,
where you do the actual processing.
And there are two problems with that,
which I don't think are solved in either Yocto or Biltroot.
The first one is that you actually, to do the downloads,
you need some tools which you have to build.
You need to do the cargo downloads,
you have to build cargo first.
Yeah, but that means that you don't completely separate your build,
you can't completely separate your build in your downloads step,
because you need to build something to download something.
And the second issue, I forgot what the second issue was.
Yeah, the built hermetic builds should be available for everybody.
But we need the tools to support it.
People don't realize that you need your tools to support that.
Yeah, so you can maybe do something like a download step,
a build step, a download step, a build step,
but it's getting complicated.
We're out of time.
Thank you.