Yeah, yeah, exactly. Okay, good afternoon. Yeah, so I'm going to be talking about compiler
intrinsics in sickle in DPC plus plus specifically. This is Intel's open source sickle implementation.
This is what I work on. Yeah, so hopefully I'll be able to say something without saying
too much in 10 minutes. Yeah, so code play. I work for code play. We had the first sickle
implementation, compute CBP. We now work. We were acquired by Intel. So now we work
on the Intel sickle implementation DPC plus plus. That's what I work on. We have lots
of partners, you know, hardware companies, that kind of thing, whoever needs an open
CL implementation, sickle implementation, and so on, come to us.
Yeah, so sickle is a single source heterogeneous programming API. So you can write single source
code that can run on NVIDIA, Intel, AMD GPUs, close to the mic. Okay, voice up. Yeah, so
it's great for someone who's developing scientific applications to be able to write single source
code that runs on whatever GPU the implementation enables, such as CUDA, level zero for Intel,
AMD GPUs, and so on. Yeah, this is a really good thing. So I work specifically on the
NVIDIA and the HIP, the AMD backends for DPC plus plus. Okay, so yeah, I just want to talk
a little bit about compiler intrinsics and how kind of, you know, math function calls
work in sickle and DPC plus plus at the moment, and how we can hopefully improve them so that
we're contributing upstream. So what happens to sickle cause? So essentially, you get your
sickle cause in your source code. This is redirected to spear V open CL cause F, you
compile the spear V, you make a spear V module, this is a symbol within the spear V module,
and then that is the implementation is provided by a CL level zero Vulkan driver. Okay, as
I said, I don't work on the spear V backend at all. I work the PTX, the CUDA or the AMD
GPU backends. So what do we do with these symbols so that we get to the native implementations?
We're not trying to reinvent the wheel. We're not trying to do anything that the people
who are making the GPUs aren't doing already. We're just trying to redirect to that. So
how do we go from this to that, and then compile to our PTX module, our AMD GPU module, HSA
module, and so on. So, yeah, how do we go from spear V OCL to NV cause F? So use the
shim library, easy peasy, that's fine. Okay, you just redirect it, you compile it to bitcode,
you link it, a compilation time, and you get to this, this native bitcode implementation.
This is great. Okay, so we use libclc for this. So libclc is written in open CL. Okay, open
CL does lots of stuff that SQL doesn't expose as easily like address spaces, that kind of
thing. So we write an open CL. This is great. This makes our lives really, really easy.
We can do it. Say, before we get into this, just why do we want to use a BC library in
the first place? Why don't we use a.so? Why don't we just resolve to some symbol that
then a runtime is called and we don't care about it? So on a GPU, the overhead of a function
call is really high. Okay, it's because we lose information about, say, address spaces,
that kind of thing. The GPU memory hierarchy is a bit more complex than, say, for CPU.
So we really, really need to worry about this. We want to inline everything so we don't lose
any information about our memory hierarchies. We also allow compile time branching of code
based on the architecture, based on the back end, that kind of thing. We don't want to
have these checks at runtime. We want high performance. That's the name of the game for
what we're doing. This gives us greater optimization opportunities as well. You can do lots of
dead code elimination, lots of fun stuff in the middle end because you're doing all these
checks at the IR level. Okay, so this is just kind of what it looks like. So we just have
Spirvio CR-Casef. We return NV-Casef. Great. Amazing. That's so easy. And then this is
the implementation which is provided by NVIDIA. This is in-bit code. We link this, and then
this is just in-lined into our IR. This is great. Okay. Yes, so we're linking this echo
code with LipsyLC. Then we link that with the vendor-provided BC library. So we're linking,
linking. We get to the implementation. It's all in-lined. It's all great. We love it.
So this works well, but so this is a bit of code from LipsyLC. Because we're dealing
in OpenCL-C, we could choose something else. We could write a native IR. We find that OpenCL
is actually easier to use than an easier to maintain than writing a native IR. So we end
up with some funny kind of problems with mangling and all this kind of thing. This isn't nice.
Sometimes we need manual mangling. It's got to do with namespaces when they're interpreted
by the OpenCL mangler, unfortunately. Yes, we need to sometimes as well. Sometimes OpenCL
isn't as good as we want it to be. So we need to actually write a native IR as well. So
it's a mix of LVM IR, LipsyLC. It's a bit messy. It's not great. Yes, so also we're
exposing some compiler internals here. This is the NVVM reflect pass, which essentially
takes your function call for NVVM reflect, replaces it with a numeric value. This is
totally done at the IR level, so you can branch at the IR level based on this is a high architecture,
a newer architecture. Do this new implementation, do this new built-in. There's an old architecture,
as well for things like rounding modes. This pass is used. We're exposing this in source
code through hacks. This isn't really, you know, it's not, it's not kosher. But it works.
Who cares? Okay, but consider the new proposal to add FP accuracy attributes to math built-ins.
This is where we have, say, FP built-in cars, and we specify the accuracy in ULP that we
want it to be computed to. Okay, this is totally lost on us. Okay, so this is what it would
look like. Yeah, you have this attribute here. You've, FP max error. This is really, really
needed in SQL because SQL is targeting lots and lots of different platforms. All these platforms
have different numerical accuracy guarantees. We really, really need this, but we don't use
built-ins at all. We're sorry, we don't use LVM intrinsics at all. So this is, we need to get to
a point where we can start using this compiler infrastructure. We're not using it as much as
you want to. So we could do this using a libclc compiler kind of hack workaround. We do another,
you know, pass, you just say compiler precision value. If it's that, do some precise square root.
If it's not, do some approximate thing. Yeah, we could do that. Okay, the problem with libclc
and this stuff, it's not upstreamable. Okay, it's, it's a collection of hacks. It's not totally
hacked, but like it's a little bit messy. It's not written in the same API. It's lib, it's OpenCL
and it's, it's LVM IR. It's messy. We can upstream this. We can all benefit from this.
Okay, so the pro about doing some, another, adding another hack to the, the kind of passes,
another hack to the bunch is that it's easy to do. Okay, we can do this and we can keep going with our
libclc implementation. It's pretty straightforward. We've been doing this the whole time. Yeah,
fine. We don't need to worry about the broader LVM concerns. However, we miss out on LVM community
collaboration, which is why we're, we're here. And then how many of these workarounds do we
actually need in order to keep up with the latest trends and then libclc as bad as it could be now,
like it just degenerates into an absolute mess and we don't want that.
Okay, so we think the answer for this is to try and redirect, try and, try and actually
have it calling the compiler intrinsic. Okay, we want to use compiler intrinsic and then have
some generic behavior of these intrinsics for offload targets. Okay, and this would be used by
say OpenMP by, by, you know, CUDA Clang and so on, all these different targets, but we don't have
this transformation. We, we're not comfortable with this connection. Okay, from an intrinsic to a
vendor provided BC built in. Okay, why is that? Essentially, this needs to happen as early as
possible in the, at the IR level. So we're adding an external dependency in our LLVM kind of,
you know, pipeline. We need to link this BC library early on in our, in our, yeah, pipeline.
We don't do this. We're not comfortable with doing this. We need to figure out a way that people
will be happy with us doing this. Okay, obviously we're used to things resolving to external symbols,
but then that's a runtime thing. It's not, it's not a compile time thing. Okay, this needs to be
inline. We need to do lots and lots of stuff with this at the IR level. Okay, so there will still be
cases where we need libclc potentially. It's not going to, you know, just disappear from our
SQL implementation, hopefully, but we need to start pushing towards better kind of resolution,
better use of these intrinsics in LLVM for offload in general. Okay, so why?
Why? Share infrastructure, keep an eye, keep on the cutting edge of new developments,
less compiler hacks, and we make SQL compilation eventually work upstream. It doesn't at the
moment, but eventually we want it to, of course. We're trying to upstream as much as possible,
but libclc is not upstreamable, and that's a problem.
Okay, so the first step, try and have this discussion about making the intrinsics work
for offload. Okay, so time, okay, time's up. So we need to have this link step at the IR level
early on in the IR kind of pipeline. This is problematic for some people, but we need to talk
about this. So please join in the discussion here. This is NVPTX co-gen for LLVM sign-in friends,
if you have any opinions on this. Sorry, I kind of ran over a little bit, but yeah, any questions?
Yeah, I was wondering, would it make sense to try to get rid of the mess by going to an MLIR
type of approach, or like, what are the benefits or downsides to MLIR?
So I'm not an expert. So the question was, are there benefits? Can we avoid this by going to MLIR?
I think it's more fundamental than MLIR. I'm not an expert on MLIR, but I think we need basic
resolution of intrinsics. Presumably with MLIR, you'll have, you know, other MLIR intrinsics that
will need the same kind of treatment. We'll have the same questions there. So this is the first
case study. This is the most simple case. We're not trying to implement the new
FU built-ins with the accuracy thing. We're just trying to decide how do we make this dependency
on this external BCLib work, and do it in a very, very confined sort of way. Yeah, thank you.
Yeah. Two questions. First one is a tutorial to generate NVPTX from MLIR. There is a work section
about linking with the Bitcoin library from NVIDIA. So what's the difference with this?
And the second question is, you mentioned NVM, which is the closed source
ptx generator from NVIDIA, and there is also the LLVM NVPTX backend.
Are we reaching speed parity with the closed source one?
It depends on the application. We find that with, so the second question first,
is there still a big performance gap between the native, say, NVCC compiler and LLVM client? So
in terms of DPC++, which is a fork of LLVM, we're attaining, say, roughly comparable performance,
whether you're using SQL or you're using CUDA with NVCC, and then any improvements that we make to
the kind of compiler or whatever, they're shared by client CUDA as well. So the first question again
was, how is this different from? So essentially, when you're linking Bitcode or whatever,
you're not using any LLVM intrinsics. You're just redirecting things yourself. You're not
using intrinsics. So you need to do everything explicitly. You need to either have a specific
kind of driver path that will do this for you or you need to specifically say, I want to link this
in at this time or whatever. And so it's more manual. It's not happening automatically. It's
not happening really within the compiler. It's happening at link time, LLVM link time.
All right. Thank you, Hugh. Thank you.