Yeah, I hope you had a great Boston so far.
I'm happy to talk about how to build your own MLIR dialect.
So just as a first question, who is aware of what MLIR actually is, who have heard of
the MLIR subproject?
Awesome.
So it's not the whole audience, so I'm going to talk a little bit more about what MLIR
is.
So my outline is, yeah, what is MLIR actually, but I only have a really short slide on that.
I will show you the standalone example, which exists in the MLIR, or in the LLVM repository
as part of the MLIR project.
And I will tell you a little bit more about how you can extend it, how you can build your
own dialect, because following the discussions on discourse and discourse, it always seems
like people hitting the same pain points, at least we did several times.
So that's why I set up this kind of tutorial to show you some of the tricks behind, mainly
from the CMake perspective, which is some kind of tricky sometimes.
So beside that, how to build it, I show you how you can combine it with other dialects.
And last but not least, how to extend your dialect with types or attributes.
So and just as a side note, all code snippets are, of course, licensed under Apache 2 with
LLVM exceptions.
So what is MLIR?
MLIR is actually a reusable compiler infrastructure that was introduced by Google in 2019, early
2019, and at the end of 2019, Google donated it to the LLVM foundation.
So it's officially part of the LLVM project, and there it lives in the mono repo and MLIR,
and what it allows you is to define operations, attributes and types, which are grouped in
so-called dialects, and that let you define your own intermediate representation.
Later on in the session, we will also have an update about the Flang compiler, which
also uses MLIR to define its own intermediate representation.
And these dialects that can be part either of MLIR core, meaning they are in upstream,
like the hung dialect, which gives you the ability to define what a function is, or there's
also an LLVM IR dialect, which actually mirrors what LLVM IR is.
But it is modeled in MLIR, sorry, what LLVM IR is, but it is modeled in MLIR.
There are tons of other dialects, like a GPU dialect, a Tosa dialect, which is the
tensor-operate set architecture, or MTC, which I am one of the developers behind.
And there are also many, many out-of-tree dialects, like the SORC project is using it,
or Torch MLIR, which is actually modeling PyTorch in MLIR.
Many, many more, and these are considered as out-of-tree.
So when we look at the standalone example, which is really a brilliant starter when you
want to create your own dialect, you find it as part of the LLVM Mono repository, and
you can just build it against an installed LLVM.
You can just run CMake, configure it accordingly.
You just need to pass where you find your installed MLIR and where the LLVM external
lit is present.
And actually, then you can just build your target, which is here's a check standalone.
It builds all the tools and further runs some tests.
This actually assumes, as I have mentioned, that LLVM and MLIR are built here and built
here, and then they are installed to prefix.
And that corresponds to out-of-tree somehow.
And for me, when I began with LLVM or MLIR, I was not a compiler developer, but I had
some experience in CMake and how these terms out-of-tree are used in LLVM and MLIR and
the outer world are sometimes confusing, so I want to give at least a short definition.
So in the LLVM world, entry also often or nearly every time refers to a monolithic build.
That means you build LLVM or your LLVM subproject plus your dialects or whatever.
Entry can refer to the source location.
So here we have an out-of-tree dialect, which is however part of the LLVM monorepo, but
it's considered out-of-tree because you can pull it out and you don't need to have it
in the monorepo.
So out-of-tree normally refers to work with a separate repository.
However, there is also a mechanism which you can use to build your project with this LLVM
external projects mechanism, and projects using this, and if you look into their CMake configuration
or into other tutorials, either they call it out-of-tree monolithic build.
So it's not a component build like you have against an installed MLIR or LLVM, or they
even call it entry, which is somehow confusing because when you look to CMake, normally infantry
just means you're building where your source code lives, which is actually a bad practice.
You shouldn't do this.
Normally you do out-of-tree builds.
It just means you create a separate directory where you set up your configuration and where
you do your build.
This can also be a sub-directory in the source tree, but it's a separate directory not checked
into your Git later on.
So within this talk, I just call it the external project mechanism.
For me, it's always an out-of-tree build, regardless of what I do.
Even if I build LLVM, I wouldn't call it personally entry because I'm using the CMake notation
normally just to make it clear when you look into some of the projects and don't get confused.
So what we can do is we can extend the standalone project by this LLVM external projects mechanism
and the question is, why should we do this?
So Stephen Noindover gave a great tutorial about how to architecture LLVM projects at
the LLVM DevMitting 2021, which is available on YouTube.
I also have the link in my references.
Here we are referring to a monolithic build and in his tutorial, he says, use the component
build.
That is what the standalone project already gives you.
But there are some benefits when you maybe want to use the LLVM external projects.
So what we actually do is when we developed the IMHC dialect, we developed this as an
out of three dialect, completely independent or buildable against an installed MLIR version.
IMHC is now part of the MLIR core, so it's upstreamed.
And what we normally do is, or what's quite nice is, sometimes we want to look into when
we change our dialect upstream or when we extend it, how does it behave together with
these out of three source, which we still have, all our conversions, all our transformations
are not upstreamed yet.
And it is quite nice to build it as a run project because you can easily debug into,
you don't have to keep your installation and what you're building out of source, you don't
have to keep this in sync, you just have a monolithic build.
So there are some benefits and we just want to look into what do we need to do to build
it with the LLVM external project mechanism.
So we are creating our build directory again, then we have to define the LLVM targets to
build.
So here you need to specify for which architect you want to build LLVM.
So here it's just host or x86, which is also an option.
You must specify the build type, either release, debug, minstars with relinfo, whatever.
And we need to enable our project MLIR, otherwise it's not build.
And in addition to that, as we want to build our standalone project, we specify LLVM external
projects, standalone dialect, which is our project name.
And furthermore, we specify LLVM external standalone dialect source tree to specify where
do we find our source, that are the two additional parameters you need to pass.
And here LLVM source tier, actually, we assume that it points to the root of our monorepo
checked out.
So that is what we want to have later on.
Right now the standalone example can't do this.
What do we need to change to make this possible?
So right now it's looking like the following, looking to the main CMake configuration and
what is important here is we have find package, we call find package MLIR, and find package
in general imports information which were exported by a project.
So here find package imports information from the installed MLIR version.
And furthermore, the find package MLIR also calls find package LLVM for us, so we don't
need to care about this.
So then just the MLIR config CMake is actually parsed as well as the LLVM config CMake parsed
and we can gladly just do our includes, which adds some further code for us.
So for the external project mechanism build, we don't need to do this.
So what we need to change is we only need to call find package.
If there is an installed MLIR otherwise there won't be one because we're just building it
as part of our build process.
So in that case CMake source here normally is equal to CMake current source here.
If it's not the case we have a different build type and we're just adding this if else block
and we don't have the need no longer for our other project to load the CMake models with
include.
And the code we're adding is we're just setting the variables which are not available as export
a project settings by yourself.
So MLIR main source here, main include here and that's actually it.
So that are the few lines we need to make it buildable.
However, your LIT tests will fail.
So there is a little bit more code that we need to modify.
Here we just define a standalone source here and standalone binary variables which are
then later on used also for include directories.
And we adjust our LIT side CFG upon pi accordingly.
So here we actually need to change CMake binary D or CMake source here by our newly set variable.
Otherwise, yeah, the LIT tests are the location of LIT CFG is assumed in the wrong place.
So we just fix that here.
That's nearly it.
So when you now want to use a dialect with other dialects and you have these in several
repositories or with several projects at least, you can either use LLVM external projects
to build multiple dialects.
Torch MLIR for example is doing exactly this.
Another option is to use CMake's external project at which is considered as the cleanest
way as it really keeps the projects enclosed and doesn't transfer variables between the
projects.
However, what I normally do is I use a sub directory, but in addition with the exclude
from awesome no only require the build targets I really use are exported or transferred to
the other project.
And we do this in our MLIR MLC repository and to do this we actually have an option just
embedded which changes our source code a little bit.
So only when we want to call it embedded then we check is it the case or not because the
find package is already done by the other project.
We don't need to call this.
We only do the includes which we don't need for the external project mechanism.
So getting to types.
This is how the standalone dialect is currently structured or at least most of it.
There are also some tools standalone ops standalone translate which are considered here.
And you see we have multiple finds and types could be specified in standalone ops.td in
our table definition file.
However, it's quite nice to not put it into all into one file but to use separate files
for it.
So what we're doing is we're adding new files.
We're adding a table gen file standalone types.
We're adding a header file and we're adding the CPP for our implementation.
And what you need to put into those are actually the following.
Let's start with the table gen file.
First of all, we include the attribute type base and the dialect itself because the dialect
has some definitions and then we can define our standalone types class which is the base
class for types.
And in addition to that, we can define a custom type.
Actually this is a simple copy of a mid-seas or park type.
Quite straightforward but here we use a standard assembly format so no custom parser and printer
and it just holds a string of parameters.
So nothing special just to illustrate the example.
So that is how the table gen file could look like.
Getting to standalone ops, we can just replace the include of standalone dialect by standalone
types.
And this is because the types already includes the table gen TDE file so that's fine and
that's it.
Regarding the CMAC list, we don't need anything to change.
Why is that?
Actually at MLIR dialect already calls MLIR table gen for you with gen type decals and
type definitions so that's fine.
We don't need to change anything here.
Whatever for attributes that would be different because for attributes, the at MLIR dialect
doesn't call MLIR table gen for you to just set the LRM target definitions by yourself,
call MLIR table gen by yourself, add a public table gen target and that's it.
So attributes are quite close related to, are quite similar except for that you need
to adjust your CMAC configuration by yourself.
For the header file, just include the auto-generated type dev classes in the header, that's it,
add the define, the include, nothing more to do.
For our implementation, we need to make sure that the types can actually be registered
by the parent dialect.
So what we do is we have a define here, get type dev classes, we include then our generated
code, generated by table gen and then we implement or we write a function register types which
actually calls the method add types plus some of the auto-generated code and this needs
to be called in our standalone dialect.cpp.
So we just add the register types here and that's nearly the real trick.
You can do the same not with ad operands or add types but with add attributes for attributes
and to register your attributes.
For the CMAC list itself, just add this to your MLIR standalone live or MLIR dialect
library target, that's it, nothing more to do.
For attributes, you can also just add your source code or you must add your source file
here of course.
But in addition, you also need a dependency on MLIR standalone attributes, ink gen, the
target we generate or we create it by hand because it's not auto-generated just to make
sure that table gen generates the code before CMAC tries to, or before the MLIR standalone
target is built.
You might be lucky otherwise you might have a race condition in your build system.
I experienced that several times, tried to fix it or just keep it in mind and that's
mainly it.
For the standalone dialect, here we use the default printer and parser.
Just let us tell table gen to generate those.
And for register types, actually we need of course a declaration.
We have the implementation but we also need the extra cards declaration generated by table
gen.
Otherwise, yeah, we cannot use it in our standalone ops.cpp.
So all the examples are available in my fork of the LLVM project.
I couldn't make it to senses, we are fabricated to be reviewed through upstream inclusion
prior to my talk.
But I will do so, I will add some more documentation to this, that's at least my goal.
So when I planned this talk, I thought maybe some hints which could have one or the other
and hopefully it's even more helpful if you not only find it in the slidespot also in
the upstream example.
And there are many good resources out there.
So the talk given by media mini and river riddle, the MLIR primer, the MLIR tutorial
at the 2020 LLVM deaf meeting.
We have some great docs at MLIRLLVM.org, here how to create a dialect, the toy example
for example, how to combine it, how to add attribute and types if you want to get more
into the details what you can do all in the table gen world.
Thanks, last but not least, the tutorial given by Steven Neuerhender at the LLVM 2021 deaf
meeting.
Yeah, so that's it from my side.
So if you have questions, please let me know and I try to answer them.
Hey, I just recently got into learning how to develop compilers, even less how to create
languages.
I've been focusing mostly on the front end part, like lexing and parsing.
And my idea was to use LLVM as the back end for a really abstracted C type language which
I'm working on and use LLVM as a back end to generate machine code for x86.
And my understanding was that I only had to use or generate an IR for LLVM, the LLVM
IR.
And now you mentioned that LLVM IR can be described or is somehow related with the MLIR, right?
So I was wondering if I'm still in the correct track to try to generate the parse tree and
use LLVM and try to generate the standard LLVM IR and target the x86 or an x86 platform
or do I need to learn something about the MLIR?
So to try to summarize the question, the question is when as a compiler starter you're mostly
focusing on parsing an abstract C type like language and want to know if you can just
go through the ordinary LLVM IR way or if you need to plug in switch over to MLIR to
do what you want.
So in real short.
So you can do this definitely.
You can go the way you're right now doing.
So MLIR actually is a little bit different.
So if we are looking to clang, if you're talking about an abstract C language, looking into
clang, there is clang AST and then we directly or more or less go to LLVM IR and that is
one of the things which yeah isn't that nice or if you look into other compilers they introduce
more intermediate representations in between like we will see later on in the session the
flang app that for example or even Swift has two intermediate representations for example.
So MLIR just gives you the ability to define additional intermediate representations.
So you can also write a front end for your language, parse it into MLIR, convert it to
the LLVM IR dialect and then translate it to LLVM IR.
So that would be identical.
It really depends on what you want to do, what kind of infrastructure you want to use.
But you can go the way you're already going.
So hopefully that somehow at least answers your question.
Okay, the question is not directly related to the talk but as I'm one of the developers
behind MLC, why we developed MLC?
Sometimes you cannot compile with clang or directly or with LLVM at all to your target.
So the idea was to get something independent of the compiler and when we actually generate
C code with MLC you can have send the freedom to choose which compiler you want to use to
translate for your final target.
So we are in the domain of compilers for machine learning and sometimes we have some very exotic
targets where clang unfortunately is not the option to use it as a compiler.
So that's the simple reason.
Hi, I was coming from the opposite side of the spectrum, I was looking into doing some
sort of just in time compilation but I also wanted to define my own types and my own let's
say things.
My question is would MLIR be a good fit for that or would possibly just see with some
sort of, I don't know, C++ with templates and some sort of, I don't know, dynamic language
or something like that.
So the question is when coming from the other side, so for JIT essentially if MLIR might
be a good fit to define your own types and attributes and well I'm not an expert regarding
JIT but MLIR provides you, I think most of the codes are upstream provides you the possibility
to register types and attributes I'm quite sure at least at runtime.
So you can extend your dialect after you compiled MLIR.
So that is maybe, yeah, depending on what you want to do.
If you really want to modify it during runtime, that should be possible with MLIR.
So I'm not 100% sure but at least worth a look, I think.
Well, I'm partly aware of IRDL but I don't know, you mean how you composite VRC make
into targets or would, yeah, probably you, as you with IRDL as far as I know as you do
most of the time at runtime, you wouldn't need to build it in advance.
So yeah, the CMake stuff would be somehow obsolete, yeah, I think so.
All right, if we're out of questions, thank you Marius, thank you.