Okay, then let's get started. So we have one on stage James Loden, yet again about
Bitcoin. Some of you will remember from your compiler classes maybe, or from other words.
He's one of the team working on a Kobo front end in GCC. And now you may think, okay, Kobo
is what my grandfather used maybe or something. But Kobo is still alive. Did it have a new
standard release recently? Yeah, 23. There you go. And you use it. Diversity in programming
languages. You use it every day indirectly probably for financial transactions. You may
well have used it this morning. I tend to think that the three years I've spent on this is a lot
of work. But I've learned today to appreciate how much other work has gone in to the thing that
we are contributing to. We set out some years ago to add to we decided it's time for a real
free Kobo compiler. And I proposed we should add it to GCC and that is what led to this
presentation. I'm going to talk to you about why that is a good idea and why I had no idea what
I was getting into and what I've learned in the process. All of the large firms, I would say,
in the 1970s and 80s wrote their Books and Records software in Kobo and that's how we run our
financial businesses everywhere in the world yet now. The chances are that your ATM transaction
yesterday went through a Kobo program that was written in 1980 or so. There are estimated to be
billions of lines in use still today. And there was a period some years ago in the 90s when some
Kobo applications, for example, were moved to Java because that was the new thing. Those were the
easy programs. The hard ones are one mass of spaghetti that you don't even want to know what
it looks like. And the only way you can move it to a new system is by taking that source code and
making it run on a different computer. You're never going to re-engineer it. If it were, I wouldn't
be here. The banks or any of these institutions spend a fortune running their proprietary systems on
emulations provided by vendors for machines they haven't made for decades. That costs a lot of
money. I just want 10% of that money. So this is an ongoing thing. We're working now actually with
someone else to go into more detail later. This idea is you just take the code as you find it, however
it may be, you compile it and you run it into a different machine. So we are targeting ISO as a
standard because you have to go somewhere plus whatever additions are needed by some particular
house to build what they need for their purposes. It's so old, Bacchus now hasn't started yet.
So yes, we have a grammar. No, it was not defined for anything like an LALR machine. So
it is an all-encompassing thing, but there is no place to go to get a library, a standard library.
When COPL was defined originally there were no functions, there was no recursion, there were no
local variables. Your program had stuff, they worked on it and produced things. It's also a
gigantic language because it takes all of the problems that you have as an application programmer
and puts it in the compiler. So if you want to convert something there's no printf string. You
just move your variable from one thing to another, that's a compiler job. It's very fast because if
you aren't doing anything in particular to check for say runtime errors, right, like length of the
variable or something, that's the default behavior because 1957, we run as quick as C at the bottom.
There are other features that can slow it down, but that's to take care of things that you would
like to take care of, like that I write to a place that doesn't belong to me.
Somebody suggested that C++ is an easy language to compile and I think there's reason to believe
that's true. This is a big language. And that's not the biggest one. I checked Ganooko Bowl recently.
Their sizes are about twice that now, those numbers in terms of terminals and rules inside the grammar.
And this is just one of the verbs. You don't just name the things past the arguments, you say by which
way you want to do it. And if something goes wrong, you have a way to capture that too. That's also
handled inside the system. It's not try and catch, but it's another such exception system.
Yeah, it's work. So, yeah, I've been doing this for a while. That's the price tag on my version of
C programming language. I entered into this saying, okay, we're going to write a compiler.
And I understood at the beginning it was just the front end. It turns out that my work is chapter
four, section four. Everything else is the compiler. And I had to, I entered into this not really
knowing, oh, 15 minutes, okay, that's good. Not really knowing what this was going to mean.
And so, and I had never written such a thing as I'm working on now. I had used Bison for smaller
tasks. But I didn't really, you know how it is, right? You start out trying to figure out how to
do something. That's where that's the way I worked on it. And the there's some distance between
knowing what you have to do, reading about how it's done, and then actually getting there. And this
is the sort of answer I had to find my way through in order to learn what we were doing. And there's
plenty and how to say this, there is no Royal Road, meaning to say, no one will tell you how to do
this. All you can do is pick up the pieces and plow through it. I recommend as a life proposition
that you begin every problem by knowing the complete domain, the problem domain perfectly, and the
tools you're going to need perfectly. That I think you can guarantee will lead to success. In my case,
I substituted what most programmers have, which is 90% of programmers think that they're above
average. So why not just do that? There is a relatively small number of people in the in the
Bison world. It's not a it was not has not been easy, I don't think as a person using that project to
find people who can help me understand how to solve the problems I'm having. So that that's just a
qualifier for like what's life like in the in the world of a guy who's writing a parser. Also, I had to
learn that the Bison and Flex, the Lexar and the parser. It's really an odd thing in our world where
you've got two projects that I don't know to what degree they talk to each other. They communicate and
cooperate. It's not evident from what I've seen that there's very much communication like that. And
but they share global variables. They share functions. They talk about each other a little bit. But they
don't say the same things. So that was a that's a stumbling block that I have no idea how to solve. But I
think it's a it's a gap in in the world that we work in. When you are writing your parser, there are
two levels you're working in or maybe more. You are defining the metadata for your for your language in C.
And then you're using C to tell Bob Dubner to generate the code for that stuff, please. So it was not
clear when I began. And it's still a little bit fuzzy for me where those definitions have to lie and why it's
sometimes difficult to understand why Bison doesn't understand what we're talking about. But I have found
that we were able to solve almost all the problems in in the Kobo grammar, just using good old precedence, that
is to say, you're always defining one thing in terms of another. If you are, you know, you could it's a little bit
like a make file, right? It's work your way up to the left. And then what do I spend my do my day, my day I spend
tracing, I just looking at the results. What did the machine do? We move from state here to state there, we have
this, we're looking for that. It's not there. Oh, that's the error. Okay. I also discovered that if you read these
books at the beginning, they say, Oh, well, we have identifiers, how do we change how we know how do we distinguish a
function name from a variable name? Well, in my case, the function names are all defined. They're all statements,
it's call, it's, it's read, it's, it's inspect. So I don't have that I know the names. And I know all my variable
names, because Kobal has four sections and one of them is the data section of us where you put your variable names. So
those are all. So I didn't have the problem of here's a string, I'll pass that off to the parser and let him figure
out like I could support them out and have different kinds of tokens for different individual pieces. And that's
this magic in that that's what you want to do. The more types you got, the less you're going to have to worry.
Bison itself is a complex beast. It's not clear to me who's at the helm. There are a lot of pieces being added in
different ways to do different things. And they all look very interesting. I just don't know which ones I want. I
learned that there were some things that were quite useful. And, and then not. So if you have an optional term in the
grammar, and you, and that thing, you know, obviously can be substituted for the thing that could be there, then, then
you can use the precedent thing to to convert the empty version of it to the same presidents as the thing that is
that would be there if it was present. That worked great. If you have a conflict and you simply say, Oh, well, that
rule needs higher presidents than the other one. Very often that won't work. Somebody in this room might be able to tell
me why. But I that has has been a dead end more than once. There were a lot of work went into Bison counter
examples. It's something that I think someone spent a lot of time on. I myself have tried that feature several times hoping for a
magic solution, because I use those need those frequently. But eventually I've come to find that, at least for myself, it
doesn't help me that much. It's it you all it does is produce for you the path that you can actually trace through yourself back
through the report. If you look at the state machine. So yes, it could be resolved different ways. Sometimes it helps me to think,
doesn't matter. It's okay, we can take this shift. We don't have to worry about the reduce. I did try to run the graph on my
grammar. I never saw it never came back. It did. I gave it 24 hours, but I never got an answer. So I'm not sure why that
features there. I guess as a tutorial thing, if you had a seven line grammar and you want to understand how things operate. But I had to
went my way through all of the different things that it could do this feature set that I was talking about is and I don't know the answer.
Why I only know the answer I found. And I think that's true when you're in a complex environment, you just pick the pieces that work for you.
And that's how you land there. So Tim toad, you will remember that from Pearl, there's more than one way to do it. There you can put options
into the grammar itself or you can put options on the command line. I chose the to think that you could probably run the same source file with
different options. So why put them in the source file put them on the command line so you can choose it. That's that's the route I went down. But
there are pure parsers, you can push them, you can use the gc++ interfaces, you've got the general something.
Parcer, you've got different versions of parsers that can be produced from the same text file. And you have YAC emulation. And so I had to like, decide what not to use.
And I think what we want to do is fairly vanilla. There is a really cool feature though of the way they've separated out the pieces of C code that have to go into the grammar.
And they don't do a great job if you ask me of describing how that works. But these these things of separating out what the early part of the the metaflight metadata needs in order to describe the data as versus the way we're going to generate the code.
That requires and provides is very, very useful. And locations, you can't write a debugger if you don't have locations. So I would recommend if perhaps I'll send this to the bison folks that you you separate out your code that's going to the pieces that you need into different files and use those
brackets to to to tell by some what the way they belong. It's really handy actually if you got enough C code in your in your YAC file to put it in some include somewhere because your editor will make you a lot happier.
If you use these two, if you use a if you've got a printer type for every element, every semantic type in your union, then you get nice outputs like this. And there ought to be a rule that says, or there'll even ought to be a warning that says, Hey, you define a type where's the string for it? How come I can't see what this looks like? You're going to want it. And that's what you see here is a reduction for the varying part of the perform a part of the
one of the four types of loops. And we know we know that we've got some keywords we've got varying we have from we have we were it's coming from a numeric display that's a bunch of digits in a row that we can use like a number. And and it's got a name. And this was tricky because names don't usually have hyphens in them when you get to the assembler. And there's a literal so that that was a that was probably a three or something. And boom, and so I'm able to look at this trace and refer right
back to the source code and see the pieces that are are being operated on. Do not open up your debugger on the outputted on the on the object file that is compiled from the seek that was generated by the bison. You just won't like it. You don't run GDB on make. You don't run GDB on on that either. It's just awful. The what you do want to know is what the rules mean. You're using a system that is
declarative. So you have to think about what the rule says and how they operate. That's the way you get to to the answer. And I think I'm just about done except to ask you to help me solve my problem. What you see here is what you would like to write in a lot of languages, right? The second thing is much more easily typed as the first thing, but it easily parsed. I'm on version eight of my
my test for that one because when you get to see you don't know if C belongs to be as means it's relating back to a or whether C is going to be followed by a relational operator and be just an ordinary expansion. So you hit that spot and you think what I need is L.
R.
Two, but I don't have it. So but maybe there's someone here who does know. So that's why I'm here and I'm done. Thank you very much.
I'm glad you asked that question. The man asked why didn't I write a handwritten parser. And the reason is because bison has saved my bacon. The the I don't know how to do that. I do know this many times every week, not every day, but every week.
The the the bison output tells me I have an ambiguity in the grammar. I'm not that can't be parsed. It finds the mistakes that I would be putting in freely if I was writing it by hand.
Any more questions?
Or suggestions for this issue?
I'm I'm here all day.