Okay, hello everybody, welcome. Thank you for coming to my talk. My name is Brian Duggan.
I'm going to be talking today about something called TERME, which is Practical and Fun Automation
for all your terminal sessions. I'd like to thank my employer Instacart and the Pearl Foundation
for helping me to be here. I'm on the logistics team at Instacart. Okay, so here's an outline of
the talk. I'm going to give a quick overview of the concept of TERME, what it does, what it's all
about, go through some of the features, explain the scripting capabilities, and then a little bit
about why it's written in Raku, which is usually the first question I get, but I'm saving it for last.
Okay, so here's the basic concept of TERME, works like this. So you have your shell,
you type TERME to start the session, it starts a T-Mux session, how many T-Mux users in here?
Oh good, okay, screen. Okay, you guys can fight it later. Okay, so it starts a T-Mux,
starts a T-Mux session, it puts you in the bottom half, and basically anything that you type into
the bottom half goes into both the bottom half and the top half. Okay, so you type, what is TERME?
And it sends it to the bash shell session at the top, which doesn't know what what means,
depending on your environment, you'll get some strange error messages about,
you know, what command are you trying to type. So, you know, I did this and since this is the
automation room, I thought, you know, probably, you know, maybe I could just automate this talk
completely, so I found a command line version of ChatGPT on the internet that had a command line
wrapper, I thought maybe I could just get ChatGPT to write my entire talk for me,
automate it away, and then I would be done. Okay, so I did the pip install, which sends
a lot of things to the terminal, as everybody probably knows, lots of recommendations about
what to upgrade. Finally, I have the ChatGPT executable, so I typed that and I said, what is
TERME? But did not get very much information since it didn't know about the talk that I hadn't given
yet. So then I said, you know, TERME is being presented at FOSSTEM, which was a little bit more,
a little bit better, but still I had to add a little more substance to the talk. And then,
then the program hung, so I had to interrupt it with control C. And the way you do that with
TERME is you use a backslash, which starts the command, anything that starts with a backslash,
you kind of like the Postgres command line interface is a directive to TERME, so stop says
send a control C signal to the other pane. Okay, so I got a keyboard interrupt, but that wasn't
enough to stop it because that was trapped by the Python interpreter. So then I sent another one,
and then that finally gave me a stack trace, which everybody who uses Python sees a lot.
And then finally, you know, I was done with this session. Okay, so the basic concept here
is simple, you know, you have something on the top, something on the bottom, it's the same,
and the things on the bottom go to the top, so you have kind of an interactive session.
And then you can also send these additional commands to the top. Okay, so now I'm going to go through
some of the features. Okay, so as you saw from the last one, you have everything sort of organized
on the bottom, even if you have stack traces and things on the top, you still have a nice little
session that shows you what you're doing, and you have, you can set up macros, you can run scripts,
you can wait for things, and I'm going to go through a few more of these in detail now in
the next few minutes. So it has a new read line built in, it has a few ways of getting history,
right, read line, there's a last command, there's also fuzzy find, fzf, anybody use fuzzy find for
things, yep, so you can search your history for that. And it searches, right, even if you have
several different sessions on the top, maybe you're on different machines or maybe part of it is in
some other application, some of it is in a shell, it'll search your local history.
So for instance, let's say you're using PSQL, you're connected to a remote database and you have,
you know, your local history and maybe you run some sequel that's going to show you the long
running queries. So you can write a macro to send all of this, and the way you do that is you say
slash edit, then you have a text editor, put your file in an SQL file, and then slash alias
will create a macro that says, you set the name of the macro and then slash run says run this,
run this little script, which will send it to the other console. And then after that,
you can just type slash find queries and the top will get the SQL that you put into the file.
Okay, so it can be convenient for things like that, you know, or you could use it with Redis or,
you know, building your, doing your kernel testings, they saw in the first talk or,
you know, whatever, just any, any sort of session you can just make a macro and send it.
So here's, here's another example. In this case, instead of using a macro, we're going to send
standard out from a command that we run locally to the other pane. So on the bottom, I say delay
three, which means wait three seconds between every line that you send to the top, and then slash
shell means just run this, run this command, and then show me the output. So I say slash shell,
cat, e.g. simple.bash, and here you can see my bash script, which does an echo,
echo docker run, and then echo hostname. So then it runs that command when I say slash do,
it runs that command, and the output from that command gets sent to the top, and it's sent,
you know, after every line, it waits three seconds. So it sort of throttles the output,
you know, which we, which we might need because it might take docker, you know, a few seconds to
start before you run the hostname command on, you know, on the shell inside the container.
So, so the bottom is what you're typing, the top is what you see. So here's another,
here's another feature. So in addition to standard out, you can take, take the output of the top
and send it to standard in of anything that you write. So in this example, I'm using the nl
command, which the standard in, it basically takes standard in and outputs line numbers for,
for the commands that are coming in. So I say sleep three and head user share addict words.
The reason I do sleep three, if you think about it for a second is because when I do the exec
command, I need time to type it, right? So I type sleep three and head, and that gets sent
immediately to the shell, which waits a few seconds, then I type exec nl, and then standard
in comes in, and then it prints out what goes out. So, you know, in the real world, you probably
won't have to sleep because there will be constantly stuff coming through the top terminal.
A few other interesting commands. So await, we'll just wait for either a string or a regular
expression to appear in the top. In queue is something, is a way to in queue a command
after you're finished awaiting it. Grab, repeat, send the same thing over and over,
maybe add an interval, send a file, and we already saw what delay does, setting the delay.
And there are actually a lot of commands. There are 43. Whenever I needed to do something,
I added a new one. So if you have any ideas, send me a PR or send a request. There are 43
different commands right now. Actually, 44. I think I added one this morning.
Okay, so script, anybody here use expect? A few people. Okay, so expect is been around for a
long time, 1993, but it's still pretty useful if you have to interact with a program that requires
a TTY. So here is an example of an expect script on the left. In this, in this case,
what we're going to do is we're going to start a Docker container again, and then we're going to run
user add to add a user. And then we'd like to set a password for the user. So we're going to run
the password command, and we're also going to look for the prompts that are coming back. Okay, so
on the left we see the way expect works is you say spawn, and then expect takes a pattern. So
root at is what comes back in the prompt. And then you send user add, termy, we're going to add a
username to termy. Then it has a regular expression expect dash re that you can then capture with
the expect out. And then finally at the bottom, we're going to print out what we caught, like
we captured the fact that the host name was something that was in the prompt. So you can do
the same thing in termy. You can say a user bin end of termy and then set it to be an executable
file. The default is to just send everything. So it's just kind of like you're interacting from
the console. All the lines just get sent directly to the other pane. So you just say docker run,
and then backslash expect is just kind of like the expect command. There's a little subtlety in
there that you usually don't think about because it's sort of intuitive as a human, you know,
you type it and you're expecting something. But really there's a race condition there, right?
Because between the time that you send your command and the time you send the expect command,
the output might have already happened. So the way that expect deals with it is it keeps track of
the stream and then it kind of goes back and there's sort of this running, you know, a pointer to the
output stream. And the way we do that with termy is we basically run the expect before we run the
command. So it'll basically say now I'm starting to watch the output and then it'll send the output
and then it'll capture it. So when we run this and it runs interactively in Tmux,
so you can see on the top it sends docker run, pulls the image and it's waiting now to see
the prompt. So it waits for the root and then after that it sends the user add, waits for new
password and then finishes successfully. So the output here is in the test anything protocol.
You may or may not be familiar with it, it's just okay and then the number of the test.
And then currently if it doesn't get what it expects then it aborts the tests.
Okay, so quickly I'm just going to say why it's written in Raku. And the main idea here is that
Raku has a lot of very nice ways to do asynchronous programming and also interact with other
commands. So you can quickly open a command, open a pipe to Tmux and interact with it using
asynchronous processes. You can run things synchronously. It's got good not just inter-process
but inter-thread communication. It has built-in constructs that like supplies and channels
and promises which you may be familiar with from other languages. So this is like an example of
how you could tail a file and create a supply which is a built-in type in Raku. And finally,
this is kind of like the implementation of expect. So it's really, it's really pretty
straightforward. You set, you can basically set up an event loop in a separate thread using this
construct, you know, start, starts a thread, react whenever it says here's an event loop.
And then when, if it's a string, we look to see if it contains the target, if it's a regex,
then we send it to a channel so that then we can have it available to use locally. So even if
you're not interested in using Termi, you might find some value in using Raku for automation.
That's the end. Thank you for listening.
Questions? I think I have a few minutes. Yeah, that's a good question. So different shells
do different things that you kind of are not really aware of, even just the simple like printing
of a prompt. They don't always send a new line character. Sometimes they'll send escape sequences
that go to the beginning of the line and then go down the line or sometimes they'll even redraw
the line above it. So it works fine. But you just have to be aware of the idiosyncrasies of the
various shells in terms of what they do to the terminal. It does interfere. You can, one of the,
one of the commands is to buffer the lines and it does get tricky to split up the lines when
there's a lot of cursor movement. Sorry, with serial consoles. So that, that aspect is basically
taken care of by Tmux. So it doesn't do the direct communication with the serial console the way
that expect would. Yep. We have one more minute. Last question. So we only have a handful of users.
So now is your opportunity to request features. We don't have to worry too much about backwards
compatibility. Yep. That's it. Thank you.