Good afternoon everybody. Who is a Go developer? Very well. Very nice to meet you. My name
is Giuseppe Magia. I work with VMware. I am not the creator of this thing that I'm presenting
today. My company is not involved in this. I'm just a user and since it makes my life
easier, I decide to share with you what I do with it. So the thing that I want to do
theoretically is the things that you see on the screen. But practically what I want to
do is to make you curious about Test Script. So you will try and eventually see how good
it is and what you can do with that. The important thing is that you will learn a few basic things.
I mean, I could talk about Test Script for three hours and probably wouldn't exhaust the
argument. But since we have only 20 minutes, this is what we are going to do. We are going to
show the basics of Test Script so you will know it. We started with why. Why do we need to have
this kind of tool? It's because we have a problem. When we do a command line program, we need to test
it and to test it, we need to build it. Then we need to do something with this build to shake it up
and check that it's doing what it's supposed to do. You can do a lot of things instead of
testing directly the command line program in the shell. For example, you could test the single
functions that are inside the program and you should do that. But this is not the same as
testing the program. To test the program, you need to make sure that the function that works well in
your tests also is linked to the command line command or option that you hope is linked correctly,
but not always. Also, the input that you put in the function that works beautifully and since it has
space in between, it doesn't work on the command line. You need really to test the real thing.
The problem that you have is that to achieve this goal, you need first to compile the program
and second to find a way of testing that program in such a way that it works well with your go
code and it is checked in the right way. By checking the right way, I mean that
you are sure that what you hope to achieve is what exactly happens. Doing this kind of thing
in a shell script is not always easy. Let's talk about this test script and what is it?
It's a Go library. It's also a standalone tool and the best thing is that this is developed directly
by the Go team. They use it to test the Go tool itself and all the tools that come with the Go
program. It has been recently, a few years ago, released in the Go internal package, so you can
use it separately from the Go code. Especially, you can use it for mostly anything. If you are
developing your command line code, a command line program in Go, it's much better, but you could
also use it to test mostly any command line program, even if it's not written in Go. Of course, if it's
written in Go, it helps. Let's see a first example. To test something with test script,
you need two components. The first one is a script. It's a script that says something about
what you want to test and what you want to get. In the script that you see on the
upper part of the screen, there is an echo, a hello world and a keyword, exec, before.
Exec is an internal command of test script that will run something. Then there is a standout
with confirmation. The confirmation is that you should receive something that says hello world
and a new line. Then there is an exclamation point that says standard error and a dot. This
line means I don't want a standard error. I don't want anything in a standard error.
More about this later. Then you need a component in your Go code. The component will just call
the standard, the test script dot run, which contains at least one piece of information,
meaning in which directory you find the scripts. I say scripts plural because in that directory,
you can have one or a thousand scripts that do different things to your program.
Let's modify the first script a little bit. Instead of expecting hello world with standout,
we expect h. Then you have this strange thing that is a regular expression. If you know regular
expression, what we are saying here is I just want two words, one that starts with h and one
that starts with w. Like before, I want the standard error to be nil. This standard error with a dot
suggests that what we are expecting here is not a dumb piece of text but a regular expression.
You can use a dumb piece of text if it suits you, but you can have much more powerful type of
information. For example, you can use several statements to describe better what you expect
from the output of the program. In this case, instead of putting everything in one line,
I put that in two lines. This is often useful if you want to make your test more readable
to express exactly what you are expecting.
More important, the test script environment includes one thing that is called text par.
Text par is a very simple way of encoding files. To encode files, you just put the name of the
file between double dashes and then you put the content of the file there. The file will be
magically created in the environment where the test is executed. The thing that happens there
is that the script will use a different temporary directory to each script. Every script can run
in parallel and it will be more or less isolated from the rest. This data.txt will exist in the
temporary directory created for this script only.
You have some built-in commands that you can use directly to do your tests.
Exactly what we have seen already. Then standard out and standard error, it will
check what happens after you have run your command. A standard input will
create the input for the next command. Then there is the command exists, the checks that the file
exists and stop and skip will interrupt the test. If you put the exclamation point before
the command, it will negate the command. Meaning that you expect that command to fail.
Other commands are compare and compare with environment and then you have n that will set
variables and this can be useful. Then you have something that are also available in shell scripts
like cat, cd, cp, check, change mode and make directory move and remove that works like in a shell.
Then you have conditions. Conditions is like a command but is within square brackets
and you are telling the program that you expect something to happen. For example,
exec file name, you are saying I want to make sure that this program is in the path.
Unix says this will only be true if you are running in a Unix system.
And after that condition, you put a command that will run if the condition is true.
And you can check other things like if you have a network, if you are running a specific
version of Go and so on. There is a specific environment, some specific environment variables
that you have work is where you are running the test in practical. The home doesn't exist but you
can set it if you want. And then there is a temp directory that it created for each script but
you can change it if you need something different.
So if you run the test with verbosity, you will get a lot of information that tells you
what is the environment where you are running and what is everything that is executed. If you
don't put the verbose, the test is silent, it will just succeed silently. You will see some
output only in case the test fails. Let's see some more examples with command,
second condition. So the first line says if it's not Unix, skip it. The second line says if it's
Linux, say exec, good choice. Exec, sec means if the exec doesn't exist, then say the command echo,
the command sec was found and so on.
Remember I mentioned something about compiling the executable and one thing that the test script
can do for you is having a transparent executable. How does it work? Let's say I have this word count
command that I have created in go and I want to test it. So I run exec word count and this command
may fail or may succeed whether word count exists or not. So if word count is in the path, it will
succeed. If it's not, it will fail. But we want to make sure that it always succeeds so we need to take
to tell test script this word count not only I want to exist but I want to be the one that I have
created for which I have the code and to be fresh, not stale. How do we do that? In the test we use
the test main in case you don't remember the test main is something that you put in your test code
and runs before any test function that you may have in that directory, in that package.
So the test main contains a call to test script dot run main which has a map of functions that you
can associate with a name. In this case we have a name that is word count and we have
command run main that returns one integer. In the main code you will have the main that
doesn't run directly the code but will call run main and it will exit with the integer that the
run main returns. So what happens here is that your word count that is in the script is in reality
a call to this function and the funny thing is that there is no separate executable. If you remember
go is a compiled language so whenever you run a test nothing is running like in Python,
it's not interpreted. There is a compilation and the compilation happens in a hidden place
and you will have a piece of code that has been compiled that is a binary and that
binary will be available for your test and the good thing is that there is no additional
binary is the same binary that is used for the test. I'm going to show you an example later.
So let's see something more. I said before that we have built in commands but we may want to have
something more profound like custom commands. So for example I want a command that will sleep
for let's say three seconds and this command is not available because it's not one of the
built in commands but I can build it. So I also can have a command that says I want to check all
the files in a directory and I want all these files to exist. So I want to check files that has
first argument the name of the directory and the rest of the arguments are the file names.
How do I do this? When I run a script run I can have a call an indication of a map of functions
that will produce these custom commands. So the custom commands are a map of functions and each
function has a test script object a negation in case I put a bang there and a list of string
arguments. If the command succeed I do nothing if the command doesn't succeed I call test script
dot fatal and I fail. So for example this is how implement the custom commands in my word count.
I call check files and I call sleep for and if we look at the implementation of each one you see
that the sleep for is a function that accepts a test script a negation and a list of arguments
and it takes that argument to say to determine how many seconds I want to sleep for and then
it calls time sleep. If the first argument was not a number it will fail and the command will
not succeed. A similar thing I do for check files so the first argument will be the directory
and then I will check that the files exist for each one of the of the arguments.
A similar thing I can do for custom conditions so in addition to the conditions that we have
I can implement conditions that suit my environment better for example I may want a condition that
says the version of this particular program must be at least 0.2 and how do I do that I cannot do
this with the the building syntax of test script so I can implement a custom condition.
To do a custom condition is is similar to what we do for the custom commands. I have a function
that function receives one string and will parse that string to determine what we do with that
condition. In addition to that test script allows us to pass arguments that are variable
depending on the environment. So for example I want to pass the current version to the test
or I want to pass the home directory to the version and I can do that with a set environment
function. So back to the custom conditions we receive a string and return a boolean and an error.
What do we do inside that function we parse the string and the string could be a simple
condition or it could be a condition with some arguments that are that we need to parse inside
and see if the the condition is true. In this case we have a version is at least and you see I
have created a condition a function that will check that with the elements that are parsed
just in the first line of the of the function. So I assume that the function the the arguments are
separated by a column and I use them. For example this version is at last we'll check that we have
at least two arguments the first argument will be the version and the second argument will be the
compared version. In the same way I can have this condition exist within seconds that checks that a
file exists at least after at maximum a number of seconds that I wait. This is useful for example
when I test in a database system that is supposed to create something but it doesn't create it
instantly. So I say I want to to see this log file at maximum 10 seconds after the database starts
and if not I get an error. So I'm going to show you a quick demo of something that happens when we run
test script.
So if I run go help test flags
sorry
I get a bunch of options and these are used always by the test. Now if I run
word count minus h this is the real executable that I built with go and I you see I have these
options that are just the options of the of the program but if I run something a little bit different
so let's see a test that I have inside here where I running word count minus h like I did on the
command line and you see I have here the options that are made by the executable but in addition to
that I have also the option that belong to the test. This to show you that what we are running here
is an executable but is the executable that go builds for the test itself and the side effect is
that it contains the the command line options that belong to the test itself. Now back to the
to the presentation what we have learned today is that using test script you can simplify
the testing of any command line program and programs that manipulate tests are
extremely suitable for this kind of testing because test script was created for the go tools
which manipulate tests. You don't need to have a separate executable because the test script
environment will create one for you and you can build the commands and conditions if the
built-in ones are not suitable. If you want to see the slides and if you want to see a full example
of how to use test script to test a common command that I was created with go you can go to
github.com data charmer word count and there is the code for this word count and all the example
that I have shown here and a lot more that are testing the word count in most the conditions
that you may have. So you can see how to test this kind of program in reality. Well in reality
I can have a lot more than that but it will be too too long to show. So this is the beginning of
a project that I have to to illustrate all the characteristics of test script using code
and the first step is to show a simple command line program and all the tests that are needed.
Here you will see also more that you will see in the in that on github all the resources that
you can use to learn more and if you want to learn more right now you can ask me outside
and I may show you some more examples. Do we have time for questions? Three minutes. Any questions?
Yes.
When you still want to do unit tests like is it for CLI to know it for like not click input or would
you always create custom commands for that? You can use so the question was whether I can use
test script for unit tests. So you can use test script for mostly anything. I use it for unit test
and I use it for integration test. For integration test I just put some logic before the test to
create the environment and so it will run a little bit slower but mostly it will I mean doing unit
tests is the easiest thing in the world. So if you if you look at the at the go code most of the unit
tests for the go tool itself are run with test script but also the integration test can be run in that.
More questions?
Okay thanks a lot.