We are starting, please be quiet.
Couple of things, first of all, welcome.
Here I am to introduce our next speaker, Pablo, who is going to talk about to go.
Just a few practical things.
When you exit, please exit from the back.
And if you want to go out in between the talk, please be quiet.
Because this diaphragm is very loud and these chairs are very squeaky.
So, yes.
That being said, enjoy.
Hello people. How are you today?
It's very fun. Because first two talks were not really crowded.
But mine is good already.
My name is Pablo Golub. I'm working for Cybertech.
So, I'm a senior consultant in the body of a young developer.
So, today, a couple of words about my company. Cybertech is purely PostgreSQL company.
We work with clients only if they have PostgreSQL.
If they don't, we install it and they can work with us from that point.
We are having several branches all over, not all over, but the world.
Some of them in South America, most of them are in Europe.
Not now. I wanted to restart.
So, some of our customers, so choose your fighter.
Some of our products. So, we are not only consulting people, but we do products.
Yeah. Why PostgreSQL? You know why. It's cool. Absolutely.
So, what I'm talking about today.
So, first of all, I want to introduce you to the Go language.
How many of you have an idea of what the Go language is?
Okay. So, I don't need to start from the beginning explaining what the compiler is.
So, yeah. Okay.
Then, I will say a couple of words about IDEs and editors we are using.
Then, I will describe drivers for PostgreSQL specifically.
Some useful extensions. How we do testing, how we do releases,
how we do continuous integration, development, etc.
And then, probably, I hope, we will have a question session.
Okay. So, why Go?
First of all, when I start my first project,
I like that Go produces the native one binary for every platform.
I said, wow. Wow.
I just can build everything from the same command line for every operating system
and architecture I want to. And I don't need virtual machines and other crap.
Just like Go. It's simple enough.
It has a good community. It now has already very comprehensive tools
support by GitHub, GitLab, etc., etc.
Yeah. So, cross-platform is somehow connected with the native binaries for every architecture.
So, this is the last developer survey for Go asking how people are using Go language
for what kind of. So, speaking about PostgreSQL,
I would say that data processing is somehow connected.
Before that, we had, like, these answers where you can see that databases
is like a half of the projects people were using.
Probably, you know, if not, so the top products written Go are Kubernetes and Docker,
OpenShift, then Hugo, the fastest framework for building
statistical sites, etc., etc.
Postgres-related Go products which are written in Go are the CockroachDB,
both Postgres operators from Zalando and from Crunchy,
WallG, PgCenter, PgWatch, PgTimeTable, so on and so on.
So, if you want to find more projects, more applications written in Go,
please go to this site, to this repo.
There are a lot of them. A lot of them, yeah.
Okay, so what about tooling?
Because when I was started, I used sublime text.
There was no proper IDE to work with, no debugger,
no these kind of things.
Right now, according to the last year developer Sari,
the most used IDE is VS Code, then GoLand by JetBrains,
and then Vim, and sublime text is 1%.
I was saying at the conference that they talked that I will try GoLand
and will tell you about how it is different from VS Code.
No, still didn't try it, but I think it's good.
So these are the answers from the previous years.
So as you can see, the intention is the same.
So, VS Code on top, and GoLanguage, et cetera, et cetera.
GoLand, sorry.
What I'm using, I'm using VS Code with the official plugin installed.
Then I'm using the tipwars command line utility
to make these fancy tables out of test output.
Linter, of course, tip nine.
I tried GitLab co-pilot.
It's not bad, but I don't want to spend my money on this.
I have my own head, you know.
GoReleaser to produce the packages and binaries like in one go.
Then PostgreSQL, of course.
And the last, but not least, the Gitpod.io,
which is pretty much the same thing as a Git Hub workspace,
but it's free for open source developers.
So it's good if you want to try something new
and you don't want to install everything on your machine
or to set up the virtual machine.
You can go there, run it in your browser,
drink a beer on the beach, and try something new.
It's okay. If you're done, you just close your browser,
close your tab, and it's gone.
Now, about drivers, about the PostgreSQL part of this talk.
The whole idea of this talk started after I tried to find
good tutorials about how to start to work with PostgreSQL.
A lot of them, not all of them, but a lot of them say,
okay, just use ORM and you're fine.
Well, why?
If I need to create utility to use three commands,
why should I use ORM?
And yeah, don't get me wrong.
ORMs are fine if you know what you are doing.
They solve the problems, but you don't need to put it everywhere
and you don't need to start from it
because otherwise you will be learning ORM
but not learning PostgreSQL itself, right?
Yeah. We have SQL for that.
So, during this talk, I will not be explaining ORMs
on how to work with that.
I will explain the basic drivers.
Anyway, ORMs are using drivers on the low level
to speak to the PostgreSQL, right?
So, we should know how to use them.
So, the thing is, in Go,
that we have these databases QL interfaces.
Interfaces is just like an agreement
on what methods are available from, I don't know,
from object, from structure, or whatever.
And for each database, there should be a special driver,
an implementation for these interface, right?
So, the first official implementation for the PostgreSQL
in the Go world was Leap2Q.
It's good. It's proven.
It's a long time on the market,
but it is in maintenance mode right now for two years, probably.
It's not bad. It's okay.
You can be sure that this functionality is solid,
but if you want more, and if you start a new project,
JXC-PGX is the way to go,
because you can use it with whatever you want.
I will show you later.
So, yeah, we are scientists, and we do graphs.
So, unlike of Python, or not in Python,
but NPM, JavaScript work,
we don't have this statistic for how many times
a particular package was downloaded, used, whatever.
So, the most funny way is to use GitHub stars
and, yeah, to build this stuff.
So, as you can see, the PGX started one year later,
but now it's going to be very popular.
So, in 2019-20, there was an announcement
that the LPQ is going to maintain smart,
and the PGX started to grow.
So, what if your project is using already databases QL,
or you want to follow tutorials or techniques
and to use these databases QL, standard de facto interfaces?
For that purpose, PGX has a special wrapper.
So, you can still use databases QL interface,
but you can, underneath, on the low level,
you will use the PGX package.
So, you should use PGX solely if you are starting a new project,
and your project is aiming only POSQS QL.
Then you don't need databases QL,
but if you are dependent on Orm,
or you are dependent on other package
which wants you to use databases QL,
you go for wrapper.
In that case, the dependency will use this wrapper
to speak to databases QL,
and you will use the power of PGX.
So, there are a lot of unique features
that are not implemented in the standard library,
and they cannot be implemented
because this interface is the same for every possible database.
So, for example, you cannot add methods with a copy support
because only POSQS QL do copy support, right?
So, the most cool feature is that PGX supports binary
transfer protocol, and it supports natively all built-in
POSQS types.
And if you create user types,
it will support them out of the box
unless they are very complicated.
But even in that case, you can create a special interface
or a special object structure
that will tell PGX how to encode, decode the developers
of your types.
So, as I said, yeah, copy protocol, logging,
yeah, and for connection pooling,
you have this after connect hook.
So, the idea in girl language is that the database object
that you have is pooled by default.
So, it can create additional connections,
and you never know how many active...
Well, you can know, but you can never tell
how many connections you have right now.
And this, for example, after connect hook
helps you to prepare your session,
prepare your new open connection for something,
like add some identifier, login, or whatever.
Yeah, listen.notify is implemented natively.
What else?
Yeah, different JSON, HStore, large objects.
So, everything you need is already there.
Nice thing about PGX is in December,
probably the new major version 5 release was...
And it was a huge step forward,
especially in the term of dependencies.
v4, version 4, was good,
but it has so many external dependencies
that, for example, Magnus Agander said,
no, we will not rewrite our internal tool into PGX
because it's too much dependencies.
It's not the thing anymore, and it's very cool.
So, yeah, hello world.
You probably all know how to write it in the database SQL
interface, so you're just using that import package,
but instead of using libpq,
you are specifying PGX as the libp,
which is a wrapper for the standard library.
And then all things are the same.
No difference.
So, if you want to update your project,
you just change the import part,
the import of libpq to the PGX standard lib,
and you are fine.
If you want to use PGX directly,
you are fine to do that.
The thing is here that the PGX return
the connect method return the one connection only.
So, if you don't need a pool of connections,
or if you want to be sure that only one connection is live,
one connection is used, you are going with PGX connect, right?
But please remember that you cannot use this structure,
this connection in parallel,
so you need to know that at one point in time,
only one thread or one go routine can talk to the database.
Otherwise, you're good.
But if you want a pool of connections,
you are going with PGX pool.
And I don't know what can I add here.
It's obvious.
You can pass it to the go routines,
and it will create additional connections as you go,
and you can limit up a number of connections, etc., etc.
It's very, very, very flexible.
Okay, about useful extensions.
For my first project, I started with the libpq as well.
It's way to go.
That's how we grow.
And later, I understand that I want this copy functionality badly.
I need that.
So I started to look to the PGX, how to switch it,
and I didn't want to lose these SQL things
when you're encoding, decoding your structures, slices, arrays,
whatever, right from rows or two rows, right?
That's what most people think the ORM do.
It just translates the rows into the structures.
But, yeah, it's very useful.
So, like, if you are working with an old database SQL or libpq,
you are importing this SQL thing,
and you can have a lot of new methods,
like you can struct the row into the structure,
or you can scan the scalar into the variable,
or you can create a slice from your rows, etc., etc.
It's very cool.
PGX, at that time, didn't provide that.
But with the latest version 5, everything is already there.
Better is included.
Probably you can find cases where you want more control
over decoding and coding,
but after this guy was introduced,
wrote to struct by name,
written by me, by the way,
yeah, everything became very easy.
So, before that, we had only row to struct by position, right?
So, if your structure fields are in the same position
as your field in your row result set,
you're fine.
You're just, like, doing the back and forth.
But if you want to skip some fields,
or, for example, if you have some non-public fields
in the structure, but still want to use this functionality
to decode and code from the result set,
this is the way to go.
Yeah.
Now about testing.
We all know it's very essential, right?
And I heard a lot this statement
that you need to write your tests first,
and only then implementation.
I never did.
Maybe one of us tried it.
Oh, go, go, go.
I'm too lazy, because I never know
where at the end I will go with my code.
I'm starting like, okay, I will implement this thing
that will return this integer,
and then I'm like, oh, no, let's do this.
CTE with a lot of things, yeah.
And if I write a test before I need to follow it, right?
No, it's not funny.
How we do testing?
So I would say there are three main approaches.
The first one, obviously,
is to start a real PostgreSQL server.
You can have your local installation
on the test environment,
or you can download it during the test running,
install it, initialize, et cetera,
then cache, but it's still the same.
It's a real PostgreSQL server.
The second approach would be Mockin libraries.
For database SQL,
that would be Datadog,
go SQL Mock.
And for PGX, I created this PGX Mock,
which is the brother of the SQL Mock,
but, yeah, works with PGX.
I hope you know what is Mockin
and how these things are working, right?
We are pretending that we are a PostgreSQL server,
and our application, our tests,
are thinking that they are speaking to the real server,
but in fact, we just throw the needed answers
to the application.
So, do we need rows?
Okay, this is row, this is the answer.
Oh, no, that one is a row.
Let's see how you will react with that, et cetera, et cetera.
But if you want to test on the protocol level,
there is also some very low libraries,
like PG Mock,
which is just like the real low-level Mockin protocol.
KacrosDB has its own test server,
which is just like an import KacrosDB test server
and use it in your tests.
And another library is copied.
Let's try to maybe...
It's not very useful.
No, let's get back.
Can it work?
Okay, so how to create a test,
how to use this PGX Mock thing.
So, if you read me on the repository,
you will see that now change is required
to your application.
That's a lie.
You need to provide an interface
because the PGX return structures
is a connection or a pool.
We cannot mock structures.
We can mock interfaces.
So, I am defining PGX interface here
and say to my method, to my function,
that I will use this interface.
Please use that.
And for my function, it doesn't care
whether it be a real connection
or whether it be Mockin or anything.
It just knows that this object has this method
and it's enough for that, okay?
So, yeah, we write a code, kind of shit even, okay?
We are trying to call me, we are trying to roll back,
et cetera, et cetera, et cetera, how to test it.
So, I will always start with a successful test cases.
I am a very positive person.
First thing, first though,
I am creating the Mockin object.
PGX mock new pool.
Then I will tell my Mockin object
how should my session looks like, right?
So, I am saying I am expecting
that we will start a transaction.
I am expecting to begin.
Then I am expecting that the code
will try to execute update statement, right?
And when this happens,
please return to this code
the new result that update was successful
and we updated one row, right?
After that, I expect that the code
will try to insert something.
And I expect that the arguments for this statement
would be two and three.
If that is the case,
please tell that everything is good.
We insert one row.
And after all that,
I expect that the code will commit the transaction, right?
That is what I am expecting from the code.
Then I am calling my function record starts
and instead of the PGX, I am passing the Mockin object, Mock.
And two and three arguments, right?
And if anything goes wrong,
the taste case is failed, right?
But another thing I want to check
is every expectation I set were met.
For example, after the commit,
my code might want to write a log to the database
or do other things.
I don't expect that thing from it
and these expectations were met
will fail if something else happens inside this function
which is not being expected, right?
So for fail, for failure is pretty much the same.
We are telling that we expect to start transaction,
we expect to start update statement,
but let's pretend we want to test how our code will behave
if the insert statement will fail.
So we are telling,
when insert statement is coming with the arguments two and three,
let's pretend that error happened.
Return error to our code
and the error with some error text.
Very beautiful.
We are starting our function,
but in that case, we know that it should fail.
That's why we are checking our error to be not new.
We are waiting error, right?
And the same for expectations were met.
So for example, if we failed
and our code tries to do more than we are expecting,
we say, no, please don't.
Please don't.
Yeah, so then you are just using go test
with the t-parts thingy.
I just love how the tables look after this output.
So like for this case, we have like one package
and we have two test cases, right?
But in real application, you might have hundreds,
hundreds of test cases and dozens of packages.
They all be listed
and you can see a coverage for every package
and you can see probably coverage for every test case,
how many passes, how many fails, et cetera, et cetera.
Also, you want to probably investigate
what is the coverage.
For that, you are using the built-in go to cover two
test cases that will produce temporary HTML files
and will open them in your browser.
So you see a combo box with the list of files
in your application and you just go through
all your code.
The red one is not covered by our test cases.
The green one covered by our test cases.
For example, in this case, our main function
is to test it all because it's tricky to test
the main function.
That's why you should put it outside of everything
and make it like two, three lines
and only use your packages inside.
Okay, so time for continuous integration
and continuous delivery.
As a company, we are working in GitHub,
but I'm pretty sure that the same functionality
is available on GitLab and BigBucket and everywhere.
So for every my project, I want to have at least
five actions, GitHub actions.
First one is a dependent bot.
I want my repository is constantly being checked
if any package I'm dependent on is updated.
So it will update, okay, we have a new
minor or whatever version of this package.
It will automatically create the pull request.
I will check the output.
I will check if tests are fine, if everything is okay.
Okay, very good.
I like this because you can do three pull requests
per day and you are super productive.
Then what I want to also always have is a code QL
which will build your sources
and will instigate the possible security vulnerabilities.
Building tests, I'm using that for pull requests only
because if you have fired them on every push
and the test is heavy, it's not fine.
Release will produce the binaries and packages
when the new release is created.
And the docker is the same like for release,
it will produce a special tag and an image and push it.
But for every commit, it will produce a special docker image
which you can just try immediately, right?
Like a night build or whatever.
Dependable is very simple.
For example, for almost all my repositories,
I first want to check the Go code itself,
so package ecosystem Go mode.
And I want to use the latest GitHub actions as well.
That's important.
So I check them daily and that's usually enough.
For code QL, nothing special.
When you create these actions from the GitHub interface,
it will fill all the fields for you.
I never changed the important thing there,
only removed some comments and we are fine.
Building tests, I hope you can see what is going on there.
No. Sorry, I don't want to switch to the editor
because I cannot work like that.
Yeah, so I will tell you.
So I usually run all my tests on three different platforms,
Windows, MacOS and Linux.
The good thing is all the workers already have
PostgreSQL installed on them,
but the thing is that it's not started.
So for you, essentially it's to start PostgreSQL
and then run your tests and you are fine.
Usually the version of Postgres is behind
two or three minor versions, but it's okay.
If you want just like the latest one,
you can go with the Docker images instead of that one.
Yeah.
So there in the build action we have Linter,
so no, without Linter we cannot accept any changes
or pull requests, pull requests.
And yeah, and then we are using,
we are generating the coverage report
to put them everywhere.
See, 99% of code is covered.
Yeah, let's lie.
Okay, so release is a little bit simpler.
As I said, we are using Go Releaser.
It's absolutely fabulous piece of software
that may produce everything for everything.
So the GitHub action code is simple
because everything is stored in the YAML configuration file
where you set up the name, the architecture,
the OSIS, everything.
And then you just like, okay,
let's check out our code and let's release it.
And the cool thing about it,
this is the Go Releaser,
will create a change lock automatically for you
based on your pull requests.
So when I'm releasing, it's just like,
I copy paste it, just sort it by the,
like, what's added, what's fixed, and whatever.
And I'm done.
The release is very simple for me.
Absolutely.
Before that I may spend two days on each release
to produce all these binders, et cetera, et cetera.
Okay, Docker.
Go Releaser can produce Docker images.
I'm too lazy to rewrite this.
And yeah, I'm using like another special,
special GitHub actions to build a Docker.
You can build them for every possible platforms.
And this Apple M1, M2 silicon thing is whatever,
it's just working.
Okay, takeaways.
The Go is popular.
Devrim doesn't like Go.
Yes?
So maybe this is the last time you see this talk,
like when everything goes right,
he said that I need to switch to Rust.
So maybe next, I will go out when the Rust goes right.
So no, no.
Should I stick to the Go?
Okay, we can do both.
Yeah, so a lot of developers are using databases with Go.
Of course.
You can use whatever it did or whatever operating system you are.
By the way, a lot of people are using Windows,
which is not common for like Postgres community, for example.
But it's fine if your system can produce whatever you want.
You don't care.
You just work on what you have, right?
So Kubernetes, you can use whatever you want with the Postgres.
If you want to use Orm, please do.
But remember, you're responsible for that.
Use it wisely.
Otherwise, you can use LITPQ package or new PGX.
And you can use whatever GitHub, GitHub, Bitbucket you want.
And the most amazing thing about Go is the backward compatibility.
You can build whatever in the future.
Yeah, in the future.
Whatever code you want.
And it still will be compatible with the oldest something written ages before.
Even now, when we have generics, when we have a cool thing is in Go,
they are still compatible with that old things.
Okay, so yeah, don't be stranger, check my GitHub account,
check our blog.
Yeah, some of our projects.
And if you have a question, or maybe you have a question to Devrim
why he hates Go.
No, I don't hate Go.
It says I don't package.
I'm not sure what the problem is.
I provide everything you need.
Take my binders and put them in your packaging, whatever.
These conditions don't like to be in Go,
because either it should be a binary or the long internet.
Not internet.
No, no, not internet.
There are only four dependencies for a new created application
to connect to the possible scale.
There are only one direct dependency is PGX,
and there is only four indirect dependency.
Two of them are libraries from Google.
One of them library from KOROS.
I don't remember.
And one of them is by the same author,
because he's using this package in other way.
Yeah, so, questions?
Hi.
A couple of slides back.
You went to a certain functionality where you were asking,
do you do select, and then you put in a structure
in order to get the results from?
Yeah, couple of back.
I have a question on that one.
Yeah, that's one.
Exactly.
So you're asking a select star, and then some stuff,
and then you ask it by the rows.
Does the actual connection, so if you're running this
on one machine, you're running Postgres on another machine,
there is a network, of course, in between.
There's data going back and forth.
Is all the data being sent from Postgres SQL server
to your Go program or not?
The reason I'm asking this is because, let's say,
there's a fourth field, which is, let's say,
a huge JSON field or a huge binary field,
which contains like a megabyte of data per record.
Is it then sending, if I'm asking 100 records,
a thousand records, is it sending a gigabyte over the network,
or is it only sending those three fields?
Okay, I got you.
Yeah, thank you.
Very good question.
So in this particular case, there shouldn't be star,
first of all.
We should always list columns we want to have.
I'm just too lazy, and this is not my code.
But it's like, you know, it shows.
Yeah.
So in this case, yes, everything.
Can you do it automatically?
Can we add columns from a database automatically?
Yes, we can.
But for that, we should use SQLC package,
which is exactly for that.
So it is pre-built or hooks or whatever.
So when you say, go build, you have special SQL files.
Like, I want this field from that and that,
and it will go through it, and it will build automatically
the appropriate structures for go, and then you can use it.
Yeah, it's just a lot of information.
I cannot, like, put it into the one talk.
Yeah, but yeah.
About if we are loading at once, yes, we do in this case.
But the Postgres protocol itself supports row by row functionality,
and it's possible to use that functionality with this package.
So you can, like, yeah, say that.
Hello.
Thank you for the talk.
I have one question about this driver.
Actually, it's kind of only way to work with Postgres,
for my opinion.
And I'm a little bit worried that this driver is not supported
by Postgres community, let's say.
It's supported by someone.
And what is the life cycle of this software?
And maybe it will die.
You say, how is about new features to it,
and all this question arises when you work with it,
because if your management says, okay, let's use Java,
and in Java it's kind of stable, this Postgres driver,
and you know that you always have the new version.
What is about this software?
Yeah, thank you for the question.
So versioning and owner, who is owner, and that kind of stuff.
So as a Postgres community, we support on the C library,
which is live PQ, and Java, which is JDBC, right?
That's all.
All others?
Yeah.
Who uses C++?
All other libraries are maintained by someone.
By the way, live PQ is not a standard library
in terms of made by go.
It's also maintained by one person.
So how we do in this case, we just fork it and use it
if you want something new.
If I did everything better than the maintainer,
the owner of the PGX will accept my progress proposals
and we are stinked, right?
If not, I will beat them and I will be popular.
I have a question regarding testing strategies in CI CD.
So you have shown that it's possible to mock Postgres scale, right?
But sometimes you are relying on some feature of Postgres scale,
or possibly you are relying on some extension Postgres.
Do you want to test with a real Postgres scale?
Yes.
What you can recommend, so I have seen in your CI CD,
you are executing some peer scale comments.
Do you have a dedicated instance for running tests?
No.
So my GitHub actions are using pre-installed Postgres QL
on the GitHub workers.
They already have Postgres 15.1 probably nowadays.
So I'm okay if they are behind several versions.
I don't need to test for a specific feature or bug or whatever.
But if I want to, in my GitHub action,
I may specify the Docker image against which I want to test.
So for example, if I want to test the latest master
from the Postgres QL community,
I will build my own image docker and will run my test against it.
And I'm fine.
Okay, thank you.
Excellent from the back. Yes.