[00:00.000 --> 00:12.000] We are starting, please be quiet. [00:12.000 --> 00:14.000] Couple of things, first of all, welcome. [00:14.000 --> 00:21.000] Here I am to introduce our next speaker, Pablo, who is going to talk about to go. [00:21.000 --> 00:23.000] Just a few practical things. [00:23.000 --> 00:26.000] When you exit, please exit from the back. [00:26.000 --> 00:31.000] And if you want to go out in between the talk, please be quiet. [00:31.000 --> 00:36.000] Because this diaphragm is very loud and these chairs are very squeaky. [00:36.000 --> 00:38.000] So, yes. [00:38.000 --> 00:47.000] That being said, enjoy. [00:47.000 --> 00:50.000] Hello people. How are you today? [00:50.000 --> 00:56.000] It's very fun. Because first two talks were not really crowded. [00:56.000 --> 01:01.000] But mine is good already. [01:01.000 --> 01:04.000] My name is Pablo Golub. I'm working for Cybertech. [01:04.000 --> 01:11.000] So, I'm a senior consultant in the body of a young developer. [01:11.000 --> 01:20.000] So, today, a couple of words about my company. Cybertech is purely PostgreSQL company. [01:20.000 --> 01:25.000] We work with clients only if they have PostgreSQL. [01:25.000 --> 01:32.000] If they don't, we install it and they can work with us from that point. [01:32.000 --> 01:41.000] We are having several branches all over, not all over, but the world. [01:41.000 --> 01:50.000] Some of them in South America, most of them are in Europe. [01:50.000 --> 01:54.000] Not now. I wanted to restart. [01:54.000 --> 01:58.000] So, some of our customers, so choose your fighter. [01:58.000 --> 02:07.000] Some of our products. So, we are not only consulting people, but we do products. [02:07.000 --> 02:14.000] Yeah. Why PostgreSQL? You know why. It's cool. Absolutely. [02:14.000 --> 02:18.000] So, what I'm talking about today. [02:18.000 --> 02:20.000] So, first of all, I want to introduce you to the Go language. [02:20.000 --> 02:26.000] How many of you have an idea of what the Go language is? [02:26.000 --> 02:33.000] Okay. So, I don't need to start from the beginning explaining what the compiler is. [02:33.000 --> 02:35.000] So, yeah. Okay. [02:35.000 --> 02:43.000] Then, I will say a couple of words about IDEs and editors we are using. [02:43.000 --> 02:51.000] Then, I will describe drivers for PostgreSQL specifically. [02:51.000 --> 02:56.000] Some useful extensions. How we do testing, how we do releases, [02:56.000 --> 02:59.000] how we do continuous integration, development, etc. [02:59.000 --> 03:06.000] And then, probably, I hope, we will have a question session. [03:06.000 --> 03:14.000] Okay. So, why Go? [03:14.000 --> 03:17.000] First of all, when I start my first project, [03:17.000 --> 03:22.000] I like that Go produces the native one binary for every platform. [03:22.000 --> 03:25.000] I said, wow. Wow. [03:25.000 --> 03:31.000] I just can build everything from the same command line for every operating system [03:31.000 --> 03:36.000] and architecture I want to. And I don't need virtual machines and other crap. [03:36.000 --> 03:42.000] Just like Go. It's simple enough. [03:42.000 --> 03:49.000] It has a good community. It now has already very comprehensive tools [03:49.000 --> 03:56.000] support by GitHub, GitLab, etc., etc. [03:56.000 --> 04:05.000] Yeah. So, cross-platform is somehow connected with the native binaries for every architecture. [04:05.000 --> 04:16.000] So, this is the last developer survey for Go asking how people are using Go language [04:16.000 --> 04:20.000] for what kind of. So, speaking about PostgreSQL, [04:20.000 --> 04:23.000] I would say that data processing is somehow connected. [04:23.000 --> 04:32.000] Before that, we had, like, these answers where you can see that databases [04:32.000 --> 04:41.000] is like a half of the projects people were using. [04:41.000 --> 04:48.000] Probably, you know, if not, so the top products written Go are Kubernetes and Docker, [04:48.000 --> 04:58.000] OpenShift, then Hugo, the fastest framework for building [04:58.000 --> 05:02.000] statistical sites, etc., etc. [05:02.000 --> 05:14.000] Postgres-related Go products which are written in Go are the CockroachDB, [05:14.000 --> 05:21.000] both Postgres operators from Zalando and from Crunchy, [05:21.000 --> 05:28.000] WallG, PgCenter, PgWatch, PgTimeTable, so on and so on. [05:28.000 --> 05:33.000] So, if you want to find more projects, more applications written in Go, [05:33.000 --> 05:37.000] please go to this site, to this repo. [05:37.000 --> 05:42.000] There are a lot of them. A lot of them, yeah. [05:42.000 --> 05:47.000] Okay, so what about tooling? [05:47.000 --> 05:52.000] Because when I was started, I used sublime text. [05:52.000 --> 05:58.000] There was no proper IDE to work with, no debugger, [05:58.000 --> 06:01.000] no these kind of things. [06:01.000 --> 06:06.000] Right now, according to the last year developer Sari, [06:06.000 --> 06:16.000] the most used IDE is VS Code, then GoLand by JetBrains, [06:16.000 --> 06:22.000] and then Vim, and sublime text is 1%. [06:22.000 --> 06:26.000] I was saying at the conference that they talked that I will try GoLand [06:26.000 --> 06:30.000] and will tell you about how it is different from VS Code. [06:30.000 --> 06:35.000] No, still didn't try it, but I think it's good. [06:35.000 --> 06:40.000] So these are the answers from the previous years. [06:40.000 --> 06:44.000] So as you can see, the intention is the same. [06:44.000 --> 06:48.000] So, VS Code on top, and GoLanguage, et cetera, et cetera. [06:48.000 --> 06:52.000] GoLand, sorry. [06:52.000 --> 06:57.000] What I'm using, I'm using VS Code with the official plugin installed. [06:57.000 --> 07:03.000] Then I'm using the tipwars command line utility [07:03.000 --> 07:10.000] to make these fancy tables out of test output. [07:10.000 --> 07:14.000] Linter, of course, tip nine. [07:14.000 --> 07:17.000] I tried GitLab co-pilot. [07:17.000 --> 07:21.000] It's not bad, but I don't want to spend my money on this. [07:21.000 --> 07:24.000] I have my own head, you know. [07:24.000 --> 07:31.000] GoReleaser to produce the packages and binaries like in one go. [07:31.000 --> 07:34.000] Then PostgreSQL, of course. [07:34.000 --> 07:37.000] And the last, but not least, the Gitpod.io, [07:37.000 --> 07:42.000] which is pretty much the same thing as a Git Hub workspace, [07:42.000 --> 07:45.000] but it's free for open source developers. [07:45.000 --> 07:48.000] So it's good if you want to try something new [07:48.000 --> 07:51.000] and you don't want to install everything on your machine [07:51.000 --> 07:54.000] or to set up the virtual machine. [07:54.000 --> 07:58.000] You can go there, run it in your browser, [07:58.000 --> 08:02.000] drink a beer on the beach, and try something new. [08:02.000 --> 08:07.000] It's okay. If you're done, you just close your browser, [08:07.000 --> 08:10.000] close your tab, and it's gone. [08:10.000 --> 08:18.000] Now, about drivers, about the PostgreSQL part of this talk. [08:18.000 --> 08:25.000] The whole idea of this talk started after I tried to find [08:25.000 --> 08:30.000] good tutorials about how to start to work with PostgreSQL. [08:30.000 --> 08:34.000] A lot of them, not all of them, but a lot of them say, [08:34.000 --> 08:39.000] okay, just use ORM and you're fine. [08:39.000 --> 08:42.000] Well, why? [08:42.000 --> 08:46.000] If I need to create utility to use three commands, [08:46.000 --> 08:50.000] why should I use ORM? [08:50.000 --> 08:52.000] And yeah, don't get me wrong. [08:52.000 --> 08:57.000] ORMs are fine if you know what you are doing. [08:57.000 --> 09:06.000] They solve the problems, but you don't need to put it everywhere [09:06.000 --> 09:09.000] and you don't need to start from it [09:09.000 --> 09:12.000] because otherwise you will be learning ORM [09:12.000 --> 09:17.000] but not learning PostgreSQL itself, right? [09:17.000 --> 09:22.000] Yeah. We have SQL for that. [09:22.000 --> 09:28.000] So, during this talk, I will not be explaining ORMs [09:28.000 --> 09:30.000] on how to work with that. [09:30.000 --> 09:32.000] I will explain the basic drivers. [09:32.000 --> 09:36.000] Anyway, ORMs are using drivers on the low level [09:36.000 --> 09:38.000] to speak to the PostgreSQL, right? [09:38.000 --> 09:42.000] So, we should know how to use them. [09:42.000 --> 09:46.000] So, the thing is, in Go, [09:46.000 --> 09:50.000] that we have these databases QL interfaces. [09:50.000 --> 09:53.000] Interfaces is just like an agreement [09:53.000 --> 09:58.000] on what methods are available from, I don't know, [09:58.000 --> 10:01.000] from object, from structure, or whatever. [10:01.000 --> 10:09.000] And for each database, there should be a special driver, [10:09.000 --> 10:12.000] an implementation for these interface, right? [10:12.000 --> 10:17.000] So, the first official implementation for the PostgreSQL [10:17.000 --> 10:23.000] in the Go world was Leap2Q. [10:23.000 --> 10:26.000] It's good. It's proven. [10:26.000 --> 10:29.000] It's a long time on the market, [10:29.000 --> 10:34.000] but it is in maintenance mode right now for two years, probably. [10:34.000 --> 10:36.000] It's not bad. It's okay. [10:36.000 --> 10:41.000] You can be sure that this functionality is solid, [10:41.000 --> 10:46.000] but if you want more, and if you start a new project, [10:46.000 --> 10:50.000] JXC-PGX is the way to go, [10:50.000 --> 10:57.000] because you can use it with whatever you want. [10:57.000 --> 11:00.000] I will show you later. [11:00.000 --> 11:05.000] So, yeah, we are scientists, and we do graphs. [11:05.000 --> 11:08.000] So, unlike of Python, or not in Python, [11:08.000 --> 11:11.000] but NPM, JavaScript work, [11:11.000 --> 11:15.000] we don't have this statistic for how many times [11:15.000 --> 11:19.000] a particular package was downloaded, used, whatever. [11:19.000 --> 11:24.000] So, the most funny way is to use GitHub stars [11:24.000 --> 11:28.000] and, yeah, to build this stuff. [11:28.000 --> 11:36.000] So, as you can see, the PGX started one year later, [11:36.000 --> 11:40.000] but now it's going to be very popular. [11:40.000 --> 11:47.000] So, in 2019-20, there was an announcement [11:47.000 --> 11:50.000] that the LPQ is going to maintain smart, [11:50.000 --> 11:54.000] and the PGX started to grow. [11:54.000 --> 12:00.000] So, what if your project is using already databases QL, [12:00.000 --> 12:04.000] or you want to follow tutorials or techniques [12:04.000 --> 12:09.000] and to use these databases QL, standard de facto interfaces? [12:09.000 --> 12:14.000] For that purpose, PGX has a special wrapper. [12:14.000 --> 12:20.000] So, you can still use databases QL interface, [12:20.000 --> 12:24.000] but you can, underneath, on the low level, [12:24.000 --> 12:30.000] you will use the PGX package. [12:30.000 --> 12:36.000] So, you should use PGX solely if you are starting a new project, [12:36.000 --> 12:41.000] and your project is aiming only POSQS QL. [12:41.000 --> 12:44.000] Then you don't need databases QL, [12:44.000 --> 12:47.000] but if you are dependent on Orm, [12:47.000 --> 12:50.000] or you are dependent on other package [12:50.000 --> 12:53.000] which wants you to use databases QL, [12:53.000 --> 12:57.000] you go for wrapper. [12:57.000 --> 13:02.000] In that case, the dependency will use this wrapper [13:02.000 --> 13:05.000] to speak to databases QL, [13:05.000 --> 13:10.000] and you will use the power of PGX. [13:10.000 --> 13:13.000] So, there are a lot of unique features [13:13.000 --> 13:19.000] that are not implemented in the standard library, [13:19.000 --> 13:21.000] and they cannot be implemented [13:21.000 --> 13:26.000] because this interface is the same for every possible database. [13:26.000 --> 13:32.000] So, for example, you cannot add methods with a copy support [13:32.000 --> 13:36.000] because only POSQS QL do copy support, right? [13:36.000 --> 13:43.000] So, the most cool feature is that PGX supports binary [13:43.000 --> 13:48.000] transfer protocol, and it supports natively all built-in [13:48.000 --> 13:50.000] POSQS types. [13:50.000 --> 13:54.000] And if you create user types, [13:54.000 --> 13:58.000] it will support them out of the box [13:58.000 --> 14:00.000] unless they are very complicated. [14:00.000 --> 14:03.000] But even in that case, you can create a special interface [14:03.000 --> 14:08.000] or a special object structure [14:08.000 --> 14:13.000] that will tell PGX how to encode, decode the developers [14:13.000 --> 14:15.000] of your types. [14:15.000 --> 14:21.000] So, as I said, yeah, copy protocol, logging, [14:21.000 --> 14:23.000] yeah, and for connection pooling, [14:23.000 --> 14:26.000] you have this after connect hook. [14:26.000 --> 14:31.000] So, the idea in girl language is that the database object [14:31.000 --> 14:35.000] that you have is pooled by default. [14:35.000 --> 14:39.000] So, it can create additional connections, [14:39.000 --> 14:41.000] and you never know how many active... [14:41.000 --> 14:45.000] Well, you can know, but you can never tell [14:45.000 --> 14:47.000] how many connections you have right now. [14:47.000 --> 14:50.000] And this, for example, after connect hook [14:50.000 --> 14:53.000] helps you to prepare your session, [14:53.000 --> 14:56.000] prepare your new open connection for something, [14:56.000 --> 15:00.000] like add some identifier, login, or whatever. [15:00.000 --> 15:08.000] Yeah, listen.notify is implemented natively. [15:08.000 --> 15:10.000] What else? [15:10.000 --> 15:15.000] Yeah, different JSON, HStore, large objects. [15:15.000 --> 15:19.000] So, everything you need is already there. [15:19.000 --> 15:26.000] Nice thing about PGX is in December, [15:26.000 --> 15:34.000] probably the new major version 5 release was... [15:34.000 --> 15:38.000] And it was a huge step forward, [15:38.000 --> 15:42.000] especially in the term of dependencies. [15:42.000 --> 15:47.000] v4, version 4, was good, [15:47.000 --> 15:51.000] but it has so many external dependencies [15:51.000 --> 15:54.000] that, for example, Magnus Agander said, [15:54.000 --> 15:59.000] no, we will not rewrite our internal tool into PGX [15:59.000 --> 16:03.000] because it's too much dependencies. [16:03.000 --> 16:07.000] It's not the thing anymore, and it's very cool. [16:07.000 --> 16:10.000] So, yeah, hello world. [16:10.000 --> 16:16.000] You probably all know how to write it in the database SQL [16:16.000 --> 16:24.000] interface, so you're just using that import package, [16:24.000 --> 16:28.000] but instead of using libpq, [16:28.000 --> 16:32.000] you are specifying PGX as the libp, [16:32.000 --> 16:36.000] which is a wrapper for the standard library. [16:36.000 --> 16:39.000] And then all things are the same. [16:39.000 --> 16:41.000] No difference. [16:41.000 --> 16:46.000] So, if you want to update your project, [16:46.000 --> 16:49.000] you just change the import part, [16:49.000 --> 16:53.000] the import of libpq to the PGX standard lib, [16:53.000 --> 16:56.000] and you are fine. [16:56.000 --> 17:00.000] If you want to use PGX directly, [17:00.000 --> 17:04.000] you are fine to do that. [17:04.000 --> 17:10.000] The thing is here that the PGX return [17:10.000 --> 17:15.000] the connect method return the one connection only. [17:15.000 --> 17:18.000] So, if you don't need a pool of connections, [17:18.000 --> 17:22.000] or if you want to be sure that only one connection is live, [17:22.000 --> 17:26.000] one connection is used, you are going with PGX connect, right? [17:26.000 --> 17:32.000] But please remember that you cannot use this structure, [17:32.000 --> 17:35.000] this connection in parallel, [17:35.000 --> 17:40.000] so you need to know that at one point in time, [17:40.000 --> 17:46.000] only one thread or one go routine can talk to the database. [17:46.000 --> 17:48.000] Otherwise, you're good. [17:48.000 --> 17:51.000] But if you want a pool of connections, [17:51.000 --> 17:56.000] you are going with PGX pool. [17:56.000 --> 18:00.000] And I don't know what can I add here. [18:00.000 --> 18:02.000] It's obvious. [18:02.000 --> 18:05.000] You can pass it to the go routines, [18:05.000 --> 18:09.000] and it will create additional connections as you go, [18:09.000 --> 18:13.000] and you can limit up a number of connections, etc., etc. [18:13.000 --> 18:16.000] It's very, very, very flexible. [18:16.000 --> 18:21.000] Okay, about useful extensions. [18:21.000 --> 18:26.000] For my first project, I started with the libpq as well. [18:26.000 --> 18:28.000] It's way to go. [18:28.000 --> 18:30.000] That's how we grow. [18:30.000 --> 18:37.000] And later, I understand that I want this copy functionality badly. [18:37.000 --> 18:39.000] I need that. [18:39.000 --> 18:44.000] So I started to look to the PGX, how to switch it, [18:44.000 --> 18:51.000] and I didn't want to lose these SQL things [18:51.000 --> 18:56.000] when you're encoding, decoding your structures, slices, arrays, [18:56.000 --> 19:01.000] whatever, right from rows or two rows, right? [19:01.000 --> 19:05.000] That's what most people think the ORM do. [19:05.000 --> 19:09.000] It just translates the rows into the structures. [19:09.000 --> 19:12.000] But, yeah, it's very useful. [19:12.000 --> 19:17.000] So, like, if you are working with an old database SQL or libpq, [19:17.000 --> 19:20.000] you are importing this SQL thing, [19:20.000 --> 19:22.000] and you can have a lot of new methods, [19:22.000 --> 19:30.000] like you can struct the row into the structure, [19:30.000 --> 19:36.000] or you can scan the scalar into the variable, [19:36.000 --> 19:41.000] or you can create a slice from your rows, etc., etc. [19:41.000 --> 19:43.000] It's very cool. [19:43.000 --> 19:54.000] PGX, at that time, didn't provide that. [19:54.000 --> 20:03.000] But with the latest version 5, everything is already there. [20:03.000 --> 20:05.000] Better is included. [20:05.000 --> 20:13.000] Probably you can find cases where you want more control [20:13.000 --> 20:15.000] over decoding and coding, [20:15.000 --> 20:21.000] but after this guy was introduced, [20:21.000 --> 20:23.000] wrote to struct by name, [20:23.000 --> 20:25.000] written by me, by the way, [20:25.000 --> 20:29.000] yeah, everything became very easy. [20:29.000 --> 20:37.000] So, before that, we had only row to struct by position, right? [20:37.000 --> 20:43.000] So, if your structure fields are in the same position [20:43.000 --> 20:47.000] as your field in your row result set, [20:47.000 --> 20:48.000] you're fine. [20:48.000 --> 20:50.000] You're just, like, doing the back and forth. [20:50.000 --> 20:53.000] But if you want to skip some fields, [20:53.000 --> 20:58.000] or, for example, if you have some non-public fields [20:58.000 --> 21:01.000] in the structure, but still want to use this functionality [21:01.000 --> 21:05.000] to decode and code from the result set, [21:05.000 --> 21:08.000] this is the way to go. [21:08.000 --> 21:10.000] Yeah. [21:10.000 --> 21:15.000] Now about testing. [21:15.000 --> 21:20.000] We all know it's very essential, right? [21:20.000 --> 21:24.000] And I heard a lot this statement [21:24.000 --> 21:27.000] that you need to write your tests first, [21:27.000 --> 21:30.000] and only then implementation. [21:30.000 --> 21:32.000] I never did. [21:32.000 --> 21:37.000] Maybe one of us tried it. [21:37.000 --> 21:41.000] Oh, go, go, go. [21:41.000 --> 21:44.000] I'm too lazy, because I never know [21:44.000 --> 21:48.000] where at the end I will go with my code. [21:48.000 --> 21:50.000] I'm starting like, okay, I will implement this thing [21:50.000 --> 21:52.000] that will return this integer, [21:52.000 --> 21:55.000] and then I'm like, oh, no, let's do this. [21:55.000 --> 21:58.000] CTE with a lot of things, yeah. [21:58.000 --> 22:02.000] And if I write a test before I need to follow it, right? [22:02.000 --> 22:04.000] No, it's not funny. [22:04.000 --> 22:06.000] How we do testing? [22:06.000 --> 22:09.000] So I would say there are three main approaches. [22:09.000 --> 22:12.000] The first one, obviously, [22:12.000 --> 22:18.000] is to start a real PostgreSQL server. [22:18.000 --> 22:20.000] You can have your local installation [22:20.000 --> 22:22.000] on the test environment, [22:22.000 --> 22:25.000] or you can download it during the test running, [22:25.000 --> 22:28.000] install it, initialize, et cetera, [22:28.000 --> 22:31.000] then cache, but it's still the same. [22:31.000 --> 22:34.000] It's a real PostgreSQL server. [22:34.000 --> 22:38.000] The second approach would be Mockin libraries. [22:38.000 --> 22:43.000] For database SQL, [22:43.000 --> 22:46.000] that would be Datadog, [22:46.000 --> 22:49.000] go SQL Mock. [22:49.000 --> 22:53.000] And for PGX, I created this PGX Mock, [22:53.000 --> 22:56.000] which is the brother of the SQL Mock, [22:56.000 --> 23:00.000] but, yeah, works with PGX. [23:00.000 --> 23:03.000] I hope you know what is Mockin [23:03.000 --> 23:06.000] and how these things are working, right? [23:06.000 --> 23:10.000] We are pretending that we are a PostgreSQL server, [23:10.000 --> 23:14.000] and our application, our tests, [23:14.000 --> 23:17.000] are thinking that they are speaking to the real server, [23:17.000 --> 23:20.000] but in fact, we just throw the needed answers [23:20.000 --> 23:22.000] to the application. [23:22.000 --> 23:24.000] So, do we need rows? [23:24.000 --> 23:27.000] Okay, this is row, this is the answer. [23:27.000 --> 23:29.000] Oh, no, that one is a row. [23:29.000 --> 23:33.000] Let's see how you will react with that, et cetera, et cetera. [23:33.000 --> 23:39.000] But if you want to test on the protocol level, [23:39.000 --> 23:45.000] there is also some very low libraries, [23:45.000 --> 23:49.000] like PG Mock, [23:49.000 --> 23:55.000] which is just like the real low-level Mockin protocol. [23:55.000 --> 24:00.000] KacrosDB has its own test server, [24:00.000 --> 24:03.000] which is just like an import KacrosDB test server [24:03.000 --> 24:06.000] and use it in your tests. [24:06.000 --> 24:13.000] And another library is copied. [24:13.000 --> 24:19.000] Let's try to maybe... [24:19.000 --> 24:24.000] It's not very useful. [24:24.000 --> 24:27.000] No, let's get back. [24:27.000 --> 24:31.000] Can it work? [24:31.000 --> 24:37.000] Okay, so how to create a test, [24:37.000 --> 24:41.000] how to use this PGX Mock thing. [24:41.000 --> 24:45.000] So, if you read me on the repository, [24:45.000 --> 24:47.000] you will see that now change is required [24:47.000 --> 24:49.000] to your application. [24:49.000 --> 24:53.000] That's a lie. [24:53.000 --> 24:56.000] You need to provide an interface [24:56.000 --> 25:00.000] because the PGX return structures [25:00.000 --> 25:03.000] is a connection or a pool. [25:03.000 --> 25:05.000] We cannot mock structures. [25:05.000 --> 25:07.000] We can mock interfaces. [25:07.000 --> 25:13.000] So, I am defining PGX interface here [25:13.000 --> 25:17.000] and say to my method, to my function, [25:17.000 --> 25:21.000] that I will use this interface. [25:21.000 --> 25:23.000] Please use that. [25:23.000 --> 25:28.000] And for my function, it doesn't care [25:28.000 --> 25:30.000] whether it be a real connection [25:30.000 --> 25:32.000] or whether it be Mockin or anything. [25:32.000 --> 25:37.000] It just knows that this object has this method [25:37.000 --> 25:40.000] and it's enough for that, okay? [25:40.000 --> 25:46.000] So, yeah, we write a code, kind of shit even, okay? [25:46.000 --> 25:50.000] We are trying to call me, we are trying to roll back, [25:50.000 --> 25:53.000] et cetera, et cetera, et cetera, how to test it. [25:53.000 --> 25:58.000] So, I will always start with a successful test cases. [25:58.000 --> 26:01.000] I am a very positive person. [26:01.000 --> 26:04.000] First thing, first though, [26:04.000 --> 26:09.000] I am creating the Mockin object. [26:09.000 --> 26:13.000] PGX mock new pool. [26:13.000 --> 26:16.000] Then I will tell my Mockin object [26:16.000 --> 26:20.000] how should my session looks like, right? [26:20.000 --> 26:23.000] So, I am saying I am expecting [26:23.000 --> 26:27.000] that we will start a transaction. [26:27.000 --> 26:29.000] I am expecting to begin. [26:29.000 --> 26:32.000] Then I am expecting that the code [26:32.000 --> 26:35.000] will try to execute update statement, right? [26:35.000 --> 26:38.000] And when this happens, [26:38.000 --> 26:41.000] please return to this code [26:41.000 --> 26:44.000] the new result that update was successful [26:44.000 --> 26:47.000] and we updated one row, right? [26:47.000 --> 26:50.000] After that, I expect that the code [26:50.000 --> 26:54.000] will try to insert something. [26:54.000 --> 26:57.000] And I expect that the arguments for this statement [26:57.000 --> 27:00.000] would be two and three. [27:00.000 --> 27:02.000] If that is the case, [27:02.000 --> 27:05.000] please tell that everything is good. [27:05.000 --> 27:08.000] We insert one row. [27:08.000 --> 27:10.000] And after all that, [27:10.000 --> 27:14.000] I expect that the code will commit the transaction, right? [27:14.000 --> 27:19.000] That is what I am expecting from the code. [27:19.000 --> 27:23.000] Then I am calling my function record starts [27:23.000 --> 27:29.000] and instead of the PGX, I am passing the Mockin object, Mock. [27:29.000 --> 27:32.000] And two and three arguments, right? [27:32.000 --> 27:35.000] And if anything goes wrong, [27:35.000 --> 27:38.000] the taste case is failed, right? [27:38.000 --> 27:42.000] But another thing I want to check [27:42.000 --> 27:45.000] is every expectation I set were met. [27:45.000 --> 27:49.000] For example, after the commit, [27:49.000 --> 27:54.000] my code might want to write a log to the database [27:54.000 --> 27:57.000] or do other things. [27:57.000 --> 28:00.000] I don't expect that thing from it [28:00.000 --> 28:04.000] and these expectations were met [28:04.000 --> 28:10.000] will fail if something else happens inside this function [28:10.000 --> 28:14.000] which is not being expected, right? [28:14.000 --> 28:20.000] So for fail, for failure is pretty much the same. [28:20.000 --> 28:25.000] We are telling that we expect to start transaction, [28:25.000 --> 28:29.000] we expect to start update statement, [28:29.000 --> 28:35.000] but let's pretend we want to test how our code will behave [28:35.000 --> 28:39.000] if the insert statement will fail. [28:39.000 --> 28:41.000] So we are telling, [28:41.000 --> 28:45.000] when insert statement is coming with the arguments two and three, [28:45.000 --> 28:48.000] let's pretend that error happened. [28:48.000 --> 28:51.000] Return error to our code [28:51.000 --> 28:55.000] and the error with some error text. [28:55.000 --> 28:57.000] Very beautiful. [28:57.000 --> 29:00.000] We are starting our function, [29:00.000 --> 29:03.000] but in that case, we know that it should fail. [29:03.000 --> 29:09.000] That's why we are checking our error to be not new. [29:09.000 --> 29:12.000] We are waiting error, right? [29:12.000 --> 29:15.000] And the same for expectations were met. [29:15.000 --> 29:17.000] So for example, if we failed [29:17.000 --> 29:21.000] and our code tries to do more than we are expecting, [29:21.000 --> 29:24.000] we say, no, please don't. [29:24.000 --> 29:26.000] Please don't. [29:26.000 --> 29:31.000] Yeah, so then you are just using go test [29:31.000 --> 29:36.000] with the t-parts thingy. [29:36.000 --> 29:44.000] I just love how the tables look after this output. [29:44.000 --> 29:48.000] So like for this case, we have like one package [29:48.000 --> 29:51.000] and we have two test cases, right? [29:51.000 --> 29:54.000] But in real application, you might have hundreds, [29:54.000 --> 29:59.000] hundreds of test cases and dozens of packages. [29:59.000 --> 30:02.000] They all be listed [30:02.000 --> 30:07.000] and you can see a coverage for every package [30:07.000 --> 30:13.000] and you can see probably coverage for every test case, [30:13.000 --> 30:18.000] how many passes, how many fails, et cetera, et cetera. [30:18.000 --> 30:22.000] Also, you want to probably investigate [30:22.000 --> 30:24.000] what is the coverage. [30:24.000 --> 30:30.000] For that, you are using the built-in go to cover two [30:30.000 --> 30:33.000] test cases that will produce temporary HTML files [30:33.000 --> 30:37.000] and will open them in your browser. [30:37.000 --> 30:41.000] So you see a combo box with the list of files [30:41.000 --> 30:44.000] in your application and you just go through [30:44.000 --> 30:46.000] all your code. [30:46.000 --> 30:50.000] The red one is not covered by our test cases. [30:50.000 --> 30:53.000] The green one covered by our test cases. [30:53.000 --> 30:58.000] For example, in this case, our main function [30:58.000 --> 31:03.000] is to test it all because it's tricky to test [31:03.000 --> 31:05.000] the main function. [31:05.000 --> 31:08.000] That's why you should put it outside of everything [31:08.000 --> 31:10.000] and make it like two, three lines [31:10.000 --> 31:17.000] and only use your packages inside. [31:17.000 --> 31:26.000] Okay, so time for continuous integration [31:26.000 --> 31:32.000] and continuous delivery. [31:32.000 --> 31:36.000] As a company, we are working in GitHub, [31:36.000 --> 31:39.000] but I'm pretty sure that the same functionality [31:39.000 --> 31:47.000] is available on GitLab and BigBucket and everywhere. [31:47.000 --> 31:51.000] So for every my project, I want to have at least [31:51.000 --> 31:56.000] five actions, GitHub actions. [31:56.000 --> 31:59.000] First one is a dependent bot. [31:59.000 --> 32:03.000] I want my repository is constantly being checked [32:03.000 --> 32:08.000] if any package I'm dependent on is updated. [32:08.000 --> 32:12.000] So it will update, okay, we have a new [32:12.000 --> 32:15.000] minor or whatever version of this package. [32:15.000 --> 32:17.000] It will automatically create the pull request. [32:17.000 --> 32:19.000] I will check the output. [32:19.000 --> 32:23.000] I will check if tests are fine, if everything is okay. [32:23.000 --> 32:26.000] Okay, very good. [32:26.000 --> 32:29.000] I like this because you can do three pull requests [32:29.000 --> 32:33.000] per day and you are super productive. [32:33.000 --> 32:38.000] Then what I want to also always have is a code QL [32:38.000 --> 32:45.000] which will build your sources [32:45.000 --> 32:53.000] and will instigate the possible security vulnerabilities. [32:53.000 --> 32:59.000] Building tests, I'm using that for pull requests only [32:59.000 --> 33:03.000] because if you have fired them on every push [33:03.000 --> 33:10.000] and the test is heavy, it's not fine. [33:10.000 --> 33:15.000] Release will produce the binaries and packages [33:15.000 --> 33:18.000] when the new release is created. [33:18.000 --> 33:22.000] And the docker is the same like for release, [33:22.000 --> 33:26.000] it will produce a special tag and an image and push it. [33:26.000 --> 33:33.000] But for every commit, it will produce a special docker image [33:33.000 --> 33:37.000] which you can just try immediately, right? [33:37.000 --> 33:41.000] Like a night build or whatever. [33:41.000 --> 33:43.000] Dependable is very simple. [33:43.000 --> 33:47.000] For example, for almost all my repositories, [33:47.000 --> 33:51.000] I first want to check the Go code itself, [33:51.000 --> 33:54.000] so package ecosystem Go mode. [33:54.000 --> 33:59.000] And I want to use the latest GitHub actions as well. [33:59.000 --> 34:01.000] That's important. [34:01.000 --> 34:06.000] So I check them daily and that's usually enough. [34:06.000 --> 34:10.000] For code QL, nothing special. [34:10.000 --> 34:14.000] When you create these actions from the GitHub interface, [34:14.000 --> 34:17.000] it will fill all the fields for you. [34:17.000 --> 34:22.000] I never changed the important thing there, [34:22.000 --> 34:25.000] only removed some comments and we are fine. [34:25.000 --> 34:30.000] Building tests, I hope you can see what is going on there. [34:30.000 --> 34:37.000] No. Sorry, I don't want to switch to the editor [34:37.000 --> 34:40.000] because I cannot work like that. [34:40.000 --> 34:42.000] Yeah, so I will tell you. [34:42.000 --> 34:48.000] So I usually run all my tests on three different platforms, [34:48.000 --> 34:54.000] Windows, MacOS and Linux. [34:54.000 --> 34:59.000] The good thing is all the workers already have [34:59.000 --> 35:01.000] PostgreSQL installed on them, [35:01.000 --> 35:04.000] but the thing is that it's not started. [35:04.000 --> 35:08.000] So for you, essentially it's to start PostgreSQL [35:08.000 --> 35:11.000] and then run your tests and you are fine. [35:11.000 --> 35:15.000] Usually the version of Postgres is behind [35:15.000 --> 35:20.000] two or three minor versions, but it's okay. [35:20.000 --> 35:23.000] If you want just like the latest one, [35:23.000 --> 35:28.000] you can go with the Docker images instead of that one. [35:28.000 --> 35:31.000] Yeah. [35:31.000 --> 35:37.000] So there in the build action we have Linter, [35:37.000 --> 35:42.000] so no, without Linter we cannot accept any changes [35:42.000 --> 35:45.000] or pull requests, pull requests. [35:45.000 --> 35:49.000] And yeah, and then we are using, [35:49.000 --> 35:52.000] we are generating the coverage report [35:52.000 --> 35:54.000] to put them everywhere. [35:54.000 --> 35:57.000] See, 99% of code is covered. [35:57.000 --> 36:01.000] Yeah, let's lie. [36:01.000 --> 36:05.000] Okay, so release is a little bit simpler. [36:05.000 --> 36:09.000] As I said, we are using Go Releaser. [36:09.000 --> 36:12.000] It's absolutely fabulous piece of software [36:12.000 --> 36:15.000] that may produce everything for everything. [36:15.000 --> 36:19.000] So the GitHub action code is simple [36:19.000 --> 36:24.000] because everything is stored in the YAML configuration file [36:24.000 --> 36:27.000] where you set up the name, the architecture, [36:27.000 --> 36:30.000] the OSIS, everything. [36:30.000 --> 36:33.000] And then you just like, okay, [36:33.000 --> 36:40.000] let's check out our code and let's release it. [36:40.000 --> 36:43.000] And the cool thing about it, [36:43.000 --> 36:46.000] this is the Go Releaser, [36:46.000 --> 36:51.000] will create a change lock automatically for you [36:51.000 --> 36:54.000] based on your pull requests. [36:54.000 --> 36:57.000] So when I'm releasing, it's just like, [36:57.000 --> 37:02.000] I copy paste it, just sort it by the, [37:02.000 --> 37:07.000] like, what's added, what's fixed, and whatever. [37:07.000 --> 37:08.000] And I'm done. [37:08.000 --> 37:10.000] The release is very simple for me. [37:10.000 --> 37:12.000] Absolutely. [37:12.000 --> 37:16.000] Before that I may spend two days on each release [37:16.000 --> 37:20.000] to produce all these binders, et cetera, et cetera. [37:20.000 --> 37:23.000] Okay, Docker. [37:23.000 --> 37:27.000] Go Releaser can produce Docker images. [37:27.000 --> 37:30.000] I'm too lazy to rewrite this. [37:30.000 --> 37:34.000] And yeah, I'm using like another special, [37:34.000 --> 37:38.000] special GitHub actions to build a Docker. [37:38.000 --> 37:42.000] You can build them for every possible platforms. [37:42.000 --> 37:47.000] And this Apple M1, M2 silicon thing is whatever, [37:47.000 --> 37:50.000] it's just working. [37:50.000 --> 37:53.000] Okay, takeaways. [38:01.000 --> 38:04.000] The Go is popular. [38:04.000 --> 38:07.000] Devrim doesn't like Go. [38:07.000 --> 38:08.000] Yes? [38:08.000 --> 38:14.000] So maybe this is the last time you see this talk, [38:14.000 --> 38:17.000] like when everything goes right, [38:17.000 --> 38:20.000] he said that I need to switch to Rust. [38:20.000 --> 38:30.000] So maybe next, I will go out when the Rust goes right. [38:30.000 --> 38:32.000] So no, no. [38:32.000 --> 38:34.000] Should I stick to the Go? [38:34.000 --> 38:36.000] Okay, we can do both. [38:36.000 --> 38:43.000] Yeah, so a lot of developers are using databases with Go. [38:43.000 --> 38:46.000] Of course. [38:46.000 --> 38:50.000] You can use whatever it did or whatever operating system you are. [38:50.000 --> 38:53.000] By the way, a lot of people are using Windows, [38:53.000 --> 38:58.000] which is not common for like Postgres community, for example. [38:58.000 --> 39:06.000] But it's fine if your system can produce whatever you want. [39:06.000 --> 39:07.000] You don't care. [39:07.000 --> 39:11.000] You just work on what you have, right? [39:11.000 --> 39:17.000] So Kubernetes, you can use whatever you want with the Postgres. [39:17.000 --> 39:20.000] If you want to use Orm, please do. [39:20.000 --> 39:24.000] But remember, you're responsible for that. [39:24.000 --> 39:26.000] Use it wisely. [39:26.000 --> 39:35.000] Otherwise, you can use LITPQ package or new PGX. [39:35.000 --> 39:41.000] And you can use whatever GitHub, GitHub, Bitbucket you want. [39:41.000 --> 39:49.000] And the most amazing thing about Go is the backward compatibility. [39:49.000 --> 39:54.000] You can build whatever in the future. [39:54.000 --> 39:55.000] Yeah, in the future. [39:55.000 --> 39:56.000] Whatever code you want. [39:56.000 --> 40:06.000] And it still will be compatible with the oldest something written ages before. [40:06.000 --> 40:13.000] Even now, when we have generics, when we have a cool thing is in Go, [40:13.000 --> 40:18.000] they are still compatible with that old things. [40:18.000 --> 40:27.000] Okay, so yeah, don't be stranger, check my GitHub account, [40:27.000 --> 40:31.000] check our blog. [40:31.000 --> 40:36.000] Yeah, some of our projects. [40:36.000 --> 40:40.000] And if you have a question, or maybe you have a question to Devrim [40:40.000 --> 40:43.000] why he hates Go. [40:43.000 --> 40:45.000] No, I don't hate Go. [40:45.000 --> 40:48.000] It says I don't package. [40:48.000 --> 40:50.000] I'm not sure what the problem is. [40:50.000 --> 40:52.000] I provide everything you need. [40:52.000 --> 40:56.000] Take my binders and put them in your packaging, whatever. [40:56.000 --> 40:59.000] These conditions don't like to be in Go, [40:59.000 --> 41:04.000] because either it should be a binary or the long internet. [41:04.000 --> 41:06.000] Not internet. [41:06.000 --> 41:09.000] No, no, not internet. [41:09.000 --> 41:14.000] There are only four dependencies for a new created application [41:14.000 --> 41:16.000] to connect to the possible scale. [41:16.000 --> 41:20.000] There are only one direct dependency is PGX, [41:20.000 --> 41:23.000] and there is only four indirect dependency. [41:23.000 --> 41:27.000] Two of them are libraries from Google. [41:27.000 --> 41:32.000] One of them library from KOROS. [41:32.000 --> 41:34.000] I don't remember. [41:34.000 --> 41:38.000] And one of them is by the same author, [41:38.000 --> 41:44.000] because he's using this package in other way. [41:44.000 --> 41:49.000] Yeah, so, questions? [41:59.000 --> 42:00.000] Hi. [42:00.000 --> 42:01.000] A couple of slides back. [42:01.000 --> 42:05.000] You went to a certain functionality where you were asking, [42:05.000 --> 42:08.000] do you do select, and then you put in a structure [42:08.000 --> 42:11.000] in order to get the results from? [42:11.000 --> 42:13.000] Yeah, couple of back. [42:13.000 --> 42:16.000] I have a question on that one. [42:16.000 --> 42:17.000] Yeah, that's one. [42:17.000 --> 42:18.000] Yeah, that's one. [42:18.000 --> 42:19.000] Exactly. [42:19.000 --> 42:22.000] So you're asking a select star, and then some stuff, [42:22.000 --> 42:24.000] and then you ask it by the rows. [42:24.000 --> 42:28.000] Does the actual connection, so if you're running this [42:28.000 --> 42:31.000] on one machine, you're running Postgres on another machine, [42:31.000 --> 42:33.000] there is a network, of course, in between. [42:33.000 --> 42:35.000] There's data going back and forth. [42:35.000 --> 42:38.000] Is all the data being sent from Postgres SQL server [42:38.000 --> 42:40.000] to your Go program or not? [42:40.000 --> 42:42.000] The reason I'm asking this is because, let's say, [42:42.000 --> 42:44.000] there's a fourth field, which is, let's say, [42:44.000 --> 42:47.000] a huge JSON field or a huge binary field, [42:47.000 --> 42:50.000] which contains like a megabyte of data per record. [42:50.000 --> 42:52.000] Is it then sending, if I'm asking 100 records, [42:52.000 --> 42:55.000] a thousand records, is it sending a gigabyte over the network, [42:55.000 --> 42:57.000] or is it only sending those three fields? [42:57.000 --> 42:58.000] Okay, I got you. [42:58.000 --> 43:00.000] Yeah, thank you. [43:00.000 --> 43:01.000] Very good question. [43:01.000 --> 43:06.000] So in this particular case, there shouldn't be star, [43:06.000 --> 43:08.000] first of all. [43:08.000 --> 43:11.000] We should always list columns we want to have. [43:11.000 --> 43:16.000] I'm just too lazy, and this is not my code. [43:16.000 --> 43:21.000] But it's like, you know, it shows. [43:21.000 --> 43:22.000] Yeah. [43:22.000 --> 43:25.000] So in this case, yes, everything. [43:25.000 --> 43:35.000] Can you do it automatically? [43:35.000 --> 43:43.000] Can we add columns from a database automatically? [43:43.000 --> 43:44.000] Yes, we can. [43:44.000 --> 43:53.000] But for that, we should use SQLC package, [43:53.000 --> 43:55.000] which is exactly for that. [43:55.000 --> 44:00.000] So it is pre-built or hooks or whatever. [44:00.000 --> 44:06.000] So when you say, go build, you have special SQL files. [44:06.000 --> 44:10.000] Like, I want this field from that and that, [44:10.000 --> 44:14.000] and it will go through it, and it will build automatically [44:14.000 --> 44:19.000] the appropriate structures for go, and then you can use it. [44:19.000 --> 44:21.000] Yeah, it's just a lot of information. [44:21.000 --> 44:25.000] I cannot, like, put it into the one talk. [44:25.000 --> 44:26.000] Yeah, but yeah. [44:26.000 --> 44:32.000] About if we are loading at once, yes, we do in this case. [44:32.000 --> 44:40.000] But the Postgres protocol itself supports row by row functionality, [44:40.000 --> 44:45.000] and it's possible to use that functionality with this package. [44:45.000 --> 44:53.000] So you can, like, yeah, say that. [44:53.000 --> 44:54.000] Hello. [44:54.000 --> 44:55.000] Thank you for the talk. [44:55.000 --> 44:58.000] I have one question about this driver. [44:58.000 --> 45:02.000] Actually, it's kind of only way to work with Postgres, [45:02.000 --> 45:03.000] for my opinion. [45:03.000 --> 45:08.000] And I'm a little bit worried that this driver is not supported [45:08.000 --> 45:11.000] by Postgres community, let's say. [45:11.000 --> 45:13.000] It's supported by someone. [45:13.000 --> 45:17.000] And what is the life cycle of this software? [45:17.000 --> 45:19.000] And maybe it will die. [45:19.000 --> 45:22.000] You say, how is about new features to it, [45:22.000 --> 45:25.000] and all this question arises when you work with it, [45:25.000 --> 45:29.000] because if your management says, okay, let's use Java, [45:29.000 --> 45:32.000] and in Java it's kind of stable, this Postgres driver, [45:32.000 --> 45:35.000] and you know that you always have the new version. [45:35.000 --> 45:37.000] What is about this software? [45:37.000 --> 45:38.000] Yeah, thank you for the question. [45:38.000 --> 45:43.000] So versioning and owner, who is owner, and that kind of stuff. [45:43.000 --> 45:47.000] So as a Postgres community, we support on the C library, [45:47.000 --> 45:55.000] which is live PQ, and Java, which is JDBC, right? [45:55.000 --> 45:56.000] That's all. [45:56.000 --> 45:59.000] All others? [45:59.000 --> 46:00.000] Yeah. [46:00.000 --> 46:03.000] Who uses C++? [46:03.000 --> 46:08.000] All other libraries are maintained by someone. [46:08.000 --> 46:13.000] By the way, live PQ is not a standard library [46:13.000 --> 46:15.000] in terms of made by go. [46:15.000 --> 46:19.000] It's also maintained by one person. [46:19.000 --> 46:24.000] So how we do in this case, we just fork it and use it [46:24.000 --> 46:26.000] if you want something new. [46:26.000 --> 46:30.000] If I did everything better than the maintainer, [46:30.000 --> 46:36.000] the owner of the PGX will accept my progress proposals [46:36.000 --> 46:38.000] and we are stinked, right? [46:38.000 --> 46:50.000] If not, I will beat them and I will be popular. [46:50.000 --> 46:54.000] I have a question regarding testing strategies in CI CD. [46:54.000 --> 46:58.000] So you have shown that it's possible to mock Postgres scale, right? [46:58.000 --> 47:01.000] But sometimes you are relying on some feature of Postgres scale, [47:01.000 --> 47:04.000] or possibly you are relying on some extension Postgres. [47:04.000 --> 47:06.000] Do you want to test with a real Postgres scale? [47:06.000 --> 47:07.000] Yes. [47:07.000 --> 47:09.000] What you can recommend, so I have seen in your CI CD, [47:09.000 --> 47:12.000] you are executing some peer scale comments. [47:12.000 --> 47:15.000] Do you have a dedicated instance for running tests? [47:15.000 --> 47:16.000] No. [47:16.000 --> 47:23.000] So my GitHub actions are using pre-installed Postgres QL [47:23.000 --> 47:25.000] on the GitHub workers. [47:25.000 --> 47:34.000] They already have Postgres 15.1 probably nowadays. [47:34.000 --> 47:38.000] So I'm okay if they are behind several versions. [47:38.000 --> 47:43.000] I don't need to test for a specific feature or bug or whatever. [47:43.000 --> 47:46.000] But if I want to, in my GitHub action, [47:46.000 --> 47:54.000] I may specify the Docker image against which I want to test. [47:54.000 --> 47:58.000] So for example, if I want to test the latest master [47:58.000 --> 48:01.000] from the Postgres QL community, [48:01.000 --> 48:07.000] I will build my own image docker and will run my test against it. [48:07.000 --> 48:15.000] And I'm fine. [48:15.000 --> 48:27.000] Okay, thank you. [48:27.000 --> 48:45.000] Excellent from the back. Yes.