[00:00.000 --> 00:11.720]  First thing first, the dagger team was kind enough to send me some stickers.
[00:11.720 --> 00:13.640]  This is really the reason why I go to conferences.
[00:13.640 --> 00:17.360]  So if you want to pick some up, I will leave it somewhere here.
[00:17.360 --> 00:20.640]  Okay, cool.
[00:20.640 --> 00:23.960]  So, good evening, everyone.
[00:23.960 --> 00:27.120]  Thank you for joining my presentation today.
[00:27.120 --> 00:33.360]  My name is Mark, and for the last couple years or maybe for the better part of the decade,
[00:33.360 --> 00:37.920]  I've been helping engineering teams focus on their business, building their business
[00:37.920 --> 00:44.400]  applications instead of worrying about things like deployments or CI CD or stuff like that.
[00:44.400 --> 00:50.520]  And I'm currently, my current title at Cisco is Tech Lead, but by the way, is there anyone
[00:50.520 --> 00:55.200]  here who saw my presentation in the morning in the go-to room?
[00:55.200 --> 00:56.200]  Okay.
[00:56.200 --> 01:00.840]  Okay, so I'm going to make it up to you guys.
[01:00.840 --> 01:05.640]  So, and please laugh at my jokes as well again, even though you already heard them.
[01:05.640 --> 01:11.680]  So I decided that I would come clean here today, that this is a completely fake title, and
[01:11.680 --> 01:16.840]  that in fact my real job is a YAML engineer.
[01:16.840 --> 01:21.880]  So anyone else want to come clean, unburden themselves?
[01:21.880 --> 01:24.000]  Okay, okay, cool.
[01:24.000 --> 01:30.760]  Oh, yeah, yeah, that's engineer probably, that's an overstatement.
[01:30.760 --> 01:37.320]  So let's talk about CI CD, and we do have a bunch of CI CD services available today,
[01:37.320 --> 01:43.840]  and it's still evolving continuously, but we do have a couple of challenges that causes
[01:43.840 --> 01:50.200]  pain to developers and others, people every day, and I've already kind of hinted at one
[01:50.200 --> 01:58.440]  of them, YAML, like, man, you put a space in the wrong place and it just breaks over,
[01:58.440 --> 02:03.120]  and you don't even know why, because the CI tool may not even tell you where that extra
[02:03.120 --> 02:04.920]  space is.
[02:04.920 --> 02:10.880]  So YAML is really one of the core pains of all the CI solutions today, and yeah, I know
[02:10.880 --> 02:16.880]  there is Jenkins and Groovy, which is inverse, but YAML is really the standard of CI CD languages
[02:16.880 --> 02:17.880]  these days.
[02:17.880 --> 02:22.480]  There are some places, like, there are a couple of solutions where it works kind of okay for
[02:22.480 --> 02:29.960]  simple pipelines, but for more complex cases it's just a nightmare.
[02:29.960 --> 02:38.960]  Then CI has this tendency to break for no obvious reason, like, one day the pipeline
[02:38.960 --> 02:46.920]  works and the other way it just doesn't, and, well, for operations, for deploying your application
[02:46.920 --> 02:52.480]  to a production environment, you can always say, okay, OAPS problem, let them solve, but
[02:52.480 --> 02:58.680]  for CI that's not really a case, like, developers have to interface with interactive CI, and
[02:58.680 --> 03:04.440]  if the CI is breaking, then it's probably the developers who have to fix it.
[03:04.440 --> 03:09.200]  And the problem with current CI solutions today is that we don't really have, like,
[03:09.200 --> 03:15.120]  an easy way to debug CI issues, like, if there is something wrong, you probably have to guess
[03:15.120 --> 03:20.720]  where the problem is, maybe add a few echo lines to the YAML file, push the whole thing
[03:20.720 --> 03:26.840]  to the repository, wait for the CI to get triggered by the repository, and then go through
[03:26.840 --> 03:35.040]  this whole long feedback loop over and over again, you see a lot of people nodding.
[03:35.040 --> 03:41.280]  So when something goes wrong, it's your job to fix it, and it takes a lot of time, and
[03:41.280 --> 03:43.120]  it's just painful.
[03:43.120 --> 03:51.880]  And sometimes, sometimes, it's actually not the CI's fault, but your fault.
[03:51.880 --> 03:59.800]  The code doesn't, or the test doesn't pass, the linter doesn't pass, and that's often
[03:59.800 --> 04:05.520]  caused by things like having different versions in the CI and different versions in your development
[04:05.520 --> 04:12.280]  environment, and there are tools, and there are, like, ways to make those as close to
[04:12.280 --> 04:18.320]  each other as possible, but still this is happening very often, like, I don't know, I had this
[04:18.320 --> 04:21.000]  problem like a week ago.
[04:21.000 --> 04:26.920]  So sometimes it's just your code that's not working with the CI, and you have to go through
[04:26.920 --> 04:33.120]  the same feedback loop trying to push a change, hoping that it will fix your problem, and
[04:33.120 --> 04:39.280]  of course, it doesn't work for the first time, so you do it over and over again, until after
[04:39.280 --> 04:46.880]  an hour, maybe, maybe if you sacrifice something to the CI gods, it works.
[04:46.880 --> 04:53.640]  So I'm pretty sure there are other challenges with CI, but let's see how Dagger may be able
[04:53.640 --> 04:56.120]  to solve some of these challenges here.
[04:56.120 --> 04:59.200]  So first of all, who has heard about Dagger?
[04:59.200 --> 05:02.960]  Who knows what Dagger is, oh, cool.
[05:02.960 --> 05:08.960]  So Dagger is a program about portable CI-CD solution, and portable is a pretty great
[05:08.960 --> 05:14.120]  feature here, because instead of going through the total feedback loop I was talking about,
[05:14.120 --> 05:20.040]  you can run your CI-CD pipeline on your own local machine and figure out what's wrong
[05:20.040 --> 05:25.360]  much sooner than by pushing to the Git repository and waiting for the CI over and over again.
[05:25.360 --> 05:32.080]  So it's much quicker that way to debug issues either related to the CI or your code, and
[05:32.080 --> 05:35.840]  it's also much easier to build the pipeline in the first place, like when you build a
[05:35.840 --> 05:40.800]  new CI pipeline for a new project, you have to go through the same feedback loop, because
[05:40.800 --> 05:46.000]  you have to add new steps and figure out if it works or not, and if it doesn't, then you
[05:46.000 --> 05:48.640]  have to figure out how to add the right parameters.
[05:48.640 --> 05:55.040]  So even building new pipelines is way easier, because the whole thing is portable.
[05:55.040 --> 06:00.600]  The other thing that makes Dagger great is that you can basically write your pipelines
[06:00.600 --> 06:03.280]  in any language.
[06:03.280 --> 06:10.640]  Dagger officially supports a couple of languages, like Go, Python, TypeScript, and Q, but basically
[06:10.640 --> 06:16.240]  any language that can talk to a GraphQL API, because that's what's under the hood, any
[06:16.240 --> 06:22.080]  language that can talk to a GraphQL API basically can be used to build your own pipelines with
[06:22.080 --> 06:23.560]  Dagger.
[06:23.560 --> 06:28.680]  And if you combine these two traits, like being portable and being able to write pipelines
[06:28.680 --> 06:33.440]  in your own language, it also points to the fact that you can completely avoid vendor
[06:33.440 --> 06:39.360]  locking, like you can, obviously you would still need some sort of CI service and you
[06:39.360 --> 06:45.120]  would need like a thin layer of integration that would run Dagger itself, but once you
[06:45.120 --> 06:50.960]  have a portable pipeline written in your own format in your own language, not in a proprietary
[06:50.960 --> 06:57.280]  or CI-specific general format, you are not logged into the CI vendor you are using right
[06:57.280 --> 06:58.280]  now.
[06:58.280 --> 07:04.760]  And that you don't really switch like CI providers often, but that happens, like when they buy
[07:04.760 --> 07:08.520]  your company and then you have to move from one provider to another and then you have
[07:08.520 --> 07:13.400]  to move again because reasons.
[07:13.400 --> 07:20.040]  And the fourth reason or the fourth thing that makes Dagger great is caching.
[07:20.040 --> 07:27.160]  Now, most CI services already have some sort of caching solution that you can use to cache
[07:27.160 --> 07:32.560]  like intermediary artifacts or dependencies or whatever you want to store in a cache that
[07:32.560 --> 07:37.760]  you don't want to download or compile every single time when your CI pipeline runs.
[07:37.760 --> 07:41.800]  But you still have to configure it properly and you have to make sure that you have the
[07:41.800 --> 07:47.400]  right caching keys, you have to add the right paths to the caching configuration.
[07:47.400 --> 07:52.440]  And you may either end up with a huge cache at the end of the day or you may not use cache
[07:52.440 --> 07:53.440]  at all.
[07:53.440 --> 07:56.360]  So if you don't configure it properly, it may not work.
[07:56.360 --> 07:59.960]  With Dagger, you get caching by default.
[07:59.960 --> 08:06.280]  And by default, I mean every single step in your pipeline, the result of that run will
[08:06.280 --> 08:11.880]  be cached similarly how a Docker file works, like every single instruction, the result
[08:11.880 --> 08:19.080]  of that instruction will be cached if there are no changes before that step, actually.
[08:19.080 --> 08:26.640]  So similarly to that, Dagger caches every step in your CI pipeline.
[08:26.640 --> 08:27.640]  Now how does it do that?
[08:27.640 --> 08:28.640]  How does it work?
[08:28.640 --> 08:29.640]  How is this portable?
[08:29.640 --> 08:30.640]  Any guesses?
[08:30.640 --> 08:31.640]  One word?
[08:31.640 --> 08:36.080]  Yeah, Docker, yeah, that's right.
[08:36.080 --> 08:38.240]  So containers, of course, containers.
[08:38.240 --> 08:43.600]  So in order to be portable and to do all this magic that Dagger does, it needs to have like
[08:43.600 --> 08:49.280]  a reasonable level of isolation so that you can be confident that it will run on your local
[08:49.280 --> 08:51.160]  machine and on your CI the same way.
[08:51.160 --> 08:56.480]  So it runs your builds in containers.
[08:56.480 --> 09:02.240]  And I already mentioned that Dagger has a few official SDKs that you can use to build
[09:02.240 --> 09:05.280]  the pipeline in your own code.
[09:05.280 --> 09:11.360]  Using that Dagger SDK, you can talk to the so-called Dagger engine, which is the API
[09:11.360 --> 09:15.240]  that implements the GraphQL specification.
[09:15.240 --> 09:21.000]  And the Dagger SDK will call this API with the steps in your pipeline.
[09:21.000 --> 09:26.320]  And the Dagger engine will build a DAG from these steps.
[09:26.320 --> 09:31.000]  And then we'll pass that to basically to run through a container runtime.
[09:31.000 --> 09:35.080]  And that's how your pipeline will run.
[09:35.080 --> 09:39.720]  And the good thing about this is that you can actually change this pipeline so the output
[09:39.720 --> 09:42.720]  of one pipeline can be the input of another.
[09:42.720 --> 09:46.440]  And this whole thing goes through a single thing called session.
[09:46.440 --> 09:51.280]  So in a single session, you can have multiple, like, these container executions.
[09:51.280 --> 09:55.360]  And you can change the results into each other if you want to.
[09:55.360 --> 10:00.680]  Now let's actually take a look at how these things run.
[10:00.680 --> 10:04.880]  And the reason why I asked if there is anyone here who was in my presentation in the morning
[10:04.880 --> 10:08.920]  because I completely botched the demo and it didn't work at all.
[10:08.920 --> 10:12.880]  So let's hope it works this time.
[10:12.880 --> 10:15.120]  So the example is in Go.
[10:15.120 --> 10:21.640]  But again, it could be, like, TypeScript and Python or even Q. And I'm not going to go
[10:21.640 --> 10:25.080]  into that much detail about the Go specific here.
[10:25.080 --> 10:31.200]  But basically, you need to import this Dagger SDK in order to, by the way, can you see the
[10:31.200 --> 10:34.720]  screen or the code?
[10:34.720 --> 10:35.720]  Make it bigger?
[10:35.720 --> 10:36.720]  OK.
[10:36.720 --> 10:37.720]  Better?
[10:37.720 --> 10:38.880]  Cool.
[10:38.880 --> 10:42.720]  So you have to import this Dagger SDK if you want to use Go.
[10:42.720 --> 10:46.320]  And then the first thing you need to do is connect to the Dagger Engine.
[10:46.320 --> 10:53.000]  Now if the Dagger Engine doesn't run locally, then the SDK will actually run it using, as
[10:53.000 --> 10:55.320]  a simple Docker container.
[10:55.320 --> 11:00.960]  So the first thing you need to do is connect to this Dagger Engine.
[11:00.960 --> 11:04.960]  And then you can start launching these containers and start building your pipelines.
[11:04.960 --> 11:10.240]  And if it looks very similar, it's because it's basically the same, uses the same language
[11:10.240 --> 11:14.840]  and it looks very similar to Docker files.
[11:14.840 --> 11:18.640]  And it works basically the same way.
[11:18.640 --> 11:20.960]  So you have, like, a base image.
[11:20.960 --> 11:24.480]  You have a bunch of mounted volumes for caching.
[11:24.480 --> 11:30.280]  And then you mount the source code and you run some sort of command.
[11:30.280 --> 11:31.280]  And that's it.
[11:31.280 --> 11:32.280]  That's your pipeline.
[11:32.280 --> 11:33.760]  Now let's see if it actually runs.
[11:33.760 --> 11:39.880]  So I use this make file authority for Go called mage.
[11:39.880 --> 11:45.240]  So this is how I have this whole code implemented in a test function.
[11:45.240 --> 11:50.440]  Let's see if it runs.
[11:50.440 --> 11:55.200]  Okay.
[11:55.200 --> 11:57.200]  So it did run.
[11:57.200 --> 11:58.200]  Cool.
[11:58.200 --> 12:06.760]  Let's try just for, let's debug to see what's happening in the background.
[12:06.760 --> 12:12.200]  So it pulls an image, the goal and image.
[12:12.200 --> 12:13.200]  It mounts the code.
[12:13.200 --> 12:15.400]  It mounts the volumes.
[12:15.400 --> 12:22.360]  And then runs my Go test on the mounted code.
[12:22.360 --> 12:29.400]  And then basically exits and outputs the result of the test.
[12:29.400 --> 12:33.800]  So, well, that's it.
[12:33.800 --> 12:37.040]  If you want to get started with Dagger, check out the documentation.
[12:37.040 --> 12:38.720]  It's getting better by the day.
[12:38.720 --> 12:43.400]  They actually released, well, either today or yesterday, a new quick start guide, which
[12:43.400 --> 12:44.880]  is pretty awesome.
[12:44.880 --> 12:49.840]  It has all the three or four supported languages in a single document.
[12:49.840 --> 12:52.320]  So you can switch between languages if you want to.
[12:52.320 --> 12:55.240]  There is even a playground for the lower level GraphQL API.
[12:55.240 --> 12:59.680]  So if you don't want to start a new project, you can play directly with the GraphQL API
[12:59.680 --> 13:03.040]  with the hosted version of the Dagger engine.
[13:03.040 --> 13:04.160]  So thank you for your attention.
[13:04.160 --> 13:17.920]  If you have any questions, I'm happy to answer if you have time for that.
[13:17.920 --> 13:18.920]  Thank you so much.
[13:18.920 --> 13:19.920]  Awesome.
[13:19.920 --> 13:23.800]  I have a question with regards to implementation, so let's say that you roll your pipeline,
[13:23.800 --> 13:28.000]  you commit that you want to run that somewhere like in a CI environment.
[13:28.000 --> 13:32.160]  GitHub Actions, or Gaffer Bids, Jenkins, or whatever.
[13:32.160 --> 13:33.480]  How do you go about that?
[13:33.480 --> 13:38.480]  I can imagine that you need to expose the Docker socket to the pipeline, or how does
[13:38.480 --> 13:39.480]  it work?
[13:39.480 --> 13:43.440]  Yeah, so basically, if you have Docker running in your environment, you can run this pipeline.
[13:43.440 --> 13:46.160]  And you can run Docker anywhere, basically, today.
[13:46.160 --> 13:48.560]  You can run it in Jenkins or GitHub.
[13:48.560 --> 13:50.920]  You have it in GitHub Actions, actually.
[13:50.920 --> 13:53.440]  And you probably have it on your machine, as well.
[13:53.440 --> 13:58.120]  So wherever you have Docker running today, this pipeline will run.
[13:58.120 --> 14:02.720]  So you just invoke the Dagger command, that command that you just showed us.
[14:02.720 --> 14:04.560]  Yeah, it's not even a Dagger command.
[14:04.560 --> 14:05.800]  This is entirely my code.
[14:05.800 --> 14:07.800]  This is my go binary, basically.
[14:07.800 --> 14:08.800]  Right, okay.
[14:08.800 --> 14:17.680]  And it will find the Docker API socket, and if you just start containers there, yeah.
[14:17.680 --> 14:21.520]  Before I, very cool stuff.
[14:21.520 --> 14:28.600]  So before I switch all my CI to Dagger, let's frame it like this.
[14:28.600 --> 14:34.440]  What would be the two things that you would really love to see an improved implementation
[14:34.440 --> 14:37.800]  of in the next version?
[14:37.800 --> 14:38.960]  Can you repeat the question?
[14:38.960 --> 14:44.480]  What are the two things that really need to be improved about the current state of Dagger?
[14:44.480 --> 14:46.160]  What improves?
[14:46.160 --> 14:52.600]  To improve Dagger, in your opinion, what are the two things that need the most improvement?
[14:52.600 --> 14:56.240]  Okay, so one thing is secret management.
[14:56.240 --> 15:01.920]  Right now, Dagger, it's not that easy to work with secrets, so that needs to be improved,
[15:01.920 --> 15:04.080]  and they are actually working on it, so that's great.
[15:04.080 --> 15:08.360]  The other thing is that right now, if you build something in one language, for example,
[15:08.360 --> 15:15.480]  if I build a reusable library in Go to run my pipelines, I can't reuse it in TypeScript,
[15:15.480 --> 15:16.680]  for example, today.
[15:16.680 --> 15:23.320]  And for that, there is actually a feature called extensions, so they are working on a
[15:23.320 --> 15:28.600]  feature so you can build extensions to the Dagger engine, so you can build these reliable
[15:28.600 --> 15:35.000]  or reusable pipeline PCs, like running liters and stuff like that, so you don't have to
[15:35.000 --> 15:38.560]  build that in your own code, you just have to build the extensions, and you can call
[15:38.560 --> 15:41.160]  it from whatever language you want to call them.
[15:41.160 --> 15:43.760]  Basically GraphQL API extensions.
[15:43.760 --> 15:46.080]  Thank you.
[15:46.080 --> 15:51.480]  Last question.
[15:51.480 --> 16:00.440]  Hi, does Dagger support spinning up Service A concurrently with Service B, because the
[16:00.440 --> 16:04.320]  tests need something else to run while the test is running, and then afterward you can
[16:04.320 --> 16:06.000]  continue to other stuff?
[16:06.000 --> 16:07.880]  Right now, I don't think it does.
[16:07.880 --> 16:14.480]  Again, this is something that they are thinking about, but it's not a trivial thing to do,
[16:14.480 --> 16:15.480]  so no.
[16:15.480 --> 16:16.480]  Currently.
[16:16.480 --> 16:21.480]  Okay, someone is working on it.
[16:21.480 --> 16:22.480]  Thank you.
[16:22.480 --> 16:23.480]  Of course.
[16:23.480 --> 16:24.480]  Thank you.
[16:24.480 --> 16:39.440]  Thank you very much.