Introducing the concept of Dagger in CI/CD

Table of contents

Reading Time: 5 minutes

Introduction

Dagger is an open-source devkit for CICD. It works using two open-source tools Cuelang and BuildKit from Docker. In addition, architecture is of the same concept as that of docker, Client-Server architecture. But, instead of docker, dagger daemon engine runs on any container runtime.

Prerequistes

A computer system with any OS with Internet.
Docker daemon installed on system

Installing Dagger

Here we will be exploring linux OS, you can visit this page to install dagger on other OS.

enable root user

knoldus@1708knoldus:~$ sudo su

move to /usr/local

( This step is used as primary linux search for binaries is done here only)

root@1708knoldus:/home/knoldus# cd /usr/local

Install dagger binary

root@1708knoldus:/usr/local# curl -L https://dl.dagger.io/dagger/install.sh | sh

Find current version and verify installation

root@1708knoldus:/usr/local# dagger version

Understanding the problem in CI/CD

At present, the IT industry is booming. This leads to innovation and scaling of the platform on multiple tech tools. Therefore a dream time for every Software developer. But this leads a problems for the Devops and SRE engineers. Why?

There is demand of same CI/CD efficiency in this complex environment as in a small infrastructure.
Variety of tools and scripts are to be stiched together with various pipelines.
Every deployment requires its own scripting and configuration.

Concept of Dagger CI/CD

Dagger environment consider pipelines as softwares and divides everything in building blocks. The Buildkit provides the containerization build benefit to dagger projects. Dagger does replace the CI pipeline but to its benefit.The has its own benefits as explained in this blog.

Dagger provides following benefits to CI/CD

Unification

The development and CI environment is unified. Before developer write code and configurations like shell scripts. And then again by Devops engineer write same for pipeline scripts and compose files. In Dagger, simply write a Dagger plan and use for both places. For example in this blog.

package daggerapp

import (
  "dagger.io/dagger"
  "universe.dagger.io/docker"
)

dagger.#Plan & {
  client: {
    env: {
      DH_CREDS_USR: string
      DH_CREDS_PSW: dagger.#Secret
    }
  }
  actions: {
    build: docker.#Build & {
      steps: [
        docker.#Pull & {
          source: "alpine"
        },
      ]      
    }
    push: docker.#Push & {
      auth: {
        username: client.env.DH_CREDS_USR
        secret: client.env.DH_CREDS_PSW
      }
      image: build.output
      dest: "vaibhavkuma779/jenkins-dagger:latest"
    }
  }
}

This dagger plan is used by developer as well as the CI pipeline script.

Localisation

Pipelines can be run locally and tested before deploying on a cloud platform. The containerization enables to create and run the deployment as container image on a local system. For example in this github template.

package app

import (
  "dagger.io/dagger"
  "universe.dagger.io/docker"
)

dagger.#Plan & {
  client: {
    env: {
      DH_CREDS_USR: string
      DH_CREDS_PSW: dagger.#Secret
    }

    filesystem: ".": read: contents: dagger.#FS
  }

  

  actions: versions: {

    "latest": _
    "3.14": _
    "3": _
    
    // The cue template type format used here for various versions
    [tag=string]:{
      build: docker.#Build & {
        steps: [
          docker.#Pull & {
            source: "alpine:\(tag)"
          },
          
          docker.#Copy & {
            contents: client.filesystem.".".read.contents
            source: "index.html"
            dest: "/var/www/html/"
          },

          docker.#Copy & {
            contents: client.filesystem.".".read.contents
            source: "default.conf"
            dest: "/etc/nginx/conf.d/"
          },

          docker.#Set & {
            config: {
              expose: ["80"]:{} 
              cmd: ["nginx","-g","daemon off;"]
            }
          },
        ]      
      }
      push: docker.#Push & {
        auth: {
          username: client.env.DH_CREDS_USR
          secret: client.env.DH_CREDS_PSW
        }
        
        image: build.output
        dest: "vaibhavkuma779/jenkins-dagger:\(tag)" // Provide your registry name here
   
      }
    }

    
  }
}

The action of build allows to build and run as docker images. And the image built can be run locally as well.

Reusability

Dagger works on concept of packages as in java, so boiler templates are provided by dagger team under Universe of actions in dagger repository. For example in the dagger script from this example.

package todoapp

import (
	"dagger.io/dagger"
	"dagger.io/dagger/core"
	"universe.dagger.io/alpine"
	"universe.dagger.io/bash"
	"universe.dagger.io/docker"
	"universe.dagger.io/netlify"
)

dagger.#Plan & {
	_nodeModulesMount: "/src/node_modules": {
		dest:     "/src/node_modules"
		type:     "cache"
		contents: core.#CacheDir & {
			id: "todoapp-modules-cache"
		}

	}
	client: {
		filesystem: {
			"./": read: {
				contents: dagger.#FS
				exclude: [
					"README.md",
					"_build",
					"todoapp.cue",
					"node_modules",
				]
			}
			"./_build": write: contents: actions.build.contents.output
		}
		env: {
			APP_NAME:      string
			NETLIFY_TEAM:  string
			NETLIFY_TOKEN: dagger.#Secret
		}
	}
	actions: {
		deps: docker.#Build & {
			steps: [
				alpine.#Build & {
					packages: {
						bash: {}
						yarn: {}
						git: {}
					}
				},
				docker.#Copy & {
					contents: client.filesystem."./".read.contents
					dest:     "/src"
				},
				bash.#Run & {
					workdir: "/src"
					mounts: {
						"/cache/yarn": {
							dest:     "/cache/yarn"
							type:     "cache"
							contents: core.#CacheDir & {
								id: "todoapp-yarn-cache"
							}
						}
						_nodeModulesMount
					}
					script: contents: #"""
						yarn config set cache-folder /cache/yarn
						yarn install
						"""#
				},
			]
		}

		test: bash.#Run & {
			input:   deps.output
			workdir: "/src"
			mounts:  _nodeModulesMount
			script: contents: #"""
				yarn run test
				"""#
		}

		build: {
			run: bash.#Run & {
				input:   test.output
				mounts:  _nodeModulesMount
				workdir: "/src"
				script: contents: #"""
					yarn run build
					"""#
			}

			contents: core.#Subdir & {
				input: run.output.rootfs
				path:  "/src/build"
			}
		}

		deploy: netlify.#Deploy & {
			contents: build.contents.output
			site:     client.env.APP_NAME
			token:    client.env.NETLIFY_TOKEN
			team:     client.env.NETLIFY_TEAM
		}
	}
}

The todo app is build on javascript and react. This dagger script simply uses the boilerplates of alpine, netlify. the localisation and reusaability both are shown with this example.

If you run “dagger do –help“, it shows 4 actions. for more information on actions read following blog.

these four commands are composite actions made with actions from universe packages. like when we run “dagger do build“, we get

Now just run “xdg-open _build/index.html“. You get the deployed output:

Multi platform compatiblity

Dagger is compatible for many automation servers like github actions, jenkins, circle ci, etc. youc refere to this page to learn more. This prevents vendor lock-in.

Conclusion

Dagger is a way to standardize current CI/CD development. This work on environment building with unifying of dev with devops, one step up of devops culture which was for development and operations unification. This is in beta version and developing with time. This blog was an approach to accustom people to dagger and its prospects.