Core Building Blocks and its concepts in concourse

Concourse:

Concourse CI is a system built with a loosely coupled microservices architecture; a database node using PostgreSQL is the only state saved by the system. All build histories, stored pipelines, as well as user and group access privileges are stored here.

Concourse building blocks:

Resource:

Resources are fundamental building blocks of pipeline. Resource is an entity which stores data.Resources are the heart and soul of Concourse. They represent all external inputs to and outputs of jobs in the pipeline. These are external inputs and outputs of a job in a pipeline.

resources:
- name: demo  #This name will be referenced by build plans of jobs in the pipeline.
  type: git  #The resource type implementing the resource.
  source:    #This varies by resource type, and is a black box to Concourse; it is blindly passed to the 
    uri: https://github.com/concourse/examples.git   #resource at runtime.

Resource Type:

Each resource in a pipeline has a type . The resource’s type determines what versions are detected, the bits that are fetched when the resource’s get step runs and the side effect that occurs when the resource’s put step runs.

resource_types:
- name: rss    #The name of the resource type. This should be short and simple.
  type: registry-image   #The type of the resource used to provide the resource type's container image.
  source:   #The configuration for the resource type's resource. This varies by resource type, and is a 
    repository: ashidubey/concourse   #black box to Concourse; it is blindly passed to the resource at 
    tag: latest   #runtime.

Task:

Task is the smallest unit of work in pipeline and these tasks are put together to make up jobs. It is the execution of script in a container.Tasks are similar to a functions with inputs and outputs and focus on one specific thing. Tasks run in temporary containers. 

---
platform: linux   #The platform the task should run on. This determines the pool of workers that the task      #can run against.

image_resource:   #The container image to run with, as provided by an anonymous resource definition.
  type: registry-image
  source:
    repository: golang
    tag: '1.6'

params:    #A map of arbitrary configuration to forward to the resource. Refer to the resource type's 
  SOME_PARAM: some-default-value   #documentation to see what it supports.

run:    #The command to execute in the container.
  path: sh
  args:
  - -exc
  - |
    whoami
    env
    go version
    find .
    touch some-output/my-built-artifact

Job: 

The tasks make up a job. A job is connecting tasks and resources. You might have a task building a source code and that task is pulling resources of git type. Job is collection of tasks that make up a piece of work.

jobs:
  - name: pull-source               # The job is called pull source
    plan:
      - get: bold-shopify-toolkit  # This job will get bold-shopify-toolkit
        trigger: true               # Whenever the resource changes trigger 
                                    # this job. This setting is completely 
                                    # optional
      - task: list-directories.     # The job has one task to list all  
                                    # directories
        config:
          platform: linux           # The platform task should run on
          image_resource:    # Which container image the task should run in
            type: docker-image
            source: {repository: ubuntu}  # repository in required  
                                          # parameter and optional ones are
                                          # like tag and username/password     
                                          # are based on resource type
          outputs:
            - name: output-folder   # Outputs are artifacts produced by  
                                    # task which might be used in later 
                                    # steps by other tasks.
                                    # these are auomated generated 
                                    # directories which are passed as 
                                    # inputs to other tasks.
                                    # tasks run in container which go away 
                                    # after each run and the only way to 
                                    # pass the data between tasks is
                                    # through these inputs/outputs
          run:
            path: ls    # This run is the most important part of task which 
                        # can run command/script or
            args: [.]   # call to a file. The `path` can point to file or 
                        # bash command

Pipeline: 

Jobs form a pipeline. Jobs, resources and all components put together makes up a pipeline.

Build:

A build is an execution of a build plan, which is either

• configured as a sequence of steps in a job
• generated by the Resource Checker to run a check
• submitted directly to Concourse as a one-off build via fly execute

Difference between Resource and resouce Types:

In the term of the container:

A resource type is an image and we need to config the repository and tag in its source so that the concourse can locate/download it.

A resource is a container which is an instance of that image and can be used in the jobs when the pipeline is running. Its source that we configure is the common parameters which will be passed on the stdin to the check, in and out scripts when the resource is configured in a get or put step.
I think it’s a little similar to the comparison between the class and object.

Tasks run in ephemeral containers:

Concourse pipelines are container based. All the activities like pull source code, build code or run tasks are run in containers which are ephemeral i.e these containers go away after task is done. This means there are repeatable builds and there is nothing saved on the host machine.

Declarative pipelines:

There is no GUI for configuring Concourse. Instead, pipelines are defined as YAML files.

Basic example of a pipeline:

Let’s now put some core building blocks in a yml file to start building a pipeline.

Create file called pipeline.yml with following contents.

resources:
- name: demo
  type: git
  source:
    uri: https://github.com/concourse/examples.git

jobs:
- name: pipe
  plan:
  - get: demo
    trigger: true
  - task: task1
    config:
      platform: linux
      image_resource:
        type: registry-image
        source:
          repository: bash
      inputs:
      - name: demo
      run:
        path: cat
        args: ["demo/README.md"]  

Command to create a pipeline in concourse 

fly -t {your_target_name} set-pipeline -c pipeline.yml -p {your_pipeline_name}