Continuous Integration : Integrating BuildKite with Your Scala Project

In a very broad sense, BuildKite (earlier called BuildBox) is a continuous integration server that allows you to keep working on your code while there is a CI box which is reporting about any issues. The problem with most web based CI servers is that they have to do a lot of magic beneath the hood to let you build your custom system on their environment.

What does this mean? This means that if your product uses the Riak database and mine uses PostgreSQL then a web based CI system would have to give me default installations of both of them so that both our products can be supported. Remember this is just 2 databases that we talked about. Now, bring in more databases, more external integration’s and to cap it, different versions of all of these for different products. Suddenly having a hosted CI is not really the space you want to be 🙂

Contrary to this, BuildKite seems to have pushed down these problems back to the development team. Ok, this is not necessarily as bad as it sounds. What this means is that now you have control over your build environment. You know exactly what instances and what versions you would be working with.

Conceptually, it is quite similar to any other CI environment that you would have worked with.
You have a Project on which you want to run CI. This could be your GitHub, Bitbucket or any other repository project. You need to specify a file which would get executed when the CI would trigger. For example in our case the file is build.sh It is IMPORTANT to note that this should be an executable script and buildkite should have rights to execute this script.

Selection_011
Now comes the interesting part. Which is the Agents. Agent is responsible for running the CI job on your CI server. The agent polls Buildkite looking for work. When a new job is ready to run, the agent will run the bootstrap.sh script with all the environment variables required to run the job.
This script is responsible for creating the build directory, cloning the repo, running the build script, and uploading artifacts.

Agent would do the following for any repository that it works with as a part of setting up the environment.

Once it has downloaded the latest code onto your CI box then it would run the script that you have mentioned. In our case, it ends up being src/main/resources/build.sh

For example, refer to the diagram below. We have 2 projects and 2 CI servers. On one CI server we have the agent running for Project 1 and on the other, we have the agent running for Project 2. Now when BuildKite gets the trigger for building then it invokes the running agent on either of the CI servers as necessary.
Selection_012

What I like?

  1. Controlled environment – We can set up our environment
  2. Ease of use
  3. Has Bells and whistles like build passing badges etc

What I dont like?

  1. It should be free for public projects on Github

Our public project which has been integrated with BuildBox is present here.

Written by 

Vikas is the CEO and Co-Founder of Knoldus Inc. Knoldus does niche Reactive and Big Data product development on Scala, Spark, and Functional Java. Knoldus has a strong focus on software craftsmanship which ensures high-quality software development. It partners with the best in the industry like Lightbend (Scala Ecosystem), Databricks (Spark Ecosystem), Confluent (Kafka) and Datastax (Cassandra). Vikas has been working in the cutting edge tech industry for 20+ years. He was an ardent fan of Java with multiple high load enterprise systems to boast of till he met Scala. His current passions include utilizing the power of Scala, Akka and Play to make Reactive and Big Data systems for niche startups and enterprises who would like to change the way software is developed. To know more, send a mail to hello@knoldus.com or visit www.knoldus.com

2 thoughts on “Continuous Integration : Integrating BuildKite with Your Scala Project

Leave a Reply

%d bloggers like this: