In our current project we were using managed service as our primary CICD solution. Since the last few months our managed service provider’s servers went down a few times which impacted our client’s business. So we decided to move to a self managed CICD solution. For our case Jenkins turns out to be the best tool to implement the same.
Requirements
- High availability
- Scalability
- Security
- Cost effective
High Availablity
To make out infrastructure highly available we have decided to deploy our master server in an auto scaling group in two different AZs. So even if our single master server deployed in one AZ goes down due to an AZ outage.
It will be re-spawned in another AZ. This will make our servers highly available but there is a problem i.e how we going retain the Jenkins master configuration in case of an scaling event?
To retain installed plugins , jobs and other jenkins configurations. We decided to mount the Jenkins home on AWS EFS with a lifecycle policy for 15 days backup retention with 12hours interval backup schedules.
Scalability
In order to scale our jenkins slave instances we have used the AWS EC2 plugin in jenkins. As a result we can scale our worker nodes as our job queue size grows.
Security
For providing a more secure connection we have implemented a free openVPN server on AWS for accessing our jenkins dashboad in AWS private network.
Cost Effective
We have used spot instances with maximum spot price as their current on demand price. This will let you utilise the resource with less chances of getting it reclaimed by AWS.
Setup guide
Create an AMI for your Jenkins Master
- Create EFS volume in 2 Availablity Zones.
- Launch an instance with an ubuntu 20 ami.
- Install Jenkins and other utilities using the below links on the created instance.
- https://www.digitalocean.com/community/tutorials/how-to-install-jenkins-on-ubuntu-20-04 (Make sure you have already installed the java before)
- Install aws cli and nfs utility
Create a Launch Configuration or a Launch template with Jenkins Master AMI.
- Use the below user-data and earlier created Jenkins Master AMI for Launch Configuration.
#cloud-config
runcmd:
- echo "<your efs file-system-id>.efs.<your aws region>.amazonaws.com:/ /var/lib/jenkins nfs4 nfsvers=4.1,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2 0 0" >> /etc/fstab
- mount -a -t nfs4
- chown jenkins:jenkins /var/lib/jenkins/
Create an Auto Scaling Group with above Launch Configuration or Template.
Once Auto Scaling Group successfully launches your Jenkins Master instance. You can use ALB to redirect your requests to your Jenkins dashboard and try accessing it.
