Terraform: Enabling developer to create and manage deployment through code

In this blog post, We will walk you through Terraform which is a tool for building, changing, and versioning infrastructure safely and efficiently.

Terraform enables the developers to properly manage the infrastructure through code.

The set of files used to describe infrastructure in Terraform is simply known as a Terraform configuration. These files have extension .tf.

Configuration files describe to Terraform the components needed to run a single application or your entire datacenter. Terraform generates an execution plan describing what it will do to reach the desired state, and then executes it to build the described infrastructure. As the configuration changes, Terraform is able to determine what changed and create incremental execution plans which can be applied.

Here, we will show you how to build architecture on AWS, but Terraform can also manage other providers including Azure, DataDog, Bitbucket, DNS, Google Cloud and many more.

provider "aws" {
  access_key = "${var.access_key}"
  secret_key = "${var.secret_key}"
  region     = "${var.region}"

The provider block is used to configure the named provider, in our case “aws”. A provider is responsible for creating and managing resources.

Continue reading

Posted in Amazon, Architecture, AWS, Cloud, Devops, knoldus | Tagged , | 1 Comment

Introduction to Google Kubernetes

Over the past two years, Containerization has provided a lot of flexibility to the developer in which most popular container technology is docker. Container provide the developer a virtual environment for isolated process or application on the host system. Before everything else lets take a look what used to happen in past.

Earlier when the number of systems for running application were limited they were mostly specified by name so that user can easily identify which software or application is running on which machine. If the server dies everyone used to rush for the backup and admin  felt lucky if he got an up-to-date backup. Obviously, the same won’t work for scaling the application and the situation with this approach is not feasible when there are hundreds or thousands of servers. There should be an automation which takes full responsibility of allocating resources to the application on the specific machine, not only this  continuous monitoring and resilience is also required.

One of the main reason for deploying a service on the container is that they are flexible, lightweight, and easily scalable while deploying on hundreds of machine. Now, the question arises who is going to manage the containers on the large set of machines. that’s, why kubernetes and other container orchestration came into picture Lets, understand what it is. Continue reading

Posted in Devops, Scala | Leave a comment

Quick Start with Finagle

Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers. Finagle implements uniform client and server APIs for several protocols, and is designed for high performance and concurrency. Most of Finagle’s code is protocol agnostic, simplifying the implementation of new protocols.

Today here I am going to implement the Finagle example using Scala where I am sending the request with some message & get future of response using Finagle.

Continue reading

Posted in Scala, Web Services | Tagged , | Leave a comment

Java8 Bliss: Optional

Java8 is not so new now. Everybody is well aware and probably using it in one or the other project. Certainly, Java8 is a big leap for Java since it makes Java functional.
(though its procedural internally). Anyway, it successfully got interested in the IT industry.

In this blog, I am going to talk about one of the vital features of Java8 i.e. Optional. Optional, as the name suggests gives an option to use/operate on the value. Let’s dive deep into the world of Optional.
Optional is basically a container object which may or may not have the Not-Null value. It is used for representing a value. The main use of the Optional class is to avoid “NullPointerException“.

Optional is part of the java.util package, so we need to create this import:

import java.util.Optional;

How to create Optional Object?

There are several ways of creating Optional objects.
Empty Optional Object :

public void emptyOptional_thenCorrect() {
Optional  empty = Optional.empty();

Optional Object with Static value :

public void notNullOptional_thenCorrect() {
String name = "StaticValue";
Optional value = Optional.of(name);
assertEquals("Optional[StaticValue]", value.toString());}

Advantages of Optional in Java 8 :

• There is no need for Null checks.
• No more chances for NullPointerException at runtime.
• No boilerplate in the code (neat and clean code).

Continue reading

Posted in Scala | Leave a comment

Know a Knolder – Rishi Khandelwal!


Image | Posted on by | Leave a comment

KnolX: Understanding Spark Structured Streaming

Hello everyone,

Knoldus organized a session on 05th January 2018. The topic was “Understanding Spark Structured Streaming”. Many people attended and enjoyed the session. In this blog post, I am going to share the slides & video of the session.


Continue reading

Posted in apache spark, Scala, Spark, sql, Streaming | Tagged , , | Leave a comment

A WalkThrough with AWS Lambda


Serverless Architecture

Serverless is an application framework for building a serverless application without having to worry about managing infrastructures. It is based on the principle of third party service (BaaS) and on custom codes which run on a container(FaaS). Serverless architecture doesn’t mean we don’t have a server. With the serverless architecture, we still have a server to run our application, it’s just that we don’t need to manage it on our end. We can just focus on developing our application rest will be handled automatically.


The AWS Serverless Application Model (AWS SAM) is a model to define serverless applications. AWS SAM is natively supported by AWS CloudFormation and defines a simplified syntax for expressing serverless resources. The specification currently covers APIs, Lambda functions, and Amazon DynamoDB tables. AWS SAM uses Cloudformation template to deploy the lambda in a cloud formation stack. These templates are written in JSON or YAML file which states the step to execute in the cloud stack. So our main motive of this blog is to focus on lambda so the rest of the content will be on lambda functions.

AWS Lambda

AWS Lambda is a serverless framework so we can just create an application, make an artifact out of it and just upload it. AWS Lambda will handle the rest. It runs the application based on the language, scale it whenever required and the best part of it is that we are charged for only when we run a lambda function i.e NO CHARGE for not running it.

Continue reading

Posted in AWS, AWS Services, Java | Tagged , , | Leave a comment

Introduction to AWS DynamoDB

DynamoDB is a database service provided by Amazon. Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. You can use it to store and retrieve any amount of data without worrying about hardware provisioning, setup & configuration, replication, software patching, or cluster scaling. It mostly takes care of these things itself while also providing options to configure these things manually.

DynamoDB provides good replication and traffic handling. It distributes our data on multiple servers within the availability zones in an AWS region and stores the data on Solid State Disks (SSDs). These availability zones are linked to each other with fast and inexpensive connectivity while also being isolated from each other, meaning when one availability zone goes down for some reason, it won’t affect any other zone and thus our data will be highly available to us. Our data is spread across multiple servers which provides us with good traffic handling and handles our throughput and storage requirements.

Continue reading

Posted in AWS, AWS Services, database | Tagged , , | Leave a comment

Blockchain Nuggets – Distributed Systems

The first law of distribution is “Don’t Distribute“. If everything can be done on a single machine then nothing beats that. But, lets cut to reality.

Today’s computing and information systems are inherently distributed. Starting from large data centers to cloud storage to your mobile phone, everything is a part of the distributed infrastructure. Distribution, just like distributed development teams is here to stay because of

  • Geography – We don’t live in the same location
  • Parallelism – speed up computation
  • Reliability – replication, and redundancy to avoid loss
  • Availability – replication leads to faster access, fault tolerance, and low latency

Even though distributed systems lead to multiple benefits, there are a large number of issues that need to be tackled. Even though every single node of a distributed system will only fail once every few years, with millions of nodes, you can expect a failure every minute. Did not see that coming!

The basis of this failure is that the participants or their communication medium may experience failures.

  • Processors are prone to failures because, each participants processor would work at a different speed, may crash, may rejoin after the crash
  • The network might result in message loss, delays, lost, reordered, duplicated, there may be a network partition with n/2 nodes forming a separate network.

What are the issues that plague a distributed system?

I) A simple example, One client and One server

Screenshot from 2018-03-01 20-49-31

A simple scenario with just a single client and a server are fraught with challenges such as Message Loss. The client has no way to know if the message reached the server. Even if it reached the server, was it corrupted or in the original order. For the client to know that the message reached, the server can send out an ACK but what if the ACK is lost?

Continue reading

Posted in Architecture, Blockchain, Scala | Tagged , | Leave a comment

Being a Software Consultant: Knoldus way

I have been now working as a Consultant for 2 years and I wanted to document my insights and understanding towards the designation. I want to take you through the benefits and the sporadic hardships (luckily!) that have shaped my understanding of the work we do.

As a Software Consultant, our involvement in the decision making for the project has been immense, compared to what I hear from some outside world consultants who rant that their opinions and suggestions are not heard, even though they are called Consultants. Their work is more or less limited to development. Why? Because Knoldus puts us in front of the clients as people who are able to make sustainable decisions and not just merely as coders. That increases the potential of a person in decision making. It also increases the confidence and the connection with what we do. With that increased connection, our focus is on building a sustainable solution for a problem statement, which in turn ensures the longevity of the work.
This, in turn, churns out good faith from the client in us (also we get praises from them!) and they also hear our suggestions, concerns, opinions, and decisions. Finally, we achieve the goal of building the right product right!
Continue reading

Posted in knoldus | Tagged , | 1 Comment