How to setup Envconsul for dynamic configuration in your application(or services).


So, You want to integrate Envconsul into your application for dynamic configuration.Let’s see How it can be achieved and what all needs to be done.

Using Envconsul you can have the dynamic configuration in your services without taking them down i.e Services will never go down and you can make changes in your services(dynamic configuration).You can change things like the PORT number of any service, db password, access keys and IP address on runtime and You don’t have to worry about the service downtime.

Step1. The very first job would be to setup Consul. In consul, we will be dealing with Consul KV store to allow to define KV(Key Value) pairs that can be pulled into running services.Let’s see How we can setup Consul and then add KV pairs into Consul KV store.

1. First, install the consul from here https://www.consul.io/intro/getting-started/install.html
2. Run the Consul Agent using this command “consul agent -dev”
3. Then add KV pairs into Consul KV store using the following commands

consul kv put redis/config/minconns 1

//where redis/config is the folder and minconns is the key and 1 is the value for the key minconns.

For more information check here https://www.consul.io/intro/getting-started/kv.html

Now our Consul has been setup, Now You can check the KV pairs and running services at

http://localhost:8500

For KV pair visit

http://localhost:8500/ui/#/dc1/kv/

Step2. Setup Envconsul
Envconsul is a tool which is basically used to read and set environmental variables for processes from Consul.Envconsul provides a convenient way to populate values from Consul into a child process environment using the envconsul daemon.
The daemon envconsul allows applications to be configured with environment variables, without having knowledge about the existence of Consul. This makes it especially easy to configure applications throughout all your environments: development, testing, production, etc.

Now you can download the Envconsul from here https://releases.hashicorp.com/envconsul/

Once the envconsul is downloaded unzip it and put it in the /usr/local/bin. It is the same path where we put consul at.

Now both Consul and Envconsul are setup.

Step3. Access Environment variables set by envconsul into the application.
I am using Java.So, There are basically two ways in Java.

1.The java.lang.System.getenv(String name) method gets the value of the specified environment variable. An environment variable is a system-dependent external named value.

For Example
String value = System.getenv(“TEST”); //where TEST is the key and we are trying to fetch the value of key “TEST” over here.

2.Another way is to access the environment variables through application.conf.

In application.conf do the following
test = test
test = ${?TEST}//TEST is an environment variable.

After this you can access the test variable in your application like below

String testEnvVariable = ConfigFactory.load().getString("test");

Step4. Now finally we will run our application using envconsul like below

$envconsul -prefix=Test env mvn lagom:runAll

// Test is the folder where all the key value pairs reside and env is where all the environment variables reside and mvn lagom:runAll is the way to run my application

Now, You are done and everthing is in its place. To check everything works fine and we are able to make dynamic configuration, You can make changes to Consul’s KV store values and see if the updated values are reflected in your application while it is running,If it does you have achieved it cheers!
If the updated values do not reflect into the running application then you might have to go back to previous steps and ensure everything is correct and configured properly. That’s it!

If you find any challenge, Do let me know in the comments.
If you enjoyed this post, I’d be very grateful if you’d help it spread.Keep smiling, Keep coding! Cheers!

Posted in Scala | Leave a comment

Knolx – Intoduction to AWS IAM


Hi all,

Knoldus has organized a 30 min session on 3rd March 2017 at 4:50 PM. The topic was Introduction to AWS IAM. Many people have joined and enjoyed the session. I am going to share the slides here. Please let me know if you have any question related to linked slides.

 

 

Posted in Amazon, AWS, Cloud, Scala, Security, Security Controls | Leave a comment

Jenkins – Integrating Email Service


Jenkins is one open source tool to perform continuous integration and build automation. Using it, all development work can be integrated as early as possible. The resulting artifacts are automatically created and tested and as a result the process of identification of errors becomes faster. But there must be a way so that build status and testing results can be reported to the team. And then email service come into picture.

To dig further into the details of jenkins and building jobs in it, you can refer to Introduction to Jenkins and Building Jobs with Jenkins.

Now the next question comes i.e “ How email service can be integrated with Jenkins? ”

So I will be discussing two approaches to configure email service:

1) Default Email Notifier: This is what comes default with Jenkins. It contains a default message containing the Build number and its status.

2) Email Extension Plugin: This plugin allows you to configure every aspect of email notifications. You can customize when an email is sent, who should receive it, and what the email says.

Next step will be, what all configurations we require to use the above two.

Continue reading

Posted in testing, Tutorial | Tagged | Leave a comment

Web Designing using Pure.css


In today’s blog I will be introducing Pure.css (simply referred to as “Pure”), it’s use case and advantages over its counterparts. The blog will get you acquainted with the basics of the Pure. I shall discuss the basic idea behind its inception and what are its different components and finally how to integrate and implement it in web projects.

What is Pure.css?

Pure.css a set of responsive CSS modules build by YUI team.It is a small framework made up of CSS modules. All its modules put together weigh less than 3.8KB if served minified and gzip’d. One can save even more space(bytes) if we decide to use only one or two of the modules.

Pure’s minimal styling interference lets one ensure that one can customise the look and feel of web elements to meet the needs without much fuss.

Whose alternative is Pure.css?

Twitter Bootstrap, Zurb Foundation etc.

How is it Better?

There are several popular and widely accepted CSS frameworks available, but they are very heavy for they contain lots of CSS code.  Leaving that , one needs to customise the vanilla CSS code with personal styling making it even more stuffed.

The Pure fixes this  issue by making only requisite CSS modules required by almost every web projects. Pure allows writing of own style for customisation on top of it.

  • Pure is ridiculously tiny. The entire set of modules clocks in at 3.8KB minified and gzipped. That is pretty tiny compared to other CSS frameworks.
  • Unlike other popular CSS frameworks like Bootstrap, Pure does not come with any JavaScript Plugin out of the box. In fact, there only a few instance where it uses JavaScript, like drop down menus and fixed top menu.     

Continue reading

Posted in Play Framework, Scala | Tagged , , , | Leave a comment

Hello React: Creating a Single Page Application with React


Few days ago we were looking for a JavaScript library which is flexible and can be used in a variety of projects. Basically, something with which we can create new apps, or introduce into an existing project without rewriting it. However, we came across React, a JavaScript library for building user interfaces and AngularJS, a JavaScript MVW Framework.

Before we start creating a single page application with React we need to install NodeJS and NPM. After you have installed them, we are ready to create our first application with React.

Create React App is the best way to start building a new React single page application. So, we will be using it to create out first single page application with React.

sudo npm install -g create-react-app
create-react-app hello-world
cd hello-world
npm start

create-react-app will create a hello-world application for us with following directory structure.

hello-world/
  README.md
  node_modules/
  package.json
  .gitignore
  public/
    favicon.ico
    index.html
  src/
    App.css
    App.js
    App.test.js
    index.css
    index.js
    logo.svg

To see the application running open http://localhost:3000/. The page will reload if we make any edits in the app. Also, to run the tests just run command:

npm test

It will run the test watcher in an interactive mode.

react-blog-pic-2

At last to deploy the application, we can bundle it in production mode and optimize it for the best performance.

react-blog-pic-3

The build is minified and the filenames include the hashes. Finally, our app is ready to be deployed. Note that we have not described different components of the app yet. So, stay tuned !


KNOLDUS-advt-sticker

Posted in JavaScript, Node.js, React, ReactJS | Tagged , | Leave a comment

Reading data from different sources using Spark 2.1


Hi all,

In this blog, we’ll be discussing on fetching data from different sources like csv, json, text and parquet files.

So first of all let’s discuss what’s new in Spark 2.1. In previous versions of Spark, you had to create a SparkConf and SparkContext to interact with Spark whereas in Spark 2.1 the same effects can be achieved through SparkSession, without explicitly creating the SparkConf, SparkContext or SQLContext, as they’re encapsulated within the SparkSession.

SparkSession provides a single point of entry to interact with underlying Spark functionality and allows programming Spark with DataFrame and Dataset APIs.

To create an sbt Spark2.1 project you need to add the following dependencies in your build.sbt file

libraryDependencies += "org.apache.spark" % "spark-core_2.11" % "2.1.0"
libraryDependencies += "org.apache.spark" % "spark-sql_2.11" % "2.1.0"

and your scala version should be 2.11.x . For this project I am using:

scalaVersion := "2.11.8"

To initialise the spark session you can use :

val spark = SparkSession
  .builder()
  .master("local")
  .appName("ReadDataFromCsv")
  .getOrCreate()

Reading data from a csv file

val spark = SparkSession
  .builder()
  .master("local")
  .appName("ReadDataFromCsv")
  .getOrCreate()

val df = spark.read.csv("./src/main/resources/testData.csv")
  //To display dataframe data
  df.show()

Reading data from a Json file

val spark = SparkSession
  .builder()
  .master("local")
  .appName("ReadDataFromJson")
  .getOrCreate()

val df = spark.read.json("./src/main/resources/testJson.json") 
//To display dataframe data 
df.show()

Reading data from a text file

val spark = SparkSession
  .builder()
  .master("local")
  .appName("ReadDataFromTextFile")
  .getOrCreate()

val df = spark.read.text("./src/main/resources/textFile.txt") 
//To display dataframe data 
df.show()

Reading data from a parquet file

val spark = SparkSession
  .builder()
  .master("local")
  .appName("ReadDataFromParquet")
  .getOrCreate()

val df = spark.read.parquet("./src/main/resources/testJson.parquet") 
//To display dataframe data 
df.show()

You can also create a temporary table from a dataframe and perform sql queries on it using the following code:

df.registerTempTable("tempTable")
spark.sqlContext.sql("select * from tempTable").show

Here is the link for a demo project on Spark2.1: ReadingDataUsingSpark2.1

Happy coding !!

Posted in apache spark, sbt, Scala, Spark | Leave a comment

Automation Testing of AngularJS Application with Protractor


Protractor is testing framework for AngularJS application .It play the important role to check the angular application.Its a combination of selenium webdriver,nodejs,Jasmine, Cucumber.

Now a time most of the application are using javascript so due to using of javascript it increases tester  difficulty to check the web application.So basically when we use the selenium its difficult to us to identify the web element.

Protractor is nodejs program which is written on javascript so it can easily identify web element in angular js application.Angular js application extends the HTML attribute and some extra attribute like ng-model,ng-repeater etc which is not include in selenium webdriver locators. so selenium is not able to identify these locators thats why we are use the Protractor for angularJs application.

So for the Automation of angular application we need the following things.

1- Selenium web-driver.

2-node js.

Installation:

For Protractor Installation we type some commands.

sudo npm install –g protractor

It will install the protractor in your system and we can check the protractor version.

$ protractor –version

After that we will install web-driver manager by following command.

sudo npm install -g webdriver-manager

We just update the webdriver manager using following command.

webdriver-manager update

Then we will start the web driver manager.

webdriver-manager start

webdriver-start

If we type the url like http://localhost:4444/wd/hub/static/resource/hub.html  

it will give you the session page.

protractor-session

So for running the angular js application we need two file one is configuration file which is describe which file will be run and selenium server address and other file is main file where we describe locators which will be interact with the application.Protractor by default using chrome driver for automation.

So we will make two file conf file and filename.js

conf.js

exports.config = {
seleniumAddress: ‘http://localhost:4444/wd/hub’,
specs: [‘filename.js’]
};

 

filename.js

describe(‘angularjs homepage todo list’, function() {
it(‘should add a angular’, function() {
browser.get(‘https://angularjs.org’);
element(by.model(‘todoList.todoText’)).sendKeys(‘please fill you first test’);
element(by.css(‘[value=”add”]’)).click();
expect(browser.getTitle()).toContain(‘AngularJS’);
});
});

Now we run the application.

$ protractor conf.js

It will open the chrome browser and run the application also show the result in terminal.

protractor-result

Fell free to ask me any question regarding automation testing.

Thanks.

Posted in Scala | Tagged , , , , , , , , , , , , , | 1 Comment

Appium-Mobile App Automation[Test an app]Part-2


In our last blog we describe Installation and configuration of Appium on Ubuntu machine.Now we explain how to run automation test case on android device using appium.

So for test the app we have to put the app apk in android device which we want to test.After that we start the writing the code using selenium webdriver.We are using TestNg famework with selenium webdriver.

So when we run the program, appium server automatically start the mapping with emulator and run the script on emulator.

appiumcode

appium-server-start

Here is the result of automation test case.

report

Feel free to ask me any question.

Thanks.

Posted in Scala | Tagged , , , , , , , , | Leave a comment

Appium-Mobile App Automation[configuration and Installation]Part-1


Appium is open source test automation framework which is used for testing the hybrid app,native app and mobile web app on Android and iOS using webdriver.

Appium work on distributed open source mobile UI testing framework. It run the automation test on real devices, emulator, simulator. Appium translate the selenium web-driver command into UIautomation like android device. Appium support lot of language like java,php,ruby,python,JavaScript,node.js etc.

How to use Appium:

For using the Appium we have to need follwing things…

1-Android sdk

2-java

3-TestNg

4-Appium for Ubuntu

5-Selenium jar

6-Apk app

So we start from Android sdk. First of all download the android sdk. After downloading the sdk ,run the command in terminal

$android

it will download the all the android API whatever you used after that we enter emulator open command. It will open the emulator.

android-avd

emulatorstart

We start the emulator which work on android api 5.0.1

android-emulaoropen

After that we will install the Appium on Ubuntu machine.

> brew install node      # get node.js
> npm install -g appium  # get appium
> npm install wd         # get appium client
> appium &               # start appium
> node your-appium-test.js

After installation the appium we start the appium server.

appium

we  can see our connected device listed in Command Prompt type adb devices on terminal

adbdevice

Appium does not support android version lower than 4.2 So if you want to test the lower version of android you can use Selendroid.

If you have any query feel free to ask me.

Thanks.

Posted in Scala | Tagged , , , , | 2 Comments

Introduction to Scala Parser And Combinators


Scala parser combinators are a Powerful way to build parsers that can be used in everyday programs. But it’s hard to understand the plumbing pieces and how to get started. After you get the first couple of samples to compile and work, the plumbing starts to make sense,this is very easy way to design your own programming language, using its parser library

so lets get started first thing is that we need to extend the regex parser

to use scala parser combinators you need to add following dependencies in your build.sbt

libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.5"

sbt version is 2.11.3

EBNF grammar for this language would look something like this:

def symbol: Parser[Any] = "+" | "-" | "*"

def number: Parser[Int] = """(0|[1-9]\d*)""".r ^^ { _.toInt }

symbol is our parser of type any that state that it will contain either the +,-, * symbol using | operator

number is parser of type int looking at it you might get confused ,it states that it will evaluate a regular expression of type int,then you might  think what is is  ^^ symbol is doing it denotes the operation that needs to be performed when the left hand side expression is evaluated so here it is converting this expression value to integer

def expression = {
  number ~ symbol ~ number ^^ { case firstOperand ~ operator ~ secondOperand =>
    validateAndExtractFirstValue(firstOperand) + validateAndExtractSecondValue(secondOperand)
  }

now what is this method is doing

~ symbol in scala parser is used to seperate out the token in scala,it will parse the expression if the expression contain a number followed by a symbol followed by another number,then it simply add those two numbers

to be more concise here is the full code

import scala.util.Try
import scala.util.parsing.combinator.RegexParsers

class ScalaParser extends RegexParsers {

  def expression = {
    number ~ symbol ~ number ^^ { case firstOperand ~ operator ~ secondOperand =>
      validateAndExtractFirstValue(firstOperand) + validateAndExtractSecondValue(secondOperand)
    }
  }

  def symbol: Parser[Any] = "+" | "-" | "*"

  def number: Parser[Int] = """(0|[1-9]\d*)""".r ^^ { _.toInt }

  def validateAndExtractFirstValue(firstOperand: Any): Int = {
    val firstValue: Try[Int] = Try(firstOperand.toString.toInt)
    firstValue match {
      case util.Success(value) => value
      case util.Failure(exception) => throw new Exception("can not convert values to integer")
    }
  }

  def validateAndExtractSecondValue(secondOperand: Any): Int = {
    val secondValue = Try(secondOperand.toString.toInt)
    secondValue match {
      case util.Success(value) => value
      case util.Failure(exception) => throw new Exception("can not convert values to integer")
    }
  }
}

object TestSimpleParser extends ScalaParser {
  def main(args: Array[String]) = {
    parse(expression, "5 + 4") match {
      case Success(result, _) => println(result)
      case Failure(msg, _) => println("FAILURE: " + msg)
      case Error(msg, _) => println("ERROR: " + msg)
    }

  }
}

to be clear to call the parser you have to call the parse method it takes two arguments one is type of parser and second one is statement to be parsed

i think this blog will help to getting started with scala parser and combinators

KNOLDUS-advt-sticker

Posted in Scala | Leave a comment