Insights and perspectives to keep you updated.

Start/Deploy Apache Spark application programmatically using spark launcher

Reading Time: < 1 minute Sometimes we need to start our spark application from the another scala/java application. So we can use SparkLauncher. we have an example in which we make spark application and run it with another scala application. Let see our spark application code. import org.apache.spark.SparkConf import org.apache.spark.SparkContext object SparkApp extends App{ val conf=new SparkConf().setMaster(“local[*]”).setAppName(“spark-app”) val sc=new SparkContext(conf) val rdd=sc.parallelize(Array(2,3,2,1)) rdd.saveAsTextFile(“result”) sc.stop() } This is our simple spark Continue Reading

Stateful transformation on Dstream in apache spark with example of wordcount

Reading Time: 2 minutes Sometimes we have a use-case in which we need to maintain state of paired Dstream to use it in next Dstream . So we are taking a example of stateful wordcount in socketTextStreaming. Like in wordcount example if word “xyz” comes twice is in first Dstream or window, it reduce it and its value is 2 but its state will lost in the next Dstream Continue Reading

Apache spark + cassandra: Basic steps to install and configure cassandra and use it with apache spark with example

Reading Time: 3 minutes To build an application using apache spark and cassandra you can use the datastax spark-cassandra-connector to communicate with spark. Before we are going to communicate with spark using connector we should know how to configure cassandra. So following are prerequisite to run example smoothly. Following steps to install and configure cassandra If you are new to cassandra first we nee to install cassandra on our Continue Reading

Easiest Way to connect with Couchbase using Scala

Reading Time: 2 minutes In this blog, we have explained how easily we can connect with Couchbase, store and fetch data  using scala through an example. Tools and technologies used : SBT 0.13.8 Scala 2.11.6 Couchbase Server 4.0 Beta Eclipse JDK 1.7 Please follow below instructions to use this example:- 1) Install and configure Couchbase a) Download the couchbase server from here. b)  If you are using ubuntu, then Install the Continue Reading

Shufflling and repartitioning of RDD’s in apache spark

Reading Time: 3 minutes To write the optimize spark application you should carefully use transformation and actions, if you use wrong transformation and action will make your application  slow. So when you are writing application some points should be remember to make your application more optimize. 1. Number of partitions when creating RDD By default spark create one partition for each block of the file in HDFS it is Continue Reading

Play 2.4.X : Microservice Architecture using Play and Scala

Reading Time: < 1 minute Playing-Microservices This blog describes a basic Microservice architecture design using Play 2.4.X and Scala. The big idea behind microservices is to architect large, complex and long-lived applications as a set of cohesive services that evolve over time. The term microservices strongly suggests that the services should be small. In short, the microservice architectural style is an approach to developing a single application as a suite Continue Reading

Data Science & Spark :- Logistic Regression implementation for spam dataset

Reading Time: 3 minutes We all are bit familiar with term Data Science, as it is turning out to be a field with potential of new discoveries. Challenges of Data Science seem to be evolutionary, given the amount of data in the world is only going to increase. However, the development of tools and libraries to deal with these challenges can be somewhat called revolutionary. One such product of Continue Reading

Elasticsearch: CURD Operations and sorting documents by time stamp with scala using java api of elastic search

Reading Time: 6 minutes Elasticsearch is an open-source search engine built on top of Apache Lucene™, a full-text search-engine library. You can read it more on their website. Elasticsearch is also written in Java and uses Lucene internally for all of its indexing and searching, but it aims to make full-text search easy by hiding the complexities of Lucene behind a simple, coherent, RESTful API. In this post, we Continue Reading

Play 2.3.x PDF : Basic example for generating PDF documents dynamically from Play Application with the help of play2-pdf module

Reading Time: 2 minutes In this blog I have created a basic example for generating PDF documents dynamically from Play Application with the help of play2-pdf module Play 2.3.x PDF module: This module helps generating PDF documents dynamically from your Play! web application. It simply renders your HTML- and CSS-based templates to PDF. It is based on the Flying Saucer library, which in turn uses iText for PDF generation. Continue Reading

Play framework security with Silhouette

Reading Time: 6 minutes Silhouette is a security library for Play framework. Its basically a core only fork of Secure Social, which is another security library for play framework. Secure Social is a great framework for lots of user but my experience with secure social is not that good, nor with some of my friends and colleagues who use Play framework in their application. It’s been thrown out from Continue Reading

Playing Gravatar: A basic example to generate gravatar using email address with Play Framework

Reading Time: 2 minutes Playing Gravatar Image In this blog I have created a basic example to generate gravatar using email address with Play Framework Gravatar: An “avatar” is an image that represents you online—a little picture that appears next to your name when you interact with websites. A Gravatar is a Globally Recognized Avatar. You upload it and create your profile just once, and then when you participate Continue Reading

Tutorial #3: Getting started with HTTP Programming in Play Framework

Reading Time: < 1 minute We have already discussed about the Play development environment in Tutorial #1 and use of WebJars, jQuery, Bootstrap & Bootswatch with Play in Tutorial #2 So, In this blog we would discuss about HTTP programming in Play Framework which would drive us through rest of the tutorial series.  We would be running this blog as a series and we would be looking at various aspects Continue Reading