Could not instantiate SVGImageReader Exception in Scrimage


Scrimage is a powerful image processing scala library . Basically Scrimage’s functionality based on java.awt.* along with others third party library like “metadata-extractor” , “xmlgraphics” etc .

The use case of this library is optimizing the image without loosing the original  image quality . We can apply processing like  resize, rotate, filter, crop, convert between formats on image .

To know more about Scrimage , you can visit on https://github.com/sksamuel/scrimage .

That was about Scrimage but what actually we face during the processing of image “Could not instantiate SVGImageReader” error message on terminal .It does not affect processing of image  but no one would like gaint error message on console like :

scrimageError

Found  root cause :

rootcause

Deep into looking , i found Apache Batik – Common runtime exceptions list . On searching in this list i came across “Fixed by adding batik-codec.jar to class path” .

A qucik glance in maven repository and update build file by adding


libraryDependencies += "org.apache.xmlgraphics" % "batik-codec" % "1.7"

Then the console error disappear . ohh Great !

Hope , this would help someone else who are facing same issues :)

Posted in Scala | Leave a comment

Testing Scala Applications with In-memory mongoDB


Is your test suite taking a large amount of time to run just because your methods need some database queries to be handled? Testing with In-memory databases can save a hell lot of time. In-memory database makes the queries readily available to methods in a matter of milliseconds. In case you are using mongoDB, this blog can help you run your test suite much faster than currently it is.

Adding Configuration

I assume you are already interacting with mongoDB in your application and have included mongoDB driver for that. This plug-in for in-memory mongoDB works with scalatest. I also assume you are using Scalatest as test Toolkit. If you are not, please include the following dependencies to your build to include embedded mongoDB and scalatest.

resolvers ++= Seq(
"Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots"
)

libraryDependencies ++= Seq(
"org.scalatest" %% "scalatest" % "1.9.2",
"com.github.simplyscala" %% "scalatest-embedmongo" % "0.2.3-SNAPSHOT"
)

Once you have added these lines into your build, resolve these dependencies by running ‘sbt clean compile’ before you proceed.

Preparing the Test Suite

I hope you have written a Test Class to test your code. If you’ve not created any, create one. Your test class must have the following imports:

import org.scalatest.FunSuite
import org.scalatest.BeforeAndAfter
import com.github.simplyscala.MongodProps
import com.github.simplyscala.MongoEmbedDatabase

These imports are necessary to start an in-memory instance of mongoDB before running your Test Suite. Extend your test class with FunSuite (ignore if import is already there) and mix with the traits BeforeAndAfter and MongoEmbedDatabase. For example, have a look at the following code snippet.

class InsertPersonTest extends FunSuite with BeforeAndAfter with MongoEmbedDatabase

Here InsertPersonTest is my test class and I’ve extended it with FunSuite and mixed the traits BeforeAndAfter and MongoEmbedDatabase.

Starting Embedded Mongo

Now the goal is to start the mongoDB instance before running the Test Suite and Stopping it afterwards. To hold the reference of In-memory mongoDB instance, let us declare a variable in the test class as mongoInstance and then include the following before and after statements in your test class. Follow the given code snippet:

var mongoInstance: MongodProps = null //reference holder variable

before {
try{ mongoInstance = mongoStart(27017) } //Try to start mongoDB on default port
catch { case ex:Exception => }
}

after {
mongoStop(mongoInstance) //Stop mongoDB Instance
}

This will start an in-memory instance of mongoDB on the given port, i.e. (27017: default port of mongoDB). Please ensure that no local instance of mongoDB server is running on your machine before running your test suite. Stop the mongoDB server if it is running already and then run your test cases. I hope this will accomplish your purpose of in-memory mongoDB in testing environment.

Example Code

You can clone the example code from the EmbededMongoExample repository on the github. This project illustrates the working of in-memory mongoDB instances with example test cases.

Happy Testing!!

Posted in Scala | Tagged , , , , , , ,

Asynchronous Programming with Scala’s Future & Promises


In this presentation, we are going to discuss about Scala’s Future and Promise,their usage and Future composition and brief overview of Thread pool Executors.

Posted in Scala | 3 Comments

Web Application using rest in AngularJS – Part-2


As discussed in last blog , intro to AngularJS, its directives and Routing. Let us continue here with Angular Validations and how to define a controller in AngularJS.

Validations :

Angular provides various form validations. We will discuss a basic one here.

Keep in mind to give a form name to your form. Also, add novalidate flag to your form element. This will remove all browser validations.

<form name="userForm" novalidate>

Next to add any of the form validation remarks:

  • Required – required, to make the value of an element mandatory.
  • Minlength – ng-minlength=20, to give a minimum count of entered value.
  • Maxlength – ng-maxlength=20, to give a maximum count of entered value.
  • Email – type=”email”, will add email validation.
  • Number – type=”number”, will add number validation.
  • URL – type=”Url”, will accept value in URL format.
<form class="form-horizontal" ng-submit="submitForm()"  role="form" name="userForm" novalidate>
	<div class="form-group">
      <label class="control-label col-sm-2" for="email">Name:</label>
      <div class="col-sm-5">
        <input type="text" ng-minlength="3" ng-model="name" class="form-control" id="name" placeholder="Enter name" required>
      </div>
    </div>
    ...
</form>

Form name is ‘userForm‘ and we will call function submitForm() on submission. The input box has some validations placed.
We are having an input field with its name bound by ng-model to the object name on the $scope object.
ng-model – Binds the view into the model,using controller.
Scope is the glue between application controller and the view. It refers to the application model.

Sign Up Form with validations binded with a controller can be viewed here.

Now, to submit the form only when all fields are valid, add formName.$valid:
ng-submit=”submitForm(userForm.$valid)”

<div ng-controller="FormController">
<form class="form-horizontal" ng-submit="submitForm(userForm.$valid)"  role="form" name="userForm" novalidate>
...
</form>

Next, to add an action notifying the user that his form is successfully submitted, we hit an angular controller.

app.controller('FormController' , function($scope){
    $scope.submitForm = function(isValid) {
	if(isValid) {
		alert('Congratulations! you registration has been successful')
			
		$scope.name="";  //to empty the input box after successful submit
		$scope.username="";
		$scope.email="";
		$scope.pwd="";
		$scope.password="";
	}
    }
});

We created a function for form submit within an angular Controller and pass the same controller into the DOM element we wish to take into consideration.

Angular Controller is build with the following syntax :

angular.module(‘firstApp’, [])
.controller(‘CtrlName’, function($service_object){
$service_object.functionName = function(){
……..
}
}

blog

You can even add custom validation messages on the form :

<input type="text" name="userName" ng-model="username" ng-minlength=3 ng-maxlength=10 required />
    <div class="validate" ng-show="userForm.username.$dirty && userForm.username.$invalid">
      <small class="errorMessage" ng-show="userForm.username.$error.required">
        UserName is required.
      </small>
      <small class="errorMessage" ng-show="userForm.username.$error.minlength"> 
        UserName should be at least 3 characters
      </small>
      <small class="errorMessage" ng-show="userForm.username.$error.maxlength">
        UserName cannot exceed 8 characters
     </small>
   </div>
<button type="submit" ng-disabled="userForm.$invalid">Submit</button>

ng-show/ng-hide : show/hide the DOM elements, if the form is invalid.
ng-disabled : To disable the button till all the input fields get valid.

Some other angular properties used here are :

$valid – formName.elementName.$valid : True, if form valid
$invalid – formName.elementName.$invalid : True, if form invalid
$error – formName.elementName.$error.validation : includes all validations residing within it.
$dirty – formName.elementName.$dirty : To display errors only if user modified the input field.

and even more to explore.

Reference : http://www.ng-newsletter.com/posts/validations.html

and for above code can checkout here.

Watch out for the next part of this blog – the RESTful Angular. Coming soon !!

Posted in AngularJS | 1 Comment

Web Application using rest in AngularJS – Part-1


You know what is AngularJS famous for ?

So before beginning to develop an example web app for you using angular, let us start knowing about some angular features.

  • Angular gives you the feature of two way data binding :With reducing the code change in model, it shows the change in model over DOM element in fraction of seconds.
  • AngularJS is a MVVM (Model-View-ViewModel) : ViewModel here is the $scope of angular, with an API to detect and broadcast changes.
  • and many more …

Here, I will divide the web application under 3 groups(explaining first 2 in this blog):

1. Developing a static web application -> using angular Directives.
2. Adding actions -> using Angular Routing.
3. Handling RESTful backened -> with angular $resouce – $http service

Let us start with building some static web pages using Angular Directives.

You will require the following libraries of Bootstrap and Angular :

index.html :

&lt;head&gt;
&lt;link rel="stylesheet" href="../css/bootstrap.min.css"&gt;
&lt;script src="../jquery/1.11.3/jquery.min.js"&gt;&lt;/script&gt;
&lt;script src="../bootstrap/3.3.5/js/bootstrap.min.js"&gt;&lt;/script&gt;
&lt;script src="../angular/angular.min.js" type="text/javascript"&gt;&lt;/script&gt;
&lt;script src="../angular/angular-route.min.js" type="text/javascript"&gt;&lt;/script&gt;
&lt;/head&gt;
&lt;body&gt;
&lt;/body&gt;

Create a Navbar for your web application (You can add your own multi view tab designs as per your requirement).

header.html

&lt;nav class="navbar navbar-inverse"&gt;
  &lt;div class="container-fluid"&gt;
    &lt;div class="navbar-header"&gt;
      &lt;a class="navbar-brand" href="#"&gt;Brand&lt;/a&gt;
    &lt;/div&gt;

    &lt;div class="collapse navbar-collapse" id="bs-example-navbar-collapse-1"&gt;
      &lt;ul class="nav navbar-nav navbar-right"&gt;
        &lt;li&gt;&lt;a href="#"&gt;Sign Up&lt;/a&gt;&lt;/li&gt;
      &lt;/ul&gt;
    &lt;/div&gt;
  &lt;/div&gt;
&lt;/nav&gt;
&lt;div ng-view&gt;&lt;/div&gt;

Don’t forget to add a view to show your contents – ng-view.

You can add ng-view anywhere you wish your data to be reflected. It acts as a view space for your reflected data.

A form for your Sign Up page :

signUp.html

&lt;form class="form-horizontal" role="form"&gt;
	&lt;div class="form-group"&gt;
      &lt;label class="control-label col-sm-2" for="email"&gt;Name:&lt;/label&gt;
      &lt;div class="col-sm-5"&gt;
        &lt;input type="email" class="form-control" id="email" placeholder="Enter email"&gt;
      &lt;/div&gt;
    &lt;/div&gt;
    &lt;div class="form-group"&gt;
    ......
&lt;/form&gt;

Similarly, design a Sign In page too using basic Html – SignIn.html

Now let’s start connecting your view files together using few angular Directives :

We start with ng-app (Directive to define your application as an angular application):

&lt;body ng-app="firstApp"&gt;  //bind your angular application with a name - firstApp

Next :In your custom main.js file, add the following –

var app = angular.module('firstApp', []); //Declare an angular module for your app

Now, its time to include your files.
ng-include : It Fetches, compiles and includes an external HTML fragment.

&lt;body ng-app="firstApp"&gt;
&lt;div ng-include="'header.html'"&gt;&lt;/div&gt;
&lt;/body&gt;

Did you noticed something weird above ? The ng-include directive is adding the file name within both the double quote – ” ” , and a single quote – ‘ ‘ .

ng-include is expecting a variable with the name of the file to include.
To pass the name directly as a string, we use single quotes (‘…’)

Now, let’s add our Sign Up page :
Angular Routing will not only give you the form but reflects it without refreshing your page.

Start with Angular Routing :
In your header.html :

 &lt;li&gt;&lt;a href="#signUp"&gt;Sign Up&lt;/a&gt;&lt;/li&gt;

You will notice ‘#signUp’ above. Yes, Angular redirection URL includes # (Hash) to be appended before the new redirection link.

Now, in your main.js

var app = angular.module('demo', ['ngRoute']);

Add, ngRoute in your array of dependencies for providing routing. Next, create a config function –

app.config(function($routeProvider) {
    $routeProvider.
    when('/', {
      templateUrl: 'main.html'
    }).
    when('/signUp', {
      templateUrl: 'signUp.html'
    }).
    when('/login', {
      template: '&lt;div class="alert alert-success fade in"  style="margin: 86px 281px 7px; width: 50%"&gt;&lt;strong&gt;Congrats!&lt;/strong&gt; You successfully logged In&lt;/div&gt;'        
        }).
    otherwise({
      redirectTo: '/'
    });
});

here ,
app – is your angular module variable that you created above.
.config – is an angular function run when page loads. It configures routes.
$routeProvider – is the angular service. Controller calls angular injector to provide him with the service and Angular DI ( Dependency injector) returns back the routing service in the form of parameter to the controller.
when – block matches with the URL after ‘/’. Redirects to angular template associated with the respective matching condition.

Few angular routing properties :

  • template : where you can directly write a template
  • templateURL : where you give the URL
  • controller : where you can define the controller name , to which it would hit and get the corresponding action.

This way you are redirected with your SignUp form using angular Redirection.

For the code, you can refer here.

Handling of angular controllers and doing form validations, will be continued in the next part of this Blog.

Posted in AngularJS | 1 Comment

Tuning apache spark application with speculation


What happen if spark job will be slow its a big question for application performance so we can optimize the jobs in spark with speculation, Its basically start a copy of job in another worker if the existing job is slow.It will not stop the slow execution of job both the workers execute the job simultaneously.

To make our job speculative we need to set configuration as one of follows :-

we can set speculation configuration in our code as follows

val conf=new SparkConf().set("spark.speculation","false")

Or, We can add configuration dynamically at time for deploy application with the help of flag as follows

--conf spark.speculation=false

Or, We can also add it in properties file with the space delimiter as follows

spark.speculation false

Now its time to deep dive, with speculation we have lot of questions like:
It will produce redundant result?
In what time speculation will happen?
If most of our job were completed then its reliable to speculate job?

So, firstly actions in spark are always idempotent, spark handle it for us that even task will be executed for multiple times but only add it in result once.Spark processes the result generated from each partition by passing them to resultHandler and it is always equal to partition.size

We also need to define how often the task will be check for the speculation for that we can define configuration as

spark.speculation.interval (time in milliseconds)

By default it is 100 ms

The speculation take place at the end of the stages is not make sense so for that we can also set configuration that speculation the stage only if particular percentage of tasks were not completed. For that we need to set configuration as follow

spark.speculation.quantile (value in percentage)

By default its value is 0.75

Last last we can set configuration to set at how much level the tasks will slow will be speculate for that we define the configuration

spark.speculation.multiplier

By default its value is 1.5 so if task is 1.5 times slower than it will be consider for the speculation.

Posted in apache spark, big data, Scala, Spark | Tagged , , , , , | 3 Comments

Spray – Build RESTfull services in scala


For Demo you can clone the below git repository and run it with sbt

https://github.com/phalodi/spray-demo

Posted in Scala | Leave a comment

An introduction to BackboneJs


Posted in Scala | 1 Comment

Introduction to Play Framework


Posted in Scala | Tagged , , , , , , , , , , , | Leave a comment

Introduction to Couchbase


Posted in couchbase, Scala | Tagged , , ,