Reading Time: 3 minutes Transformation is one of the RDD operation in spark before moving this first discuss about what actual Spark and RDD is. What is Spark? Apache Spark is an open-source cluster computing framework. Its main objective is to manage the data created in real time. Hadoop MapReduce was the foundation upon which Spark was developed. Unlike competing methods like Hadoop’s MapReduce, which writes and reads data Continue Reading
Reading Time: 4 minutes What’s AWS ? Amazon Web Services is a comprehensive, evolving cloud computing platform provided by Amazon that includes a mixture of infrastructure-as-a-service (IaaS), platform-as-a-service (PaaS) and packaged-software-as-a-service (SaaS) offerings. AWS services can offer an organization tools such as compute power, database storage and content delivery services. AWS (Amazon Web Services) is a Cloud Provider. It provide you with servers and services that you can use on demand Continue Reading
Reading Time: 2 minutes What exactly is Tail Recursion? A Recursive Function is tail-recursive when the function executes a recursive call. Evolution of Tail Recursion : What does Tail Recursion mean in SCALA Language? Syntax : tailrec def Func(P1, P2, …): type = … Example of a Tail Recursive Program : import scala.annotation.tailrec object Article { def GCD(n: Int, m: Int): Int = { @tailrec def gcd(x:Int, y:Int): Int= Continue Reading
Reading Time: 3 minutes Join in Spark SQL is the functionality to join two or more datasets that are similar to the table join in SQL based databases. Spark works as the tabular form of datasets and data frames. The Spark SQL supports several types of joins such as inner join, cross join, left outer join, right outer join, full outer join, left semi-join, left anti join. Joins scenarios Continue Reading
Reading Time: 2 minutes Overview In this tutorial, we’ll learn how to convert an RDD to a DataFrame in Spark. We’ll look into the details by calling each method with different parameters. Along the way, we’ll see some interesting examples that’ll help us understand concepts better. RDD and DataFrame in Spark RDD and DataFrame are two major APIs in Spark for holding and processing data. It provides us with low-level APIs for processing distributed data. Continue Reading
Reading Time: 2 minutes Higher order function is a function that either takes a function as argument or returns a function. In other words we can say a function which works with function is called higher order function. Higher order function allows you to create function composition, lambda function or anonymous function etc. Let’s see an example. Passing a Function as Parameter in a Function Output: 50 Function Composition Continue Reading
Reading Time: 5 minutes Introduction Lagom is a highly opinionated framework for building flexible, resilient, and responsive systems in Java and Scala. It’s an open-source framework maintained by Lightbend. It offers libraries and development environments to build systems based on reactive microservices with best practices. It supports multiple aspects from development to deployment by leveraging other reactive frameworks from Lightbend like Play and Akka: We design microservices to be isolated and autonomous, with Continue Reading
Reading Time: 2 minutes In Akka, you can stop Actors by invoking the stop() method of either ActorContext or ActorSystem class. ActorContext is used to stop child actor and ActorSystem is used to stop top level Actor. The actual termination of the actor is performed asynchronously. There are some other methods available in Akka, which are used to stop Actor. Some of which are PoisonPill, terminate() and gracefulStop() are Continue Reading
Reading Time: 4 minutes In this blog, we’ll find out what rich wrappers are and how we can use them in Scala. The definition of “rich wrappers” lies in the name itself. First, they enrich some other class, that is, they add features the “original” class lacks. Secondly, they wrap the other class, meaning that they should, as much as possible, behave as the class they wrap. Ideally, there should be Continue Reading
Reading Time: 4 minutes Introduction to Apache Spark Big Data processing frameworks like Apache Spark provides an interface for programming data clusters using fault tolerance and data parallelism. Apache Spark is broadly used for the speedy processing of large datasets. Apache Spark is an open-source platform, built by a broad group of software developers from 200 plus companies. Over 1000 plus developers have contributed since 2009 to Apache Spark. Continue Reading
Reading Time: 4 minutes Overview In earlier blogs I discussed about play framework now lets we move to further topics on play. For building simple, CRUD-style REST APIs in Scala, the Play Framework is a good solution. It has an uncomplicated API that doesn’t require us to write too much code. In this blog, we’re going to build a REST API in Scala with Play. We’ll use JSON as Continue Reading
Reading Time: 3 minutes What Play is ? Play makes you more productive. Play is also a web framework whose HTTP interface issimple, convenient, flexible, and powerful. Most importantly, Play improves on themost popular non-Java web development languages and frameworks—PHP and Rubyon Rails—by introducing the advantages of the Java Virtual Machine (JVM). Project Structure Now, it’s time to load the project code into the IDE and look at the directory Continue Reading
Reading Time: 6 minutes Introduction Scala Future represents a result of an asynchronous computation that may or may not be available yet. When we create a new Future, Scala spawns a new thread and executes its code. Once the execution is finished, the result of the computation (value or exception) will be assigned to the Future. Type Future Operations Map When we have a Future instance, we can use the map method to transform its successful result Continue Reading
