Author: Meenakshi Goyal

Different Types of JOIN in Spark SQL

Reading Time: 3 minutes Join in Spark SQL is the functionality to join two or more datasets that are similar to the table join in SQL based databases. Spark works as the tabular form of datasets and data frames. The Spark SQL supports several types of joins such as inner join, cross join, left outer join, right outer join, full outer join, left semi-join, left anti join. Joins scenarios Continue Reading

RDD to DataFrame Conversion in Spark

Reading Time: 2 minutes Overview In this tutorial, we’ll learn how to convert an RDD to a DataFrame in Spark. We’ll look into the details by calling each method with different parameters. Along the way, we’ll see some interesting examples that’ll help us understand concepts better. RDD and DataFrame in Spark RDD and DataFrame are two major APIs in Spark for holding and processing data. It provides us with low-level APIs for processing distributed data. Continue Reading

Scala Higher Order Functions

Reading Time: 2 minutes Higher order function is a function that either takes a function as argument or returns a function. In other words we can say a function which works with function is called higher order function. Higher order function allows you to create function composition, lambda function or anonymous function etc. Let’s see an example. Passing a Function as Parameter in a Function Output: 50 Function Composition Continue Reading

Getting Started with Lagom Framework

Reading Time: 5 minutes Introduction Lagom is a highly opinionated framework for building flexible, resilient, and responsive systems in Java and Scala. It’s an open-source framework maintained by Lightbend. It offers libraries and development environments to build systems based on reactive microservices with best practices. It supports multiple aspects from development to deployment by leveraging other reactive frameworks from Lightbend like Play and Akka: We design microservices to be isolated and autonomous, with Continue Reading

Akka Stopping Actors

Reading Time: 2 minutes In Akka, you can stop Actors by invoking the stop() method of either ActorContext or ActorSystem class. ActorContext is used to stop child actor and ActorSystem is used to stop top level Actor. The actual termination of the actor is performed asynchronously. There are some other methods available in Akka, which are used to stop Actor. Some of which are PoisonPill, terminate() and gracefulStop() are Continue Reading

Rich Wrappers in Scala

Reading Time: 4 minutes In this blog, we’ll find out what rich wrappers are and how we can use them in Scala. The definition of “rich wrappers” lies in the name itself. First, they enrich some other class, that is, they add features the “original” class lacks. Secondly, they wrap the other class, meaning that they should, as much as possible, behave as the class they wrap. Ideally, there should be Continue Reading

Getting Started with Spark 3

Reading Time: 4 minutes Introduction to Apache Spark Big Data processing frameworks like Apache Spark provides an interface for programming data clusters using fault tolerance and data parallelism. Apache Spark is broadly used for the speedy processing of large datasets. Apache Spark is an open-source platform, built by a broad group of software developers from 200 plus companies. Over 1000 plus developers have contributed since 2009 to Apache Spark.  Continue Reading

Build REST API in Scala with Play Framework

Reading Time: 4 minutes Overview In earlier blogs I discussed about play framework now lets we move to further topics on play. For building simple, CRUD-style REST APIs in Scala, the Play Framework is a good solution. It has an uncomplicated API that doesn’t require us to write too much code. In this blog, we’re going to build a REST API in Scala with Play. We’ll use JSON as Continue Reading

Introduction to the Play Framework

Reading Time: 3 minutes What Play is ? Play makes you more productive. Play is also a web framework whose HTTP interface issimple, convenient, flexible, and powerful. Most importantly, Play improves on themost popular non-Java web development languages and frameworks—PHP and Rubyon Rails—by introducing the advantages of the Java Virtual Machine (JVM). Project Structure Now, it’s time to load the project code into the IDE and look at the directory Continue Reading

Future Type Operation with Scala

Reading Time: 6 minutes Introduction Scala Future represents a result of an asynchronous computation that may or may not be available yet. When we create a new Future, Scala spawns a new thread and executes its code. Once the execution is finished, the result of the computation (value or exception) will be assigned to the Future. Type Future Operations Map When we have a Future instance, we can use the map method to transform its successful result Continue Reading

How To Use Pattern Matching

Reading Time: 3 minutes Overview In this blog, we will show a functional feature of Scala which is pattern matching. If you have used Java or .NET in the past, it may at first sight appear similar to switch statements. But, Scala’s pattern matching is a lot more powerful! Different ways of Pattern Matching 1. Pattern matching – a very basic example Suppose you want to test a variable called donutType. In the case that its value Continue Reading

Opaque Type Aliases

Reading Time: 3 minutes Introduction Scala is a statically typed programming language. This means the compiler determines the type of a variable at compile time. A type alias is usually used to simplify declaration for complex types, such as parameterized types or function types. Opaque type aliases do not allow access to their underlying type outside of the file in which they are defined. Opaque type alias is a feature that Continue Reading

Getting started with Akka Cluster

Reading Time: 3 minutes Introduction Akka cluster provides a fault-tolerant decentralized peer to peer cluster membership service with no single point of failure or single point of bottleneck. It does this using gossip protocol and an automatic failure detector. Akka Cluster allows for building distributed applications, where one application or service spans multiple nodes (in practice multiple Actor Systems). Cluster Membership Akka cluster module provides a very simple cluster membership Continue Reading