transformations

Apache Spark’s Developers Friendly Structured APIs: Dataframe and Datasets

June 15, 2022June 16, 2022Studio-ScalaApache Spark, APIs, class, DataFrame, datasets, filter, scala, Set, Spark, transformations

Reading Time: 3 minutes This is the second part of the blog series on Spark‘s structured APIs Dataframe & Datasets. In the first part we covered Dataframe and I recommend you go read that blog first if you are new to spark. In this blog we’ll cover the Spark Datasets API, so let’s get started. The Datasets API Datasets are also the combination of two characteristics: typed and untyped Continue Reading

Deep Dive into Apache Spark Transformations and Action

October 21, 2019October 21, 2019Apache Spark, Big Data and Fast Data, Cloud, Database, HDFS, linux, NoSql, Spark, SQL, Streaming, Streaming Solutions, Studio-Scalaactions, Big Data, Big Data Analytics, data analysis, data engineering, fast data, RDD partitions, RDDs in Spark, Spark, Spark with Scala, transformations

Reading Time: 4 minutes In our previous blog of Apache Spark, we discussed a little about what Transformations & Actions are? Now we will get deeper into the topic and will understand what actually they are & how they play a vital role to work with Apache Spark? What is Spark RDD? Spark introduces the concept of an RDD (Resilient Distributed Dataset), an immutable fault-tolerant, distributed collection of objects Continue Reading