Partitions

Controlling RDD Partitions in Apache Spark

July 13, 2016September 10, 2018Apache Spark, Spark, Studio-ScalaApache Spark, partitioning in apache spark, Partitions, RDD partitions, shffle operation in spark, shuffle operation

Reading Time: 2 minutes In this blog, we will discuss What is RDD partitioning, why Partitioning is important and how to create and use spark Partitioners to minimize the shuffle operations across the nodes in a distributed Spark application. What is Partitioning? Partitioning is a transformation operation which is available on all key value pair RDDs in Apache Spark. It is required when we try to group values on the basis Continue Reading