Catalyst Optimizer

Optimizations In Spark: For BETTER OR For WORSE

July 11, 2020October 23, 2020Apache Spark, Big Data and Fast Data, Spark, Studio-Scala, Tech BlogsApache Spark, Catalyst Optimizer, Spark, Spark SQL, SPARK SQL OPTIMIZATIONS, sql

Reading Time: 5 minutes This blog focuses on some of the problems faced while working with the Spark SQL

Logical and Physical Plan in Spark

Understanding Spark’s Logical and Physical Plan in layman’s term

November 4, 2019March 12, 2021Apache Spark, Big Data and Fast Data, Spark, SQL, Studio-ScalaCatalyst Optimizer, DAG in Spark, Lineage, Logical Plan, Physical Plan, Tungsten

Reading Time: 6 minutes This blog pertains to Apache SPARK 2.x, where we will find out how Spark SQL works internally in layman’s terms and try to understand what is Logical and Physical Plan. Also, we will be looking into Catalyst Optimizer. So let’s get started. First, let’s see what Apache Spark is. The official definition of Apache Spark says that “Apache Spark™ is a unified analytics engine for large-scale Continue Reading