Spark Job

Cluster vs Client: Execution modes for a Spark application

May 19, 2020August 30, 2022Apache Spark, Big Data and Fast Data, Java, Studio-ScalaBig Data Analytics, resourece manager, spark ecosystem, Spark Job, yarn

Reading Time: 3 minutes Whenever we submit a Spark application to the cluster, the Driver or the Spark App Master should get started. And the Driver will be starting N number of workers. Spark driver will be managing spark context object to share the data and coordinates with the workers and cluster manager across the cluster. Cluster Manager can be Spark Standalone or Hadoop YARN or Mesos. Workers will Continue Reading

Apache Spark: Tricks to Increase Job Performance

February 19, 2020Analytics, Apache Spark, Big Data and Fast Data, Database, ML, AI and Data Engineering, NoSql, Spark, SQL, Streaming, Streaming Solutions, Studio-ScalaApache Spark, Big Data, Big Data Analytics, data analysis, data engineering, fast data analytics, Spark Job, Spark Performance, Spark SQL, Spark Streaming

Reading Time: 2 minutes Apache Spark is quickly adopting the Real-world and most of the companies like Uber are using it in their production. Spark is gaining its popularity in the market as it also provides you with the feature of developing Streaming Applications and doing Machine Learning, which helps companies get better results in their production along with proper analysis using Spark. Although companies are using Spark in Continue Reading