spark 2.1

Application compatibility for different Spark versions

Reading Time: 3 minutes Recently spark version 2.1 was released and there is a significant difference between the 2 versions. Spark 1.6 has DataFrame and SparkContext while 2.1 has Dataset and SparkSession. Now the question arises how to write code so that both the versions of spark are supported. Fortunately maven provides the feature of building your application with different profiles. In this blog i will tell you guys how to Continue Reading