Databricks Deployment via Jenkins
Reading Time: 3 minutes In this blog, We will learn how do we create the Databricks Deployment pipelines to deploy databricks components(Notebooks, Libraries, Config files and packages) via a Jenkins.
Migrating to Cloud: Inhouse Hadoop to Databricks
Reading Time: 6 minutes Migration of applications is a good thing. It forces the organization to clean up junk, that is never used. It adds a lot of innovation and new ideas to your engineering teams. It is important to build confidence in our teams that future migrations are not stressful and pushes teams to design systems to be flexible. It sends a message to vendors that you are Continue Reading
Databricks: Make Log4J Configurable
Reading Time: 3 minutes The goal of this blog is to define the processes to make the log4j configuration file configurable for debugging purpose.
Collecting logs in Azure Databricks
Reading Time: 3 minutes Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. In this blog, we are going to see how we can collect logs from Azure to ALA .Before going further we need to look how to setup spark cluster in azure Create a Spark cluster in Databricks In the Azure portal, go to the Databricks workspace that you created, Continue Reading