Introduction to Apache Airflow and its Components

Reading Time: 3 minutes What is Apache Airflow ? Apache Airflow is a free and open-source application for managing complicated workflows and data processing pipelines. It’s a platform for automating and monitoring workflows for scheduled jobs. It allows us to configure and schedule our processes according to our needs while simplifying and streamlining the process. Why do we need Apache Airflow ? Lets us assume a use case where Continue Reading

Apache Airflow – A Workflow Manager

Reading Time: 4 minutes As the industry is becoming more data driven, we need to look for a couple of solutions that would be able to process a large amount of data that is required. A workflow management system provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application. Workflow management has become such a common need that most Continue Reading