workflows

An introduction to Apache Airflow : An Ultimate Guide for Beginners.

January 31, 2022January 31, 2022Apache Airflow, Big Data and Fast DataDAG, workflows

Reading Time: 3 minutes It is one of the most popular open-source workflow management platforms within data engineering to manage the automation of tasks and their workflows. Apache Airflow is written in Python, which enables flexibility and robustness. What is Apache Airflow? Apache Airflow is a robust scheduler for programmatically authoring, scheduling, and monitoring workflows. It is a workflow engine that will easily schedule and run your complex data pipelines. It Continue Reading

Self-hosted runners for GitHub Actions

September 5, 2021September 5, 2021github, Studio-DevOpscicd in github, github, github-actions, runners, self-hosted, workflows

Reading Time: 4 minutes Self-hosted runners offer more control of hardware, operating system, and software tools than GitHub-hosted runners. It provide us with ability to create custom hardware and software configuration with more processing ability to run larger jobs. Runners can be physical, virtual, in a container, on-premises, or in a cloud. In this blog, we will be adding a runner in cloud. For introduction to CI/CD using GitHub Continue Reading