Reading Time: 4 minutes If you are wondering how to start working with Apache Airflow for small development or academic purposes here you will learn how to. Well deploying Airflow on GCP Compute Engine (self-managed deployment) could cost less than you think with all the advantages of using its services like BigQuery or Dataflow. Table of Content What is apache airflow cloud composer overview Google cloud composer benefit Composer Continue Reading
Reading Time: 3 minutes Installation of Airflow The more preferable approach to installing Apache-Airflow is to install it in a virtual environment. Airflow requires the latest version of PYTHON and PIP (package installer for python). Below are the steps to install it on your system To set up a virtual environment for Apache Airflow : virtualenv apache_airflow To activate the virtual environment navigate to the “bin” folder inside the Continue Reading
Reading Time: 4 minutes As the industry is becoming more data driven, we need to look for a couple of solutions that would be able to process a large amount of data that is required. A workflow management system provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application. Workflow management has become such a common need that most Continue Reading
