Vidushi Bansal, Author at Knoldus Blogs

Creating a DAG in Apache Airflow

May 19, 2021May 19, 2021Apache Airflow, DevOpsairflow, apache-airflow, dags

Reading Time: 4 minutes In my previous blog, I have discussed Airflow – A workflow Manager. In this blog, we will write a DAG for Airflow that would define a workflow of tasks and their dependencies. Before writing a DAG file, we will first look into the operators that can be used while writing a DAG. Airflow Operators An operator represents a single, ideally idempotent, task. Operators determine what actually Continue Reading

Apache Airflow – A Workflow Manager

May 16, 2021May 16, 2021Apache Airflow, Big Data and Fast Data, DevOpsairflow, airflow-setup, apache, data-pipelines

Reading Time: 4 minutes As the industry is becoming more data driven, we need to look for a couple of solutions that would be able to process a large amount of data that is required. A workflow management system provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application. Workflow management has become such a common need that most Continue Reading

Introduction to a Modern Reverse-Proxy: Traefik

February 6, 2021March 16, 2021DevOpsIntroduction, reverse-proxy, traefik

Reading Time: 3 minutes Traefik is an open source API gateway, written in Golang and was developed in a Unix-centric way. It is designed to simplify the complexity regarding microservices operations. Traefik performs auto-configuration of services, which means that the developer only needs to worry about developing and deploying applications. Traefik auto-configures with sensible defaults and sends a request to the said service. With changing requirements and needs of Continue Reading

Clusters

January 7, 2021Scala

Reading Time: 4 minutes Microservices is a software architecture style that advocates for many granular services. Each of these services perform a single business function. Each microservice is a self-contained, independently deployable piece of a larger application. It interacts with other components, typically via well-defined REST APIs. Clusters is an important concept that has been introduced to implement microservices in a more efficient way. Adopting a microservices architecture consisting Continue Reading

Kubernetes: Best Practices

January 7, 2021Scala

Reading Time: 3 minutes Kubernetes is an open source platform for automating deployments, scaling, and operations of application containers across clusters of hosts. It provides a platform for container-centric infrastructure. It addresses all the challenges that were described above. Among its key features are: Automated deployment and replication of containers Online scale-in and scale-out of container clusters Load balancing over groups of containers Rolling upgrades of application containers Resiliency, Continue Reading

RBAC in kubernetes

December 15, 2020December 15, 2020Scala

Reading Time: 4 minutes Starting with Kubernetes version 1.6.0, cluster security was ramped up considerably. In earlier versions, if you managed to acquire the authentication token from one of the pods, you could use it to do anything you want in the cluster. RBAC in kubernetes prevents unauthorized users from viewing or modifying the cluster state. The default ServiceAccount isn’t allowed to view cluster state, let alone modify it Continue Reading