Reading Time: 4 minutes In this blog we will go over the core concepts basic you must understand if you want to use Apache airflow. In this article, you will learn: What is Airflow Architecture Overview Dag Task Operator Dag Run Execution Date Airflow Airflow was started in October 2014 and developed by Maxime Beauchemin at Airbnb. It is a platform for programmatically authoring, scheduling, and monitoring workflows. It Continue Reading
Reading Time: 4 minutes Introduction One of the biggest challenges associated with big data is, analyzing the data. But before we get to that part, the data has to be first collected, and also for a system to process impeccably it should be able to grasp and make the data available to users. This is where Apache Kafka comes in handy. Let’s briefly understand how Kafka came into existence? Continue Reading
Reading Time: 4 minutes As the industry is becoming more data driven, we need to look for a couple of solutions that would be able to process a large amount of data that is required. A workflow management system provides an infrastructure for the set-up, performance and monitoring of a defined sequence of tasks, arranged as a workflow application. Workflow management has become such a common need that most Continue Reading
Reading Time: 3 minutes Service mesh is a dedicated infrastructure layer for handling service to service communication.
Basically, it’s a way to control how different micro services deployed on kubernetes will manage secure communication and traffic between them with lot’s of cross-cutting concerns like logging, security, etc.
Reading Time: 4 minutes In our previous blog post, Understanding Failures in Akka Cluster, we came to know how an Akka Cluster reacts in case of a failure and what are the reasons behind the failures. Now, whenever a failure will occur in an Akka Cluster, we would need a way to heal the cluster, so that we can restore it back to its normal working condition. Hence, in Continue Reading
Reading Time: 4 minutes Hi folks, in this blog, we will be discussing what challenges we face when we want to view logs with respect to metrics and how we can achieve this using Grafana, Loki, and Promtail. Problem Statement When using monitoring solutions like Grafana, Prometheus, Elatic Stack etc for our infrastructure, we have metrics and logs decoupled to an extent that it is very difficult to look Continue Reading
Reading Time: 2 minutes Kubernetes API is growing day by day and they are adding new features every day. The latest feature they added was DaemonSet. Although they had ReplicaSet, DaemonSet is the K8 object they added. Let’s see the difference. ReplicasSet will ensure that the number of pods (defined in our config file) is always running in our cluster. Does not matter in which worker node they are Continue Reading
Reading Time: 4 minutes Prometheus is an open-source monitoring system for processing time series metric data. It collects, organizes, and stores metrics using unique identifiers and timestamps. DevOps teams and developers query that data using PromQL and then visualize it in a UI such as Grafana. Basic Terminologies of Prometheus Monitoring : Monitoring is a systematic process of collecting and recording the activities taking place in a target project, Continue Reading
Reading Time: 4 minutes Both Lagom and Kubernetes are gaining popularity quite fast. Lagom is an open source framework for building reactive microservice systems in Java/Scala. And, Kubernetes (or K8s in short) is an open-source system for automating deployment, scaling, and management of containerized applications. Together they make an excellent stack for developing Reactive microservices of production grade. We have already seen a lot of blogs on Lagom on this Continue Reading
Reading Time: 5 minutes What is Vault? Hashicorp Vault is a tool for managing our our secrets. It has support for multiple secrets and we can enable access to both humans and machines separately. Secrets can be stored, dynamically generated, and in the case of encryption, keys can be consumed as a service without the need to expose the underlying key materials. In terms of its architecture, it has Continue Reading
Reading Time: 7 minutes Jenkins – Introduction Jenkins is an open-source automation server that is used to automate all sorts of tasks related to building, testing, and delivering, or even deploying software. CD pipelines refers to the process of getting the application / software from the source control and delivering it to end-users.Few of the features of Jenkins that make it the first choice for a user are : Continue Reading
Reading Time: 3 minutes These days, digital transformation is grabbing a lot of attention that trumpet how it’s radically changing customer behaviours. This typically means that IT departments have to deliver new features faster even in the face of more demanding requirements for availability (24/7) and security. DevOps is one such important facet of digital transformation and promises to do exactly, by fostering a high degree of collaboration across the Continue Reading
All Categories Brochure Case Study Datasheet eBook Infographic PDF Video Webinar All Technologies Akka AI/ML Big Data Cloud Native Data Science Data Warehouse DevOps Digital Transformation Ensemble Learning Event Streaming Fast Data IoT Knime Lagom Machine Learning Microservices Reactive Application Scala Transfer Learning All Technologies AI-ML-for-business-decision-making – Don’t rely on guesswork for arriving at important strategic decisions for your business. Let AI empower you with Continue Reading
