Reading Time: 6 minutes In this blog, we will get the opportunity to come across what Kafka is, and explain how Kafka works from the inside out. How does it replicate data between nodes, what happens if replication fails, and how do consumers scale-out automatically? Insights of Apache Kafka Kafka is a statistics streaming system that permits builders to react to new activities as they arise in real-time. Kafka Continue Reading
Reading Time: 4 minutes Reactive programming is a programming paradigm that focuses on asynchronous and event-driven programming. Reactive programming is designed to handle streams of data and events, and it has become popular in recent years due to the rise of modern web applications and the need for real-time data processing. Java, being a popular language for enterprise applications, has several libraries and frameworks that support reactive programming. In Continue Reading
Reading Time: 3 minutes Kafka Connect Kafka Connect is an open-source component and framework to get it connected with external systems, including the databases. There are connectors that help to move huge data sets into and out of the Kafka system. Kafka Connect is only used to copy the streamed data, thus its scope is not broad. It executes as an independent process for testing and a distributed, scalable Continue Reading
Reading Time: 3 minutes Overview In this blog, we are going to discuss Kafka connect/connectors in detail. If you want a basic introduction to Kafka Connect then you can refer to this blog. Now we will understand concepts such as Kafka Connect, Kafka connectors, and Kafka Convertors. Kafka Connect Kafka Connect is a framework to stream data into and out of Apache Kafka®. The Confluent Platform ships with several built-in Continue Reading
Reading Time: 4 minutes Airflow dynamic DAGs can save you a ton of time. As you know, Apache Airflow is written in Python, and DAGs are created via Python scripts. That makes it very flexible and powerful (even complex sometimes). By leveraging Python, you can create DAGs dynamically based on variables, connections, a typical pattern, etc. This very nice way of generating DAGs comes at the price of higher Continue Reading
Reading Time: 5 minutes In my previous blog, I have discussed about the Introduction to the Apache Airflow. In this blog, we will learn how to create a DAG for Airflow that would define a workflow of tasks and their dependencies. What is DAG? First of all the question that comes to our mind is that what is this DAG .So in Airflow, a DAG – or a Directed Acyclic Graph – Continue Reading
Reading Time: 4 minutes What is Apache Airflow? Airflow is a platform to programmatically author, schedule and monitor workflows.These functions achieved with Directed Acyclic Graphs (DAG) of the tasks. It is an open-source and still in the incubator stage. It was initialized in 2014 under the umbrella of Airbnb since then it got an excellent reputation with approximately 800 contributors on GitHub and 13000 stars. The main functions of Apache Airflow is to schedule workflow, monitor Continue Reading
Reading Time: 4 minutes OOP Concept Object-Oriented Programming (OOP) concept is a programming methodology that uses objects and classes and is also used to implement real world scenarios. Most important concepts of OOP are as follows:- 1.Class :- A class is a user defined blueprint or prototype from which objects are created. It also represents the set of properties or methods that are common to all objects of one Continue Reading
