kafka

Spring Cloud Pub/Sub

Reading Time: 2 minutes Cloud Pub/Sub Google Cloud Pub/Sub allows services to communicate asynchronously, with latency on the order of 100 milliseconds. Pub/Sub is used for streaming analytics and data integration pipelines to ingest and distribute data. It is equally effective as a messaging- oriented middleware for service integration or as a queue to parallelised tasks. Pub/Sub enables to create systems of event producers and consumers, called publishers and subscribers. Publishers communicate Continue Reading

Apache Kafka Connect – Basic Introduction

Reading Time: 3 minutes We use Apache Kafka Connect for streaming data between Apache Kafka and other systems, scalably as well as reliably. Moreover, connect makes it very simple to quickly define Kafka connectors that move large collections of data into and out of Kafka. Kafka Connect collects metrics or takes the entire database from application servers into Kafka Topic. It can make available data with low latency for Continue Reading

Getting started with Confluent Hub

Reading Time: 4 minutes As we know, We use connectors to copy data between Apache Kafka and other systems that we want to fetch or send data to. We can download the connector from Confluent Hub. So, in this blog, we will see how we can set up Confluent Hub on our local system. And how we can run confluent services like Kafka, zookeeper, schema registry, etc, etc. The first primary step is, if we have a windows Continue Reading

Introduction to Kafka Connect/Connectors

Reading Time: 3 minutes What is Kafka Connect? Kafka Connect is an open-source component and framework to get Kafka connected with the external systems, including the databases. There are connectors that help to move huge data sets into and out of the Kafka system. Kafka Connect is only used to copy the streamed data, thus its scope is not broad.It executes as an independent process for testing and a Continue Reading

Security & SSL Setup in Confluent Kafka

Reading Time: 2 minutes What is SSL ? Secure Socket Layer (SSL) is a security protocol for the transport layer. In SSL Protocol data is divided into fragments. The fragments are compressed and encrypted Message Authentication Code (MAC) generated by algorithms like Secure Hash Protocol(SHA) and MD5(Message Digest) is appended. SSL is the predecessor of Transport Layer Security(TLS) . After encryption of data, finally, the SSL header is appended Continue Reading