Knoldus Blogs

Streaming Kafka Messages to Google Cloud Pub/Sub

May 25, 2022May 25, 2022Apache Kafka, GCPkafka, pub-sub

Reading Time: 3 minutes In this blog post i present an example that creates a pipeline to read data from a single topic/multiple topics from Apache Kafka and write data into a topic in Google Pub/Sub. The example provides code samples to implement simple yet powerful pipelines.also provides an out-of-the-box solution that you can just ” compatiable.This consicutive example is build in Apache Beam.And it can be downloaded here.So, we hope you will find this Continue Reading

Google Cloud Pub/Sub: Asynchronous Messaging Service

March 3, 2022March 3, 2022Studio-DevOpsAsynchronous messaging, GCP services, pub-sub

Reading Time: 3 minutes Introduction Google Cloud Pub/Sub is reliable, scalable, fully-managed asynchronous messaging service for exchanging event data among applications and services. By decoupling senders and receivers, it allows for secure and highly available communication between independently written applications. Google Cloud Pub/Sub delivers low-latency/durable messaging, and is commonly used by developers in implementing asynchronous workflows, distributing event notifications, and streaming data from various processes or devices. Pub/Sub Core Continue Reading

Hands-on: Apache Kafka with Scala

February 25, 2019February 25, 2019Apache Kafka, Studio-Scalaapache, Apache Kafka, kafka, kafka consumer, kafka producer, pub-sub, scala

Reading Time: 4 minutes Apache Kafka is an open sourced distributed streaming platform used for building real-time data pipelines and streaming applications. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies. Before the introduction of Apache Kafka, data pipleines used to be very complex and time-consuming. A separate streaming pipeline was needed for every consumer. You can guess the complexity of it with Continue Reading