Google Cloud Pub/Sub: Asynchronous Messaging Service

pubsub gcp
Reading Time: 3 minutes

Introduction

Google Cloud Pub/Sub is reliable, scalable, fully-managed asynchronous messaging service for exchanging event data among applications and services. By decoupling senders and receivers, it allows for secure and highly available communication between independently written applications. Google Cloud Pub/Sub delivers low-latency/durable messaging, and is commonly used by developers in implementing asynchronous workflows, distributing event notifications, and streaming data from various processes or devices.

Pub/Sub Core Components

  • Topic: A named resource which acts as an alias to which the publishers send messages.
  • Publisher: An application that creates and sends a message to a Pub/Sub topic.
  • Subscriber: An application that registers itself with the desired topic to receive the appropriate messages
  • Subscription: It represents a stream of messages from a single, specific topic, which is to be delivered to the subscribing application.
  • Message: It is a combination of data and other attributes which is sent by a publisher to a topic and is delivered to the subscribers at the end.
  • Message attribute: Depending on the receiver’s language, the publisher can define a key-value pair for the message.
  • Acknowledgement (“ack”): A signal sent by a subscriber to Pub/Sub after it has received a message successfully.
  • Push and pull: The two message delivery methods. A subscriber receives messages either by Pub/Sub pushing them to the subscriber’s chosen endpoint, or by the subscriber pulling them from the service.

Data flow in Pub/Sub

Data flow in pub/sub service is mainly dependent on the publisher application, topic and a subscribing application.

A publisher application that can be any application which makes a HTTPs requests to the web services, app engine, publishes the messages to the topic.

A subscribing application at subscribing end, subscribes to that topic. A subscriber can be a pull subscriber or a push subscriber. A pull subscriber requests a Hypertext Transfer Protocol Secure request to the google APIs and a push subscriber accepts the data which is streaming over HTTPs.

Pub/Sub sends out the messages from the topic to all of its subscribers and waits for the acknowledgement. Once a message is received, it is acknowledged by the subscriber, it is deleted from the queue of the subscription message.

Enough Talk – Let’s Get Hands-on

Create a topic:

  1. In the google cloud console, navigate to the Pub/Sub topic page.
  2. Click create topic and enter a unique name.

Once the topic is created, you can export it to the GCP cloud storage such as the BigQuery or Cloud storage.

Subscribe to the Topic:

  1. Once a Pub/Sub topic is created, add a subscriber to this. Go to Subscriptions and then Click: Create subscriptions.
  2. Name the subscription and select one of the options as push or pull and then click the create button.

Publish a message to the topic:

To publish a message to the topic. For the topic we just created, click on the Publish a message button and enter a suitable message in the message field and finally, publish it with the help of the publishing button.

Pull the data published:

Go to the subscriptions and in the messages section, click on pull to view the messages.

Use cases

  • Event Ingestion and delivery for streaming analytics pipelines.
  • Real-time personalization in gaming.
  • Fast reporting, targeting and optimisation in advertising and media.
  • Syndicating market-related data streams for financial services.
  • Notifications can be distributed.
knoldus

Written by 

I am an DevOps engineer having experience working with the DevOps tool and technologies like Kubernetes, Docker, Ansible, AWS cloud, prometheus, grafana etc. Flexible towards new technologies and always willing to update skills and knowledge to increase productivity.