Knoldus Blogs

Google Cloud Platform: Migrating Data to New Schemas on Big Query Using Dataflow

February 1, 2023February 14, 2023AI, GCP, ML, AI and Data EngineeringApache Beam, BigQuery, Data, Data Migration, Dataflow, Google Cloud, Pipelines, python

Reading Time: 6 minutes Migrating data on Google Cloud BigQuery may seem like a straightforward task, until you run into having to match old data to tables with different schemas and data types. There are many approaches you can take to moving data, perhaps using SQL commands to transform the data to be compatible with the new schema. However, SQL has limitations as a programming language, being a query-centric Continue Reading

Publish and receive messages in Google Cloud Pub/Sub

November 28, 2022November 30, 2022GCP, JavaFunctional Programming, Gcp

Reading Time: 4 minutes What is Pub/Sub? Cloud Pub/Sub is a message queuing service that allows you to exchange messages between applications and microservices. It’s a scalable, durable, and highly available message-passing system that helps you build event-driven architectures. In this tutorial, we will show you how to use the Google Cloud console to publish and receive messages in Cloud Pub/Sub. We will also provide some tips on how Continue Reading

Digital engineer working on virtual blueprint building

Big Query DML Statements Technique: A small Guide

August 8, 2022August 10, 2022AI, Big Data and Fast Data, Database, GCP, ML, AI and Data Engineeringbig query, Database

Reading Time: 3 minutes In this blog we are going to learn about some of the key Big Query DML statements. Data plays an integral part in any organisation. With the data-driven nature of modern organisations, almost all businesses and their technological decisions are based on the available data. Let’s assume that we have an application distributed across multiple servers in different regions of a cloud service provider, and Continue Reading

gray laptop computer showing html codes in shallow focus photography

How to Create our first Cloud Function in Google Cloud Platform

June 28, 2022June 28, 2022Cloud, GCP, Studio-DevOps

Reading Time: 4 minutes Hello Readers! In this blog we we are going to see How to create our first Cloud Function in Google Cloud Platform. Let’s See first what is google cloud function? Let’s get started with Google Cloud Functions Cloud Function Cloud Functions will be responsible for scaling up your code to handle the millions of requests come in. You don’t really need to worry about where this Continue Reading

Spring Cloud GCP – Cloud Spanner

June 27, 2022June 28, 2022GCP, Java, Spring Cloud GCP, Tech BlogsGcp, java, Spring Cloud GCP

Reading Time: 5 minutes Spring framework is very popular and most widely used across the world. It is also one of the oldest frameworks in Java. The Spring Cloud GCP project makes the Spring Framework a first-class citizen of the Google Cloud Platform (GCP). Cloud Spanner is a fully managed relational database with unlimited scale, strong consistency, and up to 99.999% availability. It is like the big brother of Cloud Continue Reading

Spring Cloud GCP and Cloud Vision: A New Record Chain

June 22, 2022June 24, 2022GCP, Java, java11, Spring Cloud, Spring Cloud GCP, Tech BlogsGoogle cloud Vision, java, Spring Cloud GCP

Reading Time: 3 minutes Introduction In this blog post, we will explore Spring Cloud GCP’s support for Spring Boot and Cloud Vision. Spring Cloud GCP Spring Cloud GCP is a suite of libraries that help you build and manage applications on the Google Cloud Platform. Applications built using Spring Cloud GCP have the following characteristics: They are resilient, with a robust design and fault tolerance features like Circuit Breaker Continue Reading

Spring Cloud GCP – Send Pub/Sub Messages and its implementation with SpringBoot

June 20, 2022June 21, 2022GCP, Spring Cloud, Spring Cloud GCP, Tech BlogsSpring Cloud GCP

Reading Time: 5 minutes Overview Pub/Sub is an asynchronous communications system that is both dependable and scalable. The service is based on a Google infrastructure component that has been used by numerous Google products for over a decade.This infrastructure is used by Google products such as Ads, Search, and G-mail to send over 500 million messages per second, totaling over 1TB/s of data. Basics of a Publish/Subscribe Service Pub/Sub Continue Reading

Make better decisions with Google Cloud Document AI

June 20, 2022June 21, 2022AI, Artificial intelligence, GCP, Java, ML, AI and Data Engineering, python, python, Studio-ScalaAI, AMP, Applicatio, Classification, dabl, Document AI, f, Google Cloud, text processing, thre, types, UI, understanding

Reading Time: 3 minutes Nearly all business processes today begin, include or end with a document. Most companies are sitting on the document goldmine. Thinking of which some are PDFs, emails, customer feedback, patents, contracts, technical documents, sensitive documents, HR files and the list goes on. These documents are only going to grow with time. Making sense of each document is difficult since a lot of these documents are Continue Reading

Spring Cloud Pub/Sub

June 16, 2022June 21, 2022Cloud, GCP, Java, kafka, spring boot, Spring Cloud, Spring Cloud GCP, Tech Blogs

Reading Time: 2 minutes Cloud Pub/Sub Google Cloud Pub/Sub allows services to communicate asynchronously, with latency on the order of 100 milliseconds. Pub/Sub is used for streaming analytics and data integration pipelines to ingest and distribute data. It is equally effective as a messaging- oriented middleware for service integration or as a queue to parallelised tasks. Pub/Sub enables to create systems of event producers and consumers, called publishers and subscribers. Publishers communicate Continue Reading

Google Cloud Pub/Sub: Asynchronous Messaging Service

June 10, 2022June 13, 2022GCP, Studio-DevOps, Tech Blogs

Reading Time: 3 minutes What Is Cloud Pub/Sub? Cloud Pub-Sub is a real-time messaging service offered by Google for applications to publish and subscribe to events. Google Cloud Pub/Sub provides messaging between applications and is designed to provide reliable, many-to-many, asynchronous messaging between applications. Publisher applications can send messages to a “topic” and other applications can subscribe to that topic to receive the messages. Pub/Sub= Publisher-Subscriber One can publish the message Continue Reading

black samsung tablet display google browser on screen

Spring Cloud GCP- Cloud Pub/Sub Receiving Messages

June 2, 2022June 3, 2022GCP, Java, Spring Cloud GCP, Tech BlogsGcp, java, Spring Cloud GCP

Reading Time: 3 minutes This is the second blog in the series Spring Cloud GCP- Cloud Pub/Sub. If you have not gone through the first part please navigate to Spring Cloud GCP- Cloud Pub/Sub Sending Messages. In this blog, we will create an application to receive messages that are sent by the sender application that we created in the last blog. We will use Google Cloud Pub/Sub as the Continue Reading

Spring Cloud GCP- Cloud Pub/Sub Sending Messages

May 27, 2022May 30, 2022GCP, Google Cloud Storage, Spring, Spring Cloud, Spring Cloud GCP, Tech BlogsSpring Cloud GCP

Reading Time: 4 minutes Spring framework is very popular and most widely used across the world. It is also one of the oldest frameworks in Java. The Spring Cloud GCP project makes the Spring Framework a first-class citizen of the Google Cloud Platform (GCP). In this blog series, we will create an application to send messages and another for receiving messages. We will use Google Cloud Pub/Sub as Continue Reading

Google Cloud Virtual Private Cloud – VPC

May 27, 2022May 30, 2022GCP, Google Cloud Storage, Java, Studio-DevOps, Tech Blogs

Reading Time: 3 minutes Virtual Private Cloud – VPC provides networking functionality for the cloud-based resources and services that is global, scalable, and flexible. What is the Virtual Private Cloud – VPC network in GCP? A VPC network is a global resource that consists of a list of regional virtual subnetworks (subnets) in data centers, all connected by a global wide area network. VPC networks are logically isolated from each Continue Reading