Services

Go to Overview

 High performance systems Data strategy & Analytics Intelligence Driven Decisioning Platform strategy Blockchain Expert Services Academy, Audit & Consulting

Accelerators

Go to Overview

Platforms

KDP KDSP

Products

PremonR CodeSquad Studio9

TechHub

Akka Scala Rust Spark Functional Java Kafka Flink ML/AI DevOps Data Warehouse

Industries

Go to Overview

Travel

Retail Finance Healthcare Media and Publishing Consumer Internet Hi-tech & IOT

Insights

Go to Overview

 Case Studies Blogs Books Conferences & Events Resources OS contributions Webinars Knolx
CONTACT US

Stream Processing

Flink: Time Windows based on Processing Time

Apache Flink, Big Data and Fast Data, Flink, Java, ML, AI and Data Engineering, , , , , , , ,

Reading Time: 4 minutes In the previous blog, we talked about Flink’s windows operator, a heart of processing infinite streams. Generally in Flink, after specifying that the stream is keyed or non keyed, the next step is to define a window assigner. The window assigner defines how elements are assigned to windows. Flink provides some useful predefined window assigners like Tumbling windows, Sliding windows, Session windows, Count windows, and Continue Reading

Basic Anatomy of a Flink Program

Apache Flink, Big Data and Fast Data, Flink, Java, , , , , , ,

Reading Time: 3 minutes Hi Folks! Hope you all are safe in the COVID-19 pandemic and learning new tools and tech while staying at home. I also have just started learning a very prominent Big Data framework for stream processing which is  Flink. Flink is a distributed framework and based on the streaming first principle, means it is a real streaming processing engine and implements batch processing as a special case. In Continue Reading

Windows operator: Heart of processing infinite streams in Flink

Apache Flink, Big Data and Fast Data, Flink, Scala, Streaming, Streaming Solutions, , , , , ,

Reading Time: 3 minutes Apache Flink is an open-source, distributed, Big Data framework for stream and batch data processing. Flink is based on the streaming first principle which means it is a real streaming processing engine and implements batching as a special case. Flink is considered to have a heart and it is the “Windows” operator. It makes Flink capable of processing infinite streams quickly and efficiently. Windows split Continue Reading

Streaming from Kafka to PostgreSQL through Spark Structured Streaming

Apache Kafka, Apache Spark, Scala, , , , ,

Reading Time: 3 minutes Hello everyone, in this blog we are going to learn how to do a structured streaming in spark with kafka and postgresql in our local system. We will be doing all this using scala so without any furthur pause, lets begin. Setting up the necessities first: Dependencies Set up the required dependencies for scala, spark, kafka and postgresql. 2. PostgreSQL setup Lets start fresh by Continue Reading

Delta Lake To the Rescue

Apache Spark, Big Data and Fast Data, Java, python, Scala, Spark, Streaming, , , ,

Reading Time: 4 minutes Welcome Back. In our previous blogs, we tried to get some insights about Spark RDDs and also tried to explore some new things in Spark 2.4. You can go through those blogs here: RDDs – The backbone of Apache Spark Spark 2.4: Adding a little more Spark to your code In this blog, we will be discussing something called a Delta Lake. But first, let’s Continue Reading

Flinkathon: What makes Flink better than Kafka Streams?

Apache Flink, Apache Kafka, Big Data and Fast Data, cluster, Flink, Scala, Streaming, Streaming Solutions, , , , , , , ,

Reading Time: 2 minutes Initially, I would like you all to focus on a few questions before comparing the frameworks:1. Is there any comparison or similarity between Flink and the Kafka?2. What could be better in Flink over the Kafka?3. Is it the problem or system requirement to use one over the other? Before talking about the Flink betterment and use cases over the Kafka, let’s first understand their Continue Reading

Kafka Streams : More Than Just a Dumb Storage

Apache Kafka, Scala, , ,

Reading Time: 5 minutes Whenever we hear the word Kafka, all we think about it as a messaging system with a publisher-subscriber model that we use for our streaming applications as a source and a sink. So we can say that Kafka is just a dumb storage system that stores the data provided by a producer for a long time (configurable) and it can provide it to some consumer whenever Continue Reading

OUR OFFERINGS

High Performace System Data Strategy & Analytics Intelligence Driven Decisioning Platform Strategy Blockchain Expert Services Academy, Audit & Consulting KDP KDSP PremonR

COMPANY

About Knoldus Knolway Case Studies Testimonials Careers Tech Expertise Conferences and Events News Room Our Partners CSR

LEARN

Why Knoldus? Blog Resources KnolX Webinars and Videos Podcast Rust Times Innovation Labs Knoldus Tech Hub Open Source Contributions

CONNECT

Contact Us Engagement Model Follow us Here:
Subscribe to our newsletter

Partners

lightbend
databricks
confluent
docker
hashicorp
ibm
© 2020 Knoldus Inc. All Rights Reserved.
Privacy Policy | Sitemap