Reading Time: 2 minutes Have you ever thought about the Push vs Pull approach for the system, which one suits or solves which problem? Another Question why did Kafka choose Pull over Push design for Consumers? Before talking about the Kafka approach, whether the Broker should push the data to consumer or consumer should pull from Kafka? Let’s first understand both of the approaches, as each one has its Continue Reading
Reading Time: 3 minutes By now you must be familiar with KSQL and how to get started with it. If not, check out the Part1 KSQL: Getting started with Streaming SQL for Apache Kafka of this series. In this blog, we’ll move one step forward to get an understanding of the Dual streaming model to see what abstractions does KSQL use to process the data. All the data that we Continue Reading
Reading Time: 3 minutes tuning distributed kafka cluster to attain consistency along with high availability of the system.
Reading Time: 5 minutes A few days back, we got the requirement that we need to monitor a Kafka stream application using JMX metrics. We looked for the solution and reached to the conclusion which we will discuss in this blog. I will try to explain each and every component of the solution along with the setup and the integration part of the whole system. Proposed solution: Service (application) exposes Continue Reading
Reading Time: 4 minutes Apache Kafka is an open sourced distributed streaming platform used for building real-time data pipelines and streaming applications. It is horizontally scalable, fault-tolerant, wicked fast, and runs in production in thousands of companies. Before the introduction of Apache Kafka, data pipleines used to be very complex and time-consuming. A separate streaming pipeline was needed for every consumer. You can guess the complexity of it with Continue Reading
Reading Time: 4 minutes In our previous blog, we had a look at what Akka streams are and how they are different from the other streaming mechanisms we have. In this blog, we will be taking a little step forward into the world of Akka Streams. In order to work with Akka streams, we need a mechanism to connect Akka Streams to the existing system components. That is where Alpakka Continue Reading
Reading Time: 4 minutes Kafka’s exactly once semantics was recently introduced with the version 0.11 which enabled the message being delivered exactly once to the end consumer even if the producer retries to send the messages. This major release raised many eyebrows in the community as people believed that this is not mathematically possible in distributed systems. Jay Kreps, Co-founder on Confluent, and Co-creator of Apache Kafka explained its Continue Reading
Reading Time: 2 minutes Have you ever tried to access Kafka running on mesos on top of DCOS, and figure out that you end up with no latest Kafka client image in the docker hub? I have uploaded a new image with the Latest Kafka Stable Version 2.0.0, and one can get it easily – docker pull piyushdocker/kafka-client-2.0.0-image If you want to create your own image with any other Continue Reading
Reading Time: 6 minutes In a Knolx session organized by Knoldus, we discussed the idea of following Saga Patterns. For that to be more accessible, I’d like to share the session with the help of this blog. Service-oriented architecture has given us enough advantages to be a predominant architecture in our Industry, but it can’t be all sunshine and rainbows. There are use cases where monoliths are not only Continue Reading
Reading Time: 4 minutes “If you can dream it, you can do it. ” -Walt Disney For some coding is a job. For some, it is an exercise. But for us folks here at Knoldus, it’s a Passion. So in order to bring a twist in the daily work schedule, Knoldus held an overnight Hackathon competition within the organization on 18th May 2018 which presented an opportunity for every Knolder(employees Continue Reading
Reading Time: 4 minutes Apache Kafka v0.10 introduced a new feature Kafka Streams API – a client library which can be used for building applications and microservices, where the input and output data can be stored in Kafka clusters. Kafka Streams provides state stores, which can be used by stream processing applications to store and query data. Every task in Kafka Streams uses one or more state stores which Continue Reading
Reading Time: 3 minutes With the advent of streaming frameworks like Spark Streaming, Flink, Storm etc. developers stopped worrying about issues related to a streaming application, like – Fault Tolerance, i.e., zero data loss, Real-time processing of data, etc. and started focussing only on solving business challenges. The reason is, the frameworks (the ones mentioned above) provided inbuilt support for all of them. For example: In Spark Streaming, by just adding Continue Reading
Reading Time: 2 minutes How to deploy a Lagom Service without ConductR? This question has been asked and answered by many, on different forums. For example, take a look at this question on StackOverflow – Lagom without ConductR? Here the user is trying to know whether it is possible to use Lagom in production without ConductR or not. To which the best answer that came up was – “Yes, it is Continue Reading
Aakash Jain
Abhishek Baranwal
Akshansh Jain
Alok Jha
Anmol Sarna
Anuj Saxena
Ayush Hooda
Ayushi Hasija
Ayush Mishra
Ayush Prashar
Ayush_Singhal
Azmat Hasan
Bhavya Aggarwal
Bhawna Sharma
Piyush Rana
Pawan Bisht
Shubham Verma
Charmy Garg
Harshit Daga
Rudar
Deepak Mehra
Deepankar
Divyansh Jain
Ayush Tiwari
girdharshubham
Girish Bharti
Harmeet Singh(Taara)
Himani Arora
Himanshu Gupta
Himanshu Rajput
Sonu Mehrotra
Nancy Jain
Jashan Goyal
Jasmine Kaur
Jatin Demla
Juoko Virtanen
Rachel Jones
JustinB
Jyoti Sachdeva
Kamlesh prajapati
Rishi Khandelwal
Manish Mishra
Vikas Hazrati
Kritika Khandelwal
Sahil Sawhney
Megha Jatana
Mahesh Chand
Mayank Bairagi
Miral Gandhi
Mukesh Kumar
Narayan Kumar
Neelam Lakra
Neelu Aggarwal
neerajchugh
Neha Bhardwaj
Niharika Datta
Nitin Arora
Pallavi Singh
Prabhat Kashyap
Pranjut Gogoi
Prashant Sharma
Priyanka Thakur
Rahul Khanna
Ramandeep
Ram Indukuri
Randhir kumar
Joseph Ross
Ruchika Dubey
Rajeev Thakur
Shivangi Gupta
Shubham Dangare
shubhamgoyal01
Sidharth Khattri
Sachin Slathia
Sourabh Verma
GauravShukla
Shubhrank Rastogi
Sudeep James Tirkey
Nitin Aggarwal
kunal sethi
Vandana Yadav
Vidisha Gupta
Vinay Kumar
Vinisha Sharma
Yatharth Sharma
Knoldus is the world's largest pure-play Scala and Spark company. We modernize enterprise through cutting-edge digital engineering by leveraging Scala, Functional Java and Spark ecosystem. Our mission is to provide reactive and streaming fast data solutions that are message-driven, elastic, resilient, and responsive.
+(1) 647-467-4396
hello@knoldus.com
