Reading Time: < 1 minute Knoldus organized a Meetup on Friday, 7th April 2017 at 4:00 PM which was presented by Himani Arora and me(Prabhat Kashyap). Topics which were covered in this meetup: What is Stream processing Advantages of stream processing Type of stream processing What are KStreams Use cases of KStreams Overview of Kafka Connect Slides: Video Recording:
Reading Time: 2 minutes Kafka is a messaging broker system which facilitates the passing of messages between producer and consumer whereas Spark Structure streaming consumes static and streaming data from various sources like kafka, flume, twitter or any other socket which can be processed and analysed using high level algorithm for machine learning and finally pushed the result out to external storage system. The main advantage of structured streaming Continue Reading
Reading Time: 7 minutes There is a lot of buzz going on between when to use use spark, when to use flink, and when to use Kafka. Both spark streaming and flink provides exactly once guarantee that every record will be processed exactly once thereby eliminating any duplicates that might be available. Both provide very high throughput compared to any other processing system like storm, and the overhead of Continue Reading
Reading Time: 2 minutes If you are working on huge amount of data, you might have heard about Kafka. At a very high level, Kafka is a fault tolerant, distributed publish-subscribe messaging system that is designed for fast processing of data and the ability to handle hundreds of thousands of messages. What is Stream Processing Stream processing is the real-time processing of data continuously, concurrently, and in a record-by-record Continue Reading
Reading Time: 2 minutes What is Lagom? Simply put, it is a microservices tool. Lagom is a microservices framework that guides developers in moving from monolithic to a scalable, resilient microservice based systems. It is a platform delivering you not only the microservice framework but also a whole tool set for developing applications along with creating, managing and monitoring your services. According to Lightbend the focus should not be Continue Reading
Reading Time: 3 minutes Hello guy’s, today’s we conduct short interview with SMACK about its architecture and there uses. Let’s start with of some introduction. Interviewer: How would you describe your self ? SMACK: I am SMACK (Spark, Mesos, Akka, Cassandra and Kafka) and belongs to all open source technologies. Mesosphere and Cisco collaboration bundles these technologies together and create a product called Infinity. Which is used to solved Continue Reading
Reading Time: 3 minutes This post will show you how to write and read messages in Apache Avro format to/from Kafka. Instead of using with plain-text messages, though, we will serialize our messages with Avro. That will allow us to send much more complex data structures over the wire. Avro Apache Avro is a language neutral data serialization format. A avro data is described in a language independent schema. Continue Reading
Reading Time: < 1 minute Knoldus organized a half an hour session on 29 July 2016 at 4:00 PM. It covers a brief introduction to Apache Kafka Connect, giving insights about the benefits of kafka connect, its use cases. It also covers the motivation behind building Kafka Connect and an introduction to its architecture. Here is the video for the same.