Reading Time: 4 minutes In this post, we are going to dive into the basics of the Avro Schema. We will create a sample avro schema and serialize it to a sample output file and also read the file as an example according to the avro schema. Intro to Avro Apache Avro is a data serialization system developed by Doug Cutting, the father of Hadoop that helps with data Continue Reading
Reading Time: 3 minutes Hello everyone, in this blog we are going to learn how to do a structured streaming in spark with kafka and postgresql in our local system. We will be doing all this using scala so without any furthur pause, lets begin. Setting up the necessities first: Dependencies Set up the required dependencies for scala, spark, kafka and postgresql. 2. PostgreSQL setup Lets start fresh by Continue Reading
Reading Time: 4 minutes Spark provides two types of serialization libraries: Java serialization and (default) Kryo serialization. For faster serialization and deserialization spark itself recommends to use Kryo serialization in any network-intensive application. Then why is it not set to default : Why Kryo is not set to default in Spark? The only reason Kryo is not set to default is because it requires custom registration. Although, Kryo is Continue Reading
Agnibhas Chattopadhyay
Atul Rawat
Kamlesh Mishra
Krishna Singh
Kuldeepak Gupta
Manas Kashyap
Megha Tyagi
Sonu Mehrotra
Suraj Saini
Abhinav Sharma
Addi
Aditi Varshney
Anjali Sharma
Ankur Thakur
Anshika Agrawal
Anuj Saxena
Ayush Hooda
Ayush Mishra
Azmat Hasan
Bhavya Aggarwal
Piyush Rana
Pawan Bisht
Chirag Agarwal
Harshit Daga
Deevisha Verma
Divyanshu Srivastava
etash2901
girdharshubham
Girish Bharti
gopal
Himanshu Upadhaya
Himanshu Chaudhary
Himanshu Gupta
Sonu Mehrotra
Jashan Goyal
Jasmine Kaur
Jitendra Kumar Singh
Juoko Virtanen
Rachel Jones
JustinB
Jyoti Jali
Kartik Tiwari
Manish Mishra
Vikas Hazrati
Komal Rajpal
Kritika Khandelwal
Pratik Kumar
kundankumarr
Sahil Sawhney
Lokesh Aggarwal
Megha Jatana
Mahesh Chand
Mansi Babbar
Mayank Bairagi
Mukesh Yadav
munandersingh
Muskan Gupta
Narayan Kumar
Neelu Aggarwal
Neha Bhardwaj
Pankaj Chaudhary
Peter Boyajian
Pinku Swargiary
Prabhat Kashyap
Praful Bangar
Priyanka Thakur
Purva Agrawal
Rachna Bulbule
Rahul Agarwal
Rahul Khanna
Ram Indukuri
Randhir kumar
Joseph Ross
Ruchika Dubey
Sakshi Gawande
Sanjana Aggarwal
Sanjoli Varshney
Rajeev Thakur
Shivani Mehrotra
Shivraj Singh
Shradha Saxena
Shubham Dangare
Sidharth Khattri
Sparsh Bhardwaj
GauravShukla
Shubhrank Rastogi
Swantika Gupta
Swapnil Gosavi
Umang Goyal
Upanshu Chaudhary
Vandana Yadav
Vidushi Bansal
Vinay Kumar
Vinisha Sharma
woodruffsw
Yamini Bansal
Yatharth Sharma
