Reading Time: 4 minutes Introduction to Streaming Why do we really need streaming? We can understand the same with the following example. Suppose we have a list of integers and we want to get all the prime numbers and do some further computation on that. In the typical application, we will use ZIO.foreachParN But when we are using the above method we will have two problems:- One Is High Continue Reading
Reading Time: 2 minutes We all know that Apache spark is a Big data processing engine that works on the model of in-memory computation. When we are dealing with extensive data even if we are able to reduce the use of even 1 MB of memory per minute it will result in thousands of dollars per month. Hence it becomes essential to learn the spark best practices and optimization Continue Reading
Reading Time: 1 minute Hello,I am assuming here that the reader is familiar with joining two tables in SQL and joining two Dataframe in spark. In case not please see the info about joining two data frames https://www.datacamp.com/community/tutorials/joining-dataframes-pandas Why do we need Broadcast Join? Broadcast join in spark is preferred when we want to join one small data frame with the large one. the requirement here is we should Continue Reading
Reading Time: 2 minutes Introduction In today’s world protecting our APIs from unwanted hits is a must, In the upcoming 5 min, we will learn how to protect our backend database of API from unwanted hits/calls with the help of the Apigee gateway tool.For basics of Apigee go through https://blog.knoldus.com/getting-started-with-apigee/ We will discuss two policies RateLimiting and Cache Mechanism here Rate Limiting in APIGEE Let’s understand the rate-limiting policy, Continue Reading
