ML, AI and Data Engineering

Using Vertica with Spark-Kafka: Write using Structured Streaming

In two previous blogs, we explored about Vertica and how it can be connected to Apache Spark. The first blog in this mini series was about reading data from Vertica using Spark and saving that data into Kafka. The next blog explained the reverse flow i.e. reading data from Kafka and writing data to Vertica but in a batch mode. i.e reading data from Kafka Continue Reading

Using Vertica with Spark-Kafka: Writing

In previous blog of this series, we took a glance over the basic definition of Spark and Vertica. We also did a code overview for reading data from Vertica using Spark as DataFrame and saving the data into Kafka. In this blog we will be doing the reverse flow i.e. working on reading the data from Kafka as a DataFrame and writing that DataFrame into Continue Reading

MachineX: Genetic Algorithm

Genetic algorithm is based on the Charles Darwin famous principle of survival of the fittest, where the fittest of the individuals are given higher importance and are chosen for reproduction in order to produce children for the new generation. The process starts by selecting the fittest individuals from a population, who then produce offspring which inherit the characteristics of the parents. Since the parents already Continue Reading

MachineX: Evaluation Metrics for a Regression ML Model

In this blog post, we will quickly look at the various metrics to evaluate our regression models. But first, let us briefly discuss one of the best-known model evaluation approach we use which is Train-Test or also known as Train-Validation split. Train-Test Split: In this approach, we split the data into two parts known as Training set and Test set. The model is then trained Continue Reading

TensorFlow for deep learning Part 1

TensorFlow is a free and Open-Source Software library for dataflow and differentiable programming across a range of tasks. It is a symbolic math library and is also used for machine learning applications such as neural networks. It is used for both research and production at Google. TensorFlow was developed by the Google Brain team for internal Google use. Deep learning is a particular kind of Continue Reading

Do you really need Spark? Think Again!

With the massive amount of increase in big data technologies today, it is becoming very important to use the right tool for every process. The process can be anything like Data ingestion, Data processing, Data retrieval, Data Storage, etc. Today we are going to focus on one of those popular big data technologies i.e., Apache Spark. Apache Spark is an open-source distributed general-purpose cluster-computing framework. Spark Continue Reading

MachineX: SVM as Non-Linear Classifiers

In our previous blogs, we have already looked and had a higher level understanding of SVM and why to choose SVM over other classifiers. In this blog post, we will look at a detailed explanation of how to use SVM for complex decision boundaries and build Non-Linear Classifiers using SVM. The primary method for doing this is by using Kernels. In linear SVM we find Continue Reading

Protein Structure determination aided by Stochastic Search (Replica Exchange Monte-Carlo Method)

Introduction Proteins are large molecules, which occur in abundance in every single living organism. They carry out vital functions such as transporting oxygen, converting the food you eat into energy your body can use, and many more. Proteins are long chains of linked units called amino acids. There are 20 types of amino acids. Proteins fold into different shapes depending upon their sequence of amino Continue Reading

MachineX :k-Nearest Neighbors(KNN) for classification

In this blog, we are going to go through about one of the widely used classification algorithm called KNN (K-Nearest Neighbors). Since I started doing data science, I observed that most of the problems end up with classification model The main reason behind this biased property is, most of the analytic problems are based on decision making. For instance, to identify loan applicants as low, Continue Reading

Data processing using ML Supervised classification algorithm to find accuracy

Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed. Machine learning focuses on the development of computer programs that can access data and use it to learn for themselves. Type of machine learning Supervised learning Unsupervised Learning Reinforcement Learning In Supervised Learning, algorithms learn from labeled data. After understanding Continue Reading

MachineX: The alphabets of Artificial Neural Network – (Part 2)

If you are reading this blog, it is supposed that you have already done with Part 1 No???? Then visit to the previous blog The alphabets of Artificial Neural Network first and comeback here for an awesome knowledge about Neural network working. We got the basic understanding of neural network so let’s get into deep. Let’s understand how neural networks work. Once you got the Continue Reading

MachineX: The inevitable Principal Component Analysis

In this blog post, we will look at an interesting feature extraction technique of Machine Learning known as Principal Component Analysis (PCA). PCA is one of the powerful techniques in dimensionality reduction, in fact, the de facto standard for human face recognition. Let’s first understand what is dimensionality reduction Dimensionality Reduction As an example let’s say we have a data set with many-many features(which is Continue Reading

MachineX: The alphabets of Artificial Neural Network

In this blog, we will talk about Neural network which is the base of deep learning which gave machine learning and ultra edge in the current AI revolution. Let’s get started!!!!!! before diving into deep learning, let’s know – Why Deep Learning ??? Well, there are plenty of reason , few of them are: Deep learning is most popular than shallow level learning once you Continue Reading

Knoldus Pune Careers - Hiring Freshers

Get a head start on your career at Knoldus. Join us!