Concept Learning: The stepping stone towards Machine Learning with Find-S

Posted on August 16, 2017 by

From our previous blog, we came across what awesome stuff a machine can do with machine learning and what all math stuff is required before you take a deep dive into machine learning. Now we all know the prerequisites for machine learning, so let’s start the journey towards machine learning with small but effective steps towards awesomeness.

Most of us always wonder how machines can learn from data, and predict future based on the available information considering facts and scenarios. Today we are living in an era where most of us are working globally on big data technologies having great efficiency and speed. But having a huge amount of data only is not the complete solution and optimal use of data until we are able to find patterns out of it and use those patterns to predict future events and identify our interest specific solutions.

girish1

Continue reading →

Advertisements
Posted in machine learning, Scala | Tagged , , , , , , | 1 Comment

Distributed Ledger – the Technology behind Blockchain

Posted on August 16, 2017 by

Distributed Ledger Technology is the concept behind the famous Blockchain technology. Let us see some of the key features of Distributed ledger, (which are all present in Blockchain as well) and how it is different from Blockchain.

In the conventional ledger systems, there is a central authority that keeps a single copy of the all the transactional records in the form of a physical or digital ledger. The authority itself holds the responsibility of maintaining and modifying the data.

Distributed ledgers on the other hand, are not dependent on any central administration or any central storage. A Distributed ledger is a database consensually shared, replicated and synchronized across multiple nodes in a peer-to-peer network. Continue reading →

Posted in Blockchain | Tagged , | Leave a comment

Getting an insight of Blockchain

Posted on August 16, 2017 by

Hi all,

blockchain

Blockchain

In today’s blog, I will be getting you acquainted with the Blockchain, its definition, inception and everything you need to know as a novice developer or user. A many of us have lately been getting very familiar with the term Bitcoin and have started to visualise it correctly as a digital currency. So getting into a bit more detail, the technology behind it is blockchain. And please keep into consideration that Bitcoin and blockchain are not interchangeable terms. The term Bitcoin is used to denote three things first, the underlying blockchain technology, second, the protocol that runs over the blockchain technology and third, it denotes a digital currency. In this blog, I shall be specifically be talking about the technology blockchain. In the blog below, I shall briefly overgo the Blockchain 1.0, 2.0 and 3.0 just to convince you of its exponential potential and categorization.

Introduction

The Blockchain originated from an innovation of designing an architecture for a new system of decentralized trustless transactions. Later on, this key innovation became its definition. The blockchain is the next big computation paradigm after the social and mobile networking. This paradigm could well be the one for this decade by establishing the “connected world of computing” by building over the blockchain cryptography. Continue reading →

Posted in Blockchain, Scala | Tagged , , , | Leave a comment

Having Issue How To Order Streamed Dataframe ?

Posted on August 14, 2017 by

A few days ago, i have to perform aggregation on streaming dataframe. And the moment, i apply groupBy for aggregation, data gets shuffled. Now the situation arises how to maintain order?

Yes, i can use orderBy with streaming dataframe using Spark Structured Streaming, but only in complete mode. There is no way of doing ordering of streaming data in append mode and update mode.

I have tried different ways to solve this issue. Like, if i go with spark structured streaming. I might sort the streamed data in batches but not across batches.

I started finding solutions with different technologies like Apache Flink, Apache storm etc. What i faced at the end is disappointment. 😦

A bit of light at the end of the tunnel

Luckily there is Apache Kafka Stream which provides the facility of accessing its StateStore.  Kafka Stream provides Processor API.

The low-level Processor API provides a client to access stream data and to perform our business logic on the incoming data stream and send the result as the downstream data. It is done via extending abstract class AbstractProcessor and overriding the init, punctuate,close and  process method which contains our logic. This process method is called once for every key-value pair.

Where the High-Level DSL provides ready to use methods with functional style, the low-level processor API provides you the flexibility to implement processing logic according to your need. The trade-off is just the lines of code you need to write for specific scenarios. For more information, refer the references.

Continue reading →

Posted in Apache Kafka, apache spark, big data, Scala, Spark, Streaming | Tagged , , , , , , , , , , | Leave a comment

Microsoft announced Coco Framework for blockchain networks

Posted on August 14, 2017 by

Microsoft

On August 10, 2017, Microsoft announced their enterprise-ready blockchain network, Coco Framework.

Blockchain technology is now not an unknown technology anymore, with each day passing the number of enterprises supporting and investing in blockchain is increasing.  There are lots of short coming in existing software like latency, performance, governance, etc.

Microsoft, one of the tech giants, investing a lot in blockchain technology.  They are trying to solve the short over comings of existing software and making blockchain more facilitate. Continue reading →

Posted in Blockchain | Tagged , , , , , , | 1 Comment

First step Towards Machine Learning

Posted on August 14, 2017 by

Machine-Learning-hero

The human pace is getting lazy day by day and wants an ease in their life. For that, we created machines, machines that are capable enough to take commands and perform tasks for us.
But what if machines can think and take decisions on their own……Sounds rubbish!!!
In this modern era of machines and technology, this thing is actually taking place. Big companies like Google and Facebook are actually doing these miracles…
Means, have you ever thought that while uploading pics on Facebook, the computer automatically detects faces and stats suggesting your friend’s name for tagging or have you ever thought while checking flights on google for a particular destination, how you start getting emails for flight related offers.

Continue reading →

Posted in machine learning, Scala | Tagged , , , | 1 Comment

Machine learning with Decision Trees

Posted on August 14, 2017 by

“Machine learning is the subfield of computer science that gives computers the ability to learn without being programmed.”    – Arthur Samuel, 1959

Machine learning is a buzzword in the technology world right now. It is fun, challenging, puzzling, and even a bit scary if you’re one of those people that believe robots will someday steal our jobs and rule the world. Whether we like it or not, we are surrounded by adaptive smart things that can fix some of our most common daily queries in a split of a second. Continue reading →

Posted in Scala | 3 Comments

Consensus Algorithms in Blockchain

Posted on August 13, 2017 by

blockchain-consensus

When we talk about the blockchain, the first thing that came up in our mind is the security and the security because of the blockchain consensus algorithm. Those who know about the blockchain know that we keep the ledger transactions synchronized across the network to ensure that ledgers only update when the appropriate participants approve transactions and that when ledgers do update, they update with the same transactions in the same order is called consensus. Here we will discuss the three different consensus algorithms.

Practical Byzantine fault tolerance Continue reading →

Posted in Blockchain | Tagged , , , , , , , | 1 Comment

Transforming to a New World with Blockchain Technology

Posted on August 13, 2017 by

The Blockchain technology is booming beyond belief. But apart from the definition or relating it to BITCOINS what else do we know about The Blockchain?

LandingPage_BlockChain

Reading articles regarding this, all I could summarize about Blockchain is that it is a-

  • Distributed and immutable (write once and read only) Database/System.
  • A public ledger of all distributed transactions.
  • Key features include Security, Integrity, Accessibility.
  • A well-known implementation of which is BITCOINS.

    Benefits of Blockchain Technology :

  • Increased time effectiveness due to real time transactions
  • Direct Transactions eliminate the overheads and intermediary costs
  • Reduced risk related to cyber crimes, frauds, and tampering
  • More transparent processes with a proper creation and tracking
  • Highly secure due to cryptographic and decentralized Blockchain protocols

Continue reading →

Posted in Blockchain, IOT, Scala | Tagged , , , , , , | 2 Comments

Self-Learning Kafka Streams with Scala – #2

Posted on August 13, 2017 by

In our previous blog – Self-Learning Kafka Streams with Scala – #1, we saw how to create a simple KStream in Scala. In this blog, we will see how to transform a KStream and create a new Stream from it.

But, before we get into the details of the KStream transformations, let’s take a look at the code:

Continue reading →

Posted in Apache Kafka, Scala | Tagged | Leave a comment