This term “Deep Learning”, is on fire for past two decades. Every machine learning enthusiast wants to work on it and many big companies are already making an impact on Data Science field by exploring it e.g. Google Brain project from Google or DeepFace from Facebook.
The reason is simple, experts say and I quote “for most flavors of the old generations of learning algorithms … performance will plateau. … deep learning … is the first class of algorithms … that is scalable. … performance just keeps getting better as you feed them more data“.
So in this image, we can see with increment in the data for learning, the performance of other algorithms becomes constant but the performance of deep learning still increases. According to a talk given by Andrew Ng from Coursera, the core of deep learning is that we now have fast enough computers and enough data to train our algorithms.
Deep learning comes into focus where we have to consider very large data that can not be fully utilized by any other algorithms.
So we know now why Deep Learning is so much famous and is really effective than any other algorithm. But wait… do we really know what it is!!! Let’s try to find out
What is it??
In my previous blog, I explained the difference between Artificial Intelligence, Machine Learning, and Deep Learning. Now let’s find out more about Deep Learning.
Simple googling the keyword “Deep Learning” will yield you this definition.
Deep learning is a subset of machine learning in Artificial Intelligence (AI) that has networks which are capable of learning unsupervised from data that is unstructured or unlabeled. Also known as Deep Neural Learning or Deep Neural Network.
Deep Learning is an algorithm for machine learning (supervised). The base for Deep Learning is Artificial Neural Networks (ANN).
ANN or Artificial Neural Networks is an algorithm which mimics the working of a human brain to process information.
As we all know that the human mind is capable of processing such large amount of data as it deals with the real environment with lots of variable information and it processes that whole information to get some knowledge out of it. The human brain does this part using the neural networks. Neural Network implies the network of billions of neurons connected with each other. These neurons forms layers and these layers are then connected to each other to process the data through each layer for necessary computing.
ANN does the same, it is made of perceptrons which mimics the behavior of a neuron. These perceptrons then form layers and with these layers interconnected we get our Artificial Neural Network. Usually, we work with two layers or three layers (very rare) while working with ANN. We’ll get to know more about Artificial Neural Network in further blogs.
Now let’s come back to the topic. Deep Learning is just an extension of ANN. Deep Learning and ANN both work on perceptron and are made of multiple layers which consist of perceptrons. Then what is the difference between them?? The key difference here is that we go deeper in ANN. That means we use more layers than usual in ANN to achieve Deep Neural Networks and we call it Deep Learning.
Advantage of Layers
So we use more layers in a multi-layered Neural Network to make it Deep Neural Network. What benefit comes from that??
Let’s take an example of a typical image recognition. In a neural net, we may add the first layer which might learn simple components of an image, like edges. Adding another layer will make our net learn more complicated features that are combinations of edges, like shapes etc. and after that one more layer can learn combinations of shapes, for example it could learn face like features if performing face detection, or it might learn components of vehicles (like wheels, headlights etc.) in a vehicle recognition task. Hence, with each layer, we are adding more computation power to our network.
So Deep neural networks are neural networks which may process more complex data to find out more information from it.
Why call it deep learning?
ANN was there in machine learning world since 1950. Why it got popular only recently and is now being used by the name deep learning instead of ANN??
A few reasons were lack of input data, lack of processing power and using the algorithm inefficiently.
In a co-authored article in Science titled “Reducing the Dimensionality of Data with Neural Networks”, Geoffrey Hinton, a pioneer in the field of artificial neural networks, says “It has been obvious since the 1980s that backpropagation through deep autoencoders would be very effective for nonlinear dimensionality reduction, provided that computers were fast enough, data sets were big enough, and the initial weights were close enough to a good solution. All three conditions are now satisfied.” So to differentiate between the old neural network and new and powerful feedforward neural networks, the term Deep Learning is used.
Conclusion
Now we know that the Deep Learning is just very huge Artificial Neural Networks which is capable to use much more volume of data to learn with increasing performance. So the requirement here is the ANN. We’ll try to understand ANN in the following blog.
I hope it cleared the definition of Deep Learning for you and helped you understand more about it.
References :
https://en.wikipedia.org/wiki/Deep_learning
https://stats.stackexchange.com/a/152491
Reblogged this on Ramandeep.