Login/Sign Up

Deep Learning

At first, Deep Learning came out as just a fancy term for training an Artificial Neural Network (ANN) with multiple hidden layers which is called Deep Neural Network (DNN). After this architecture seemed promising the research area became popular and the meaning of the term changed. Now, Deep Learning is the collective name for techniques and methods to train DNNs in supervised, unsupervised and semi-supervised manner.

Deep Learning methods include not only "learning" but also weight initialization methods. Because deep architectures have the Vanishing Gradient problem, there are some regularization and weight initialization methods specific for these DNNs. They also have the problem of high number of local minima. Due to their deep architectural nature DNNs have high number of free parameters. This leads to very high probability of "overfitting" or getting stuck in a local minimum. These problems are handled using methods such as Simulated Annealing, Dropout and Unsupervised Feature Learning.

One of the biggest steps for Artificial Intelligence. Although it got popular recently due to newly discovered algorithms and new powerful computers, deep learning has been on the field since 1980's. However, since computers were not very powerful and training was inefficiently slow it wasn't considered as a practical application in AI field.

Recent groundbreaking advancements in image and speech recognition and self driving cars are all here thanks to deep learning.

More specific examples of deep learning algorithms include Restricted Boltzmann Machine, Denoising Auto Encoder, Convolutional Neural Networks and Recurrent Neural Networks.

In some expermental classification models a Support Vector Machine with a Gaussian Kernel is used as a final layer and results are pretty good. Also there is Recursive Neural Tensor Network which is used for parsing sentences and sentamental analysis.