Unit I

Introduction to Deep Neural Networks: Feed forward Neural networks. Gradient descent and the back propagation algorithm, Intuition of Neural Networks Loss functions, Optimization, Unit saturation, aka the vanishing gradient problem, and ways to mitigate it.

Unit II

Convolutional Neural Networks, Training Neural Networks, Understanding Neural Networks Through Deep Visualization and Recurrent Neural Networks: Architectures, convolution / pooling layers, LSTM, Encoder Decoder architectures.

Unit III

Deep Unsupervised Learning: Auto encoders (standard, sparse, denoising, contractive, etc), variational auto encoders, denoising encoders, Adversarial Generative Networks.

Unit IV

Deep Belief Networks: Energy Based Models, Restricted Boltzmann Machines, Sampling in an RBM. Applications of deep neural networks in handwritten character recognition, face recognition, semantic web, social networks.

This course builds from a one node neural network to a multiple feature, multiple output neural networks. After an understanding of how neural networks work and the parameters that control deep learning systems, building of deep learning neural networks and various applications.

Course Objectives

• Understand the context of neural networks and deep learning
• Know how to use a neural network
• Understand the data needs of deep learning
• Have a working knowledge of neural networks and deep learning.
• Explore the parameters for neural networks

Course Outcomes

CO-PO Mapping

*CA – Can be Quizzes, Assignment, Projects, and Reports, and Seminar

1. Domingos, Pedro. “A few useful things to know about machine learning.” Communications of the ACM 55.10 (2012): 78-87.
2. Li Fei-Fei (Stanford), Rob Fergus (NYU), Antonio Torralba (MIT), “Recognizing and Learning Object Categories” (Awarded the Best Short Course Prize at ICCV 2005).
3. Baydin, AtilimGunes, Barak A. Pearlmutter, and Alexey AndreyevichRadul. “Automatic differentiation in machine learning: a survey.” arXiv preprint arXiv:1502.05767 (2015).
4. Bengio, Yoshua. “Practical recommendations for gradient-based training of deep architectures.” Neural Networks: Tricks of the Trade. Springer Berlin Heidelberg, 2012. 437-478.
5. LeCun, Yann A., et al. “Efficient backprop.” Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012. 9-48.
6. Simonyan, Karen, Andrea Vedaldi, and Andrew Zisserman. “Deep inside convolutional networks: Visualising image classification models and saliency maps.” arXiv preprint arXiv:1312.6034 (2013). 7. Zeiler, Matthew D., and Rob Fergus. “Visualizing and understanding convolutional networks.” Computer vision–ECCV 2014. Springer International Publishing, 2014. 818- 833.
7. Springenberg, Jost Tobias, et al. “Striving for simplicity: The all convolutional net.” arXiv preprint arXiv:1412.6806 (2014).
8. Russakovsky, Olga, et al. “Imagenet large scale visual recognition challenge.” International Journal of Computer Vision 115.3 (2015): 211-252.