Course

Unit 1

Perceptrons – classification – limitations of linear nets and perceptrons – multi-Layer Perceptrons (MLP); Activation functions – linear, softmax, tanh, ReLU; error functions; Feed-forward networks – Backpropagation – recursive chain rule (backpropagation); Learning weights of a logistic output -Loss functions – learning via gradient descent; Optimization – momentum method; Adaptive learning rates – RMSProp – mini-batch gradient descent; Bias-variance trade off – Regularization – overfitting – inductive bias – drop out – generalization.

Unit 2

Convolutional Neural Networks – Basics and Evolution of Popular CNN architectures; CNN Applications: Object Detection and Localization, Face Recognition, Neural Style Transfer
Recurrent Neural Networks – GRU – LSTM – Transformers Networks; Applications: NLP and Word Embeddings, Attention Models,

Unit 3

Restricted Boltzmann Machine, Deep Belief Networks, Auto Encoders and Applications: Semi-Supervised classification, Noise Reduction, Non-linear Dimensionality Reduction; Introduction to GAN – Encoder/Decoder, Generator/Discriminator architectures; Challenges in NN training – Data Augmentation – Hyper parameter Settings; Transfer Learning – Developing and Deploying ML Models (e.g., Tensor Flow/PyTorch)

• This course provides an introduction to deep neural network models and explores applications of these models.
• The course covers feedforward networks, convolutional networks, recurrent and recursive networks, as well as general topics such as input encoding and training techniques.

• CO1: Understand the learning components of neural networks and apply standard neural network models to learning problems.
• CO2: Analyze the learning strategies of deep learning – regularization, generalization, optimization, bias andvvariance.
• CO3: Analyze regular deep learning models for training, testing and validation in standard datasets.
• CO4: Apply neural networks for deep learning using standard tools.
• CO5: Understand the mathematics for Deep learning.

CO-PO Mapping

• Ian Goodfellow, Yoshua Bengio and Aaron Courville. “Deep Learning”, MIT Press, Second Edition; 2016.

• Koller, D. and Friedman, N. “Probabilistic Graphical Models”. MIT Press;2009.
• Hastie, T., Tibshirani, R. and Friedman, J. “The Elements of Statistical Learning”. Second edition, Springer; 2009.
• Bishop, C. M. “Neural Networks for Pattern Recognition”. Oxford University Press;1995.
• Aggarwal, Charu C. “Neural networks and deep learning.” Springer, 2018.

Evaluation Pattern: 70:30

*CAT – Can be Quizzes, Assignments, and Reports
*CAL – Can be Lab Assessments, Project, and Report
**End Semester can be theory examination/ lab-based examination/ project presentation