Syllabus
Unit 1
Perceptrons – classification – limitations of linear nets and perceptrons – multi-Layer Perceptrons (MLP); Activation functions – linear, softmax, tanh, ReLU; error functions; Feed-forward networks – Backpropagation – recursive chain rule (backpropagation); Learning weights of a logistic output -Loss functions – learning via gradient descent; Optimization – momentum method; Adaptive learning rates – RMSProp – mini-batch gradient descent; Bias-variance trade off – Regularization – overfitting – inductive bias – drop out – generalization.
Unit 2
Convolutional Neural Networks – Basics and Evolution of Popular CNN architectures; CNN Applications: Object Detection and Localization, Face Recognition, Neural Style Transfer
Recurrent Neural Networks – GRU – LSTM – Transformers Networks; Applications: NLP and Word Embeddings, Attention Models,
Unit 3
Restricted Boltzmann Machine, Deep Belief Networks, Auto Encoders and Applications: Semi-Supervised classification, Noise Reduction, Non-linear Dimensionality Reduction; Introduction to GAN – Encoder/Decoder, Generator/Discriminator architectures; Challenges in NN training – Data Augmentation – Hyper parameter Settings; Transfer Learning – Developing and Deploying ML Models (e.g., Tensor Flow/PyTorch)
Objectives and Outcomes
Course Objectives
- This course provides an introduction to deep neural network models and explores applications of these models.
- The course covers feedforward networks, convolutional networks, recurrent and recursive networks, as well as general topics such as input encoding and training techniques.
Course Outcomes
CO1: Understand the learning components of neural networks and apply standard neural network models to learning problems.
CO2: Analyze the learning strategies of deep learning – regularization, generalization, optimization, bias and variance.
CO3: Analyze regular deep learning models for training, testing and validation in standard datasets.
CO4: Apply neural networks for deep learning using standard tools.
CO5: Understand the mathematics for Deep learning.
CO-PO Mapping
| PO/PSO |
PO1 |
PO2 |
PO3 |
PO4 |
PO5 |
PO6 |
PO7 |
PO8 |
PO9 |
PO10 |
PO11 |
PO12 |
PSO1 |
PSO2 |
| CO |
| CO1 |
3 |
2 |
2 |
3 |
– |
– |
– |
– |
– |
– |
– |
– |
3 |
2 |
| CO2 |
3 |
2 |
3 |
2 |
2 |
– |
– |
– |
– |
– |
– |
– |
3 |
2 |
| CO3 |
3 |
2 |
3 |
2 |
3 |
– |
– |
– |
– |
– |
– |
– |
3 |
2 |
| CO4 |
3 |
1 |
2 |
1 |
2 |
– |
– |
– |
– |
– |
– |
– |
3 |
2 |
| CO5 |
3 |
1 |
2 |
1 |
– |
– |
– |
– |
– |
– |
– |
– |
3 |
2 |
Evaluation Pattern
Evaluation Pattern: 70:30
| Assessment |
Internal |
External |
| Midterm |
20 |
|
| Continuous Assessment – Theory (*CAT) |
10 |
|
| Continuous Assessment – Lab (*CAL) |
40 |
|
| **End Semester |
|
30 (50 Marks; 2 hours exam)
|
*CAT – Can be Quizzes, Assignments, and Reports
*CAL – Can be Lab Assessments, Project, and Report
**End Semester can be theory examination/ lab-based examination/ project presentation
Text Books / References
Textbook(s)
Ian Goodfellow, Yoshua Bengio and Aaron Courville. “Deep Learning”, MIT Press, Second Edition; 2016.
Reference(s)
Koller, D. and Friedman, N. “Probabilistic Graphical Models”. MIT Press;2009.
Hastie, T., Tibshirani, R. and Friedman, J. “The Elements of Statistical Learning”. Second edition, Springer; 2009.
Bishop, C. M. “Neural Networks for Pattern Recognition”. Oxford University Press;1995.
Aggarwal, Charu C. “Neural networks and deep learning.” Springer, 2018.
