Back close

Course Detail

Course Name Deep Reinforcement Learning
Course Code 26CS746
Program M. Tech. in Computer Science & Engineering
Credits 3
Campus Bengaluru, Coimbatore, Chennai, Nagercoil

Syllabus

Introduction to Reinforcement Learning: intelligent agents, environment, states, actions, rewards, episodes, return, and discount factor. Exploration and exploitation trade-off. Markov Decision Process (MDP): policy, value function, action-value function, and Bellman equations. Dynamic Programming approaches: policy iteration and value iteration. Model-free Reinforcement Learning: Monte Carlo methods, Temporal Difference (TD) learning, SARSA, Q-Learning, and function approximation techniques.

Introduction to Deep Reinforcement Learning (DRL): integration of deep learning and reinforcement learning. Deep neural networks for reinforcement learning. Deep Q-Networks (DQN): architecture, experience replay, target networks, and training process. Advanced value-based methods: Double DQN and Dueling DQN. Policy-based learning approaches. Actor-Critic architectures. Neural network architectures for DRL: Convolutional Neural Networks (CNNs) and Long Short-Term Memory (LSTM) networks. Training challenges, hyperparameter tuning, reward design, and performance optimization.

Advanced DRL algorithms: Proximal Policy Optimization (PPO), Deep Deterministic Policy Gradient (DDPG), and Soft Actor-Critic (SAC). On-policy and off-policy learning strategies. Introduction to Multi-Agent Reinforcement Learning (MARL): concepts, coordination, and challenges. Applications of Deep Reinforcement Learning in robotics, autonomous systems, games, healthcare, finance, recommendation systems, and other real-world domains. Case studies and analysis of DRL-based solutions.

Course Objectives

Pre-Requisite(s): Deep Learning

Course Type: Lab/Project

Course Objectives

  • To understand the fundamental concepts and algorithms of Reinforcement Learning and Deep Reinforcement Learning.
  • To apply Deep Reinforcement Learning techniques for solving complex sequential decision-making problems.
  • To develop and evaluate DRL-based solutions for real-world intelligent applications.

Course Outcomes

CO1 Understand the fundamental principles of Reinforcement Learning.
CO2 Apply classical Reinforcement Learning algorithms for sequential decision-making problems.
CO3 Implement and evaluate Deep Reinforcement Learning algorithms using deep neural networks.
CO4 Design and analyze Deep Reinforcement Learning-based solutions for real-world applications.

CO-PO Mapping

CO PO1 PO2 PO3 PO4 PO5 PO6
CO1 3 2 1 2 2 1
CO2 3 3 2 2 3 2
CO3 3 3 3 2 3 3
CO4 3 3 3 3 3 3

Evaluation Pattern

Evaluation Pattern: 70/30

Assessment Internal Weightage External Weightage
Midterm Examination 20  
Continuous Assessment (Theory) 20  
Continuous Assessment (Lab) 30  
End Semester   30

Note: Continuous assessment may include quizzes, tutorials, laboratory assessments, case studies, assignments, and other course-related evaluation components. The Mid-Term examination may be conducted as a two-hour written examination or a two-hour laboratory-integrated examination, as applicable. The End-Semester assessment may be conducted either as a two-hour written examination, a laboratory-integrated examination, or as a project presentation and evaluation, provided the project-based assessment is approved by the Department.

Text Books/ References

  1. Richard S. Sutton and Andrew G. Barto, Reinforcement Learning: An Introduction, 2nd Edition, MIT Press, Cambridge, MA, 2018.
  2. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, MIT Press, Cambridge, MA, 2016.
  3. Laura Graesser and Wah Loon Keng, Foundations of Deep Reinforcement Learning: Theory and Practice in Python, Addison-Wesley Professional, 2022.
  4. Maxim Lapan, Deep Reinforcement Learning Hands-On: Apply Modern Deep Reinforcement Learning Algorithms with PyTorch and Gym, 2nd Edition, Packt Publishing, Birmingham, UK, 2020.
  5. Sudharsan Ravichandiran, Hands-On Reinforcement Learning with Python: Master Reinforcement and Deep Reinforcement Learning Using OpenAI Gym and TensorFlow, Packt Publishing, Birmingham, UK, 2020.

DISCLAIMER: The appearance of external links on this web site does not constitute endorsement by the School of Biotechnology/Amrita Vishwa Vidyapeetham or the information, products or services contained therein. For other than authorized activities, the Amrita Vishwa Vidyapeetham does not exercise any editorial control over the information you may find at these locations. These links are provided consistent with the stated purpose of this web site.

Admissions Apply Now