Back close
Admissions

Course Detail

Course Name System Security
Course Code 26SN612
Program M. Tech. in Cyber Security Systems & Networks
Semester 2
Credits 4
Campus Amritapuri

Syllabus

Unit I

Basic operating system concepts – Processes, Threads, Virtual memory, File system.

Unit II

Security Goals, Secure Design Principles, Authentication, Linux Password scheme,
Password Security, Authorization – Access control, MAC, DAC, ACL, File Permissions, SUID bit, sticky bit, Capabilities,
Information flow control, Privilege Escalation Attacks, constraining and sandboxing users
and applications.

Unit III

Program Interaction: Linux Command Line, Binary files, Assembly Primer, Shell coding, ELF File Format, Linux Process Loading, Linux Process Interaction.

Unit IV

Memory Exploits –
Buffer Overflow, Format String Attacks, Integer Overflow, Return to Libc, ROP, ROP varaiants, Heap Overflow,
Exploit prevention mechanisms: stack canaries, Data Execution Prevention, Address
Space Layout Randomization, bypassing DEP & ASLR.

Unit V

Trusted Execution Environment – Case Study on Intel SGX. ARM TrustZone.

Unit VI

Fuzzing – Types of fuzzers, Bug detection, AI guided fuzzing for smart input generation, Case
study – AFL fuzzer. Vulnerability and exploit analysis: Spectre, Meltdown

Introduction

Preamble:

The primary objective of this course is to introduce computational problem- solving. It focuses on principles and methods thereby providing transferable skills to any other domain. The course also provides foundation for developing computational perspectives of one’s own discipline.

Objectives and Outcomes

Course Objectives

  • To understand core operating system internals — processes, threads, virtual memory, and file systems , as the foundation for system-level security analysis.
  • To develop proficiency in low-level C programming, assembly language, ELF binary internals, and Linux system interaction using industry tools including pwntools.
  • To investigate memory-based vulnerabilities including buffer overflow, format string attacks, integer overflow, ROP chains, and heap overflow, and apply exploit scripting using pwntools.
  • To explore exploit prevention mechanisms (stack canaries, DEP, ASLR) and Trusted Execution Environments (Intel SGX, ARM TrustZone) as defensive countermeasures.
  • To apply AI-guided fuzzing techniques and analyse hardware-level vulnerabilities including Spectre and Meltdown through case studies and hands-on lab exercises.

Course Outcomes

CO Description Bloom’s Level
CO1 Recall core operating system concepts — processes, threads, virtual memory, and file systems — that underpin system-level security analysis. Remember
CO2 Describe security goals, secure design principles, authentication mechanisms, authorization models (MAC, DAC, ACL), and privilege escalation defences used in building secure systems. Understand
CO3 Explain ELF binary internals, assembly language, Linux system calls, and shellcoding, and use pwntools for binary inspection and process interaction. Understand / Apply
CO4 Demonstrate exploitation of access control vulnerabilities — privilege escalation, SUID abuse, sandboxing bypass — and develop corresponding mitigations. Apply / Analyse
CO5 Construct and execute buffer overflow, format string, integer overflow, return-to-libc, ROP, and heap overflow exploits using pwntools, and evaluate countermeasures including stack canaries, DEP, and ASLR bypass. Apply / Evaluate
CO6 Analyse exploit prevention mechanisms, apply AI-guided fuzzing using AFL, and evaluate hardware-level vulnerabilities (Spectre, Meltdown) and Trusted Execution Environments (Intel SGX, ARM TrustZone). Analyse / Evaluate

Prerequisite: C Programming, Assembly Primer

Evaluation Pattern

CO-PO Mapping:

CO PO1 PO2 PO3 PSO1 PSO2 PSO3 PSO4
CO1 1       1    
CO2 3 1     2    
CO3 2 1     1   1
CO4 1 1 3 3 3    
CO5 2 1 2 3 2    
CO6 1 1 2 3 2    

Evaluation Pattern:

Mid Term Exam – 30%

Assignment – 10%

Continous Assessment(Lab) – 20%

End semester – 40%

Textbooks

  1. Andrew S. Tanenbaum, “Modern Operating Systems”, Fourth Edition, Pearson EducationIndia, 2016
  2. Neil Daswani, Christopher Kern, Anita Kesavan, “Foundations of Security, What Every Programmer Needs to Know”, Apress, 2007
  3. James C. Foster and Vincent T. Liu, “Writing Security Tools and Exploits”, Syngress Publishing
  4. Gary McGraw, John Viega, “Building Secure Software”, Addison-Wesley Professional, 2001.
  5. Jon Ericson, “Hacking: The Art of Exploitation”, Second Edition, No Starch Press, 122008, ISBN 978-1593271442
  6. Chris Anley, John Heasman, Felix Linder, Gerardo Richarte, The Shellcoder’s Handbook: Discovering and Exploiting Security Holes, Second Edition, Addison-Wiley, ISBN 978- 0470080238

DISCLAIMER: The appearance of external links on this web site does not constitute endorsement by the School of Biotechnology/Amrita Vishwa Vidyapeetham or the information, products or services contained therein. For other than authorized activities, the Amrita Vishwa Vidyapeetham does not exercise any editorial control over the information you may find at these locations. These links are provided consistent with the stated purpose of this web site.

Admissions Apply Now