Syllabus
INTRODUCTION: Concepts and Terminology – Generic Processor / ASIC Processor Architecture – Pipeline Architecture – Instruction Set Architecture –modification to the von Neumann architecture – Types of Parallelism – Flynn’s Classical Taxonomy –concurrent, parallel, distributed.
PARALLEL COMPUTER MEMORY ARCHITECTURES: Shared Memory – Distributed Memory -Hybrid Distributed-Shared Memory Multiprocessors: Communication and Memory issues – Message Passing Architectures –UMA – NUMA – Vector Processing and SIMD Architectures – Memory consistency models.
PARALLEL PROGRAMMING MODELS: Overview -Shared Memory Model – Threads Model – Message Passing Model – Data Parallel Model – Other Models. DESIGNING PARALLEL PROGRAMS: Automatic vs. Manual Parallelization – Understand the Problem and the Program – Partitioning -Communications – Synchronization -Data Dependencies – Load Balancing -Granularity -I/O -Limits and Costs of Parallel Programming – Performance Analysis and Tuning –speedup, efficiency – Amdahl’s Lab – Gustafson’s law – scalability – Parallel Examples -Array Processing – Compiler Transformation techniques for High performance computing: – Transformations for parallel Machines.
PRAM ALGORITHMS& BSP: PRAM model of computation- Work-Time formalism and Brent’s Theorem; algorithm design techniques- parallel reduction, parallel prefix, pointer jumping, Euler tours, divide and conquer, symmetry breaking; survey of data-parallel algorithms; relative power of PRAM.
HIGH PERFORMANCE COMPUTING ARCHITECTURES: Latency Hiding Architectures -Multithreading Architectures -Dataflow Architectures – GPGPU Architecture- Overview of basic Accelerators /GPU / GPGPU and its programming model – CUDA – OpenCL. (6)
DISTRIBUTED COMPUTING: Introduction -Definitions, motivation – system models – architectural model – client-server model – peer-to-peer model – distributed computing paradigm – Communication Mechanisms – Communication protocols-RPC- RMI – group communication – external data representation and marshalling – distributed file system – HDFS – DISTRIBUTED PROGRAMMING ALGORITHMS: Fundamental issues & concepts – synchronization – termination detection – clocks – event ordering – locking – snapshots – leader election – replication and coherence – consistency models and protocols – Fault Tolerance