Module I

Introduction to Probability – Random Variables, Random variable, Sample space, Conditional probability, Joint probability. Modeling of Information Sources – Self Information, Entropy, Mutual Information. Source Coding Theory and algorithms – Kraft inequality, Huffman algorithm, Arithmetic coding, Lempel Ziv coding. Modeling of Communication channels – Binary symmetric channel, Binary Erasure channel, Channel coding theorem.

Module II

Error Correction Codes – Introduction to Galois fields, polynomial arithmetic, linear block codes for error correction – Generator matrix, Encoding, Parity Check matrix, Decoding – Standard array decoding and Syndrome decoding. Cyclic Codes – Generation of codes, encoding and syndrome decoding.

Module III

BCH Codes – Minimal polynomial encoding and decoding. Convolutional encoder – Introduction to Convolutional codes, distance properties – Trellis codes, Viterbi decoder. Numerical problems and MATLAB based problem solving on selected topics of the course.

Course Objectives

• To provide an insight into the concept of information in the context of communication theory and its significance in the design of communication receivers.
• To explore in detail, the calculations of channel capacity to support error-free transmission and also, the most commonly used source coding and channel coding algorithms.
• To encourage and train to design coding schemes for data compression and error correction, and they will also get an overall perspective of how this impacts the design of an optimum communication receiver.

Course Outcomes

• CO1: Overview of Probability Theory, significance of “Information” with respect to Information Theory.
• CO2: Derive equations for entropy, mutual information and channel capacity for all kinds of channels.
• CO3: Implement the various types of source coding algorithms and analyse their performance.
• CO4: Explain various methods of generating and detecting different types of error correcting codes
• CO5: Understand the fundamentals of Field Theory and polynomial arithmetic
• CO6: Design linear block codes and cyclic codes (encoding and decoding).
• CO7: Implement and decode a sequence at the receiver using Trellis decoder and Viterbi decoder.
• CO8: Perform mathematical analysis of problems in Information Theory and Coding, Implementation and verification using MATLAB simulation

CO – PO Mapping

Textbook / References

• Ranjan Bose, “Information Theory, Coding and Cryptography”, Tata McGraw Hill, 2^nd edition.
• P.S. Satyanarayana, “Concepts of Information Theory and Coding”, Dynaram Publication, 2005
• Richard B. Wells, “Applied Coding and Information Theory for Engineers” Pearson Education, LPE 2004.
• Shu Lin and Daniel Castello, “Error Control Coding – Fundamentals and Applications”, second edition 2004
• Thomas M Cover, Joy Thomas, “Elements of Information Theory”, MGH 2006.

Evaluation Pattern