📖

Information Theory

4 courses

Algebraic Coding Theory

Construct linear codes, cyclic codes (BCH, Reed-Solomon), and LDPC codes. Analyze minimum distance, decoding algorithms, and error performance bounds (Singleton, Plotkin, Hamming bounds).

2 h advanced

Source Coding and Data Compression

Prove Shannon's source coding theorem, construct Huffman and arithmetic codes, and analyze rate-distortion theory for lossy compression.

2 h advanced

Entropy and Mutual Information

Define Shannon entropy, joint and conditional entropy, mutual information, and the data processing inequality. Prove basic properties and chain rules.

2 h advanced

Channel Capacity and Coding Theorems

Derive channel capacity for discrete memoryless channels, prove Shannon's noisy-channel coding theorem, and analyze capacity of Gaussian channels.

2 h advanced