Learn the core tools of information theory: entropy, mutual information, source coding and data compression, channel capacity and coding theorems, and algebraic coding theory. Apply these to analyze communication limits and design efficient, reliable codes.
Courses in this academy
Channel Capacity and Coding Theorems
Derive channel capacity for discrete memoryless channels, prove Shannon's noisy-channel coding theorem, and analyze capacity of Gaussian channels.
Source Coding and Data Compression
Prove Shannon's source coding theorem, construct Huffman and arithmetic codes, and analyze rate-distortion theory for lossy compression.
Algebraic Coding Theory
Construct linear codes, cyclic codes (BCH, Reed-Solomon), and LDPC codes. Analyze minimum distance, decoding algorithms, and error performance bounds (Singleton, Plotkin, Hamming bounds).
Entropy and Mutual Information
Define Shannon entropy, joint and conditional entropy, mutual information, and the data processing inequality. Prove basic properties and chain rules.