Source Coding and Data Compression

About this course

Prove Shannon's source coding theorem, construct Huffman and arithmetic codes, and analyze rate-distortion theory for lossy compression.

• 20 lessons
• 2 h
• advanced

What you'll learn

• Define self-information and entropy and interpret their units
• Explain why entropy bounds the compressibility of a discrete memoryless source
• State the AEP for i.i.d. sources
• Characterize the typical set and its probability mass
• Prove that any lossless code requires at least H(X) bits per symbol on average
• Construct an achieving code based on typical sequences

Finishing the course

• 80 knowledge-check questions along the way, so you know the ideas stuck before you move on.
• Watching is free and needs no account. Sign in to save your progress. Pass each lesson's quiz (60%+) to earn the certificate. Each certificate has its own public verification link to share, marked quiz-verified.

Created by