An introduction to Shannon’s information theory and elementary binary coding schemes with and without noise. The concept of information, entropy, simple sources, Markov sources, continuous sources, information channels, average error, ambiguity, transformation, capacity, noiseless coding, Kraft-McMillan theorem, Shannon-Fano and Huffmann coding schemes, error-correcting codes, linear codes, cyclic codes. Data Compression.
|Introduction; Related concepts.
|Entropy, relative entropy and mutual information
|Asymptotic equipartition property
|Entropy rates of stochastic processes
|Data compression -I
|Data compression -II
|The Gaussian channel
|Network Information Theory – I
|Network Information Theory – II
|Relevant topics in Information Theory today