Introducing key issues in the theory of signal transmission and their quantitative analysis. Concept of entropy and mutual information. Show the underlying algebraic structure. Apply fundamental concepts to code theory, data compression, and cryptography
teacher profile teaching materials
Reliable transmission of information. Shannon's information content. Measures of information. Entropy, mutual information, informational divergence. Data compression. Error correction. Data processing theorems. Fundamental inequalities. Information diagrams. Informational divergence and maximum likelihood.
2. Source coding and data compression
Typical sequences. Typicality in probability. Asymptotic equipartitioning property. Block codes and variable length codes. Coding rate. Source coding theorem. Lossless data compression. Huffman code. Universal codes. Ziv-Lempel compression.
3. Channel coding
Channel capacity. Discrete memoryless channels. Information transmitted over a channel. Decoding criteria. Noisy channel coding theorem.
4. Further codes and applications
Hamming space. Linear codes. Generating matrix and check matrix. Hadamard codes. Hash codes.
Fruizione: 20410442 IN420 - TEORIA DELL'INFORMAZIONE in Scienze Computazionali LM-40 BONIFACI VINCENZO
Programme
1. Introduction to information theory.Reliable transmission of information. Shannon's information content. Measures of information. Entropy, mutual information, informational divergence. Data compression. Error correction. Data processing theorems. Fundamental inequalities. Information diagrams. Informational divergence and maximum likelihood.
2. Source coding and data compression
Typical sequences. Typicality in probability. Asymptotic equipartitioning property. Block codes and variable length codes. Coding rate. Source coding theorem. Lossless data compression. Huffman code. Universal codes. Ziv-Lempel compression.
3. Channel coding
Channel capacity. Discrete memoryless channels. Information transmitted over a channel. Decoding criteria. Noisy channel coding theorem.
4. Further codes and applications
Hamming space. Linear codes. Generating matrix and check matrix. Hadamard codes. Hash codes.
Core Documentation
David J. C. MacKay. Information Theory, Inference and Learning Algorithms. Cambridge University Press, 2004.Type of delivery of the course
The Moodle website of the class is reachable at the address: https://matematicafisica.el.uniroma3.it/course/view.php?id=40 For the class' diary see the teacher's webpage: http://ricerca.mat.uniroma3.it/users/vbonifaci/in420.htmlType of evaluation
Oral exam.