Acquisition of theoretical background on information theory and methodologies and technologies for source coding of mono and multimedia signals for redundancy reduction for lossless and lossy information applications. Acquisition of theoretical background, methodologies, and technologies for channel coding, i.e., for protection of digital communications against errors caused by distortions and noise.
teacher profile teaching materials
Mutual Information and equivocation. Channel Capacity. Properties of Channel Capacity. Examples of Channel Capacity: Binary Symmetric and Gaussian Channels. Channel Coding Theorem. Fano's Inequality and the Converse to the Coding Theorem. Source-Channel Separation Theorem.
Rate Distortion Theory. Scalar Quantization. Max-Lloyd quantizer. Vector Quantization. Linde, Buzo, and Gray algorithm.
Audio signals coding. Characteristics of the human auditory sytem, perception of complex audio signals, absolute threshold of hearing, critical bandwith, Anatomy and physiology of the basilar membrane. Masking mechanisms.
Speech coding in the time domain: G.711, G.726 e G.727. Model based coding: G729. Audio coding in the transform domain: MPEG 1 layer I, II e III, MPEG-2 and MPEG-4.
Image coding elements: the Discrete Cosine Transform. JPEG encoding.
Channel Coding. Linear Block Codes. Generator Matrix. Block Codes in Systematic Form
Parity Check Matrix. Syndrome Error Detection. Minimum Distance of a Block Code. Error-Correction Capability of a Block Code. Syndrome Detection and the Standard Array. Hamming Codes. Reed-Solomon Codes.
Convolutional Codes. Convolutional Encoder Representations. Distance Properties of Convolutional Codes. Maximum Likelihood Detection. Markov chains. Decoding of Convolutional Codes: The Viterbi Algorithm. Error Probability Analysis for Convolutional Codes. Hard and Soft Decisions. Punctured Convolutional Codes and Rate-Compatible Schemes.
Concatenated encoders, Recursive Systematic Convolutional Encoders. Turbo Encoders. Construction Methods for Turbo Codes: Interleavers, Code Concatenation Methods. Turbo Code Performance as a Function of Size and Type of Interleaver . Decoding of Turbo Codes: The Turbo Decoder, Probabilities and Estimate, Symbol Detection, The Log Likelihood Ratio. The BCJR MAP Algorithm and the LLR Calculation. Turbo Decoding
Forward Error Correction and Automatic Repeat ReQuest. Forward Error Correction. Automatic Repeat ReQuest. ARQ Schemes. ARQ Scheme Efficiencies. Hybrid-ARQ Schemes.
Low-Density Parity Check Codes. Description of LDPC Codes. Construction of LDPC Codes. Decoding of LDPC Codes: The Tanner Graph. Decoding Algorithm for LDPC Codes
Packet oriented channel coding: Fountain and Raptor Codes.
Estimation Theory. Minimum Mean Square Error estimator: general solution an Gaussian in Gaussian estimate. Minimum Mean Magnitude error estimator. Maximum Posterior Probability and Maximum Likelihood estimates. Cramèr Rao lower bound for monodimensional and multidimensional estimates. Efficient estimators. Fisher’s Information Matrix. Time of arrival estimation. Joint time and Doppler shift estimation. Wavelet transform and multiresolution analysis. Signal restoration based on wavelet transform shrinking.
Additional books freely available in the e-book collection of ROMA TRE Biblioteca di Area Scientifico-tecnologica (http://host.uniroma3.it/biblioteche/page.php?page=collezion0)
Elements of information theory
Thomas M. Cover, Joy A. Thomas,
2. ed., 2006
John Wiley & Sons, Ltd.
Essentials of error-control coding
Jorge Castiñeira Moreira, Patrick Guy Farrell,
c2006
John Wiley & Sons, Ltd.
Programme
Elements of Information Theory: Entropy, Relative Entropy, and Mutual Information. Joint Entropy and Conditional Entropy. Sufficient Statistics. Fano's Inequality. Lossless source coding. Optimal Codes. Bounds on the Optimal Code Length. Kraft Inequality for Uniquely Decodable Codes. Huffman Codes. Optimality of Huffman Codes. Shannon-Fano-Elias Coding. Universal Source Coding. Universal Coding for Binary Sequences. Arithmetic Coding. Lempel-Ziv Coding.Mutual Information and equivocation. Channel Capacity. Properties of Channel Capacity. Examples of Channel Capacity: Binary Symmetric and Gaussian Channels. Channel Coding Theorem. Fano's Inequality and the Converse to the Coding Theorem. Source-Channel Separation Theorem.
Rate Distortion Theory. Scalar Quantization. Max-Lloyd quantizer. Vector Quantization. Linde, Buzo, and Gray algorithm.
Audio signals coding. Characteristics of the human auditory sytem, perception of complex audio signals, absolute threshold of hearing, critical bandwith, Anatomy and physiology of the basilar membrane. Masking mechanisms.
Speech coding in the time domain: G.711, G.726 e G.727. Model based coding: G729. Audio coding in the transform domain: MPEG 1 layer I, II e III, MPEG-2 and MPEG-4.
Image coding elements: the Discrete Cosine Transform. JPEG encoding.
Channel Coding. Linear Block Codes. Generator Matrix. Block Codes in Systematic Form
Parity Check Matrix. Syndrome Error Detection. Minimum Distance of a Block Code. Error-Correction Capability of a Block Code. Syndrome Detection and the Standard Array. Hamming Codes. Reed-Solomon Codes.
Convolutional Codes. Convolutional Encoder Representations. Distance Properties of Convolutional Codes. Maximum Likelihood Detection. Markov chains. Decoding of Convolutional Codes: The Viterbi Algorithm. Error Probability Analysis for Convolutional Codes. Hard and Soft Decisions. Punctured Convolutional Codes and Rate-Compatible Schemes.
Concatenated encoders, Recursive Systematic Convolutional Encoders. Turbo Encoders. Construction Methods for Turbo Codes: Interleavers, Code Concatenation Methods. Turbo Code Performance as a Function of Size and Type of Interleaver . Decoding of Turbo Codes: The Turbo Decoder, Probabilities and Estimate, Symbol Detection, The Log Likelihood Ratio. The BCJR MAP Algorithm and the LLR Calculation. Turbo Decoding
Forward Error Correction and Automatic Repeat ReQuest. Forward Error Correction. Automatic Repeat ReQuest. ARQ Schemes. ARQ Scheme Efficiencies. Hybrid-ARQ Schemes.
Low-Density Parity Check Codes. Description of LDPC Codes. Construction of LDPC Codes. Decoding of LDPC Codes: The Tanner Graph. Decoding Algorithm for LDPC Codes
Packet oriented channel coding: Fountain and Raptor Codes.
Estimation Theory. Minimum Mean Square Error estimator: general solution an Gaussian in Gaussian estimate. Minimum Mean Magnitude error estimator. Maximum Posterior Probability and Maximum Likelihood estimates. Cramèr Rao lower bound for monodimensional and multidimensional estimates. Efficient estimators. Fisher’s Information Matrix. Time of arrival estimation. Joint time and Doppler shift estimation. Wavelet transform and multiresolution analysis. Signal restoration based on wavelet transform shrinking.
Core Documentation
A. Neri. " Appunti dalle lezioni di Teoria dell’Informazione e Codici ". Freely available at http://elearning.dia.uniroma3.it/moodle/.Additional books freely available in the e-book collection of ROMA TRE Biblioteca di Area Scientifico-tecnologica (http://host.uniroma3.it/biblioteche/page.php?page=collezion0)
Elements of information theory
Thomas M. Cover, Joy A. Thomas,
2. ed., 2006
John Wiley & Sons, Ltd.
Essentials of error-control coding
Jorge Castiñeira Moreira, Patrick Guy Farrell,
c2006
John Wiley & Sons, Ltd.
Type of evaluation
The exam consists of a first written test consisting in answering to theoretical questions related to the whole course program. The written test can be integrated by an oral exam. Three ongoing tests are planned during the course. They consist in answering to theoretical questions related to the corresponding part of the course.