- Code COMP6261
- Unit Value 6 units
- Offered by Research School of Computer Science
- ANU College ANU College of Engineering and Computer Science
- Course subject Computer Science
Information theory studies the fundamental limits of the representation and transmission of information. This course provides an introduction to information theory, studying fundamental concepts such as probability, information, and entropy and examining their applications in the areas of data compression, coding, communications, pattern recognition and probabilistic inference.
Upon successful completion, students will have the knowledge and skills to:
Upon successful completion of the course, the student will have background knowledge necessary to understand problems in data compression, storing and communication and undertake advanced courses on statistical inference, machine learning and information engineering. In particular, the student will be able to:
- Understand and apply fundamental concepts in information theory such as probability, entropy, information content and their inter-relationships.
- Understand the principles of data compression.
- Compute entropy and mutual information of random variables.
- Implement and analyse basic coding and compression algorithms.
- Understand the relationship of information theoretical principles and Bayesian inference in data modelling and pattern recognition.
- Understand some key theorems and inequalities that quantify essential limitations on compression, communication and inference.
- Know the basic concepts regarding communications over noisy channels.
Indicative AssessmentAssignment 1 (10%) Assignment 2 (20%) Assignment 3 (20%) Final Exam (50%)
WorkloadTwenty-six one-hour lectures and five two-hour tutorial sessions.
Requisite and Incompatibility
Prescribed TextsInformation Theory, Inference, and Learning Algorithms by David MacKay, Cambridge University Press, 2003.
Additional reading: Elements of Information Theory by Cover and Thomas, 2nd Edition, New York, Wiley, 2006.
Tuition fees are for the academic year indicated at the top of the page.
- Student Contribution Band:
- Unit value:
- 6 units
Offerings, Dates and Class Summary Links
Class summaries, if available, can be accessed by clicking on the View link for the relevant class number.
|Class number||Class start date||Last day to enrol||Census date||Class end date||Mode Of Delivery||Class Summary|
|7737||27 Jul 2020||03 Aug 2020||31 Aug 2020||30 Oct 2020||In Person||N/A|