• Offered by Research School of Computer Science
  • ANU College ANU College of Engineering and Computer Science
  • Course subject Computer Science
  • Academic career UGRD
  • Course convener
    • Dr Mark Reid
  • Mode of delivery In Person
  • Offered in Second Semester 2014
    See Future Offerings

Information theory studies the fundamental limits of the representation and transmission of information. This course provides an introduction to information theory, studying fundamental concepts such as probability, information, and entropy and examining their applications in the areas of data compression, coding, communications, pattern recognition and probabilistic inference.

Learning Outcomes

Upon successful completion, students will have the knowledge and skills to:

Upon successful completion of the course, the student will have background knowledge necessary to understand problems in data compression, storing and communication and undertake advanced courses on statistical inference, machine learning and information engineering. In particular, the student will be able to:

  • Understand and apply fundamental concepts in information theory such as probability, entropy, information content and their inter-relationships.
  • Understand the principles of data compression.
  • Compute entropy and mutual information of random variables.
  • Implement and analyse basic coding and compression algorithms.
  • Understand the relationship of information theoretical principles and Bayesian inference in data modelling and pattern recognition.
  • Understand some key theorems and inequalities that quantify essential limitations on compression, communication and inference.
  • Know the basic concepts regarding communications over noisy channels.

Indicative Assessment

Assignment 1 (10%) Assignment 2 (20%) Assignment 3 (20%) Final Exam (50%)

The ANU uses Turnitin to enhance student citation and referencing techniques, and to assess assignment submissions as a component of the University's approach to managing Academic Integrity. While the use of Turnitin is not mandatory, the ANU highly recommends Turnitin is used by both teaching staff and students. For additional information regarding Turnitin please visit the ANU Online website.

Workload

Twenty-six one-hour lectures and five two-hour tutorial sessions.

Prescribed Texts

Information Theory, Inference, and Learning Algorithms by David MacKay, Cambridge University Press, 2003.
Additional reading: Elements of Information Theory by Cover and Thomas, 2nd Edition, New York, Wiley, 2006.

Assumed Knowledge

Some background in elementary statistics and probability.

Majors

Fees

Tuition fees are for the academic year indicated at the top of the page.  

If you are a domestic graduate coursework or international student you will be required to pay tuition fees. Students continuing in their current program of study will have their tuition fees indexed annually from the year in which you commenced your program. Further information for domestic and international students about tuition and other fees can be found at Fees.

Student Contribution Band:
2
Unit value:
6 units

If you are an undergraduate student and have been offered a Commonwealth supported place, your fees are set by the Australian Government for each course. At ANU 1 EFTSL is 48 units (normally 8 x 6-unit courses). You can find your student contribution amount for each course at Fees.  Where there is a unit range displayed for this course, not all unit options below may be available.

Units EFTSL
6.00 0.12500
Domestic fee paying students
Year Fee
1994-2003 $1650
2004 $2190
2005 $2190
2006 $2190
2007 $2298
2008 $2592
2009 $2850
2010 $2916
2011 $2946
2012 $2946
2013 $2946
2014 $2952
International fee paying students
Year Fee
1994-2003 $3234
2004 $3234
2005 $3288
2006 $3426
2007 $3426
2008 $3426
2009 $3426
2010 $3750
2011 $3756
2012 $3756
2013 $3756
2014 $3762
Note: Please note that fee information is for current year only.

Offerings, Dates and Class Summary Links

The list of offerings for future years is indicative only.
Class summaries, if available, can be accessed by clicking on the View link for the relevant class number.

Second Semester

Class number Class start date Last day to enrol Census date Class end date Mode Of Delivery Class Summary
7997 21 Jul 2014 01 Aug 2014 31 Aug 2014 30 Oct 2014 In Person N/A

Responsible Officer: Registrar, Student Administration / Page Contact: Website Administrator / Frequently Asked Questions