• Offered by RS Electrical, Energy and Materials Engineering
  • ANU College ANU College of Engineering and Computer Science
  • Classification Advanced
  • Course subject Engineering
  • Academic career PGRD
  • Course convener
    • Prof Parastoo Sadeghi
  • Mode of delivery In Person
  • Offered in Second Semester 2020
    See Future Offerings

This course will introduce students to the main concepts of information theory. Information theory has many applications in communications theory and practice, computer science, and statistics. For example, it quantifies fundamental limits of reliable communications in the presence of noise without restricting the treatment to a particular method of transmission or reception. Specific topics include:

  1. Entropy, Relative Entropy, Mutual Information.

  2. Entropy Rates of a Stochastic Process.

  3. Data Compression and Source Coding.

  4. Channel Capacity and Channel Coding Theorem.

  5. Differential Entropy.

  6. Capacity of Gaussian Channels.

  7. Network Information Theory (such as Multiple-access Channel, Broadcast Channel, Relay Channel).

  8. Introduction to Network Coding (if time allows).

Learning Outcomes

Upon successful completion, students will have the knowledge and skills to:

  1. Explain, model, use and evaluate the entropy for discrete as well as continuous sources, and for memoryless as well as stochastic sources
  2. Explain, model, use and evaluate the mutual information for discrete as well as continuous systems, and for memoryless as well as stochastic systems
  3. Explain, classify, and model fundamental methods that digital information is represented and compressed at the source, reliably transmitted through the channel, and finally reproduced at the destination with vanishing small probabilities of error
  4. Fully analyse and evaluate the information capacity of basic communication channels and use random channel coding techniques to prove achievability of capacity
  5. Analyse and evaluate the information capacity of selected more complicated point-to-multi-point channels such as multiple access/broadcast/relay channels and compare and critique how they differ in performance relative to the simpler point-to-point channels.
  6. Apply their general knowledge in the course to evaluate how network coding is utilisied in wireline and wireless data networks for improved throughput, robustness and delay (if time allows)
  7. implement and test real-world data compression techniques, such as Lempel-Ziv coding.
  8. Apply the broad principles of information representation, storage, and transfer learned in this course to evaluate, critique and design efficient complex wireless and wireline communication and storage systems
  9. Plan, execute and report on a group-based project and communicate professionally and effectively in written and oral form about the obtained outcomes

Indicative Assessment

  1. Assignments: (16) [LO 1,2,3,4,5,6,7,8]
  2. Active participation: (6) [LO 1,2,3,4,5,7,8]
  3. Group research project: (13) [LO 5,6,8,9]
  4. Mid-term exam: (25) [LO 1,2,3,4,5,7,8]
  5. Final exam: (40) [LO 1,2,3,4,5,6,7,8]

In response to COVID-19: Please note that Semester 2 Class Summary information (available under the classes tab) is as up to date as possible. Changes to Class Summaries not captured by this publication will be available to enrolled students via Wattle. 

The ANU uses Turnitin to enhance student citation and referencing techniques, and to assess assignment submissions as a component of the University's approach to managing Academic Integrity. While the use of Turnitin is not mandatory, the ANU highly recommends Turnitin is used by both teaching staff and students. For additional information regarding Turnitin please visit the ANU Online website.


2 x 2-hour lectures per week (4 hours total),  1 x 1-hour tutorial (or self-taught tutorial) per week, 5 hours independent/or group-based study per week.

Inherent Requirements

Not applicable

Requisite and Incompatibility

To enrol in this course you must have completed ENGN8538. You are not able to enrol in this course if you have completed COMP2610 or COMP6261.

Prescribed Texts

Textbook: 'Elements of Information Theory', Thomas Cover and Joy Thomas, Second Edition, 2006.

Optional Classic Textbook Suggestion: 'Information Theory and Reliable Communication', Robert G. Gallager, New York, Wiley, 1968

Assumed Knowledge

Knowledge of digital communications (e.g., ENGN3226 or equivalent) is advantageous but not strictly needed.


Tuition fees are for the academic year indicated at the top of the page.  

If you are a domestic graduate coursework or international student you will be required to pay tuition fees. Tuition fees are indexed annually. Further information for domestic and international students about tuition and other fees can be found at Fees.

Student Contribution Band:
Unit value:
6 units

If you are an undergraduate student and have been offered a Commonwealth supported place, your fees are set by the Australian Government for each course. At ANU 1 EFTSL is 48 units (normally 8 x 6-unit courses). You can find your student contribution amount for each course at Fees.  Where there is a unit range displayed for this course, not all unit options below may be available.

6.00 0.12500
Domestic fee paying students
Year Fee
2020 $4320
International fee paying students
Year Fee
2020 $5760
Note: Please note that fee information is for current year only.

Offerings, Dates and Class Summary Links

ANU utilises MyTimetable to enable students to view the timetable for their enrolled courses, browse, then self-allocate to small teaching activities / tutorials so they can better plan their time. Find out more on the Timetable webpage.

The list of offerings for future years is indicative only.
Class summaries, if available, can be accessed by clicking on the View link for the relevant class number.

Second Semester

Class number Class start date Last day to enrol Census date Class end date Mode Of Delivery Class Summary
7756 27 Jul 2020 03 Aug 2020 31 Aug 2020 30 Oct 2020 In Person N/A

Responsible Officer: Registrar, Student Administration / Page Contact: Website Administrator / Frequently Asked Questions