• Offered by Research School of Engineering
  • ANU College ANU College of Engineering and Computer Science
  • Classification Advanced
  • Course subject Engineering
  • Academic career PGRD
  • Mode of delivery In Person
  • Offered in Second Semester 2014
    See Future Offerings
This course will introduce students to the main concepts of information theory. Information theory has many applications in communications theory and practice, computer science, and statistics. For example, it quantifies fundamental limits of reliable communications in the presence of noise without restricting the treatment to a particular method of transmission or reception. 
Specific topics include: 
  1. Entropy, Relative Entropy, Mutual Information  
  2. Entropy Rates of a Stochastic Process 
  3. Data Compression and Source Coding 
  4. Channel Capacity and Channel Coding Theorem  
  5. Differential Entropy 
  6. Capacity of Gaussian Channels  
  7. Network Information Theory (such as Multiple-access Channel, Broadcast Channel, Relay Channel)
  8. Introduction to Network Coding (if time allows)

Learning Outcomes

Upon successful completion, students will have the knowledge and skills to:

Knowledge Base


At the end of this course students;



  1. Will gain in depth understanding of fundamental ways that digital information is a. represented and compressed at the source, b. reliably transmitted through the channel, and c. finally reproduced at the destination with vanishing small probabilities of error.
  2. Will be able to fully describe the notions of channel capacity, cut-off rate and explain how random channel coding techniques are used for proofs of achievability of capacity
  3. Will gain basic understanding of more complicated point-to-multi-point channels such as multiple access/broadcast/relay channels and how they differ from the simpler point-to-point case
  4. Will be able to describe how network coding can be utilizied in wirleline and wireless networks for improved throughput, robustness and delay (if time allows)

    Engineering and Practical Skills

    Having successfully completed this course, students should be able to:

    1. Apply the principles of information representation, storage, and transfer in real-life digital systems.
    2. Apply the principles of real-world data compression techniques, such as Lempel-Ziv coding.
    3. Develop an intuitive grasp of fundamental limits that information theory poses on capacity and data rate of wireless and wireline channels and more complicated data networks.







    Indicative Assessment

    Assignments and labs: 20%
    Preparation and active participation: 10%
    Mid-term exam: 30%
    Final exam: 40%






    The ANU uses Turnitin to enhance student citation and referencing techniques, and to assess assignment submissions as a component of the University's approach to managing Academic Integrity. While the use of Turnitin is not mandatory, the ANU highly recommends Turnitin is used by both teaching staff and students. For additional information regarding Turnitin please visit the ANU Online website.

    Workload

    • Preparation and reading: 3 hours per week
    • Lectures: 3 hours per week
    • Labs and tutorials: 1-2 hours per week
    • Assignments and follow-up work: 4 hours per week
    • Total: 11-12 hours per week



    Prescribed Texts

    Textbook: 'Elements of Information Theory', Thomas Cover and Joy Thomas, Second Edition, 2006.

    Optional Classic Textbook Suggestion: 'Information Theory and Reliable Communication', Robert G. Gallager, New York, Wiley, 1968





    Assumed Knowledge

    1.  ENGN8538 - Probability and Stochastic Processes in Engineering is pre-requisite.

    2.  Knowledge of ENGN6626- Digital Communications is not strictly needed, but would be advantageous



    Fees

    Tuition fees are for the academic year indicated at the top of the page.  

    If you are a domestic graduate coursework or international student you will be required to pay tuition fees. Students continuing in their current program of study will have their tuition fees indexed annually from the year in which you commenced your program. Further information for domestic and international students about tuition and other fees can be found at Fees.

    Student Contribution Band:
    2
    Unit value:
    6 units

    If you are an undergraduate student and have been offered a Commonwealth supported place, your fees are set by the Australian Government for each course. At ANU 1 EFTSL is 48 units (normally 8 x 6-unit courses). You can find your student contribution amount for each course at Fees.  Where there is a unit range displayed for this course, not all unit options below may be available.

    Units EFTSL
    6.00 0.12500
    Domestic fee paying students
    Year Fee
    1994-2003 $1650
    2004 $2190
    2005 $2190
    2006 $2190
    2007 $2298
    2008 $2592
    2009 $2850
    2010 $2916
    2011 $2946
    2012 $2946
    2013 $2946
    2014 $2946
    International fee paying students
    Year Fee
    1994-2003 $3450
    2004 $3450
    2005 $3534
    2006 $3618
    2007 $3618
    2008 $3618
    2009 $3618
    2010 $3750
    2011 $3756
    2012 $3756
    2013 $3756
    2014 $3762
    Note: Please note that fee information is for current year only.

    Offerings, Dates and Class Summary Links

    The list of offerings for future years is indicative only.
    Class summaries, if available, can be accessed by clicking on the View link for the relevant class number.

    Second Semester

    Class number Class start date Last day to enrol Census date Class end date Mode Of Delivery Class Summary
    9081 21 Jul 2014 08 Aug 2014 31 Aug 2014 30 Oct 2014 In Person N/A

    Responsible Officer: Registrar, Student Administration / Page Contact: Website Administrator / Frequently Asked Questions