Learning Outcomes

Upon successful completion, students will have the knowledge and skills to:

The learning outcomes change based on the area taught. For the convex analysis and optimisation topic, at the end of the course students should be able to:

• write down definitions of key concepts in convex analysis, including convexity of sets and functions, subgradients, and the convex dual

• derive basic results about convex functions such as Jensen’s inequality

• understand how Bregman divergences are constructed from convex functions and derive some of their properties

• write down a formal optimization problem from a high-level description and determine whether the problem is convex

• recognize standard convex optimization problems such as linear programs and quadratic programs

• derive the standard (dual) quadratic program for support vector machines and understand the extension to max-margin methods for structured prediction

• implement and analyse gradient descent algorithms such as stochastic gradient descent and mirror descent

For the structured probabilistic models topic, at the end of the course the student should be able to:

• write down definitions of key concepts in probabilistic graphical models, including Bayesian networks, Markov networks, probabilistic quesies, and conditional independence
• derive independence assumptions from graphical representations of a probabilistic model
• understand and implement various exact and approximate inference algorithms, including belief propagation of trees, sampling, and variational schemes
• derive maximum likelihood learning for probabilistic graphical models
• understand approximations of the likelihood function

Indicative Assessment

Assessment will be in the form of regular assignments and an open-book final examination.

The ANU uses Turnitin to enhance student citation and referencing techniques, and to assess assignment submissions as a component of the University's approach to managing Academic Integrity. While the use of Turnitin is not mandatory, the ANU highly recommends Turnitin is used by both teaching staff and students. For additional information regarding Turnitin please visit the ANU Online website.

Workload

2hrs lectures weekly

Requisite and Incompatibility

Prescribed Texts

Main text (depending on the topic taught):

• Stephen Boyd and Lieven Vandenberghe, "Convex Optimization"
• Kevin Murphy, "Machine Learning" (structured probabilistic models)

Reference texts:

• Hiriart-Urruty and Lemaréchal, “Fundamentals of Convex Analysis”
• Bertsekas, Nedic and Ozdaglar, “Convex Analysis and Optimization”
• Bertsekas, “Nonlinear Programming”
• Koller and Friedman, "Probabilistic Graphical Models"
• Bishop, "Pattern Recognition and Machine Learning"

Assumed Knowledge

• Knowledge of machine learning at the level of COMP4670 Introduction to SML
• Familiarity with linear algebra (including norms, inner products, determinants, eigenvalues, eigenvectors, and singular value decomposition)
• Familiarity with basic probablity theory
  
• Familiarity with multivariate differential calculus (e.g., derivative of a vector-valued function)
• Exposure to mathematical proofs

Offerings, Dates and Class Summary Links