Advanced Machine Learning: 2018-2019
OverviewThis is an advanced course on machine learning, focusing on recent advances in deep learning with neural networks, such as recurrent and Bayesian neural networks. The course will concentrate especially on natural language processing (NLP) and computer vision applications. Recent statistical techniques based on neural networks have achieved a remarkable progress in these fields, leading to a great deal of commercial and academic interest. The course will introduce the mathematical definitions of the relevant machine learning models and derive their associated optimisation algorithms. It will cover a range of applications of neural networks in natural language processing, including analysing latent dimensions in text, translating between languages, and answering questions.
The lectures for this course are not going to be recorded in Hilary Term 2019.
After studying this course, students will:
- Understand the definition of a range of neural network models.
- Be able to derive and implement optimisation algorithms for these models.
- Understand neural implementations of attention mechanisms and sequence embedding models and how these modular components can be combined to build state-of-the-art NLP systems.
- Be able to implement and evaluate common neural network models for language.
- Have a good understanding of the two numerical approaches to learning (optimization and integration) and how they relate to the Bayesian approach.
- Have an understanding of how to choose a model to describe a particular type of data.
- Know how to evaluate a learned model in practice.
- Understand the mathematics necessary for constructing novel machine learning solutions.
- Be able to design and implement various machine learning algorithms in a range of real-world applications.
Required background knowledge includes probability theory, linear algebra, continuous mathematics, multivariate calculus and multivariate probability theory, as well as good programming skills. Students are required to have taken the Machine Learning course. The programming environment used in the lecture examples and practicals will be Python/TensorFlow.
Mathematics of machine learning. Overview of supervised, unsupervised, and multi-task techniques. Advanced machine learning topics: Bayesian modelling and Gaussian processes, randomised methods, Bayesian neural networks, approximate inference, variational autoencoders, generative models, applications. Applications of machine learning in natural language processing: recurrent neural networks, backpropagation through time, long short term memory, attention networks, memory networks, neural Turing machines, machine translation, question answering, speech recognition, syntactic and semantic parsing, GPU optimisation for neural networks.
- Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. MIT Press 2012
- Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. MIT Press 2016
Students are formally asked for feedback at the end of the course. Students can also submit feedback at any point here. Feedback received here will go to the Head of Academic Administration, and will be dealt with confidentially when being passed on further. All feedback is welcome.
Taking our courses
Matriculated University of Oxford students who are interested in taking this course, or others in the Department of Computer Science, must complete this online form by 17.00 on Friday of 0th week of term in which the course is taught. Late requests, and requests sent by email, will not be considered. All requests must be approved by the relevant Computer Science departmental committee and can only be submitted using this form. Priority will be given to students studying for degrees in the Department of Computer Science.