Advanced Topics in Machine Learning: 2019-2020
OverviewThis is an advanced course on machine learning, focusing on recent advances in deep learning with neural networks, such as recurrent and Bayesian neural networks. The course will concentrate especially on natural language processing (NLP) and computer vision applications. Recent statistical techniques based on neural networks have achieved a remarkable progress in these fields, leading to a great deal of commercial and academic interest. The course will introduce the mathematical definitions of the relevant machine learning models and derive their associated optimisation algorithms. It will cover a range of applications of neural networks in natural language processing, including analysing latent dimensions in text, translating between languages, and answering questions.
The lectures for this course are not going to be recorded in Hilary Term 2019.
After studying this course, students will:
- Understand the definition of a range of neural network models.
- Be able to derive and implement optimisation algorithms for these models.
- Understand neural implementations of attention mechanisms and sequence embedding models and how these modular components can be combined to build state-of-the-art NLP systems.
- Be able to implement and evaluate common neural network models for language.
- Have a good understanding of the two numerical approaches to learning (optimization and integration) and how they relate to the Bayesian approach.
- Have an understanding of how to choose a model to describe a particular type of data.
- Know how to evaluate a learned model in practice.
- Understand the mathematics necessary for constructing novel machine learning solutions.
- Be able to design and implement various machine learning algorithms in a range of real-world applications.
Required background knowledge includes probability theory, linear algebra, continuous mathematics, multivariate calculus and multivariate probability theory, as well as good programming skills. Students are required to have taken the Machine Learning course. The programming environment used in the lecture examples and practicals will be Python/TensorFlow.
Mathematics of machine learning. Overview of supervised, unsupervised, and multi-task techniques. Advanced machine learning topics: Bayesian modelling and Gaussian processes, randomised methods, Bayesian neural networks, approximate inference, variational autoencoders, generative models, applications. Applications of machine learning in natural language processing: recurrent neural networks, backpropagation through time, long short term memory, attention networks, memory networks, neural Turing machines, machine translation, question answering, speech recognition, syntactic and semantic parsing, GPU optimisation for neural networks.
- Kevin P. Murphy. Machine Learning: A Probabilistic Perspective. MIT Press 2012
- Ian Goodfellow, Yoshua Bengio and Aaron Courville. Deep Learning. MIT Press 2016