Machine Learning: 2012-2013
Information
|
Degrees |
Schedule B2 — Computer Science Schedule B2 — Mathematics and Computer Science |
|
Term |
Michaelmas Term 2012 (16 lectures) |
Overview
It has long been a central concern of Computer Science to create machines capable of learning from experience. From predicting movies a customer will like, to learning to translate human languages from examples, systems that learn are becoming increasing prevalent and effective in our everyday lives. This course will introduce the field of Machine Learning, in particular focusing on the core concepts of supervised and unsupervised learning. In supervised learning we will discuss algorithms which are trained on input data labelled with a desired output, for instance an image of a face and the name of the person whose face it is, and learn a function mapping from the input to the output. Unsupervised learning aims to discover latent structure in an input signal where no output labels are available, an example of which is grouping webpages based on the topics they discuss. Students will learn the algorithms which underpin many popular Machine Learning techniques, as well as developing an understanding of the theoretical relationships between these algorithms. The practicals will concern the application of machine learning to a range of real-world problems, with particular reference to working with data on the web.
Learning outcomes
On completion of the course students will be expected to:
- Have a good understanding of the fundamental issues and challenges of machine learning: data, model selection, model complexity, etc.
- Have an understanding of the strengths and weaknesses of many popular machine learning approaches.
- Appreciate the underlying mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and un-supervised learning.
- Be able to design and implement various machine learning algorithms in a range of real-world applications.
Prerequisites
Modern Machine Learning is a mathematical discipline and as such students will benefit from a familiarity with probability theory and linear algebra, as well as some programming skills. The course is self-contained and does not require any previous knowledge of Artificial Intelligence.
Synopsis
- Introduction to Machine Learning (1 lecture): classification, regression and clustering; supervised and unsupervised learning.
- Mathematical preliminaries (1 lecture): Introduction to probability: random variables, independence, Bayes' theorem, conditional and marginal probabilities. Maximum likelihood (MLE) and Maximum a posteriori (MAP) density estimation and probability distributions.
Part 1: Classification and Regression
- Introduction to classification (1 lectures): The Naive Bayes classifier.
- Probabilistic classification and regression (3 lectures): Generative vs. Discriminative models. Logistic regression, optimisation and regularisation.
- Computational Learning Theory (1 lecture): Model selection, PAC Learnability, the Vapnik-Chervonenkis dimension.
- Non-linear classification (3 lectures): The Perceptron, Support Vector Machines and Kernels.
Part 2: Clustering and Dimensionality Reduction
- Clustering (2 lectures): Mixture models and the EM algorithm.
- Topic Models (1 lecture): Latent Dirichlet Allocation.
- Dimensionality Reduction (1 lecture): Principle Component Analysis.
Part 3: Structured Learning
- Sequential Data (2 lectures): Hidden Markov Models
Syllabus
|
General Machine Learning concepts and models, supervised and un-supervised learning; Logistic Regression, Support Vector Machines; Evaluating models and algorithms, Computational Learning Theory; Clustering, latent variable models, EM, Hidden Markov Models. |
Reading list
Primary Text
- Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
Secondary Texts
- I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition Morgan Kaufmann, 2005.
- T.M. Mitchell, Machine Learning, McGraw-Hill, 1997.