Machine Learning: 20102011
Lecturer 

Degrees 
Schedule B2 — Computer Science 
Term 
Hilary Term 2011 (16 lectures) 
Overview
MSc students will be assessed by invigilated exam lasting approximately 2 hours in week 0 of TT.
It has long been a central concern of Computer Science to create machines capable of learning from experience. From predicting movies a customer will like, to learning to translate human languages from examples, systems that learn are becoming increasing prevalent and effective in our everyday lives. This course will introduce the field of Machine Learning, in particular focusing on the core concepts of supervised and unsupervised learning. In supervised learning we will discuss algorithms which are trained on input data labelled with a desired output, for instance an image of a face and the name of the person whose face it is, and learn a function mapping from the input to the output. Unsupervised learning aims to discover latent structure in an input signal where no output labels are available, an example of which is grouping webpages based on the topics they discuss. Students will learn the algorithms which underpin many popular Machine Learning techniques, as well as developing an understanding of the theoretical relationships between these algorithms. The practicals will concern the application of machine learning techniques to a range of realworld problems, with particular reference to working with data on the web.
Learning outcomes
On completion of the course students will be expected to:
 Have a good understanding of the fundamental issues and challengesin machine learning: data, model selection, feature selection, model complexity, etc.
 Have an understanding of the strengths and weaknesses of many ofthe machine learning approaches.
 Appreciate the underlying mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and unsupervised learning.
 Be able to design and implement various machine learning algorithms in a wide range of realworld applications.
Prerequisites
Modern Machine Learning is a mathematical discipline and as such students will benefit from a familiarity with basic probability theory and linear algebra, as well as some programming skills. The course is selfcontained and does not require any previous knowledge of Artificial Intelligence.Synopsis
 Introduction to Machine Learning (1 lecture): classification, regression and clustering; supervised and unsupervised learning.
 Mathematical preliminaries (1 lecture): Introduction to probability: random variables, independence, Bayes' theorem, conditional and marginal probabilities. Maximum likelihood (MLE) and Maximum a posteriori (MAP) density estimation and probability distributions.
Part 1: Classification and Regression
 Introduction to classification (1 lectures): The Naive Bayes classifier.
 Probabilistic classification and regression (3 lectures): Generative vs. Discriminative models. Logistic regression for multiclass classification. Optimisation and regularisation.
 Computational Learning Theory (1 lecture): Model selection, feature selection, ROC analysis. PAC Learnability, the VapnikChervonenkis dimension.
 Nonlinear classification (3 lectures): The Perceptron, Support Vector Machines and Kernels.
Part 2: Clustering and Dimensionality Reduction
 Clustering (2 lectures): Mixture models and the EM algorithm.
 Topic Models (1 lecture): Latent Dirichlet Allocation.
 Dimensionality Reduction (1 lecture): Principle Component Analysis.
Part 3: Structured Learning
 Sequential Data (2 lectures): Hidden Markov Models
Syllabus
General Machine Learning concepts and models, Logistic Regression, Support Vector Machines; Evaluating models and algorithms, Computational Learning Theory; Mixtures of Gaussians, EM, LDA and PCA; HMMs. 
Reading list
Primary Text
 Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
Secondary Texts
 I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition Morgan Kaufmann, 2005.
 T.M. Mitchell, Machine Learning, McGrawHill, 1997.