Machine Learning: 20132014
Lecturer 

Degrees 
Schedule B2 — Computer Science Schedule B2 — Mathematics and Computer Science 
Term 
Michaelmas Term 2013 (16 lectures) 
Overview
It has long been a central concern of Computer Science to create machines capable of learning from experience. From predicting movies a customer will like, to learning to translate human languages from examples, systems that learn are becoming increasing prevalent and effective in our everyday lives. This course will introduce the field of Machine Learning, in particular focusing on the core concepts of supervised and unsupervised learning. In supervised learning we will discuss algorithms which are trained on input data labelled with a desired output, for instance an image of a face and the name of the person whose face it is, and learn a function mapping from the input to the output. Unsupervised learning aims to discover latent structure in an input signal where no output labels are available, an example of which is grouping webpages based on the topics they discuss. Students will learn the algorithms which underpin many popular Machine Learning techniques, as well as developing an understanding of the theoretical relationships between these algorithms. The practicals will concern the application of machine learning to a range of realworld problems, with particular reference to working with data on the web.
Learning outcomes
On completion of the course students will be expected to:
 Have a good understanding of the fundamental issues and challenges of machine learning: data, model selection, model complexity, etc.
 Have an understanding of the strengths and weaknesses of many popular machine learning approaches.
 Appreciate the underlying mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and unsupervised learning.
 Be able to design and implement various machine learning algorithms in a range of realworld applications.
Prerequisites
Modern Machine Learning is a mathematical discipline and as such students will benefit from a familiarity with probability theory and linear algebra, as well as some programming skills. The course is selfcontained and does not require any previous knowledge of Artificial Intelligence.
Synopsis
 Introduction to Machine Learning (1 lecture): classification, regression and clustering; supervised and unsupervised learning.
 Mathematical preliminaries (1 lecture): Introduction to probability: random variables, independence, Bayes' theorem, conditional and marginal probabilities. Maximum likelihood (MLE) and Maximum a posteriori (MAP) density estimation and probability distributions.
Part 1: Classification and Regression
 Introduction to classification (1 lectures): The Naive Bayes classifier.
 Probabilistic classification and regression (3 lectures): Generative vs. Discriminative models. Logistic regression, optimisation and regularisation.
 Model Selection (1 lecture): Model selection, metrics, bias variance tradeoffs.
 Nonlinear classification (3 lectures): The Perceptron, Support Vector Machines and Kernels.
Part 2: Clustering and Dimensionality Reduction
 Clustering (2 lectures): Mixture models and the EM algorithm.
 Topic Models (1 lecture): Latent Dirichlet Allocation.
 Dimensionality Reduction (1 lecture): Principle Component Analysis.
Part 3: Structured Learning
 Sequential Data (2 lectures): Hidden Markov Models
Syllabus
General Machine Learning concepts and models, supervised and unsupervised learning; Logistic Regression, Support Vector Machines; Evaluating models and algorithms; Clustering, latent variable models, EM, Hidden Markov Models. 
Reading list
Primary Text
 Kevin P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press 2012.
Secondary Texts
 Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
 I.H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition Morgan Kaufmann, 2005.
 T.M. Mitchell, Machine Learning, McGrawHill, 1997.