Machine Learning: 2015-2016
OverviewMachine learning techniques enable us to automatically extract features from data so as to solve predictive tasks, such as speech recognition, object recognition, machine translation, question-answering, anomaly detection, medical diagnosis and prognosis, automatic algorithm configuration, personalisation, robot control, time series forecasting, and much more. Learning systems adapt so that they can solve new tasks, related to previously encountered tasks, more efficiently.
This course will introduce the field of Machine Learning, in particular focusing on the core concepts of supervised and unsupervised learning. In supervised learning we will discuss algorithms which are trained on input data labelled with a desired output, for instance an image of a face and the name of the person whose face it is, and learn a function mapping from the input to the output. Unsupervised learning aims to discover latent structure in an input signal where no output labels are available, an example of which is grouping web-pages based on the topics they discuss. Students will learn the algorithms which underpin many popular Machine Learning techniques, as well as developing an understanding of the theoretical relationships between these algorithms. The practicals will concern the application of machine learning to a range of real-world problems.
On completion of the course students will be expected to:
- Have a good understanding of the fundamental issues and challenges of machine learning: data, model selection, model complexity, etc.
- Have an understanding of the strengths and weaknesses of many popular machine learning approaches.
- Appreciate the underlying mathematical relationships within and across Machine Learning algorithms and the paradigms of supervised and un-supervised learning.
- Be able to design and implement various machine learning algorithms in a range of real-world applications.
Machine Learning is a mathematical discipline, and students will benefit from a good background in probability, linear algebra and calculus. Programming experience is essential.
- 1. Introduction to machine learning
- 2. Linear prediction: Regression
- 3. Maximum likelihood
- 4. Regularizers, cross-validation, learning curves
- 5. Optimisation
- 6. Classification: Linear classification, logistic regression
- 7. Classification: Support vector machines
- 8. Kernels
- 9. Feed-forward neural networks
- 10. Backpropagation
- 11. Convolutional neural networks
- 12. Unsupervised learning: Clustering
- 13. Probabilistic modelling: EM Algorithm
- 14. Principal component analysis
- 15. Recommendation systems, collaborative filtering
- 16. Reinforcement learning
Mathematics of machine learning. Overview of supervised, unsupervised, and reinforcement learning; and important notions such as maximum likelihood, regularization, cross-validation.
- T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer 2011. (Available for download on the authors' web-page: http://statweb.stanford.edu/~tibs/ElemStatLearn/)
- Kevin P. Murphy. Machine Learning: A Probabilistic Perspective, MIT Press 2012. (Electronic copy available through the Bodleian library.)
- Christopher M. Bishop. Pattern Recognition and Machine Learning, Springer 2007.
- S. Haykin. Neural networks and learning machines. Pearson 2008.
Copies of all textbooks are available for short loan in the department library. Further copies may also be available in the RSL and college libraries.