University of Oxford Logo University of OxfordSoftware Engineering - Home
On Facebook
Facebook
Follow us on twitter
Twitter
Linked in
Linked in
Google plus
Google plus
Digg
Digg
Pinterest
Pinterest
Stumble Upon
Stumble Upon
DNN

Deep Neural Networks

This course will aim to introduce students to the core fundamentals of modern deep multi-layered neural networks, while still remaining grounded in practice. The underpinning assumption in its design is that while students may have experience (especially hands-on experience) in machine learning, data science or general software engineering — they have not worked with deep learning or taken prior courses in the area.

Frequency

This course normally runs twice a year.

Course dates

18th November 2024Oxford University Department of Computer Science - Held in the Department 0 places remaining.
10th March 2025Oxford University Department of Computer Science - Held in the Department 0 places remaining.

Objectives

At the conclusion of this course students should understand:

  1. The principles and approaches for learning with deep neural networks.
  2. The main variants of deep learning (such convolutional and recurrent architectures), and their typical applications.
  3. The key concepts, issues and practices when training and modeling with deep architectures; as well as have hands-on experience in using deep learning frameworks for this purpose.
  4. How to implement basic versions of some of the core deep network algorithms (such as back-propagation)
  5. How deep learning fits within the context of other machine learning approaches, and what tasks it is considered to be suited and not well suited to perform.

Contents

Introduction to Neural Networks and Deep Learning
Background, History and Intuition
Supervised Training Methods, and Basic Architectures
Multi-layer Perceptrons, Backpropagation, Stochastic Gradient Descent
Convolutional Networks and Image Applications
Unsupervised Methods and Related Architectures
Autoencoders and Generative Adversarial Networks
Recurrent Networks and Time-series/Sequential Applications
Modeling Audio, and Case studies in Speech and Sounds
Data Augmentation and Transfer Learning
Practical Implications due to Hardware, Systems and the Cloud
Scalability and Efficiency of Training and Inference

Requirements

Software Tools: PyTorch and TensorFlow

Prerequisites: Students registering for this course must satisfy three area of background knowledge.

  1. Mathematical Foundations — Undergraduate courses in the following topics: calculus, probability and linear algebra
  2. Programming Skills — Undergraduate level programming courses or experience that indicates programming proficiency
  3. Machine Learning Experience — Students are required to have taken at least one prior course in machine learning at an introductory level during earlier study. Such a course should have provided basic machine learning concepts and ideas, as well as described some popular modelling approaches (at the time the course was taken). There is no expectation this course would have covered any content related to deep learning or types of neural networks.

Compute Resources and Reading Material:

  • Compute resources offered via Google Cloud (free)
  • Recommended reading: Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning (http://www.deeplearningbook.org/)