DNN | Deep Neural Networks |
This course will aim to introduce students to the core fundamentals of modern deep multi-layered neural networks, while still remaining grounded in practice. The underpinning assumption in its design is that while students may have experience (especially hands-on experience) in machine learning, data science or general software engineering — they have not worked with deep learning or taken prior courses in the area.
Frequency
This course normally runs twice a year.
Course dates
18th November 2024 | Oxford University Department of Computer Science - Held in the Department | 0 places remaining. |
10th March 2025 | Oxford University Department of Computer Science - Held in the Department | 0 places remaining. |
Objectives
At the conclusion of this course students should understand:
- The principles and approaches for learning with deep neural networks.
- The main variants of deep learning (such convolutional and recurrent architectures), and their typical applications.
- The key concepts, issues and practices when training and modeling with deep architectures; as well as have hands-on experience in using deep learning frameworks for this purpose.
- How to implement basic versions of some of the core deep network algorithms (such as back-propagation)
- How deep learning fits within the context of other machine learning approaches, and what tasks it is considered to be suited and not well suited to perform.
Contents
- Introduction to Neural Networks and Deep Learning
- Background, History and Intuition
- Supervised Training Methods, and Basic Architectures
- Multi-layer Perceptrons, Backpropagation, Stochastic Gradient Descent
- Convolutional Networks and Image Applications
- Unsupervised Methods and Related Architectures
- Autoencoders and Generative Adversarial Networks
- Recurrent Networks and Time-series/Sequential Applications
- Modeling Audio, and Case studies in Speech and Sounds
- Data Augmentation and Transfer Learning
- Practical Implications due to Hardware, Systems and the Cloud
- Scalability and Efficiency of Training and Inference
Requirements
Software Tools: PyTorch and TensorFlow
Prerequisites: Students registering for this course must satisfy three area of background knowledge.
- Mathematical Foundations — Undergraduate courses in the following topics: calculus, probability and linear algebra
- Programming Skills — Undergraduate level programming courses or experience that indicates programming proficiency
- Machine Learning Experience — Students are required to have taken at least one prior course in machine learning at an introductory level during earlier study. Such a course should have provided basic machine learning concepts and ideas, as well as described some popular modelling approaches (at the time the course was taken). There is no expectation this course would have covered any content related to deep learning or types of neural networks.
Compute Resources and Reading Material:
- Compute resources offered via Google Cloud (free)
- Recommended reading: Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning (http://www.deeplearningbook.org/)