Lectures

Lecturer Varun Kanade
Hours 16h-17h30 Monday, Tuesday and Thursday
Location Lecture Theatre A (Department of Computer Science)
Notes (Draft Lecture Notes)

Note 1. I only expect there to be about 20 hours of lectures in total. I have scheduled more hours and intend to have longer lectures at the start of term to ensure that there is enough time left if rescheduling is necessary for pandemic-related reasons.

Note 2. A set of lecture notes (LN) covering the entire course will be posted. These are not meant to be used as a substitute for reading the recommended textbooks and other posted reading. There might be some delay in updating the lecture notes. Should you wish to read up ahead of lectures, you may follow the material posted on the Computational Learning Theory course from Michaelmas Term 2018. The only difference between the MT2018 notes and the new ones is an attempt at serialization. Please make sure to make use of the most updated material when revising.

Note 3. There will be parts of suggested reading material that you struggle with or may seem not directly related to what we covered in the lecture. I suggest you skip these sections and only return to them once you've understood the material covered in the lecture and want to increase your understanding further.


Lecture Schedule

Date Topic Reading

11/10/2021 Setting up the PAC Learning Framework Chap 1.1-2 (LN)
Chap 1.1 (KV)

12/10/2021 Learning Conjunctions Chap 1.2-3 (LN)
Chap 1.2-3 (KV)
Valiant's paper

14/10/2021 Hardness of Learning 3-term DNF; PAC Learning Chap 1.4-6 (LN)
Chap 1.4-5 (KV)

18/10/2021 Occam's Razor; Consistent Learning Chap 2, Chap 3.5 (LN)
Chap 2 (KV)

19/10/2021 VC Dimension, Sauer's Lemma Chap 3.1-2 (LN)
Chap 3.1-4 (KV)

21/10/2021 VC Dimension: Sample Complexity Bounds Chap 3.3 (LN)
Chap 3.5 (KV)

25/10/2021 VC Dimension: Sample Complexity Bounds
Boosting
Chap 3.4, 4.1 (LN)
Chap 3.6 (KV)

26/10/2021 Adaboost Chap 4.2 (LN)
Adaboost survey
Adaboost article

28/10/2021 Cryptographic Hardness of Learning Chap 5 (LN)
Chap 6 (KV)
Article on shallow circuits for arithmetic operations

01/11/2021 Reductions
Exact Learning using MQs + EQs
Chap 6 (LN)
Chap 7, 8 (KV)

02/11/2021 Learning DFAs using MQs + EQs Chap 6 (LN)
Chap 8 (KV)

04/11/2021 Learning with Random Classification Noise
SQ Learning
[notes]
Chap 5 (KV)

08/11/2021 SQ Learning
Hardness of Learning PARITIES
[notes]
Chap 5 (KV)

09/11/2021 Learning Real-Valued Functions
Convex Optimization
[notes]
Convex Opt. book (Chap. 3.1)

11/11/2021 Learning Real-Valued Functions
Rademacher Complexity
[notes]
Chap 3 (MRT)

15/11/2021 Rademacher Complexity [notes]
Chap 3 (MRT)

16/11/2021 Online Learning
Mistake-bounded Learning

[notes]
[older notes]
18/11/2021 Perceptron and Winnow [notes]
[older notes]
Littlestone's paper

22/11/2021 Online Learning with Expert Advice [notes]
MWUA Survey