Lecturer Varun Kanade
Hours Mon, Tue, Thu: 4-5pm
Location Lecture Theatre A
Notes (Lecture Notes)

Note 1. I only expect there to be about 20 hours of lectures in total. I have scheduled more hours and intend to have longer lectures at the start of term to ensure that there is enough time left if rescheduling is necessary.

Note 2. A set of lecture notes (LN) covering the entire course will be posted. These are not meant to be used as a substitute for reading the recommended textbooks and other posted reading. There might be some delay in updating the lecture notes. Should you wish to read up ahead of lectures, you may use the lecture notes posted for the Computational Learning Theory course from Michaelmas Term 2022. The lecture notes from MT2022 are complete, but I will add additional explanations, references and problems, and also fix typos. Please email me directly if you find any typos. Please make sure to make use of the most updated material when revising.

Note 3. There will be parts of suggested reading material that you struggle with or may seem not directly related to what we covered in the lecture. I suggest you skip these sections and only return to them once you've understood the material covered in the lecture and want to increase your understanding further.

Lecture Schedule

Date Topic Reading

09/10/2023 Setting up the PAC Learning Framework Chap 1.1-2 (LN)
Chap 1.1-2 (KV)

10/10/2023 PAC Learning Conjunctions Chap 1.2-3 (LN)
Chap 1.2-3 (KV)
Valiant's paper

12/10/2023 Hardness of learing 3-term DNF Chap 1.4-8 (LN)
Chap 1.4-7 (KV)

16/10/2023 PAC Learning; Occam's Razor Chap 1.8, Chap 2 (LN)
Chap 1.7, 2 (KV)

17/10/2023 VC Dimension; Sauer-Shelah Lemma Chap 3.1-2 (LN)
Chap 3.1-4 (KV)

19/10/2023 VC Dimension Upper Bound Chap 3.3 (LN)
Chap 3.5 (KV)

23/10/2023 VC Dimension Lower Bound; Boosting Chap 3.4, 4 (LN)
Chap 3.6 (KV)

24/10/2023 AdaBoost Chap 4 (LN)
Adaboost survey
Adaboost article

26/10/2023 Cryptographic Hardness of Learning Chap 5 (LN)
Chap 6 (KV)

30/10/2023 Exact Learning with MQ + EQ Chap 6.1-2 (LN)
Chap 8.1-2 (KV)

31/10/2023 Exact Learning DFA; L* Algorithm Chap 6.3 (LN)
Chap 8.3 (KV)

02/11/2023 Learning with RCN Chap 7.1 (LN)
Chap 5.1-2 (KV)

06/11/2023 SQ Learning; Hardness of Learning PARITIES Chap 7.2-3 (LN)
Chap 7.3 (KV)

07/11/2023 Learning Real-Valued Functions; Gradient Descent Chap 8.1-2 (LN)
Convex Opt. book (Chap. 3.1)

09/11/2023 Learning LMs, GLMs Chap 8.4-5 (LN)
Chap 3 (MRT)

13/11/2023 Rademacher Complexity Chap 8.3 (LN)
Chap 3 (MRT)

14/11/2023 (Online) Mistake-Bounded Learning Chap 9.1-3 (LN)
Littlestone's paper

16/11/2023 Perceptron and Winnow Chap 9.4-5 (LN)
Littlestone's paper

20/11/2023 Exponentially-weighted Forecaster; Regret Chap 10 (LN)
MWUA Survey