Lecturer Varun Kanade
Hours Mon, Tue, Thu: 4-5pm
Location Lecture Theatre A
Notes (Lecture Notes)

Note 1. I only expect there to be about 20 hours of lectures in total. I have scheduled more hours and intend to have longer lectures at the start of term to ensure that there is enough time left if rescheduling is necessary.

Note 2. A set of lecture notes (LN) covering the entire course will be posted. These are not meant to be used as a substitute for reading the recommended textbooks and other posted reading. There might be some delay in updating the lecture notes. Should you wish to read up ahead of lectures, you may follow the material posted on the Computational Learning Theory course from Michaelmas Term 2021. The only difference between the MT2021 notes and the new ones is an attempt at serialization. Please make sure to make use of the most updated material when revising.

Note 3. There will be parts of suggested reading material that you struggle with or may seem not directly related to what we covered in the lecture. I suggest you skip these sections and only return to them once you've understood the material covered in the lecture and want to increase your understanding further.

Lecture Schedule

Date Topic Reading

10/10/2022 Setting up the PAC Learning Framework Chap 1.1-2 (LN)
Chap 1.1-2 (KV)

11/10/2022 PAC Learning Conjunctions Chap 1.2-3 (LN)
Chap 1.2-3 (KV)
Valiant's paper

13/10/2022 Hardness of learing 3-term DNF Chap 1.4-8 (LN)
Chap 1.4-7 (KV)

17/10/2022 Occam's Razor Chap 2 (LN)
Chap 2 (KV)

18/10/2022 VC Dimension; Sauer's Lemma Chap 3.1-3.2, 3.5 (LN)
Chap 3.1-4 (KV)

20/10/2022 VC Dimension; Sample Complexity Upper Bounds Chap 3.3 (LN)
Chap 3.5 (KV)

24/10/2022 VC Dimension; Sample Complexity Lower Bounds Chap 3.4 (LN)
Chap 3.6 (KV)

25/10/2022 Boosting Chap 4 (LN)
Adaboost survey
Adaboost article

27/10/2022 Cryptographic Hardness of Learning Chap 5 (LN)
Chap 6 (KV)

31/10/2022 Cryptographic Hardness of Learning;
Exact Learning using MQ+EQ
Chap 5, 6.1-2 (LN)
Chap 6, 8.1-2 (KV)
Article on shallow circuits for arithmetic operations

01/11/2022 Learning DFAs Chap 6.3 (LN)
Chap 8.3 (KV)

03/11/2022 Reductions; Separations; RCN Chap 7.1 (LN)
Chap 7 (KV), Chap 5.1-2

07/11/2022 Learning with RCN; SQ Learning Chap 7.2 (LN)
Chap 5.3-4 (KV)

08/11/2022 Hardness of SQ Learning PARITIES Chap 7.3 (LN)

10/11/2022 Learning Real-Valued Functions Chap 8.1-2 (LN)
[older notes]
Convex Opt. book (Chap. 3.1)

14/11/2022 Learning LMs, GLMs Chap 8.4-5 (LN)
[older notes]
Chap 3 (MRT)

15/11/2022 Rademacher Complexity Chap 8.3 (LN)
[older notes]
Chap 3 (MRT)

17/11/2021 Online Learning
Mistake-bounded Learning

Chap 9.1-3 (LN)
[older notes]
[even older notes]

21/11/2021 Perceptron and Winnow

Chap 9.4-5 (LN)
[older notes]
[even older notes]
Littlestone's paper

22/11/2021 Learning with Expert Advice

Chap 10 (LN)
[older notes]
MWUA Survey