MATH/COMP 562: Theory of Machine Learning, Winter 2023

Honours Mathematics for Machine Learning

Professor: Adam Oberman

Teaching Assistant: Viet Nguyen

Class time: Tuesday and Thursday 2:35pm - 3:55 pm. Lea 14

Office Hours: Tuesday and Thursday 1:05pm-1:30pm and 4:05pm-4:30pm. Wednesday 3:30-4:00pm (by appointment). Occasionally family life interferes with my schedule, and I may miss the office hour. An email in advance is appreciated, but is not necessary.

Lecture notes and assignments:

Assigment submission:

First Day Handout 2023 MATHCOMP 562.pdf

Course Description

A mathematically rigorous approach to Machine Learning (ML), with a focus on a rigourous presentation of ML models. Proofs of in-distribution generalization bounds.


Students will be expected to have seen and coded ML models before. Experience with mathematical proofs and with probability is expected.


Learning Theory from First Principles, (January 1, 2023 edition) Free PDF, by Francis Bach

Additional references

Use Shalev-Shwartz for introduction and definitions. Use Mohri for the proofs, which are more precise.


Be sure to discuss both an early reference, and track back references to early work, which may be more clear.

Lectures and Homework

Generative Model Lectures.

Reference, Chapters 14-18

Homework 3

Reinforcement Learning Lecture March 21 (handwritten notes and references below)

Lecture March 23 (Thursday) Markov Decision Process

Making Complex Decisions.pdf

Lecture March 28 (Tuesday) and March 30 (Thursday) Markov Decision Process II

Lecture April 4 (Tuesday) and April 6 (Thursday) Contractions for Bellman Equation and error estimates


Homework 5 (on RL and final lectures)

Final Exam

Project reviewer forms