MATH/COMP 562: Theory of Machine Learning
McGill has announced online teaching for this course, at least until January 24th. I hope we can return to in person teaching, which will be better for group work and presentations.
Prerequisites: MATH 462 or COMP 451 or (COMP 551, MATH 222, MATH 223 and MATH 324) or ECSE 551.
Gabriela Moisescu-Pareja: email@example.com
Math 462, Fall 2021
To make sure that we all have the same background, there will be some overlap with this course in the first few weeks of term (of the 80+ students in 562, under 20 took 452). Students from 462 can start their projet, and be excused from the small number of HW problems which overlap.
COMP 551 Applied Machine Learning
This course focuses on implementation, rather than theory. This a complementary course.
Additional general references
ML Theory references
Update (Jan 21).
Attendance and Participation (attend group presentations, ask questions, screen on for zoom when possible): 10%
Group Project. Writeup and presentation:30%
Individual Project/Final Report: 30%
Changes due to COVID I reserve the right to make changes to the assignment structure/grading scheme in response to the COVID situation. However, significant changes will be presented to the class for feedback, and will go forward only if there is strong agreement by the class. Key Times and Dates (TBD)
McGill key Dates
Classes are every Tuesday and Thursday 11:30-1pm.
First class: Thursday Jan 6th. (11:30am-1pm, zoom link on mycourses page).
Reading Break: Feb 28 - March 4th
Classes end Tuesday April 12th.
Prerequisite material (which will be covered in more detail later in the course)
Foundational Machine Learning: Classification, binary and multi-classes. Classification losses, convex surrogate losses.
Scoring function and class probability interpretations.
Training models: Optimization, stochastic gradient descent.
Later in the course: Introduction to deep learning. Introduction and math background for computer vision/generative models, NLP, RL.
Part 1: Introduction to Machine Learning, Deep Learning. Topics in deep learning: Reinforcement Learning, Natural Language Processing, Computer Vision. Generalization in (shallow) machine learning. References: Shalev-Shwartz (first part), Mohri (latter part). PAC Learning bounds, VC dimension, Concentration of measure, Rademacher complexity.
Group and individual projects
Chapter References [Machine Learning]:
Read [SS, Chapter 1] on your own.
Types of Learning, [SS Section 1.3] :
Statistical Learning FrameworkEmpirical Risk Minimization
Overfitting, [SS Section 2.1, 2.2]
Notes from Lecture 1 Lecture 2:
Read [M, Ch 1] on your own.
Review linear predictors, linear classification, linear regression. References: [SS Ch 9] or Math 462 course notes.
Lecture 2 (class notes) Reference Mohri Ch 2. PAC Learning Framework [M 2.1]
Learning Guarantees: Finite hypothesis sets, consistent case. [M 2.2]
Equivalent references: [SS 2.3 and SS 3.1]
RKHS theory, Ref: Wainwright Ch 12 section 2 and 3, RKHS
Guest lectures from former students on RL for ventilator and word embeddings.
RKHS applications: function fitting and density estimation, section 12.5 and 12.6
Introduction to Reinforcement Learning: MDPs, Policy, Value function, the RL objective
Bellman expectation equations, Bellman optimality equations (and respective operators)
Optimal value function if and only if satisfies the optimality equations
Solving MDPs when dynamics are known: value iteration vs. policy iteration, sample efficiency
When dynamics and rewards are unknown: predicting the value function
TD learning, convergence "proof", off-policy learning
Q-learning algorithm: applying TD ideas to controlling the agent
Deep Q-learning (DQN): experience replay and stable TD targets
Introduction to exploration in model-based learning for finite-horizon MDPs
Lecture 16 (prof. Panangaden)
Bisimulation, bisimulation metrics for MDPs
Metrics on the space of probability distributions, coupling, Kantorovich metric