MATH 462: Mathematics for Machine Learning

• Fall 2021

A mathematically rigorous approach to machine learning.

Audience

Math and Stats Majors/Honours students, CS students.

Prerequisites

• Math 223/Math 248 or equivalent (linear algebra majors/honours)
• Math 222 (Calculus 3)
• Math 324/Math 357 (Statistics)
• Probability course (implied, since it is a prerequisite for 324/357)
• Math 358 Honours Vector Calculus/Math 314 Advanced Calculus
• Math 562, Winter 2022
• This is a sequel to Math 462. There will be a small amount of repetition of topics, since some students will not have taken 462. However, to avoid repeated material, students from 462 will work on their project and be excused from HW problems which overlap.
• COMP 551 Applied Machine Learning https://www.siamak.page/courses/COMP551F21/index.html
• This course focuses on implementation, rather than theory. This a complementary course.
• COMP 451 Fundamentals of Machine Learning https://cs.mcgill.ca/~wlh/comp451/
• CS theory course, not currently offered. Math 452 and Comp 451 are mutually exclusive.
• Math 308 Fundamentals of Statistical Learning
• not offered this year.

Textbook/References

• 5 HW assignments : 25%
• Group Project and Presentation: 15%
• Attendance 5%, Participation 5%.
• 2 Midterm Exams : 20%
• Final exam : 30%
• Soft grading policy: you are encouraged to make your best effort to complete all the work. However, if you need to miss anything (assignment or exam), I will institute a soft grading policy which will allow one missed assignment and one missed midterm exam, with a small penalty. Your final grade will be given by your average on the other work, with a penalty of:
• 1% (for each assignment missed),
• 2% (for a missed midterm).
• 3% (for a missed final exam)

E.g. if you missed one assignment and one midterm, with an average of 87% on the rest, then the penalty would be 87-(1+2) = 84%.

Key Dates

Refer to McGill key Dates

• Classes begin: Weds Sept 1.
• Fall reading break: Tues-Weds Oct. 12-13
• Makeup day: Oct 15, no class.
• Last class: Friday Dec 3rd.
• Midterm dates: TBD

Lecture Notes : MATH 462, Fall 2021

Lectures Part 3

Weeks 8 and 9, Oct 27 and 29th, Nov 3rd and 5th

• (Weds) Project Outline and Examples (see links above)
• (Friday) Lecture 15 Deep Neural networks.
• Additional reference for CNNs: https://www.deeplearningbook.org/ Chapter 6 and Chapter 9
• (Weds) Face Verification Problem. Generalization: in distribution and out of distribution Lecture 16
• (Friday) Unsupervised: cluster energy. Semi-supervised SVM and margin. Reinformence learning warmup. Lecture 17

Lectures Part 2

Week 3, Lecture 6, Sept 17 (F)

• Loss design for classification: zero-one loss and hinge loss.

Lectures Part 1

Week 1, lecture 1, Sept 1 (W)

• Introduction: example problems and datasets, meet and greet
• Reference: Murphy Ch1

Week 1, Lecture 2, Sept 3 (F)

• Set up the supervised learning problem for regression

Week 2, Lecture 3, Sept 8 (W)

• Regression, other losses, compare losses
• Calculus to find the minimizer of the (ELM) problem
• Reference: Course notes

Week 2, Lecture 4, Sept 10 (F)

• Review of calculus and vector calculus, chain rule, gradient
• Compute gradient of the (EL) expected loss function