MATH 462, Fall 2023

Honours Mathematics for Machine Learning,

Professor: Adam Oberman

Class: Tuesday and Thursday, 11:35 am-12:55 am

Office Hours Fall 2023: BURN 1106, 1:00pm-2:00pm and by appointment.

Audience:

Math and Stats Majors/Honours students, Computer Science students.

• Course material: https://adam-oberman.github.io/
• Assigment submission: https://mycourses2.mcgill.ca/
• Math.462.Fall.2023.First.Day.Handout.pdf updated after first class - project instead of midterm Math 462 Fall 2023 First Day Handout.pdf

Lectures

• Lecture 1, 08.31 Thursday.

  • Decision Trees Example from Russell Norvig AI book RN_Decision_trees.pdf

• Lecture 2, 09.05 Tuesday.

  • Decision Trees Theory. Classification Losses/Gains.

• Overleaf for Lecture 1 and 2 are available. Link to notes posted on mycourses

• Lecture 3, 09.07 Thursday.

  • Review Notes
  • Intro to Python
  • Intro to Numpy
  • Code for entropy, gini index
  • Sample decision tree code
  • Bayesian Models and Spam detection

• Lecture 4, 09.12 Tuesday

  • Notes Lecture 3 and 4 Deriving Losses
  • Coding

• Lecture 5, 09.14 Thursday

  • Outline and links to code and references
  • (temporary file:) Machine_Learning_in_Action Ch Trees.pdf
  • and (temporary file:) Machine_Learning_in_Action_CH_BAYES.pdf

• Lecture 6, 09.19 Tuesday: understanding Neural Networks

• Lecture 7, 09.21 : Neural Networks code

  • Notes taken from Sections 4.4, 4.5, 4.6 of (temporary link) Mitchell Ch 4.pdf
  • Notes and code also from https://towardsdatascience.com/how-neural-networks-solve-the-xor-problem-59763136bdd7
  • Course notes: Math462_NN.pdf
  • Code: Perceptron Code

• HW 2

  • Code for Bayes Classifier
  • HW 2, see PDF in MyCourses

• Lecture 8 09.26

  • Vector Calc for ML Math462_Lecture_Note_2_Calculus_Regression.pdf
  • Stochastic Optimization (Updated Oct 10) Math462_Lecture_Note_SGD.pdf

• Lecture 9, 09.28 (Cancelled)

• Lecture 10 and 11, 10.03 and 10.05 Binary Classidfication Class_Binary.pdf and Multi-Class classification losses Class_Multi.pdf

• HW 3

  • HW3.pdf
  • revised for clarity! HW3revised.pdf

• Oct 10, no lecture, thanksgiving break

• Oct 12, guest lecture.

• Lecture 12, Oct 17.Math462_Lecture_Notes_Kmeans.pdf

• Lecture 13, Oct 24. Math462_Inner_Products_Projections.pdf

• Lecture 14, PCA, Oct 26

  • Additional Reference: Jeff Calder Notes Ch 3

• Lectures 15 and 16, Oct 31, Nov 2. Convex Learning problems. Shalev-Shwartz Chapter 12.

  • Temporary link: SS Ch 12 Convex Learning.pdf

• HW 4 HW4.pdf -updated HW4.pdf

• Lecture 17 and 18, Nov 7 and Nov 9.

  • Introduction to generative models. Autoencoders Unsupervised learning Generative Models.pdf

  • Approximation Theory. Temporary Link: Approximation theory Burden Faires Numerical Analysis 9th Burden Faires.pdf

• Lecture 19 and 20, Nov 14 and 16,

  • Math462_Feature_Regression.pdf
  • Math462_Orthogonal_Function_Regression.pdf

• HW5.pdf

• (Updated) HW5.pdf

• Nov 21, homework session, and project session. Nov 23, Review and project session

• old HW_Combined.pdf

• updated HW Combined: HW_Combined.pdf