Date	Lecture	Readings
08/21	Lecture #1 (Bo Dai): Course Overview [ slides ]
Module I: Background Knowledge
08/23	Lecture #2 (Bo Dai): Optimization: convex preliminary [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 1
08/28	Lecture #3 (Bo Dai): Optimization: convex set and function [ slides | notes ]	Boyd & Vandenberghe, Convex Optimization, Section 2-3
08/30	Lecture #4 (Bo Dai): Optimization: gradient descent [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 5
09/04	No Class (Labor Day)
09/06	Lecture #5 (Bo Dai): Sampling: basic sampling method [ notes ]	Bishop, Pattern Recognition and Machine Learning, Section 11
09/11	Lecture #6 (Bo Dai): Sampling: Acceptance-Rejection & Importance Sampling [ notes ]	Bishop, Pattern Recognition and Machine Learning, Section 11
09/13	Lecture #7 (Bo Dai): Sampling: MCMC (MH, Gibbs & Hamiltonian) [ notes ]	Neal, MCMC Using Hamiltonian Dynamics Bishop, Pattern Recognition and Machine Learning, Section 11
09/18	Lecture #8 (Bo Dai): Density Parametrization [ notes ]	Bishop, Pattern Recognition and Machine Learning, Section 8 Wainwright & Jordan, Graphical Models, Exponential Families, and Variational Inference.
09/20	No Class
09/25	Lecture #9 (Bo Dai): Neural Network Revisit [ notes ]
Module II: Deep Generative Models
09/27	Lecture #10 (Bo Dai): EBM (CD, Score Matching) [ notes ]	Song & Kingma, How to Train Your Energy-Based Models
10/02	Lecture #11 (Bo Dai): Autoregressive Model [ notes ]	Larochelle & Murray, The Neural Autoregressive Distribution Estimator
10/04	Lecture #12 (Bo Dai): VAE and Diffusion [ notes ]	Kingma & Welling, Auto-Encoding Variational Bayes Rezende, Mohamed, & Wierstra, Stochastic Backpropagation and Approximate Inference in Deep Generative Models
10/09	No Class (Fall Break)
10/11	Lecture #13 (Ruiqi Gao (Virtual) - Google DeepMind): Diffusion Process [ slides | notes | recording (Canvas) ]	Ho, Jain, & Abbeel, Denoising Diffusion Probabilistic Models Song, Sohl-Dickstein, Kingma, Kumar, Ermon, & Poole, Score-Based Generative Modeling through Stochastic Differential Equations
10/16	Lecture #14 (Lingkai Kong (in-person)): Decision-Focused Learning [ slides | notes ]
10/18	Lecture #15 (Sherry Yang (Virtual) - Google DeepMind): Foundation Models for Decision Making: Problems, Methods, and Applications [ slides | notes | recording (Canvas) ]	Yang et al. 2022. Foundation Models for Decision Making &#58 Problems, Methods, and Opportunities
10/23	Lecture #16 (Bo Dai): Generative Adversarial Nets (GAN) [ notes ]	Goodfellow et al. 2014, Generative Adversarial Networks Nowozin et al. 2016, f-GAN &#58 Training Generative Neural Samplers using Variational Divergence Minimization Dai et al. 2019, Exponential Family Estimation via Adversarial Dynamics Embedding
10/25	Lecture #17 (Bo Dai): Normalizing Flow Models [ notes ]	Kobyzev et al. 2019, Normalizing Flows &#58 An Introduction and Review of Current Methods
Module III: Differentiable Programming
10/30	Lecture #18 (Bo Dai): Differentiable Algorithm I: differentiable optimizer/dynamic programming [ notes ]	Tamar et al, 2016, Value Iteration Networks Dai et al, 2016, Discriminative Embeddings of Latent Variable Models for Structured Data Amos, Tutorial on amortized optimization
11/01	Lecture #19 (Bo Dai): Differentiable Algorithm II: top-K/sorting layer [ notes ]	Xie et al, Differentiable Top-k with Optimal Transport Berthet et al, Learning with Differentiable Perturbed Optimizers Sander et al, Fast, Differentiable and Sparse Top-k, a Convex Analysis Perspective
Module IV: Reinforcement Learning
11/06	Lecture #20 (Bo Dai): MDP: Bellman Recursion [ notes ]	Puterman and Chan, 2023. Introduction to Markov Decision Processes Mohri et al, 2018. Foundations of Machine Learning
11/08	Lecture #21 (Bo Dai): DP: Value and Policy Iteration [ notes ]	Puterman and Chan, 2023. Introduction to Markov Decision Processes Mohri et al, 2018. Foundations of Machine Learning
11/13	Lecture #22 (Bo Dai): Learning with MDPs [ notes ]	Mnih et al, 2013. Playing Atari with Deep Reinforcement Learning Mohri et al, 2018. Foundations of Machine Learning
11/15	Lecture #23 (Bo Dai): Policy Gradient and Actor-Critic [ notes ]	Sutton et al, 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation Haarnoja et al, 2018. Soft Actor-Critic -- Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
11/20	Lecture #24 (Bo Dai): Imitation Learning and RLHF [ slides | notes ]	Ross et al, 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning Ouyang et al, 2022. Training language models to follow instructions with human feedback
11/22	No Class (Student Recess)
11/27	Lecture #25 (Bo Dai): Review [ notes ]
11/29	Lecture #26 : Project Presentations (online session: 4:00 - 6:15 PM)
12/11	Project Report Due