Date	Lecture	Readings
08/19	Lecture #1 (Bo Dai): Course Overview [ slides ]
Module I: Background Knowledge
08/21	Lecture #2 (Bo Dai): Optimization: convex preliminary [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 1
08/26	Lecture #3 (Bo Dai): Optimization: convex set and function I [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 2-3
08/28	Lecture #4 (Bo Dai): Optimization: convex set and function II [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 2-3
09/02	No Class (Labor Day)
09/04	Lecture #5 (Bo Dai): Optimization: gradient descent & Density Parametrization I [ notes ]	Boyd & Vandenberghe, Convex Optimization, Section 5 Bishop, Pattern Recognition and Machine Learning, Section 8 Wainwright & Jordan, Graphical Models, Exponential Families, and Variational Inference.
09/09	Lecture #6 (Bo Dai): Density Parametrization II [ notes ]	Bishop, Pattern Recognition and Machine Learning, Section 8 & 11 Wainwright & Jordan, Graphical Models, Exponential Families, and Variational Inference.
09/09	Team Formation Due
09/11	Lecture #7 (Bo Dai): Sampling: Basic Sampling, Acceptance-Rejection & Importance Sampling [ notes ]	Bishop, Pattern Recognition and Machine Learning, Section 11
09/16	Lecture #8 (Bo Dai): Sampling: MCMC (MH, Gibbs & Hamiltonian) [ notes ]	Neal, MCMC Using Hamiltonian Dynamics Bishop, Pattern Recognition and Machine Learning, Section 11
09/18	Lecture #9 (Bo Dai): Neural Network Revisit [ notes ]
Module II: Deep Generative Models
09/23	Lecture #10 (Bo Dai): EBM (CD, Score Matching) [ notes ]	Song & Kingma, How to Train Your Energy-Based Models
09/25	Lecture #11 (Bo Dai): EBM and Diffusion [ notes ]	Song et al. 2020, Score-Based Generative Modeling through Stochastic Differential Equations Shribak et al. 2024, Diffusion Spectral Representation for Reinforcement Learning Song et al. 2019, Generative Modeling by Estimating Gradients of the Data Distribution
09/30	Lecture #12 (Bo Dai): VAE and Diffusion I	Kingma & Welling, Auto-Encoding Variational Bayes Rezende, Mohamed, & Wierstra, Stochastic Backpropagation and Approximate Inference in Deep Generative Models
10/02	Lecture #13 (Bo Dai): VAE and Diffusion II	Ho, Jain, & Abbeel, Denoising Diffusion Probabilistic Models Wang, Blog-What are Diffusion Models?
10/07	Lecture #14 (Bo Dai): Autoregressive Model	Larochelle & Murray, The Neural Autoregressive Distribution Estimator
10/09	Lecture #15 (Bo Dai): Generative Adversarial Nets (GAN) and Normalizing Flow Models	Kobyzev et al. 2019, Normalizing Flows &#58 An Introduction and Review of Current Methods Goodfellow et al. 2014, Generative Adversarial Networks Nowozin et al. 2016, f-GAN &#58 Training Generative Neural Samplers using Variational Divergence Minimization Dai et al. 2019, Exponential Family Estimation via Adversarial Dynamics Embedding
10/14	Fall Break
Module III: Representation Learning
10/16	Lecture #16 ( Runyu Zhang (Harvard) ): (Zoom link: https://gatech.zoom.us/j/93227128162) Scalable Distributed Control and Reinforcement Learning for Multi-Agent Network Systems [ recording (Canvas) ]	Multi-agent network systems play a critical role in modern societal infrastructures, including smart cities, power grids, and transportation networks. This talk will explore scalable distributed control and reinforcement learning (RL) techniques tailored for such systems. In the first part, I will focus on distributed control for networks with linear dynamics, demonstrating that under certain locality assumptions, distributed controllers can achieve near-global-optimal performance. Building upon this insight, the second part will extend to systems with potentially nonlinear dynamics. By leveraging representation learning, we approximate the dynamics as a linear combination of features, and introduce an algorithm that harnesses the network structure to ensure both convergence guarantees and strong empirical performance.
10/16	Project Proposal Due
10/21	Lecture #17 ( Hanjun Dai (Google DeepMind) Hanjun is a staff research scientist and research manager at Google DeepMind. He obtained his PhD from Georgia Institute of Technology. His research focuses on efficient generative modeling for text, image and structured data, and the corresponding fundamental algorithms in sampling and optimization. His research outcome has been adopted in OSS projects and launched in Google Workspace, Gemini and Cloud AI. His work has been recognized by the 2022 Google Research tech impact award, AISTATS 2016 best student paper and best workshop papers in Recsys and Neurips. He has also served as the Area Chair in top-tier conferences including AAAI, ICML, NeurIPS, co-organized workshops and tutorials in ICML, NeurIPS, LoG.): (Zoom link: https://gatech.zoom.us/j/95094662178) Universal Query Engine	Analytics on structured data is a mature field with many successful methods. However, most real world data exists in unstructured form, such as images and conversations. We investigate the potential of Large Language Models (LLMs) to enable unstructured data analytics. In particular, we propose a new Universal Query Engine (UQE) that directly interrogates and draws insights from unstructured data collections. This engine accepts queries in a Universal Query Language (UQL), a dialect of SQL that provides full natural language flexibility in specifying conditions and operators. The new engine leverages the ability of LLMs to conduct analysis of unstructured data, while also allowing us to exploit advances in sampling and optimization techniques to achieve efficient and accurate query execution. In addition, we borrow techniques from classical compiler theory to better orchestrate the workflow between sampling methods and foundation model calls. We demonstrate the efficiency of UQE on data analytics across different modalities, including images, dialogs and reviews, across a range of useful query types, including complex search that requires deep reasoning, or the aggregated summaries that would usually require massive human efforts. Reference Dai et.al, UQE A Query Engine for Unstructured Databases, NeurIPS 2024
10/23	Lecture #18 (Bo Dai): Representation Learning from EBM view	Chen et al, 2020. A Simple Framework for Contrastive Learning of Visual Representations Radford et al, 2018. Learning Transferable Visual Models From Natural Language Supervision
10/28	Lecture #19 (Bo Dai): Representation Learning from Spectral Decomposition view	HaoChen et al, 2021, Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss Balestriero 2022, Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods
Module IV: Reinforcement Learning
10/30	Lecture #20 (Bo Dai): MDP: Bellman Recursion	Puterman and Chan, 2023. Introduction to Markov Decision Processes Mohri et al, 2018. Foundations of Machine Learning
11/04	Lecture #21 (Bo Dai): DP: Value and Policy Iteration	Puterman and Chan, 2023. Introduction to Markov Decision Processes Mohri et al, 2018. Foundations of Machine Learning
11/06	Lecture #22 (Bo Dai): Learning with MDPs	Mnih et al, 2013. Playing Atari with Deep Reinforcement Learning Mohri et al, 2018. Foundations of Machine Learning
11/11	Lecture #23 (Bo Dai): Policy Gradient and Actor-Critic	Sutton et al, 1999. Policy Gradient Methods for Reinforcement Learning with Function Approximation Haarnoja et al, 2018. Soft Actor-Critic -- Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
11/13	Lecture #24 (Bo Dai): Imitation Learning and RLHF	Ross et al, 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning Ouyang et al, 2022. Training language models to follow instructions with human feedback
11/18	Lecture #25 (Bo Dai): Review
11/20	Lecture #26 : Project Presentations
11/25	Lecture #27 : Project Presentations
11/27	No Class Student Recess
12/02	Lecture #28 : Project Presentations
12/09	Project Report Due