Lectures: MoWe 11:00–11:15, 301-302
Prof. Insoon Yang
Office hours: by appointment
This course introduces convex optimization theory and algorithms, with application to machine learning and data science. The first part will provide the basic theory of convex optimization. In the second part, application problems in many disciplines will be discussed. In the third part, basic convex optimization algorithms and acceleration methods will be covered. The last part of this course will introduce advanced algorithms including stochastic subgradient and proximal methods that are useful for solving large-scale problems arising in machine learning and data science.
Calculus, Linear Algebra, Probability
There will be 2 problem sets, a midterm test, and a final project.
Course grades: Participation (5%); Problem sets (15%); Midterm (30%); Final project (50%).
Working in groups is encouraged. However, each person must submit his/her own problem sets. Late submissions will not be accepted.
The course is based on lecture notes.
• [BV04] S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge University Press, 2004.
• [MDG18] M. W. Mahoney, J. C. Duchi, A. C. Gilbert, The Mathematics of Data, American Mathematical Society, 2018.
• [Nes04] Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Springer, 2004.
• [BNO03] D. P. Bertsekas, A. Nedic, A. E. Ozdaglar, Convex Analysis and Optimization, Athena Scientific, 2003.
• [Ber15] D. P. Bertsekas, Convex Optimization Algorithms, Athena Scientific, 2015.
• [NW05] J. Nocedal, S. Wright, Numerical Optimization, Springer, 2nd edition, 2006.
• [Ber16] D. P. Bertsekas, Nonlinear Programming, Athena Scientific, 3rd edition, 2016.
• Part I: Basic theory
– Week 1. Introduction to mathematical optimization
– Week 2. Convex sets
– Week 3. Convex functions
– Week 4. Convex optimization problems
– Week 5. Duality
• Part II: Applications
– Week 6. Approximation and fitting
– Week 7. Statistical estimation
– Week 8. Geometric problems
• Part III: Basic algorithms
– Week 9. Unconstrained minimization and Nesterov’s acceleration
– Week 10. Equality constrained minimization
– Week 11. Interior-point methods
• Part IV: Advanced algorithms for large-scale problems
– Week 12. Subgradient methods
– Week 13. Stochastic subgradient methods
– Week 14. Mirror descent methods
– Week 15. Proximal methods and ADMM