XCME005 -
Introduction to Optimization Workshop
Overview
This workshop introduces mathematical optimization and modeling, with a focus on convex optimization. We will use convexity as a starting point from which to consider some nonconvex problem types. The workshop will have a practical focus, with participants formulating and solving optimization problems early and often using standard modeling languages and solvers. By introducing common models from machine learning and other fields, the workshop aims to make participants comfortable with optimization modeling so that they may use it for rapid prototyping and experimentation in their own work.
We will cover simple but useful optimization algorithms such as gradient descent for when problems become too large for black-box solvers and cover majorization-maximization as a useful framework to view optimization algorithms for both convex and non-convex problems. Parallel and stochastic gradient descent will be covered in the contexts of distributed and large-scale optimization.
Time permitting, we will also discuss methods to solve nonconvex models for nonnegative matrix factorization, matrix completion, and neural networks.
Participants should be comfortable with linear algebra, differential multivariable calculus, and basic probability and statistics. Experience with Python will be helpful, but not required.
Topics Include
- Varieties of mathematical optimization
- Convexity of functions and sets
- Convex optimization modeling with CVXPY
- (Stochastic) gradient descent and basic distributed optimization
- In-depth examples from machine learning, statistics and other fields
- Applications of bi-convexity and non-convex gradient descent
To view other workshop descriptions, or to get general information about the 2015 ICME Summer Workshop Series, click here.
ICME Summer Workshops are open to participants 18 years and older. If you are under the age of 18 and would like to participate, please contact icme-summer-workshop@lists.stanford.edu.