Professor Bertsimas

Machine Learning and Statistics via a modern optimization lens (slides)

The field of Statistics has historically been linked with Probability Theory. However, some of the central problems of classification,  regression and estimation can naturally be written as optimization problems. While continuous optimization approaches has had a significant impact in Statistics,   mixed integer optimization (MIO)  has played a very limited role, primarily based on the belief that MIO models are computationally intractable.

In this talk, we demonstrate that modern  convex,  robust and especially  mixed integer  optimization methods, when applied to a variety of classical Machine Learning (ML) /Statistics (S) problems can lead to  certifiable optimal solutions for large scale instances   that have often significantly improved out of sample accuracy compared to heuristic methods used in ML/S.Specifically, we report results on

• The classical variable selection problem in regression currently solved by Lasso heuristically.
• We show that robustness and not sparsity is the major reason of the success of Lasso in contrast to widely held beliefs in ML/S.
• A systematic approach to design linear and logistic regression models based on MIO.
• Optimal trees for classification solved by CART heuristically.
• Robust classification including robust Logistic regression, robust optimal trees and robust support vector machines.
• Sparse matrix estimation problems: Principal Component Analysis,   Factor Analysis and Covariance matrix estimation.

In all cases we demonstrate that optimal solutions to large scale instances (a) can be found in seconds,   (b) can be certified to be optimal in minutes and (c) outperform classical approaches. Most importantly, this body of work suggests that linking ML/S to modern optimization leads to significant advances.

Bio

Dimitris Bertsimas is currently the Boeing Professor of  Operations Research and the co-director of the Operations Research Center at the Massachusetts Institute of Technology. His research interests include analytics, optimization and their applications  in a variety of industries. He has co-authored more than 200 scientific papers and recently published the book “The Analytics Edge’’.  He is a member of the US National Academy of Engineering, and an INFORMS fellow. He has received several research awards including the Philip Morse lectureship award 2013), the William  Pierskalla award for best paper in health care (2013),  the best paper award in Transportation Science (2013),  the Farkas prize (2008), the Erlang prize (1996),  the SIAM prize in optimization (1996), the Bodossaki prize (1998) and  the Presidential Young Investigator award (1991-1996).

Professor Schwartz

How is data science helping Flint recovery efforts? A Data-driven approach to infrastructure improvement

Bio