Xiaoming Huo

Some New Results on the Stochastic First-Order Methods in Parameter Estimation

Abstract:

We study the first-order stochastic methods that can be utilized to solve the optimization problems derived from parameter estimation in statistics. The stochastic algorithm has a low cost per iteration and is more suitable for a large-size dataset. The first-order method only involves gradients; therefore, its implementation is more straightforward than other methods. Within the first-order stochastic methods, the techniques of variance reduction and generalization to mirror descent are often used to enhance their performance. In the first part of this talk, we describe a discovery that by using a particular design of the first-order stochastic method, utilizing variance reduction and mirror descent, one can show that the outcome automatically has an implicit regularization property. That is, the result of the stochastic algorithm automatically is a minimum-norm solution, while no penalty has explicitly involved. Based on this property, one can build some high probability exact recovery property for the corresponding parameter estimate. In the second part, we show that the stochastic algorithm can be used to solve the optimal transport problem, which is rooted in the applications of the Wasserstein distance and has found many applications in machine learning and artificial intelligence. We show that our stochastic algorithm has the best convergence rate among all the known methods. This talk is based on joint work with Ms. Yiling Luo. 

Biography:

Dr. Xiaoming Huo is A. Russell Chandler III Professor at the Stewart School of Industrial & Systems Engineering at Georgia Tech. His research interests include statistics, machine learning, and the foundation of data science. He has made numerous contributions to sparse representation, compressive sensing, wavelets, theory of deep learning, and fast algorithms. His papers appeared in top journals, and some are highly cited. Dr. Huo received the B.S. degree in mathematics from the University of Science and Technology, China, in 1993, and the M.S. degree in electrical engineering and the Ph.D. degree in statistics from Stanford University, Stanford, CA, in 1997 and 1999, respectively. He represented China in the 30th International Mathematical Olympiad (IMO), which was held in Braunschweig, Germany, in 1989, and received a golden prize. From August 2013 to August 2015, he served the USA National Science Foundation as a Program Director in the Division of Mathematical Sciences (DMS).