Revisiting optimization: gradient descent with a general cost
12 May 2023, 14:30 — Room 322, UniGe DIBRIS/DIMA, Via Dodecaneso 35
![Pierre Cyrill Aubin Frankowski - [Fronte Guancia Sorriso]](/media/images/avatar_gdcndh_NYJyRzb.width-800.jpg){kind=avatar}
Speaker:
Pierre-Cyril Aubin-Frankowski — INRIA Paris (Sierra)
Pierre-Cyril Aubin-Frankowski — INRIA Paris (Sierra)
Abstract:
We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective function in a systematic manner, based on a chosen cost function. This surrogate is then minimized using an alternating minimization scheme. Using optimal transport theory we establish convergence rates based on generalized notions of smoothness and convexity. We provide local versions of these two notions when the cost satisfies a condition known as nonnegative cross-curvature in optimal transport. In particular our framework provides the first global rates for natural gradient descent and Newton's method. This is a joint work with Flavien Léger (INRIA).
We present a new class of gradient-type optimization methods that extends vanilla gradient descent, mirror descent, Riemannian gradient descent, and natural gradient descent. Our approach involves constructing a surrogate for the objective function in a systematic manner, based on a chosen cost function. This surrogate is then minimized using an alternating minimization scheme. Using optimal transport theory we establish convergence rates based on generalized notions of smoothness and convexity. We provide local versions of these two notions when the cost satisfies a condition known as nonnegative cross-curvature in optimal transport. In particular our framework provides the first global rates for natural gradient descent and Newton's method. This is a joint work with Flavien Léger (INRIA).
Bio:
Pierre-Cyril Aubin is a postdoctoral researcher at INRIA Paris (SIERRA team) with Alessandro Rudi. He obtained his PhD in 2021 on estimation and control under constraints through kernel methods from MINES ParisTech. His research is broadly centered on the links between kernel methods, optimal control, Kalman filtering and optimization in measure spaces, with a recent interest for max-plus spaces and c-transforms.
Pierre-Cyril Aubin is a postdoctoral researcher at INRIA Paris (SIERRA team) with Alessandro Rudi. He obtained his PhD in 2021 on estimation and control under constraints through kernel methods from MINES ParisTech. His research is broadly centered on the links between kernel methods, optimal control, Kalman filtering and optimization in measure spaces, with a recent interest for max-plus spaces and c-transforms.