Regularization properties of first order methods

Abstract: In this talk I will show that first order methods exhibit implicit regularization properties. 

This is based on the observation that beyond being convergent, they are also stable with respect to perturbations in the data.

Thanks to convergence and stability, first order algorithms can therefore be used as regularization methods, offering a provably computational efficient alternative to explicit Tykhonov regularization.