Accueil/ expose
Optimization's Hidden Gift to Learning: Implicit Regularization
mardi 01 octobre 2019

Loading the player...
Descriptif

It is becoming increasingly clear that implicit regularization afforded by the optimization algorithms play a central role in machine learning, and especially so when using large, deep, neural networks. We have a good understanding of the implicit regularization afforded by stochastic approximation algorithms, such as SGD, for convex problem, and we understand and can characterize the implicit bias of different algorithms, and can design algorithms with specific biases. But in this talk I will focus on implicit biases of local search algorithms for non-convex underdetermined problem, such as deep networks. In an effort to uncover the implicit biases of gradient-based optimization of neural networks, which holds the key to their empirical success, I will discuss recent work on implicit regularization for matrix factorization, linear convolutional networks, and two-layer ReLU networks, as well as a general bottom-up understanding on implicit regularization in terms of optimization geometry.

Thèmes : Physique
Catégories: Data Science Colloquium

Voir aussi


  • Aucun exposé du même auteur.
  • Can Big Data cure Cancer?
    Jean-Philippe Vert
  • What physics can tell us about inferenc...
    Cristopher Moore
  • Beyond stochastic gradient descent for l...
    Francis Bach
  • The brain as an optimal efficient adapti...
    Sophie Deneve
  • Cosmostatistics: Tackling Big Data from ...
    Jean-Luc Starck
  • Searching for interaction networks in pr...
    Rémi Monasson
  • Brain-computer interfaces: two concurren...
    Maureen Clerc
  • Machine learning in scientific workflow...
    Balàzs Kégl
  • Learning Graph Inverse Problems with Neu...
    Joan Bruna
  • Towards developmental AI
    Emmanuel Dupoux
Auteur(s)
Nathan Srebro
Toyota Technological Institute at Chicago

Plus sur cet auteur
Voir la fiche de l'auteur

Cliquer ICI pour fermer
Annexes
Téléchargements :
   - Télécharger la vidéo
   - Télécharger l'audio (mp3)

Dernière mise à jour : 01/10/2019