Trainability and accuracy of neural networks used in machine learning

-
Eric Vanden-Eijnden, Courant Institute, New York University
Fine Hall 214

The methods and models of machine learning (ML) are rapidly becoming de facto tools for the analysis and interpretation of large data sets. Complex classification tasks such as speech and image recognition, automatic translation, decision making, etc. that were out of reach a decade ago are now routinely performed by computers with a high degree of reliability using (deep) neural networks. These performances suggest that it may be possible to represent high-dimensional functions with controllably small errors, potentially outperforming standard interpolation methods based on Galerkin truncation or finite elements: these have been the workhorses of scientific computing but suffer from the curse of dimensionality.  By beating this curse, ML techniques could change the way we perform quantum physics calculations, molecular dynamics simulation, PDE solution, etc.  In support of this prospect, in this talk I will present results about the trainability and accuracy of neural networks, obtained by mapping the parameters of the neural network to a system of interacting particles. I will also show how these findings can be used to accelerate the training of  networks and optimize their architecture, and discuss what these results imply for applications in scientific computing.

Vanden-Eijnden's work focuses mainly on the theoretical and computational aspects of Non-Equilibrium Statistical Mechanics and Applied Probability, with applications to biomolecular systems, chemical and biological networks, materials science, atmosphere-ocean science, and fluids dynamics.  He has contributed to the development and analysis of multiscale numerical methods for systems whose dynamics span a wide range of spatio-temporal scales. Vanden-Eijnden earned his doctorate in 1997 from the Université Libre de Bruxelles under the supervision of Radu Bălescu. He moved to the Courant Institute of Mathematical Sciences NYU in 1998 where he has been Professor of Mathematics since 2003. He has held visiting positions at the Institute for Advanced Study in Princeton, the Miller Institute at the University of California, Berkeley, and Trinity College at Cambridge University. Vanden-Eijnden received a Sloan Fellowship in 2002 and a NSF Career award in 2003. He was also the recipient of the 2009 Germund Dahlquist Prize from the Society for Industrial and Applied Mathematics (SIAM)  and the 2011 J. D. Crawford Prize from SIAM SIAG/Dynamical Systems. He was invited to speak at the 2015 International Congress of Industrial and Applied Mathematics (ICIAM) in Beijing.