Theoretical Machine Learning

Compositional inductive biases in human function learning

Samuel J. Gershman
Harvard University
January 14, 2020
This talk presents evidence that humans learn complex functions by harnessing compositionality: complex structure is decomposed into simpler building blocks. I formalize this idea in the framework of Bayesian nonparametric regression using a grammar over Gaussian process kernels, and compare this approach with other structure learning approaches. People consistently chose compositional (over non-compositional) extrapolations and interpolations of functions.

How will we do mathematics in 2030 ?

Michael R. Douglas
Simons Center for Geometry and Physics, Stony Brook
December 17, 2019

We make the case that over the coming decade, computer assisted reasoning will become far more widely used in the mathematical sciences. This includes interactive and automatic theorem verification, symbolic algebra, and emerging technologies such as formal knowledge repositories, semantic search and intelligent textbooks.

Nonconvex Minimax Optimization

Chi Jin
Princeton University; Member, School of Mathematics
November 20, 2019

Minimax optimization, especially in its general nonconvex formulation, has found extensive applications in modern machine learning, in settings such as generative adversarial networks (GANs) and adversarial training. It brings a series of unique challenges in addition to those that already persist in nonconvex minimization problems. This talk will cover a set of new phenomena, open problems, and recent results in this emerging field.

Some Statistical Results on Deep Learning: Interpolation, Optimality and Sparsity

Guang Cheng
Purdue University; Member, School of Mathematics
November 13, 2019

This talk discusses three aspects of deep learning from a statistical perspective: interpolation, optimality and sparsity. The first one attempts to interpret the double descent phenomenon by precisely characterizing a U-shaped curve within the “over-fitting regime,” while the second one focuses on the statistical optimality of neural network classification in a student-teacher framework. This talk is concluded by proposing sparsity induced training of neural network with statistical guarantee. 

Rethinking Control

Elad Hazan
Princeton University, Google AI Princeton
October 2, 2019

Linear dynamical systems are a continuous subclass of reinforcement learning models that are widely used in robotics, finance, engineering, and meteorology. Classical control, since the works of Kalman, has focused on dynamics with Gaussian i.i.d. noise, quadratic loss functions and, in terms of provably efficient algorithms, known systems and observed state.