School of Mathematics

Some Recent Insights on Transfer Learning

Samory Kpotufe
Columbia University; Member, School of Mathematics
March 31, 2020
A common situation in Machine Learning is one where training data is not fully representative of a target population due to bias in the sampling mechanism or high costs in sampling the target population; in such situations, we aim to ’transfer’ relevant information from the training data (a.k.a. source data) to the target application. How much information is in the source data? How much target data should we collect if any? These are all practical questions that depend crucially on 'how far' the source domain is from the target.

CSPs with Global Modular Constraints: Algorithms and Hardness via Polynomial Representations

Sivakanth Gopi
Microsoft Researcher
March 30, 2020
A theorist's dream is to show that hard instances/obstructions for an (optimal) algorithm can be used as gadgets to prove tight hardness reductions (which proves optimality of the algorithm). An example of such a result is that of Prasad Raghavendra who showed that for any constraint satisfaction problem (CSP), there is an SDP which achieves the best possible approximation factor assuming UGC. We show that a similar phenomenon occurs in CSPs with global modular constraints.

Fragmentation pseudo-metrics and Lagrangian submanifolds

Octav Cornea
Université de Montréal
March 27, 2020
The purpose of the talk is to discuss a class of pseudo-metrics that can be defined on the set of objects of a triangulated category whose morphisms are endowed with a notion of weight. In case the objects are Lagrangian submanifolds (possibly immersed) there are a some natural ways to define such pseudo-metrics and, if the class of Lagrangian submanifolds is unobstructed, these pseudo-metrics are non-degenerate and extend in a natural way the Hofer distance.
The talk is based on joint work with P. Biran and with E. Shelukhin.

Margins, perceptrons, and deep networks

Matus Telgarsky
University of Illinois
March 26, 2020
This talk surveys the role of margins in the analysis of deep networks. As a concrete highlight, it sketches a perceptron-based analysis establishing that shallow ReLU networks can achieve small test error even when they are quite narrow, sometimes even logarithmic in the sample size and inverse target error. The analysis and bounds depend on a certain nonlinear margin quantity due to Nitanda and Suzuki, and can lead to tight upper and lower sample complexity bounds.

Joint work with Ziwei Ji.

High dimensional expanders - Part 2

Irit Dinur
Weizmann Institute of Science; Visiting Professor, School of Mathematics
March 24, 2020
In this talk I will describe the notion of "agreement tests" that are motivated by PCPs but stand alone as a combinatorial property-testing question. I will show that high dimensional expanders support agreement tests, thereby derandomizing direct product tests in a very strong way.

Sharp Thresholds and Extremal Combinatorics

Dor Minzer
Member, Institute for Advanced Study
March 17, 2020
Consider the p-biased distribution over 0,1n, in which each coordinate independently is sampled according to a p-biased bit. A sharp-threshold result studies the behavior of Boolean functions over the hypercube under different p-biased measures, and in particular whether the function experiences a phase transition between two, close p's. While the theory of sharp-thresholds is well understood for p's that are bounded away from 0 and 1, it is much less so for values of p that are close to 0 or 1.

Feature purification: How adversarial training can perform robust deep learning

Yuanzhi Li
Carnegie Mellon University
March 16, 2020
Why deep learning models, trained on many machine learning tasks, can obtain nearly perfect predictions of unseen data sampled from the same distribution but are extremely vulnerable to small perturbations of the input? How can adversarial training improve the robustness of the neural networks over such perturbations? In this work, we developed a new principle called "feature purification''.

Introduction to high dimensional expanders

Irit Dinur
Weizmann Institute of Science; Visiting Professor, School of Mathematics
March 10, 2020
High dimensional expansion generalizes edge and spectral expansion in graphs to hypergraphs (viewed as higher dimensional simplicial complexes). It is a tool that allows analysis of PCP agreement rests, mixing of Markov chains, and construction of new error correcting codes. My talk will be devoted to proving some nice relations between local and global expansion of these objects.

Your Brain on Energy-Based Models: Applying and Scaling EBMs to Problems of Interest to the Machine Learning Community Today

Will Grathwohl
University of Toronto
March 10, 2020
In this talk, I will discuss my two recent works on Energy-Based Models. In the first work, I discuss how we can reinterpret standard classification architectures as class conditional energy-based models and train them using recently proposed methods for large-scale EBM training. We find that adding EBM training in this way provides many benefits while negligibly affecting discriminative performance, contrary to other hybrid generative/discriminative modeling approaches.