In this talk, I will discuss my two recent works on Energy-Based Models. In the first work, I discuss how we can reinterpret standard classification architectures as class conditional energy-based models and train them using recently proposed methods for large-scale EBM training. We find that adding EBM training in this way provides many benefits while negligibly affecting discriminative performance, contrary to other hybrid generative/discriminative modeling approaches.
Weizmann Institute of Science; Visiting Professor, School of Mathematics
March 10, 2020
High dimensional expansion generalizes edge and spectral expansion in graphs to hypergraphs (viewed as higher dimensional simplicial complexes). It is a tool that allows analysis of PCP agreement rests, mixing of Markov chains, and construction of new error correcting codes. My talk will be devoted to proving some nice relations between local and global expansion of these objects.
We provide geometric sufficient conditions for Reifenberg flat sets of any integer dimension in Euclidean space to be parametrized by a Lipschitz map with Hölder derivatives. The conditions use a Jones type square function and all statements are quantitative in that the Hölder and Lipschitz constants of the parametrizations depend on such a function. We use these results to prove sufficient conditions for higher order rectifiability of sets and measures.
We will ask how many Lagrangian tori, say with an integral area class, can be `packed' into a given symplectic manifold. Similarly, given an arrangement of such tori, like the integral product tori in Euclidean space, one can ask about the symplectic size of the complement. The talk will describe some constructions of balls and Lagrangian tori which show the size is larger than expected.
New York University; Distinguished Visiting Professor, School of Mathematics & Natural
March 9, 2020
Striving to make contact with mathematics and to be consistent with neuroanatomy at the same time, I propose an idealized picture of the cerebral cortex consisting of a hierarchical network of brain regions each further subdivided into interconnecting layers not unlike those in artificial neural networks. Each layer is idealized as a 2D sheet of neurons, spatially homogeneous with primarily local interactions, a setup reminiscent of that in statistical mechanics. Zooming into local circuits, one gets into the domain of dynamical systems.
Machine Learning is invaluable for extracting insights from large volumes of data. A key assumption enabling many methods, however, is having access to training data comprising independent observations from the entire distribution of relevant data. In practice, data is commonly missing due to measurement limitations, legal restrictions, or data collection and sharing practices. Moreover, observations are commonly collected on a network, a spatial or a temporal domain and may be intricately dependent.
Deep neural networks have recently seen an impressive comeback with applications both in the public sector and the sciences. However, despite their outstanding success, a comprehensive theoretical foundation of deep neural networks is still missing.
An introduction to Boolean Function Analysis
We will discuss some of the basic principles and results in the study of Boolean-valued functions over the discrete hypercube using discrete Fourier analysis. In particular, we will talk about basic concepts, the hypercontractive inequality and the KKL theorem. Time permitting, we will discuss the Fourier-Entropy Conjecture and mention some recent progress towards it.
The talk is self-contained and no special background will be assumed.
University of Toronto; Visiting Professor, School of Mathematics
March 2, 2020
In this talk I will discuss the Sunflower Lemma and similar lemmas that prove (in various contexts) that a set/distribution can be partitioned into a structured part and a "random-looking" part. I will introduce communication complexity as a key model for understanding computation and more generally for reasoning about information bottlenecks.