This talk surveys the role of margins in the analysis of deep networks. As a concrete highlight, it sketches a perceptron-based analysis establishing that shallow ReLU networks can achieve small test error even when they are quite narrow, sometimes even logarithmic in the sample size and inverse target error. The analysis and bounds depend on a certain nonlinear margin quantity due to Nitanda and Suzuki, and can lead to tight upper and lower sample complexity bounds.

Joint work with Ziwei Ji.

Abstract: We will describe work in progress in which we argue that a generalization of the procedure developed by Gao-Jafferis-Wall can allow one to see the entirety of the entanglement wedge. Gao-Jafferis-Wall demonstrated that one can see excitations behind the horizon by deforming the boundary Hamiltonian using a non-local operator. We will argue in a simple class of examples that deforming the boundary Hamiltonian by a specific modular Hamiltonian can allow one to see (almost) everything in the entanglement wedge.

Why deep learning models, trained on many machine learning tasks, can obtain nearly perfect predictions of unseen data sampled from the same distribution but are extremely vulnerable to small perturbations of the input? How can adversarial training improve the robustness of the neural networks over such perturbations? In this work, we developed a new principle called "feature purification''.

High dimensional expansion generalizes edge and spectral expansion in graphs to hypergraphs (viewed as higher dimensional simplicial complexes). It is a tool that allows analysis of PCP agreement rests, mixing of Markov chains, and construction of new error correcting codes. My talk will be devoted to proving some nice relations between local and global expansion of these objects.