Tuesday, December 1, 2015 - 4:00pm
Event Calendar Category
LIDS Seminar Series
Building and Room Number
Deep neural networks have proved very successful in domains where large training sets are available, since their capacity can be increased by adding layers or by increasing the number of units in a layer. When the number of training samples is small, their performance suffers from overfitting.
In the first part of this talk we focus on a single layer and the design of linear features that are able to discriminate classes of inputs. We use wireless information theory to derive fundamental limits on the maximum number of classes that can be discriminated with low probability of error and on the tradeoff between the number of classes and the probability of misclassification.
In the second part of the talk we focus on overfitting to random error or noise instead of the underlying signal. Prior work includes methods such as weight decay, Dropout and DropConnect that are data independent and designed to make it more difficult to fit to random error or noise. We will describe GraphConnect, a complementary method that is data dependent, and is motivated by the empirical observation that data of interest typically lies close to a manifold.
This is joint work with Jiaji Huang, Matt Nokleby, Qiang Qiu, Miguel Rodrigues and Guillermo Sapiro
Robert Calderbank is Director of the Information Initiative at Duke University, where he is Professor of Mathematics and Electrical Engineering. Prior to joining Duke as Dean of Natural Sciences in 2010, he directed the Program in Applied and Computational Mathematics at Princeton University. Prior to joining Princeton in 2004 he was Vice President for Research at AT&T, in charge of what may have been the first industrial research lab where the primary focus was Big Data.
Professor Calderbank is well known for contributions to voiceband modem technology, to quantum information theory, and for co-invention of space-time codes for wireless communication. His research papers have been extensively cited and his inventions are found in billions of consumer devices. Professor Calderbank was elected to the National Academy of Engineering in 2005 and has received a number of awards, including the 2013 IEEE Hamming Medal for his contributions to information transmission, and the 2015 Claude E. Shannon Award.
Reception to follow.