# Deep Convolutional Networks are Hierarchical Kernel Machines

 Title Deep Convolutional Networks are Hierarchical Kernel Machines Publication Type CBMM Memos Year of Publication 2015 Authors Anselmi, F, Rosasco, L, Tan, C, Poggio, T Number 035 Date Published 06/17/2015 Abstract We extend i-theory to incorporate not only pooling but also rectifying nonlinearities in an extended HW module (eHW) designed for supervised learning. The two operations roughly correspond to invariance and selectivity, respectively. Under the assumption of normalized inputs, we show that appropriate linear combinations of rectifying nonlinearities are equivalent to radial kernels. If pooling is present an equivalent kernel also exist. Thus present-day DCNs (Deep Convolutional Networks) can be exactly equivalent to a hierarchy of kernel machines with pooling and non-pooling layers. Finally, we describe a conjecture for theoretically understanding hierarchies of such modules. A main consequence of the conjecture is that hierarchies of eHW modules minimize memory requirements while computing a selective and invariant representation. arXiv arXiv:1508.01084 DSpace@MIT http://hdl.handle.net/1721.1/100200