Lie symmetries in CNN architectures

Abstract: In this paper we study the spontaneous development of symmetries in the early layers of a Convolutional Neural Network (CNN) during learning on natural images. We consider a first layer with only one filter, which will show a rotational symmetry. As it is well known the filters of the next convolutional layer can be approximated by Gabor functions, which can be parametrized by rotation and translation variables. We equip this layer with a diffusion kernel, to be compared with fundamental solution of Laplacian in the group, in accordance with group-based models of V1 horizontal connectivity.