Ubiquitous Symmetry at Critical Points Across Diverse Optimization Landscapes
Irmi Schneider
对称性在理解数学结构和优化问题的特性方面起着至关重要的作用。 最近的工作在神经网络的背景下探索了这种现象,其中损失函数在网络权重的列和行排列下是不变的。 据观察,局部最小值在网络权重(行和列排列的不变性)方面表现出显著的对称性。 此外,没有发现缺乏对称性的关键点。 我们通过调查在更广泛的空间类别上定义的实值损失函数中的对称现象来扩展这一调查线。 我们将再介绍四个案例:有限场上的投射案例,八面图表壳,完美的匹配案例和粒子吸引案例。 我们表明,与神经网络的情况一样,观察到的所有临界点都有非平凡的对称性。 最后,我们引入了一种新的对称性测量,并表明它揭示了先前测量未捕获的额外对称结构。
Symmetry plays a crucial role in understanding the properties of mathematical structures and optimization problems. Recent work has explored this phenomenon in the context of neural networks, where the loss function is invariant under column and row permutations of the network weights. It has been observed that local minima exhibit significant symmetry with respect to the network weights (invariance to row and column permutations). And moreover no critical point was found that lacked symmetry. W...