Why High-rank Neural Networks Generalize?: An Algebraic Framework with RKHSs
Yuka Hashimoto, Sho Sonoda, Isao Ishikawa, Masahiro Ikeda
我们得出了一个新的Rademacher复杂性,用于使用Koopman运算符,组表示和复制内核Hilbert空间(RKHS)的深度神经网络。 拟议的绑定描述了为什么具有高重量矩阵的模型很好地推广。 虽然有现有的边界试图描述这种现象,但这些现有的边界可以应用于有限的模型类型。 我们引入了神经网络的代数表示和内核函数来构建RKHS,以便为更广泛的现实模型导出绑定。 这项工作为基于库普曼的理论铺平了道路,因为Rademacher的复杂性边界适用于更实际的情况。
We derive a new Rademacher complexity bound for deep neural networks using Koopman operators, group representations, and reproducing kernel Hilbert spaces (RKHSs). The proposed bound describes why the models with high-rank weight matrices generalize well. Although there are existing bounds that attempt to describe this phenomenon, these existing bounds can be applied to limited types of models. We introduce an algebraic representation of neural networks and a kernel function to construct an RKHS...
