Topological Signatures of ReLU Neural Network Activation Patterns
Vicente Bosca, Tatum Rask, Sunia Tanweer, Andrew R. Tawfeek, Branden Stone
本文探讨了ReLU神经网络激活模式的拓扑特征。 我们考虑具有ReLU激活函数的前馈神经网络,并分析网络诱导的特征空间的多顶点分解。 主要是,我们调查双图的Fiedler分区,并表明它似乎与决策边界相关 - 在二进制分类的情况下。 此外,我们计算细胞分解的同源 - 在回归任务中 - 在训练损失和多面体细胞计数之间绘制类似的行为模式,因为模型被训练。
This paper explores the topological signatures of ReLU neural network activation patterns. We consider feedforward neural networks with ReLU activation functions and analyze the polytope decomposition of the feature space induced by the network. Mainly, we investigate how the Fiedler partition of the dual graph and show that it appears to correlate with the decision boundary – in the case of binary classification. Additionally, we compute the homology of the cellular decomposition – in a regress...