On the Performance of Concept Probing: The Influence of the Data (Extended Version)
Manuel de Sousa Ribeiro, Afonso Leote and João Leite
概念探测最近引起了越来越多的兴趣,作为帮助解释人工神经网络的一种方式,处理它们通常的大尺寸和亚符号性质,最终使它们无法直接解释人类。 概念探测工作通过训练额外的分类器来将模型的内部表示映射到人类定义的兴趣概念中,从而使人类能够在人工神经网络中窥视。 对概念探测的研究主要集中在正在探测的模型或探测模型本身,对训练此类探测模型所需的数据的关注有限。 在本文中,我们解决了这一差距。 专注于图像分类任务中的概念探测,我们研究用于训练探测模型的数据对其性能的影响。 我们还为两个广泛使用的数据集提供概念标签。
Concept probing has recently garnered increasing interest as a way to help interpret artificial neural networks, dealing both with their typically large size and their subsymbolic nature, which ultimately renders them unfeasible for direct human interpretation. Concept probing works by training additional classifiers to map the internal representations of a model into human-defined concepts of interest, thus allowing humans to peek inside artificial neural networks. Research on concept probing h...