On the Analogy between Human Brain and LLMs: Spotting Key Neurons in Grammar Perception
Sanaz Saki Norouzi, Mohammad Masjedi, Pascal Hitzler
人工智能的构建模块人工神经网络受到人类大脑神经元网络的启发。 多年来,这些网络已经进化到复制大脑的复杂功能,使他们能够处理图像和语言处理等任务。 在大型语言模型领域,人们一直对使语言学习过程更像人类有着浓厚的兴趣。 虽然神经科学研究表明,不同的语法类别是由大脑中的不同神经元处理的,但我们表明LLM以类似的方式运作。 利用Llama 3,我们确定了与预测属于不同部分语音标签的单词相关的最重要的神经元。 利用获得的知识,我们在数据集上训练一个分类器,这表明这些关键神经元的激活模式可以可靠地预测新数据上的部分语音标签。 结果表明,LLM中存在一个子空间,专注于捕获部分语音标签概念,类似于神经科学中大脑病变研究中观察到的模式。
Artificial Neural Networks, the building blocks of AI, were inspired by the human brain's network of neurons. Over the years, these networks have evolved to replicate the complex capabilities of the brain, allowing them to handle tasks such as image and language processing. In the realm of Large Language Models, there has been a keen interest in making the language learning process more akin to that of humans. While neuroscientific research has shown that different grammatical categories are pro...