Classifying Shelf Life Quality of Pineapples by Combining Audio and Visual Features
Yi-Lu Jiang, Wen-Chang Chang, Ching-Lin Wang, Kung-Liang Hsu, Chih-Yi Chiu
使用非破坏性方法确定菠萝的保质期质量是减少浪费和增加收入的关键一步。 在本文中,构建了一个多模态和多视图分类模型,根据音频和视觉特征将菠萝分为四个质量级别。 为了研究目的,我们编译并发布了PQC500数据集,由500个菠萝组成,有两种模式:一种是利用菠萝通过多个麦克风记录声音,另一种是在不同地点由多个摄像头拍摄照片,提供多模态和多视图视听功能。 我们修改了对比式视听遮蔽自动解码器,通过丰富的音频和视觉对组合来训练基于跨模态的分类模型。 此外,我们建议对训练数据进行紧凑大小的采样,以实现高效计算。 实验在各种数据和模型配置下进行了评估,结果表明,使用音频主采样训练的拟议跨模态模型可以产生84个,优于仅音频和仅视觉的单模态模型,分别达到6个。
Determining the shelf life quality of pineapples using non-destructive methods is a crucial step to reduce waste and increase income. In this paper, a multimodal and multiview classification model was constructed to classify pineapples into four quality levels based on audio and visual characteristics. For research purposes, we compiled and released the PQC500 dataset consisting of 500 pineapples with two modalities: one was tapping pineapples to record sounds by multiple microphones and the oth...