UniFormer: Unified and Efficient Transformer for Reasoning Across General and Custom Computing
Zhuoheng Ran, Chong Wu, Renjie Xu, Maolin Che, and Hong Yan
卷积神经网络(CNN)等神经网络的成功很大程度上归功于它们有效和广泛地部署在定制的计算平台上,包括现场可编程门阵列(FPGA)和专用集成电路(ASIC)。 在当前时代,基于Transformer的架构支持大多数最先进的(SOTA)大型模型,这些模型也越来越多地部署在低功耗和实时应用的定制计算硬件上。 然而,通用和定制计算之间根本不同的并行计算范式往往导致模型转移和可部署性的妥协,这通常以复杂性,效率或准确性为代价。 此外,许多跨平台优化原则在现有研究中也没有得到充分开发。 本文介绍了UniFormer,一个统一而高效的Transformer架构,适用于通用和定制计算平台。 通过实现更高的并行性和计算存储融合,UniFormer在GPU上实现了最先进的(SOTA)精度和延迟,同时在FPGA上表现出强大的适应性。 据我们所知,本文是第一部同时考虑通用和定制计算架构的高效Transformer工作。
The success of neural networks such as convolutional neural networks (CNNs) has been largely attributed to their effective and widespread deployment on customised computing platforms, including field-programmable gate arrays (FPGAs) and application-specific integrated circuits (ASICs). In the current era, Transformer-based architectures underpin the majority of state-of-the-art (SOTA) larger models that are also increasingly deployed on customised computing hardware for low-power and real-time a...