42digest
WaLRUS:使用SSM进行长距离表示的波列

WaLRUS: Wavelets for Long-range Representation Using SSMs

Hossein Babaei, Mel White, Sina Alemohammad, Richard G. Baraniuk

arXiv
2025年5月17日

状态空间模型(SSM)已被证明是用于在顺序数据中模拟远程依赖的强大工具。 虽然最近被称为HiPPO的方法已经显示出强劲的性能,并构成了机器学习模型S4和Mamba的基础,但它仍然受到一些特定,表现良好的基础的封闭式解决方案的限制。 SaFARi框架概括了这种方法,使SSM能够从任意帧(包括非正交和冗余框架)构建SSM,从而允许SSM家族中可能存在的“物种”的无限多样性。 在本文中,我们介绍了WaLRUS(使用SSM的远程表示的Wavelets),这是由Daucheies小波构建的SaFARi的新实现。

State-Space Models (SSMs) have proven to be powerful tools for modeling long-range dependencies in sequential data. While the recent method known as HiPPO has demonstrated strong performance, and formed the basis for machine learning models S4 and Mamba, it remains limited by its reliance on closed-form solutions for a few specific, well-behaved bases. The SaFARi framework generalized this approach, enabling the construction of SSMs from arbitrary frames, including non-orthogonal and redundant o...