The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems
Petr Kasalický, Martin Spišák, Vojtěch Vančura, Daniel Bohuněk, Rodrigo Alves, Pavel Kordík
行业规模的推荐系统面临着一个核心挑战:代表具有高基本性的实体,如用户或项目,使用密集的嵌入,这些嵌入必须在培训和推理期间访问。 然而,随着嵌入尺寸的增长,内存限制使存储和访问变得越来越困难。 我们描述了一种轻量级的,可学习的嵌入压缩技术,将密集的嵌入投射到高维,稀疏激活的空间中。 我们的方法专为检索任务而设计,可降低内存需求,同时保持检索性能,可在严格的资源限制下实现可扩展的部署。 我们的结果表明,利用稀疏性是提高大规模推荐者效率的有希望的方法。 我们在https : / /github.com/recombee/CompresSAE上发布代码。
Industry-scale recommender systems face a core challenge: representing entities with high cardinality, such as users or items, using dense embeddings that must be accessible during both training and inference. However, as embedding sizes grow, memory constraints make storage and access increasingly difficult. We describe a lightweight, learnable embedding compression technique that projects dense embeddings into a high-dimensional, sparsely activated space. Designed for retrieval tasks, our meth...