CO2-Meter: A Comprehensive Carbon Footprint Estimator for LLMs on Edge Devices
Zhenxiao Fu, Chen Fan, Lei Jiang
LLM已经改变了NLP,但在边缘设备上部署它们带来了巨大的碳挑战。 先前的估计器仍然不完整,忽略了外围能源的使用,独特的预填充/解码行为以及SoC设计的复杂性。 本文介绍了CO2-Meter,一个在LLM边缘推断中估计可操作和体现碳的统一框架。 贡献包括:(1)基于方程的外围能量模型和数据集;(2)基于GNN的预测因子,具有相位特异性LLM能量数据;(3)用于SoC瓶颈分析的单位级体现碳模型;(4)验证显示优于先前方法的准确性。 案例研究表明,CO2-Meter在确定碳热点和在边缘平台上指导可持续LLM设计方面的有效性。 源代码:https://github.com/fuzhenxiao/CO2-Meter
LLMs have transformed NLP, yet deploying them on edge devices poses great carbon challenges. Prior estimators remain incomplete, neglecting peripheral energy use, distinct prefill/decode behaviors, and SoC design complexity. This paper presents CO2-Meter, a unified framework for estimating operational and embodied carbon in LLM edge inference. Contributions include: (1) equation-based peripheral energy models and datasets; (2) a GNN-based predictor with phase-specific LLM energy data; (3) a unit...
