MirrorLimb: Implementing hand pose acquisition and robot teleoperation based on RealMirror
Cong Tai, Hansheng Wu, Haixu Long, Zhengbin Long, Zhaoyu Zheng, Haodong Xiang, Tao Shen
在这项工作中,我们提出了一个基于PICO的机器人远程操作框架,可实现低成本,实时获取手部运动和姿势数据,在成本效益方面优于主流视觉跟踪和运动捕捉解决方案。 该框架与RealMirror生态系统本地兼容,在Isaac模拟环境中提供可重复使用的功能,用于稳定精确的机器人轨迹记录,从而促进Vision-Language-Action(VLA)数据集的构建。 此外,该系统还支持各种配备末端效应器的机器人的实时远程操作,包括灵巧的手和机器人夹具。 这项工作旨在降低研究上肢机器人操纵中的技术障碍,从而加速VLA相关研究的进展。
In this work, we present a PICO-based robot remote operating framework that enables low-cost, real-time acquisition of hand motion and pose data, outperforming mainstream visual tracking and motion capture solutions in terms of cost-effectiveness. The framework is natively compatible with the RealMirror ecosystem, offering ready-to-use functionality for stable and precise robotic trajectory recording within the Isaac simulation environment, thereby facilitating the construction of Vision-Languag...
