IFG: Internet-Scale Guidance for Functional Grasping Generation
Ray Muxin Liu, Mingxuan Li, Kenneth Shaw, Deepak Pathak
受过互联网规模数据训练的大型视觉模型在分割和语义理解对象部分方面表现出强大的能力,即使在杂乱,拥挤的场景中也是如此。 然而,虽然这些模型可以引导机器人朝向物体的一般区域,但它们缺乏精确控制灵巧的机器人手进行3D抓握所需的几何理解。 为了克服这一点,我们的关键见解是利用模拟,使用力闭合抓线管道,了解场景中手和物体的局部几何形状。 由于这条管道速度缓慢,需要地面实况观测,因此所得数据被提炼成在相机点云上实时运行的扩散模型。 通过将互联网规模模型的全球语义理解与基于模拟的本地感知力闭合的几何精度相结合,无需任何手动收集的训练数据即可实现高性能语义抓取。 有关此的可视化,请访问我们的网站https://ifgrasping.github.io/
Large Vision Models trained on internet-scale data have demonstrated strong capabilities in segmenting and semantically understanding object parts, even in cluttered, crowded scenes. However, while these models can direct a robot toward the general region of an object, they lack the geometric understanding required to precisely control dexterous robotic hands for 3D grasping. To overcome this, our key insight is to leverage simulation with a force-closure grasping generation pipeline that unders...