code_video_cp.mp4
- 🏆 SOTA simulation performance Seer achieves state-of-the-art performance on simulation benchmarks CALVIN ABC-D and LIBERO-LONG.
- 💪 Impressive Real-World performance Seer demonstrates strong effectiveness and generalization across diverse real-world downstream tasks.
We provide step-by-step guidance for running Seer in simulations and real-world experiments. Follow the specific instructions for a seamless setup.
For users aiming to train Seer from scratch or fine-tune it, we provide comprehensive instructions for environment setup, downstream task data preparation, training, and deployment.
This section details the pre-training process of Seer in real-world experiments, including environment setup, dataset preparation, and training procedures. Downstream task processing and fine-tuning are covered in Real-World (Quick Training w & w/o pre-training).
Relevant checkpoints are available on the website.
| Model | Checkpoint |
|---|---|
| CALVIN ABC-D | Seer (Avg.Len. : 3.98) / Seer Large (Avg.Len. : 4.30) |
| Real-World | Seer (Droid Pre-trained) |
- Release real-world expriment code.
- Release CALVIN ABC-D experiment code (Seer).
- Release the evaluation code of Seer-Large on CALVIN ABC-D experiment.
- Release the training code of Seer-Large on CALVIN ABC-D experiment (Reviewing the code).
- Release LIBERO-LONG experiment code.
All assets and code are under the Apache 2.0 license unless specified otherwise.
If you find the project helpful for your research, please consider citing our paper:
@article{tian2024seer,
title={Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation},
author={Tian, Yang and Yang, Sizhe and Zeng, Jia and Wang, Ping and Lin, Dahua and Dong, Hao and Pang, Jiangmiao},
journal={https://arxiv.org/abs/2412.15109},
year={2024}
}This project builds upon GR-1 and Roboflamingo. We thank these teams for their open-source contributions.
