Video Recording for Furniture Tasks

[chenzixuan99]

Thank you for sharing such excellent work! I have a small question: How can I save videos for each episode when fine-tuning the furniture tasks? Is there a specific parameter in the configuration file that I can modify to achieve this?

[allenzren]

Hi there! Recording Furniture-Bench video is possible if you add +env.specific.record=True to the command. Right now it only records the video of the first environment and it can be slow since the camera observations are now generated in IsaacGym. We don't recommend doing this with the default 1000 parallelized environments.

You can see the recording implemented here in our Furniture-Bench fork. You can modify the implementation to your need.

It is probably a better idea to have a script that loads the checkpoint and generates video with only one environment spawned.

[chenzixuan99]

Hi there! Recording Furniture-Bench video is possible if you add +env.specific.record=True to the command. Right now it only records the video of the first environment and it can be slow since the camera observations are now generated in IsaacGym. We don't recommend doing this with the default 1000 parallelized environments.

You can see the recording implemented here in our Furniture-Bench fork. You can modify the implementation to your need.

It is probably a better idea to have a script that loads the checkpoint and generates video with only one environment spawned.

Hi, thank you for your response. Based on your suggestion, I modified the parameters as follows:

env:
  n_envs: 1
  name: ${env_name}
  env_type: furniture
  max_episode_steps: 700
  best_reward_threshold_for_success: 1
  specific:
    headless: false
    furniture: one_leg
    randomness: low
    normalization_path: ${normalization_path}
    act_steps: ${act_steps}
    sparse_reward: true
    record: true

render:
  freq: 10
  num: 1

However, after making these adjustments, the video still hasn't been recorded during the fine-tuning process. Do you have any other suggestions to resolve this issue? Thank you!

[allenzren]

Could you try env.specific.headless=True? The instruction on the current README is for visualization with GUI, not for recording videos. You don't need to make any other changes to the config except for adding env.specific.record=True.

If it still does not work, feel free to share the log with me.