Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: RTX Lidar Support #802

Closed
wants to merge 3 commits into from
Closed

WIP: RTX Lidar Support #802

wants to merge 3 commits into from

Conversation

MoreTore
Copy link
Contributor

@MoreTore MoreTore commented Aug 7, 2024

Description

Added RTX lidar support for training and inference instances. This is a WIP but as its stands its working. Works in headless mode. Let me know if im on the right/wrong path.

Fixes #865

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Screenshots

lidar
image

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@Mayankm96 Mayankm96 requested a review from pascal-roth August 7, 2024 10:49
@Mayankm96 Mayankm96 added the enhancement New feature or request label Aug 7, 2024
rep_annotator.attach(render_prod_path)
# Debug draw
# self.writer = rep.writers.get("RtxLidarDebugDrawPointCloudBuffer")
# self.writer.attach(render_prod_path)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disable debug draw when in --headless mode

@jsmith-bdai jsmith-bdai mentioned this pull request Aug 23, 2024
@jtigue-bdai jtigue-bdai self-requested a review August 23, 2024 14:46
@jtigue-bdai
Copy link
Collaborator

jtigue-bdai commented Aug 23, 2024

@MoreTore, thanks for taking a stab at this. It looks like its pretty functional. I know this feature is of interest across the community. Are you looking for support from the Isaac Lab team to get it integrated? If so I will be a likely point person for this.

There are a few ways we can help:

  1. We can provide suggestions/reviews/requests for changes while you do most of the development.
  2. We can handle most of the development while we keep you in the loop as a reviewer.

Feel free to reach out with questions or want to discuss.

@MoreTore
Copy link
Contributor Author

MoreTore commented Sep 17, 2024

@jtigue-bdai If you guys want to take this over that would be good. I do not actually need rtx lidar sensors at this point.

@jtigue-bdai
Copy link
Collaborator

@MoreTore thanks for letting us know. We will pick this up and add it to our list.

@shaoxiang
Copy link

Hello, Thank you very much for contributing this PR.
I tested this PR and found that the LiDAR point cloud displayed in debug does not match the scene.
Is this just a display problem?
Or is it a problem with the LiDAR update rate or does the point cloud need to be transformed?
Looking forward to getting a reply, thank you.
Here is a video:
https://github.com/user-attachments/assets/5dfa8d94-8d14-44ab-9071-ead13acc6a9a

@MoreTore
Copy link
Contributor Author

Hello, Thank you very much for contributing this PR.
I tested this PR and found that the LiDAR point cloud displayed in debug does not match the scene.
Is this just a display problem?
Or is it a problem with the LiDAR update rate or does the point cloud need to be transformed?
Looking forward to getting a reply, thank you.
Here is a video:
https://github.com/user-attachments/assets/5dfa8d94-8d14-44ab-9071-ead13acc6a9a

I looks like it's just the refresh rate of the pointcloud and the debug draw points are persisting between updates in the same transform as they were captured.

@jtigue-bdai
Copy link
Collaborator

Closing this in favor of: PR 1372. Thanks for the starting point @MoreTore, please see the other PR for implementation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add RTX Lidar sensor
5 participants