Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a detail variation method of the image input #88

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

FishWoWater
Copy link

@FishWoWater FishWoWater commented Dec 24, 2024

According to Sec 3.4 of the paper, I have implemented the detail variation method given the image input.

Pipeline:

  1. Given a Z-up axis triangle mesh, it is voxelized(https://github.com/Forceflow/cuda_voxelizer) into active voxels (64^3 -> N x 4), in the same form of Sparse Structure Voxels
  2. Skip the sparse structure sampler and directly run the 2nd stage, SLAT sampler, based on the input image condition.

I have provided a example script, texgen tab, as well as example data

It can composite meshes and images of different styles, producing some creative outputs.

Example outputs:

typical_humanoid_block_robot+typical_humanoid_mech_gs.mp4
typical_creature_dragon+typical_creature_furry_gs.mp4
typical_creature_dragon+typical_creature_elephant_gs.mp4

Also other minor changes:
i.e. options to save intrinsics/extrinsics/renderings in the example
output sRGB texture
expose opt/fast mode in texture baking

…mple & Demo.

Add options to save more outputs (intrinsics/extrinsics/renderings) of the example.
@FishWoWater
Copy link
Author

@microsoft-github-policy-service agree

@YuDeng
Copy link
Contributor

YuDeng commented Dec 25, 2024

Nice implementation! The image-conditioned variation results are really cool! We have only tested text-conditioned variation in our experiments. This one seems to be more creative.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants