Multiple connected panoramas generated?

[antoinemacia]

Hello,

Wondering how multiple generated panoramas can be connected to one another (essentially maintaining the style as well as whatever artefacts/furnitures have been generated)

Use case: multiple panoramas for a home

[ArcherFMY]

Hi,

It is a very interesting application and we have discussed within the team.

We think one straightforward method is that we can generate a main pano-image (like a living room) and convert it to a 3D structure using depth. Then find a new location to render a sparse pano-image and fill the holes via inpainting.

Another data-driven idea is that we can treat it as a pano-video generation problem. The AnimateDiff (https://animatediff.github.io/) can be a try. The motion module may help to maintain the continuity of different panoramas. But we have not tried yet.

[antoinemacia]

Yeah nice!

The first option is pretty sound, this project could be of interest (I believe does more or less what you're describing)

Im curious how styled could be maintained across rooms however...

[ArcherFMY]

Yeah nice!

The first option is pretty sound, this project could be of interest (I believe does more or less what you're describing)

Im curious how styled could be maintained across rooms however...

Maybe we can train a LoRA for a certain decoration style :-)

[antoinemacia]

Yeah nice! Also realise I have shared the wrong paper previously in regards to depth conversion, this is what I meant to share https://cyhsu14.github.io/OmniNeRF/

[ArcherFMY]

Yeah nice! Also realise I have shared the wrong paper previously in regards to depth conversion, this is what I meant to share https://cyhsu14.github.io/OmniNeRF/

Interesting! Thanks for sharing!

[zhanghongyong123456]

Yeah nice! Also realise I have shared the wrong paper previously in regards to depth conversion, this is what I meant to share https://cyhsu14.github.io/OmniNeRF/

Do you have any implementation cases? I found that a video is the effect you want, but I don't know what the implementation process is.

1.19.mp4