Sat2Density: Faithful Density Learning from Satellite-Ground Image Pairs

Ming Qian, Jincheng Xiong, Gui-Song Xia, Nan Xue

IEEE/CVF International Conference on Computer Vision (ICCV), 2023

Project Page | Paper | Data | Install.md

See the Project Page for more results and a brief video introduction to Sat2Density.

Checkpoints Downloading

Two checkpoints for CVACT and CVUSA can be found from this url. You can also run the following command to download them.

bash scripts/download_weights.sh

QuickStart Demo

Video Synthesis

Example Usage

python test.py --yaml=sat2density_cvact \
  --test_ckpt_path=2u87bj8w \
  --task=test_vid \
  --demo_img=demo_img/case1/satview-input.png  \
  --sty_img=demo_img/case1/groundview.image.png  \
  --save_dir=results/case1

We visualize our .vtk shape files with Paraview.

Illumination Interpolation

python test.py --task=test_interpolation \
--yaml=sat2density_cvact \
--test_ckpt_path=2u87bj8w \
--sty_img1=demo_img/case9/groundview.image.png \
--sty_img2=demo_img/case7/groundview.image.png \
--demo_img=demo_img/case3/satview-input.png \
--save_dir=results/case2

Train & Inference

• We trained our model using 1 V100 32GB GPU. The training phase will take about 20 hours.
• For data preparation, please check out data.md.

Inference

To test Center Ground-View Synthesis setting If you want save results, please add --task=vis_test

# CVACT
python offline_train_test.py --yaml=sat2density_cvact --test_ckpt_path=2u87bj8w
# CVUSA
python offline_train_test.py --yaml=sat2density_cvusa --test_ckpt_path=2cqv8uh4

To test inference with different illumination

# CVACT
bash inference/single_style_test_cvact.sh
# CVUSA
bash inference/single_style_test_cvusa.sh

To test synthesis ground videos

bash inference/synthesis_video.sh

Training

Training command

# CVACT
CUDA_VISIBLE_DEVICES=X python train.py --yaml=sat2density_cvact
# CVUSA
CUDA_VISIBLE_DEVICES=X python train.py --yaml=sat2density_cvusa

Citation

If you use this code for your research, please cite

@InProceedings{Qian_2023_ICCV,
    author    = {Qian, Ming and Xiong, Jincheng and Xia, Gui-Song and Xue, Nan},
    title     = {Sat2Density: Faithful Density Learning from Satellite-Ground Image Pairs},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2023},
    pages     = {3683-3692}
}