【Presentation】GBD论文写作规划和进度安排

Timeline of `GBD`

Info

[!TIP] Sketch Version

GBD: A Global-Scale Building Damage Dataset based on Diffusion Model

Object Detection of Damaged Buildings in disastrous event is important for aiding and reconstruction. //Current approaches for building damage assessment include CNN-based models and transformer-based models. However, these pre-trained model is still lack of general capability which fails to timeliness of detection in terms of disastrous events. //In this work, we propose a generative model ModelName to manufacture potential post-disaster images from vulnerable regions at global scale, be it GBD. //We find that after further training SOTA models on GBD, the performance of models show great improvements.

Overview

Overview of GBD Framework

Khanna, S., Liu, P., Zhou, L., Meng, C., Rombach, R., Burke, M., Lobell, D., & Ermon, S. (2023, December 6). DiffusionSat: A Generative Foundation Model for Satellite Imagery. International Conference on Learning Representations (ICLR 2024). https://doi.org/10.48550/arXiv.2312.03606

[!IMPORTANT] Backbone Network : UNet(Learning Knowledge) + Diffusion(Generation)

Overview of Unified Conditional Framework for Diffusion-based Image Restoration (Zhang et al., 2023)

[!NOTE] Reference

Zhang, Y., Shi, X., Li, D., Wang, X., Wang, J., & Li, H. (2023). A Unified Conditional Framework for Diffusion-based Image Restoration. 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

[!IMPORTANT] Main Contributions

We propose a generative model to synthesize labeled building damage dataset.
SOTA models undergo further training on our dataset shows considerable improvement in zero-shot for downstream tasks like segmentation, change detection and scene classification, especially in regions that has never been seen in the training dataset.

[!IMPORTANT] Key Points on Architecture

Compared to natural images/synthetic images, RSIs have much more information regarding spatial relationship and so on.
The extracted features of differences between pre-event RSIs and post-event RSIs is essential for image generation.
RSIs is ideal to incorporate with contextual cues. In other words, vision-language pretraining (VLP) models would dramatically improve the performance.

Schedule

Presentation

【0425】Conception of GBD.pptx

Zhang, Y., Shi, X., Li, D., Wang, X., Wang, J., & Li, H. (2023). A Unified Conditional Framework for Diffusion-based Image Restoration. 37th Conference on Neural Information Processing Systems (NeurIPS 2023)

【0509】Introduction to FloodNet

Zhao, D., Lu, J., & Yuan, B. (2024). See, Perceive, and Answer: A Unified Benchmark for High-Resolution Postdisaster Evaluation in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14. https://doi.org/10.1109/TGRS.2024.3386934

【0516】Super-Resolution via a noval GAN & Compared to Diffusion Model to Generative Adversarial Model

Meng, F., Wu, S., Li, Y., Zhang, Z., Feng, T., Liu, R., & Du, Z. (2024). Single Remote Sensing Image Super-Resolution via a Generative Adversarial Network With Stratified Dense Sampling and Chain Training. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–22. https://doi.org/10.1109/TGRS.2023.3344112

【0523】DiffusionSAT & 【交流总结】第九届青年地学论坛会议

Khanna, S., Liu, P., Zhou, L., Meng, C., Rombach, R., Burke, M., Lobell, D., & Ermon, S. (2023, December 6). DiffusionSat: A Generative Foundation Model for Satellite Imagery. International Conference on Learning Representations (ICLR 2024). https://doi.org/10.48550/arXiv.2312.03606

Survey on Related Work

Generative Models
- Zhang, Y., Shi, X., Li, D., Wang, X., Wang, J., & Li, H. (2023). A Unified Conditional Framework for Diffusion-based Image Restoration. 37th Conference on Neural Information Processing Systems (NeurIPS 2023)
- Meng, F., Wu, S., Li, Y., Zhang, Z., Feng, T., Liu, R., & Du, Z. (2024). Single Remote Sensing Image Super-Resolution via a Generative Adversarial Network With Stratified Dense Sampling and Chain Training. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–22. https://doi.org/10.1109/TGRS.2023.3344112
- ...
Semantic Segmentation for RSI
- Cui, L., Jing, X., Wang, Y., Huan, Y., Xu, Y., & Zhang, Q. (2023). Improved Swin Transformer-Based Semantic Segmentation of Postearthquake Dense Buildings in Urban Areas Using Remote Sensing Images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 16, 369–385. https://doi.org/10.1109/JSTARS.2022.3225150
- Diakogiannis, F. I., Waldner, F., Caccetta, P., & Wu, C. (2020). ResUNet-a: A deep learning framework for semantic segmentation of remotely sensed data. ISPRS Journal of Photogrammetry and Remote Sensing, 162, 94–114. https://doi.org/10.1016/j.isprsjprs.2020.01.013
- Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. Conference on Computer Vision and Pattern Recognition (CVPR), 3431–3440. https://doi.org/10.1109/CVPR.2015.7298965
- Yang, X., Li, S., Chen, Z., Chanussot, J., Jia, X., Zhang, B., Li, B., & Chen, P. (2021). An attention-fused network for semantic segmentation of very-high-resolution remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 177, 238–262. https://doi.org/10.1016/j.isprsjprs.2021.05.004
- Zhang, C., Jiang, W., Zhang, Y., Wang, W., Zhao, Q., & Wang, C. (2022). Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery. IEEE Transactions on Geoscience and Remote Sensing, 60, 1–20. https://doi.org/10.1109/TGRS.2022.3144894
- Zheng, Z., Zhong, Y., Wang, J., & Ma, A. (2020). Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery. Computer Vision and Pattern Recognition (CVPR), 4095–4104. https://doi.org/10.1109/CVPR42600.2020.00415
Others
- Li, Q., Mou, L., Sun, Y., Hua, Y., Shi, Y., & Zhu, X. X. (2024). A Review of Building Extraction From Remote Sensing Imagery: Geometrical Structures and Semantic Attributes. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–15. https://doi.org/10.1109/TGRS.2024.3369723
- Zhao, D., Lu, J., & Yuan, B. (2024). See, Perceive, and Answer: A Unified Benchmark for High-Resolution Postdisaster Evaluation in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14. https://doi.org/10.1109/TGRS.2024.3386934
- ...

Architecture

Data

[!NOTE] Reference Zhao, D., Lu, J., & Yuan, B. (2024). See, Perceive, and Answer: A Unified Benchmark for High-Resolution Postdisaster Evaluation in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing, 62, 1–14. https://doi.org/10.1109/TGRS.2024.3386934

Gupta, R., Hosfelt, R., Sajeev, S., Patel, N., Goodman, B., Doshi, J., Heim, E., Choset, H., & Gaston, M. (2019). xBD: A Dataset for Assessing Building Damage from Satellite Imagery (arXiv:1911.09296). arXiv. https://doi.org/10.48550/arXiv.1911.09296