Paper link

Dataset link

• Introduction
• Install
• Training and Evaluation
  • Configs and pre-trained weights
  • Training
  • Evaluation
• ModelZoo
  • Models using down-sampled AODRaw
  • Models using sliced AODRaw
• Citation
• License
• Acknowledgement

Existing object detection methods often consider sRGB input, which was compressed from RAW data using ISP originally designed for visualization. However, such compression might lose crucial information for detection, especially under complex light and weather conditions. We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories, capturing a broad range of indoor and outdoor scenes under 9 distinct light and weather conditions. Based on AODRaw that supports RAW and sRGB object detection, we provide a comprehensive benchmark for evaluating current detection methods. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW, prompting us to directly pre-train on the RAW domain. However, it is harder for RAW pre-training to learn rich representations than sRGB pre-training due to the camera noise. To assist RAW pre-training, we distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain. As a result, we achieve substantial improvements under diverse and adverse conditions without relying on extra pre-processing modules.

Please refer to AODRaw to download and preprocess our AODRaw dataset.

Please refer to the README of mmdetection.

• We provide training and evaluation for RAW and sRGB object detection.
• The images in the AODRaw are recorded at a resolution of $6000\times 4000$. It is unrealistic to feed such huge images into the detectors. Thus, we adopt two experiment settings: 1) down-sampling the images into a lower resolution of $2000\times1333$, corresponding to configs, and 2) slicing the images into a collection of $1280\times 1280$ patches, corresponding to configs. Please preprocess the AODRaw dataset or directly download the processed files in datasets and downloading.

Training and evaluation using down-sampled AODRaw:

Training and evaluation using sliced AODRaw:

Pre-trained weights for ConvNeXt-T and Swin-T:

python tools/train.py ${CONFIG_FILE} [optional arguments]

bash tools/dist_train.sh ${CONFIG_FILE} ${GPU_NUM} [optional arguments]

For more training and evaluation command details, please refer to mmdetection.

python tools/test.py ${CONFIG_FILE} ${CHECKPOINT_FILE} [optional arguments]

bash tools/dist_test.sh ${CONFIG_FILE} ${CHECKPOINT_FILE} ${GPU_NUM} [optional arguments]

For more training and evaluation command details, please refer to mmdetection.

Please follow downsampling to preprocess the images or download preprocessed images in download.

The directory images_downsampled_srgb is required for the above experiments.

The directory images_downsampled_raw is required for the above experiments.

The directory images_downsampled_raw is required for the above experiments.

Please follow slicing to preprocess the images or download preprocessed images in download.

The directory images_slice_raw is required for the above experiments.

The directory images_slice_raw is required for the above experiments.

@article{li2024aodraw,
  title={Towards RAW Object Detection in Diverse Conditions}, 
  author={Zhong-Yu Li and Xin Jin and Boyuan Sun and Chun-Le Guo and Ming-Ming Cheng},
  journal={arXiv preprint arXiv:2411.15678},
  year={2024},
}

The code is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License for NonCommercial use only.

This repo is modified from mmdetection.