Learning to Select Camera Views: Efficient Multiview Understanding at Few Glances

Overview

We release code for MVSelect, a view selection module for efficient multiview understanding. Parallel to reducing the image resolution or using lighter network backbones, the proposed approach reduces the computational cost for multiview understanding by limiting the number of views considered.

Content

• Dependencies
• Data Preparation
• Training

Dependencies

Please install dependencies with

pip install -r requirements.txt

Data Preparation

For multiview classification, we use ModelNet40 dataset with the circular 12-view setup [link][download] and the dodecahedral 20-view setup [link][download].

For multiview detection, we use MultiviewX [link][download] and Wildtrack [link][download] in this project.

Your ~/Data/ folder should look like this

Data/
├── modelnet/
│   ├── modelnet40_images_new_12x/
│   │   └── ...
│   └── modelnet40v2png_ori4/
|       └── ...
├── MultiviewX/
│   └── ...
└── Wildtrack/ 
    └── ...

Training

In order to train the task networks, please run the following

# task network
python main.py

This should automatically return the full N-view results, as well as the oracle performances.

To train the MVSelect, please run

# MVSelect only
python main.py --step 2 --base_lr 0 --other_lr 0
# joint training
python main.py --step 2

The default dataset is Wildtrack. For other datasets, please specify with the -d argument.

Pre-trained models

You can download the checkpoints at this link.