Skip to content

Latest commit

 

History

History
59 lines (43 loc) · 4.32 KB

DEMO.md

File metadata and controls

59 lines (43 loc) · 4.32 KB

PartDistillation DEMO

Here we provide DEMO for PartDistillation. We use the default demo interface from detectron2. Please follow here to setup Detic, and also download our pre-trained weight here. Please save the weight under ./weights/PartProposalLearning/IN1K+Human/part_proposal_model.pth (see here).

With an image of a person and a bicycle:

Use the following command to segment each class:

python part_distillation_demo.py --input figs/input/bicycle_person.jpg --output figs/output/part_proposal/bicycle.jpg --vocabulary custom --confidence-threshold 0.1 --part-score-threshold 0.3 --custom_vocabulary bicycle --min-image-size 640 --non-overlapping

python part_distillation_demo.py --input figs/input/bicycle_person.jpg --output figs/output/part_proposal/person.jpg --vocabulary custom --confidence-threshold 0.1 --part-score-threshold 0.3 --custom_vocabulary person --min-image-size 640 --non-overlapping

If setup correctly, it should look like this:

  • --custom_vcabulary to segment different object class.
  • --non-overlapping flag to generate non-overlapping part segmentation.
  • --dcrf to further smooth out the boundary.
  • --part-score-threshold to filter-out low score parts.

Here are different examples (cat and cola):

python part_distillation_demo.py --input figs/input/cat_and_cola.jpg --output figs/output/part_proposal/cat.jpg --vocabulary custom --confidence-threshold 0.1 --part-score-threshold 0.3 --custom_vocabulary cat --min-image-size 640 

python part_distillation_demo.py --input figs/input/cat_and_cola.jpg --output figs/output/part_proposal/cola.jpg --vocabulary custom --confidence-threshold 0.1 --part-score-threshold 0.3 --custom_vocabulary cola --min-image-size 640

Here we visualize overlapping part segmentation. If setup correctly, it should look like this:

Visualizing Part Segments by Pixel Grouping

We first explore different pre-trained features and their capability of part segmentation. We provide a demo to try out:

python part_segment_demo.py --input figs/input/cat_and_cola.jpg --output figs/output/part_segment/cola.jpg --vocabulary custom --confidence-threshold 0.1 --custom_vocabulary cola --min-image-size 640 --k 4 --weight-name coco_instance_seg --dcrf

python part_segment_demo.py --input figs/input/cat_and_cola.jpg --output figs/output/part_segment/cat.jpg --vocabulary custom --confidence-threshold 0.1 --custom_vocabulary cat --min-image-size 640 --k 4 --weight-name coco_instance_seg --dcrf

Above command reads cat.jpg image as input, and use Detic to first segment instance of the prompted class (--custom_vocabulary, "cat" in this case). Then it uses the pre-trained features specified with --weight-name to cluster the features to group pixels.

  • --k is used for the number of clusters.
  • --dcrf is used for applying dense-CRF as post-processing.
  • See here to find the available weight-name options. Please download the weights from Mask2Former (here) and place them under ./weights/... (see here).

If setup correctly, the result should look like below: