Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

kartikdutt18 · 2020-05-29T10:57:48Z

Hey everyone,
This is is a WIP of image data loader for this repo.

TO DO :

Load labels into the data loader.
Add Resize function to reshape loaded images (by 31st).
(maybe another PR or in this one, kindly let me know).
Load Images. (by 1st)
Create subset of the dataset to upload on mlpack.org for testing. (By 2nd)
Write tests for the same. (By 2nd)
Repeat for classification of imagenet
(Since the code is similar it will only take a day or two after completing PASCAL VOC).

KimSangYeon-DGU · 2020-05-29T12:15:43Z

Hi @kartikdutt18. Regarding Add Resize function to reshape loaded images, if this topic is about the discussion we had in the Community Bonding period, let's put this off. Although I suggested that before, I think the feature to set the size of an image when loading is not vital for this summer work. In addition, the resizing feature is more related to image utils, not data loader.

Please let me know what you think

kartikdutt18 · 2020-05-29T12:18:31Z

Hey @KimSangYeon-DGU, This is something I missed while making the proposal, The images in PASCAL dataset have different sizes, I can preprocess the images in python and we can store that dataset or We can add a resize feature that uses Bilinear interpolation from mlpack to resize images. Kindly let me know what you think.

KimSangYeon-DGU · 2020-05-29T12:21:07Z

@kartikdutt18 Ok, then, let's do that :) I don't mind if you do this work on either this PR or another.

kartikdutt18 · 2020-05-29T12:22:57Z

@kartikdutt18 Ok, then, let's do that :) I don't mind if you do this work on either this PR or another.

Great, I'll do it in this PR first, if get a bit large I can open another one with just that. Thanks a lot.

KimSangYeon-DGU · 2020-05-29T12:29:51Z

Yes, the best is to implement and use resizing feature in mlpack :) BTW, please let me know if this work turns into a burden for the summer work. We can adjust by going another way as you said above comment.

kartikdutt18 · 2020-05-29T12:32:12Z

Yes, the best is to implement and use resizing feature in mlpack :) BTW, please let me know if this work turns into a burden for the summer work. We can adjust by going another way as you said above comment.

Thanks a lot, If things go well I should have the resize function ready by tomorrow or the day after. Also, I think it'll be nice to have that feature.

KimSangYeon-DGU · 2020-05-29T12:38:11Z

Or we can use the third library for image processing like OpenCV as well

kartikdutt18 · 2020-05-29T12:40:00Z

Or we can use OpenCV as well

Right, That would be much easier. I think have the conversion code for opencv matrix to armadillo so that should be easy to do. Thanks a lot for the amazing suggestion.

KimSangYeon-DGU · 2020-05-29T12:51:44Z

Yes, and as you know, if you need to use OpenCV, you can do that in mlpack/examples

kartikdutt18 · 2020-06-01T10:00:56Z

Hey @KimSangYeon-DGU, I completed the main function for PASCAL Dataloader i.e. LoadObjectDetectionDataset. Tested with couple of images and it works fine. I have also added a resize function and augmentation class. In augmentation class only resize is supported for now. Maybe we can open issue if anyone wants to implement some augmentation they can. I haven't used open cv yet because that was much slower than mlpack since we had to iterate over each column however here we pass a batch of images.
Things left to do are mentioned below :

Clean up.
Writing tests.
Downloading Pascal VOC dataset from constructor and calling LoadObjectDetectionDataset from constructor.
Writing Documentation for both Object Detection Dataloader and Augmentation class (Might be better to have a separate PR).
Uploading a subset of Dataset to mlpack.org for testing since its not practical to download whole dataset for testing (Roughly 2GB).

Kindly let me know what you think.
Regards.

kartikdutt18 · 2020-06-01T18:18:47Z

Hey @KimSangYeon-DGU, just another update. I have tests for both the data loader as well as augmentation class. I have created a subset that we would can upload to mlpack.org that can be used for testing. Only one thing is left i.e. Calling function through the constructor. If this makes sense I can add that in the next commit.
Regards.

KimSangYeon-DGU · 2020-06-02T02:18:44Z

Nice @kartikdutt18! Since #12 got merged, can you re-base this branch? it helps to review.

Meanwhile, I'll look through this PR :)

kartikdutt18 · 2020-06-02T03:06:49Z

augmentation/augmentation_impl.hpp

+  // Not sure how to avoid a copy here.
+  DatasetType output;
+  resizeLayer.Forward(dataset, output);
+  dataset = output;


I have resolved all conflicts. The one thing I am unsure of is how to avoid a copy here. For very large datasets resizing will simply not be possible incase the program quits as they run out of RAM.

How about you use this?

Suggested change

// Not sure how to avoid a copy here.

DatasetType output;

resizeLayer.Forward(dataset, output);

dataset = output;

// Not sure how to avoid a copy here.

DatasetType output;

resizeLayer.Forward(dataset, output);

dataset = std::move(output);

Ahh makes sense. Awesome, Thanks a lot for this.

One more thing I was planning to change was to use fields in labels. So when labels are passed in preprocessor function for YOLO model it would easier than doing look ups. Currently each labels has a different image, we could use a field vector and store multiple bounding boxes in one column rather than having a separate column for each bounding box. And the build failure is associated to zip file, I'll switch to tar / tar.gz and it should be fixed. Kindly let me know if this makes sense.

Currently each labels has a different image, we could use a field vector and store multiple bounding boxes in one column rather than having a separate column for each bounding box

Can you do that as a PoC? and then, let's see which one is better. I need to figure out how the field vector works.

At first, I thought there will be a problem in using linear algebra operations when we put multiple bounding boxes into one column. However, as you said, each label has a different shape of image, so we could do that.

Sure, I'll make the changes now.

Repeat for classification of imagenet
(Since the code is similar it will only take a day or two after completing PASCAL VOC).

I think this PR is almost ready, I am working on the last to-do now. Will complete it by tomorrow so that we can have the same functionality as FlowFromDirectory (keras) for image datasets. The Pascal dataloader is complete. It takes roughly 18 or more minutes to load all the images though. About the field vector I'll create another branch same as this one and post this link here. If that is better I can simply replace the commits.

I have added image dataloader with flow from directory functionality. We can now support voc-detection, voc-classification and cifar-10 / 100. I'll set up the constructor and do the clean up as well. The build error in linux / windows now is because they have .DS_Store file. I'll add a check for that as well. Currently just manually checking for the file. Regards.

kartikdutt18 · 2020-06-02T08:33:19Z

Also one thing that we could add maybe later (or open an issue) is addition of script to convert csv to xml files.

KimSangYeon-DGU

Left some Style related comments :)

augmentation/augmentation.hpp

augmentation/augmentation_impl.hpp

augmentation/augmentation.hpp

dataloader/dataloader.hpp

dataloader/dataloader_impl.hpp

Style fixes Style fixes

kartikdutt18 · 2020-06-08T06:05:02Z

This PR is migrated to #18. Thanks a lot.

mlpack-bot bot added s: needs review s: unanswered s: unlabeled labels May 29, 2020

kartikdutt18 marked this pull request as draft May 29, 2020 10:57

kartikdutt18 force-pushed the ImageDataloaders branch 3 times, most recently from 302375c to 94fba51 Compare May 29, 2020 11:06

KimSangYeon-DGU removed the s: unanswered label May 29, 2020

KimSangYeon-DGU added the t: added feature label May 31, 2020

kartikdutt18 force-pushed the ImageDataloaders branch from ddb2d98 to 2727851 Compare June 1, 2020 09:53

kartikdutt18 changed the title ~~[WIP] Add Image Dataloader for PASCAL VOC dataset.~~ Add Image Dataloader for PASCAL VOC dataset. Jun 1, 2020

kartikdutt18 marked this pull request as ready for review June 1, 2020 18:14

kartikdutt18 force-pushed the ImageDataloaders branch from 015e197 to c058367 Compare June 2, 2020 02:47

KimSangYeon-DGU removed the s: unlabeled label Jun 2, 2020

kartikdutt18 commented Jun 2, 2020

View reviewed changes

KimSangYeon-DGU reviewed Jun 2, 2020

View reviewed changes

kartikdutt18 changed the title ~~Add Image Dataloader for PASCAL VOC dataset.~~ Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. Jun 3, 2020

kartikdutt18 added 23 commits June 5, 2020 10:20

Augmentation work

a34d08d

Augmentation work

6a43f9d

Complete LoadObjectDetection Function

193e243

Style fixes Style fixes

Add tests for augmentation

3739acb

Add tests for dataloader

a24514b

Upload dataset temp

3668335

Resolve merge conflicts

c5828c6

Resolve merge conflicts

64c950b

Resolve merge conflicts

a6d061a

Style fixes

f45cec0

Set up constructor

c6a6819

Complete Dataloader, write function definition for image dataloader

b782bb9

Build Fixed, Style Fixed

044ffea

style fixed

553273d

Fix linux and Windows build by ignoring invalid files like .DS_Store

baae4e6

Add image dataloader, needs clean up

c5302b9

Style fixes

c914e12

Check for file type

fef6b19

Check for correct extention and filetype

90c9039

Style Fixed

17da476

Build Fixed, Nice way to avoid hidden files

7579ba2

Style Fixed

3b7bcfa

Clean Up

adade09

kartikdutt18 force-pushed the ImageDataloaders branch from e97e616 to adade09 Compare June 5, 2020 04:53

kartikdutt18 added 3 commits June 5, 2020 10:25

Why is config there even though I did a rebase

6e230b8

Add data split (internal), need implementation only for field type

1a82151

Style Fixed

0023b4a

KimSangYeon-DGU mentioned this pull request Jun 5, 2020

Image Dataloader with Field type #18

Merged

kartikdutt18 closed this Jun 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

kartikdutt18 commented May 29, 2020 •

edited

Loading

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

kartikdutt18 commented Jun 1, 2020 •

edited

Loading

kartikdutt18 commented Jun 1, 2020

KimSangYeon-DGU commented Jun 2, 2020 •

edited

Loading

kartikdutt18 Jun 2, 2020

KimSangYeon-DGU Jun 2, 2020 •

edited

Loading

kartikdutt18 Jun 2, 2020

kartikdutt18 Jun 2, 2020 •

edited

Loading

KimSangYeon-DGU Jun 2, 2020

kartikdutt18 Jun 3, 2020

kartikdutt18 Jun 3, 2020

kartikdutt18 Jun 3, 2020 •

edited

Loading

kartikdutt18 commented Jun 2, 2020

KimSangYeon-DGU left a comment

kartikdutt18 commented Jun 8, 2020

Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

Conversation

kartikdutt18 commented May 29, 2020 • edited Loading

KimSangYeon-DGU commented May 29, 2020 • edited Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 • edited Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 • edited Loading

kartikdutt18 commented May 29, 2020

KimSangYeon-DGU commented May 29, 2020 • edited Loading

kartikdutt18 commented Jun 1, 2020 • edited Loading

kartikdutt18 commented Jun 1, 2020

KimSangYeon-DGU commented Jun 2, 2020 • edited Loading

kartikdutt18 Jun 2, 2020

Choose a reason for hiding this comment

KimSangYeon-DGU Jun 2, 2020 • edited Loading

Choose a reason for hiding this comment

kartikdutt18 Jun 2, 2020

Choose a reason for hiding this comment

kartikdutt18 Jun 2, 2020 • edited Loading

Choose a reason for hiding this comment

KimSangYeon-DGU Jun 2, 2020

Choose a reason for hiding this comment

kartikdutt18 Jun 3, 2020

Choose a reason for hiding this comment

kartikdutt18 Jun 3, 2020

Choose a reason for hiding this comment

kartikdutt18 Jun 3, 2020 • edited Loading

Choose a reason for hiding this comment

kartikdutt18 commented Jun 2, 2020

KimSangYeon-DGU left a comment

Choose a reason for hiding this comment

kartikdutt18 commented Jun 8, 2020

kartikdutt18 commented May 29, 2020 •

edited

Loading

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

KimSangYeon-DGU commented May 29, 2020 •

edited

Loading

kartikdutt18 commented Jun 1, 2020 •

edited

Loading

KimSangYeon-DGU commented Jun 2, 2020 •

edited

Loading

KimSangYeon-DGU Jun 2, 2020 •

edited

Loading

kartikdutt18 Jun 2, 2020 •

edited

Loading

kartikdutt18 Jun 3, 2020 •

edited

Loading