Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENHANCEMENT] Fail gracefully when num-eval-batches > ds.size #1893

Open
jprokos26 opened this issue Mar 2, 2023 · 1 comment
Open

[ENHANCEMENT] Fail gracefully when num-eval-batches > ds.size #1893

jprokos26 opened this issue Mar 2, 2023 · 1 comment
Assignees
Labels
bug-ux Issues dealing with how the user interacts with the application enhancement

Comments

@jprokos26
Copy link
Contributor

jprokos26 commented Mar 2, 2023

When supplying --num-eval-batches from the CLI or from the config file, if it is larger than the size of the dataset armory will not fail until reaching the StopIteration error

@jprokos26 jprokos26 changed the title Fail gracefully when num-eval-batches > ds.size [ENHANCEMENT] Fail gracefully when num-eval-batches > ds.size Mar 2, 2023
@jprokos26 jprokos26 added the bug-ux Issues dealing with how the user interacts with the application label Mar 2, 2023
@jprokos26
Copy link
Contributor Author

@christopherwoodall This never got merged into tfdsv4, https://github.com/twosixlabs/armory/blob/tfdsv4/armory/utils/config_loading.py#L81-L82
Simple fix by adding suggested change

        if num_batches > dataset.batches_per_epoch:
            # since num-eval-batches only applies at test time, we can assume there is only 1 epoch
            raise ValueError(
                f"{num_batches} eval batches were requested, but dataset has only {dataset.batches_per_epoch} batches of size {dataset.batch_size}"
            )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug-ux Issues dealing with how the user interacts with the application enhancement
Projects
None yet
Development

No branches or pull requests

2 participants