Netcdf quantize #32
Merged
Netcdf quantize #32
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add the testing of the C interface to the NetCDF quantize functionality. Examples of compressed data are show for the three quantize methods:
along with two further examples:
We quantize to 3 significant figures in the Bit grooming, and granular bit round case, or 10 bits in the bit rounding case. (These should give similar behavior).
The examples shown use an analytic function, which will probably flatter the quantization slightly. Additionally as these examples are serial, we include independent timers around
nc_put_var_float()andnc_close(), to understand overhead made by quantizing, compressing (or deflating) and writing this data. The nuances of the timings are not yet fully understood, but hopefully they are enough to allow us to inform some decisions and approaches to further work.The test is run using a similar approach to the XIOS tests (although much simpler). The python unit test framework is again used, and an additional
.ymlworkflow file run_netcdf_test.yml has been added. The unit test discover mechanism has been used (again), so should help with the addition of further tests.