Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep the original dtype of the data when stretching #150

Merged
merged 7 commits into from
Nov 20, 2023

Conversation

pnuu
Copy link
Member

@pnuu pnuu commented Nov 15, 2023

The stretching forces the dtype to float64. This PR makes sure the original dtype is preserved.

  • Closes #xxxx (remove if there is no corresponding issue, which should only be the case for minor changes)
  • Tests added (for all bug fixes or enhancements)
  • Tests passed (for all non-documentation changes)
  • Passes git diff origin/master **/*py | flake8 --diff (remove if you did not edit any Python files)
  • Fully documented (remove if this change should not be visible to users, e.g., if it is an internal clean-up, or if this is part of a larger project that will be documented later)

Copy link
Member

@mraspaud mraspaud left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! thanks for tackling this issue!

@pnuu
Copy link
Member Author

pnuu commented Nov 16, 2023

This latest change to the tests made me think: do we allow integer data? If yes, should I check for that and then use float32 (or float64?) dtype for scale factor?

Copy link

codecov bot commented Nov 16, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (6617ff7) 91.23% compared to head (afc693a) 91.28%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #150      +/-   ##
==========================================
+ Coverage   91.23%   91.28%   +0.05%     
==========================================
  Files          11       11              
  Lines        3867     3890      +23     
==========================================
+ Hits         3528     3551      +23     
  Misses        339      339              
Flag Coverage Δ
unittests 91.28% <100.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@coveralls
Copy link

coveralls commented Nov 16, 2023

Coverage Status

coverage: 91.529% (+0.05%) from 91.479%
when pulling afc693a on pnuu:keep-stretch-dtype
into 6617ff7 on pytroll:main.

@djhoese
Copy link
Member

djhoese commented Nov 16, 2023

Integer data in any part of Satpy? Or a specific part? I expect all category products to be integers. And palletized images are also integers I think.

@pnuu
Copy link
Member Author

pnuu commented Nov 16, 2023

Let me rephrase: are any integer data expected to be passed to trollimage.XRImage.stretch()?

@gerritholl
Copy link
Member

I think integer data can be stretched. For example, suppose I have data that are discrete (categorical or not) and range from 0 to 10, that I want to write as a greyscale image in which the shades of grey can be seen. Then I'd want to stretch it to cover 0 to 255. In that case palettize is probably more suitable, but I don't think stretch is necessarily invalid.

@pnuu
Copy link
Member Author

pnuu commented Nov 16, 2023

Thanks @gerritholl I'll see if @djhoese or @mraspaud have thoughs on this before starting anything.

@pnuu pnuu self-assigned this Nov 16, 2023
@djhoese
Copy link
Member

djhoese commented Nov 16, 2023

Restarting failed readthedocs job as it appeared to be a download SSL certificate error.

Copy link
Member

@djhoese djhoese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as-is looks fine to me. I am concerned about the integer types that you mentioned.

I think we always want the result of stretching to be floating point though as trollimage does a 2 step process of scaling to the image format (data array -> normalized -> scale to file dtype). So maybe it is just easier to set these to 32-bit floats? Or do a check if the input array is a floating type and if not default to a 32-bit float for the scale factor.

arr = np.arange(75).reshape(5, 5, 3).astype(float)
data = xr.DataArray(arr.copy(), dims=['y', 'x', 'bands'],
coords={'bands': ['R', 'G', 'B']})
img = xrimage.XRImage(data)
img.crude_stretch(0, 74)
assert img.data.dtype == float
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how important of a check this is unless it is switched to 32-bit float, but 🤷‍♂️ not a big deal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is there just to make sure everything is not changed to float32 ("keep the original dtype").

@pnuu pnuu merged commit 9564cba into pytroll:main Nov 20, 2023
20 checks passed
@pnuu pnuu deleted the keep-stretch-dtype branch November 20, 2023 06:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants