Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Drop missing from juliaeltype if ValidityBitmap is all valid. #477

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

evetion
Copy link

@evetion evetion commented Jul 27, 2023

Fixes #384
Fixes #373

This does fail a number of tests, which we can fix after we agree on the direction of this PR.

@evetion evetion changed the title Drop missing from juliaeltype if ValidityBitmap is all valid. Drop missing from juliaeltype if ValidityBitmap is all valid. Jul 27, 2023
@evetion
Copy link
Author

evetion commented Jul 27, 2023

Fixes #283

@codecov-commenter
Copy link

codecov-commenter commented Aug 4, 2023

Codecov Report

Merging #477 (8cb11a7) into main (f8d2203) will decrease coverage by 82.39%.
Report is 3 commits behind head on main.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main    #477       +/-   ##
==========================================
- Coverage   87.50%   5.11%   -82.39%     
==========================================
  Files          26      25        -1     
  Lines        3280    3208       -72     
==========================================
- Hits         2870     164     -2706     
- Misses        410    3044     +2634     
Files Changed Coverage Δ
src/arraytypes/arraytypes.jl 0.00% <0.00%> (-90.00%) ⬇️
src/eltypes.jl 0.00% <0.00%> (-85.77%) ⬇️
src/table.jl 0.00% <0.00%> (-92.50%) ⬇️

... and 22 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@@ -200,6 +200,7 @@ end

Base.size(p::ValidityBitmap) = (p.ℓ,)
nullcount(x::ValidityBitmap) = x.nc
Base.all(x::ValidityBitmap) = x.nc == 0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't love overloading all here; can we just call this allvalid? I don't think we need to overload anything for this.

@@ -540,13 +555,13 @@ function UnionT(f::Meta.Field, convert)
UT = UnionT{
f.type.mode,
typeids,
Tuple{(juliaeltype(x, buildmetadata(x), convert) for x in f.children)...},
Tuple{(juliaeltype(x, buildmetadata(x), convert, false) for x in f.children)...},
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me understand why we hard-code false in a number of places for the allvalid argument? vs. in other places calling all(validity)?

Copy link
Member

@quinnj quinnj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'm fine going this direction. I'd like to think a bit on if there may be unintended consequences here, but it seems like it should be strictly an improvement.

@evetion
Copy link
Author

evetion commented Aug 5, 2023

Ok, I will fix the tests 😃

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Nullable fields don't always need Union{Missing, T} Addtional Missing gets injected into Schema
3 participants