Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Join bug - wrong results from exploiting DValue equality. #2794

Closed
MikeStall opened this issue Dec 29, 2024 · 0 comments
Closed

Join bug - wrong results from exploiting DValue equality. #2794

MikeStall opened this issue Dec 29, 2024 · 0 comments
Assignees

Comments

@MikeStall
Copy link
Contributor

Join implementation is using DValue/RecordValue as a dictionary key. See #2788

This can be exploited to get wrong results:
The fix here is we should never use DValue as a dictionary key. (consider using indices instead?)

>> With({T1: [10,20]}, Join(T1, T1, LeftRecord.Value = 10 &&  RightRecord.Value = 20, JoinType.Left, RightRecord.Value As R2))

  R2   Value
 ==== =======
  20   10

The correct results here would have a row for each row in Left table. (since it's a JoinType.Left), so there should be {Value:20, R2 : blank }

Using DValue as a key is really just object reference equality. Since DValue doesn't define .Equals (and it shouldn't - since we certainly don't have .equals on RecordValue either). But the join impl assumes all the dvalues are unique, which is not necessarily true.

Note that this is heavily mitigated since most builtin functions use InMemoryTable, and that always allocate new DValue. See here:

return records.Select(record => record.IsValue ? DValue<RecordValue>.Of(CompileTimeTypeWrapperRecordValue.AdjustType(((TableType)irContext.ResultType).ToRecord(), record.Value)) : record);

But we can still workaround that (see example above) - TBD whether we could come up with more mainstream examples.
And Future optimizations could significantly widen this window. Or we could also hit this with other impls of TableValue.

@MikeStall MikeStall changed the title Join bug - exploiting DValue equality. Join bug - wrong results from exploiting DValue equality. Dec 29, 2024
@anderson-joyle anderson-joyle self-assigned this Dec 30, 2024
anderson-joyle added a commit that referenced this issue Jan 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants