CASSGO-36 could map scan get nil instead of zero value for null value

[OleksiienkoMykyta]

Closes #1699

This PR adds a new feature (MapScanWithNullableValues) that allows users to receive nullable values from Cassandra.

[jameshartig]

Based on this being possible outside of the package I'm hesitant to approve this. See my comment on the original issue: #1699 (comment)

[OleksiienkoMykyta]

@jameshartig I believe we should consider implementing features directly within gocql rather than requiring external solutions. It would be much more convenient to have these features available out of the box. Furthermore, it seems like this behavior should be the default since Cassandra can return nullable values. Currently, gocql represents these as default Go values, which alters the original data. To address this, we should at least provide an alternative functionality that preserves the original data and aligns with Cassandra's behavior.

[joao-r-reis]

So it looks like @jameshartig is proposing a change to the behavior of MapScan so it fixes this issue without adding a new separate method for it if I'm understanding correctly?

With #1856 we wouldn't need this PR (CASSGO-36) right?

[joao-r-reis]

Or is #1856 just limited to byte slices and we still can't read nullable values for other types?

[jameshartig]

Or is #1856 just limited to byte slices and we still can't read nullable values for other types?

Yes, that's just for byte slices. The reason being that nil slices are equivalent to empty slices and there's no need to make an empty slice if we don't need to.

Furthermore, it seems like this behavior should be the default since Cassandra can return nullable values.

This is maybe something we can consider for 2.0 but there's no way we could make that change in the existing version.

Currently, gocql represents these as default Go values, which alters the original data.

You can use pointers if you which to get the "original data". I've seen places that utilize the default Go values as a feature because they don't have to check for nil and can rely on the default value. I also provided a code example in the original issue with how you can do this quite easily without any changes to gocql.

[joao-r-reis]

This is maybe something we can consider for 2.0 but there's no way we could make that change in the existing version.

The release that we're working on atm (main branch) is 2.0 but it might be too intrusive of a change, I'd prefer adding a method with this capability and deprecating the current one (or not deprecating at all and just having both but that might not make much sense I'm not sure)

[joao-r-reis]

So it looks like there's no consensus on whether CASSGO-36 should actually be considered, maybe this needs a [discuss] thread on the C* dev mailing list?

[OleksiienkoMykyta]

It seems like misbehavior that MapScan doesn't get the original data from the database. Sure, we can use Scan and make a custom "MapScan" as suggested in #1699 (comment), but it looks like overhead, and as mentioned in the issue, it's confusing, but it is still confusing, especially for new users, and imo MapScan doc is not clear enough when we're talking about null values. I think there is no issue with backward compatibility because this functionality will be added to 2.0.
Also, I'm a little confused about the argument that this could be implemented on the user side, for example, SliceMap, can be easily implemented by the user, but we still provide such convenient functionality.

[jameshartig]

It seems like misbehavior that MapScan doesn't get the original data from the database.

I agree with @joao-r-reis, I think we need more consensus that "original data" is not the default go values and is actually nil. Again, I've seen cases where one or the other are preferred. I don't think we can make this large of a change based on 1-2 cases. I don't even know how many devs are using MapScan outside of exoteric situations.

imo MapScan doc is not clear enough when we're talking about null values.

We can definitely clarify the docs.

I think there is no issue with backward compatibility because this functionality will be added to 2.0.

You're changing a public interface which means anything implementing TypeInfo will need to be updated to support the new behavior.

Also, I'm a little confused about the argument that this could be implemented on the user side, for example, SliceMap, can be easily implemented by the user, but we still provide such convenient functionality.

I can appreciate your frustration but we want to decrease the surface area of the API and not increase it. We're stuck with SliceMap now but I agree it should have been more scrutinized.

[joao-r-reis]

I think I misunderstood this issue because reading this comment #1785 (comment) made me believe that we actually have no way of differentiating between a NULL and empty value from the database. It looks like there is indeed a way of doing this even if it requires a bit of extra code, correct? If this is the case then I'm starting to lean with @jameshartig here and CASSGO-36 (+ this PR) should probably just be closed