Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add useDeterministicOrdering to JSONEncodingOptions #1478

Merged
merged 11 commits into from
Oct 26, 2023

Conversation

rebello95
Copy link
Contributor

@rebello95 rebello95 commented Oct 13, 2023

Adds an option to ensure that JSON serialization is deterministic when serializing Protobuf map fields. This should be the only type that needs to be sorted, since individual fields are serialized in order by the generated code that invokes try visitor.visit(...) for each field.

In order to avoid the overhead of sorting for the default case, there are no changes to the existing serialization strategy when this setting is disabled.

This new option is similar to what Protobuf provides in Go, and the docstring added in this PR matches that.

Resolves #1477

Adds an option to ensure that JSON serialization is deterministic when serializing Protobuf `map` fields. This should be the only type that needs to be sorted, since individual fields are serialized in order by the generated code that invokes `try visitor.visit(...)` for each field.

Resolves apple#1477
@rebello95
Copy link
Contributor Author

@thomasvl would you mind taking a look at this? 🙏🏽

Implements the same setting added for JSON in apple#1478 for binary serialization.

Related to apple#1477.
rebello95 added a commit to rebello95/swift-protobuf that referenced this pull request Oct 19, 2023
Implements the same setting added for JSON in apple#1478 for binary serialization.

Related to apple#1477.
@thomasvl
Copy link
Collaborator

@thomasvl would you mind taking a look at this? 🙏🏽

Sorry, been OOO for a bit, seems like these PR and issue got some discussion. Will catch up and hopefully some of the Apple folks will take a look.

If these PRs do land, there's also the question of it they can/should go back to the 1.x branch/release or just sit awaiting a 2.0 release.

@rebello95
Copy link
Contributor Author

Sorry, been OOO for a bit, seems like these PR and issue got some discussion. Will catch up and hopefully some of the Apple folks will take a look.

Thanks @thomasvl.

If these PRs do land, there's also the question of it they can/should go back to the 1.x branch/release or just sit awaiting a 2.0 release.

These are all backward-compatible and opt-in, so I think they should be okay to include in 1.x (not sure what the release plan for 2.0 is).

@thomasvl thomasvl requested a review from tbkka October 25, 2023 15:09
@thomasvl
Copy link
Collaborator

If these PRs do land, there's also the question of it they can/should go back to the 1.x branch/release or just sit awaiting a 2.0 release.

These are all backward-compatible and opt-in, so I think they should be okay to include in 1.x (not sure what the release plan for 2.0 is).

Since it is a new Option, it is sorta an ABI change, but I think the past stance has been these sorta things are ok. That's where Apple (as repo owner) gets the final call.

@FranzBusch
Copy link
Member

Since it is a new Option, it is sorta an ABI change, but I think the past stance has been these sorta things are ok. That's where Apple (as repo owner) gets the final call.

IMO this is fine to add from just a pure API/ABI compatibility perspective. We are just adding a new field to a non-frozen struct.

On the overall feature, I am +1. If other languages are providing a similar option I don't see why we shouldn't and the documentation calls out that this is unstable and break over versions.

@rebello95 rebello95 requested a review from thomasvl October 26, 2023 13:38
Copy link
Collaborator

@thomasvl thomasvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seem good to me, but I'd like one of the Apple folks to also sign off.

@thomasvl thomasvl requested review from Lukasa and FranzBusch October 26, 2023 15:30
@tbkka
Copy link
Collaborator

tbkka commented Oct 26, 2023

For the record, I'm happy with the intent here.

Adding a new field to a non-frozen struct is forward-compatible API and ABI. (Old client source/binaries work correctly with new library.) So back-porting to 1.x should be fine.

I'll do another skim over the source to see if I have any last-minute nits...

@thomasvl thomasvl merged commit 5c44631 into apple:main Oct 26, 2023
9 checks passed
@thomasvl
Copy link
Collaborator

Thank you!

thomasvl pushed a commit that referenced this pull request Oct 26, 2023
…1480)

* Add `BinaryEncodingOptions` & option for deterministic ordering

Implements the same setting added for JSON in #1478 for binary serialization.

Related to #1477.
@rebello95 rebello95 deleted the deterministic-map-json branch October 26, 2023 16:24
@rebello95
Copy link
Contributor Author

Thank you both!

rebello95 added a commit to rebello95/swift-protobuf that referenced this pull request Oct 26, 2023
* Add `useDeterministicOrdering` to `JSONEncodingOptions`

Adds an option to ensure that JSON serialization is deterministic when serializing Protobuf `map` fields. This should be the only type that needs to be sorted, since individual fields are serialized in order by the generated code that invokes `try visitor.visit(...)` for each field.

Fixes apple#1477
rebello95 added a commit to rebello95/swift-protobuf that referenced this pull request Oct 26, 2023
…pple#1480)

* Add `BinaryEncodingOptions` & option for deterministic ordering

Implements the same setting added for JSON in apple#1478 for binary serialization.

Related to apple#1477.
rebello95 added a commit to connectrpc/connect-swift that referenced this pull request Nov 13, 2023
## Summary

Implements support for Connect unary GET requests per
https://connectrpc.com/docs/protocol#unary-get-request.

By default, this behavior is disabled and can be enabled through an
option on `ProtocolClientConfig` (as demonstrated in the Eliza app).
Behavior is exercised in a new conformance test, but I do intend to add
more unit tests for this after #208.

## Linked issues

- Requires apple/swift-protobuf#1487
- Related to apple/swift-protobuf#1478
- Related to apple/swift-protobuf#1480
- Resolves #196
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Deterministic JSON serialization for map types
4 participants