Add data sources protobuf

[JAORMX]

Summary

This adds the protobuf definition of data sources. We'll re-use this
protobuf around Minder to create the CRUD interface and reason about the
rest of the implementation.

This implements https://docs.google.com/document/d/1x8ZXs81e2vB16_vnunV4eRMOklH4Dv9vP30zKYSIKMI/edit?usp=sharing

Change Type

Mark the type of change your PR introduces:

• Bug fix (resolves an issue without affecting existing features)
• Feature (adds new functionality without breaking changes)
• Breaking change (may impact existing functionalities or require documentation updates)
• Documentation (updates or additions to documentation)
• Refactoring or test improvements (no bug fixes or new functionality)

Testing

Outline how the changes were tested, including steps to reproduce and any relevant configurations.
Attach screenshots if helpful.

Review Checklist:

• Reviewed my own code for quality and clarity.
• Added comments to complex or tricky code sections.
• Updated any affected documentation.
• Included tests that validate the fix or feature.
• Checked that related changes are merged.

[coveralls]

coverage: 54.758% (-0.007%) from 54.765%
when pulling 03ad165 on JAORMX:datasource-pb
into 84d517a on mindersec:main.

[evankanderson]

This is looking good! Since this will be a public API, I gave more-detailed feedback than I might otherwise.

[evankanderson]

Do you want to use in instead of a pattern here?

[evankanderson]

Also, do we need this field now, or can we back-fill this later?

[evankanderson]

(The same could be true for version, but I'm willing to leave that one in.)

[JAORMX]

We don't trully need it now, but it does make it work with our parsing primitives.

[evankanderson]

You mean our YAML-to-foo apply methods?

Hmm -- I'd almost prefer that those dropped the type field after parsing, particularly if type must be set by all clients.

For rule types, Charlie Egan pointed me to Rego metadata yesterday, which feels like it might enhance the rego rule authoring experience by allowing you to use e.g. Rego mode in VSCode.

[evankanderson]

I assume that names are case-insensitively unique within a project?

[evankanderson]

How do data source names resolve in descendant projects?

[JAORMX]

They are case insensitive within a project. I'm thinking they'd have the same mechanics as profiles. That is, they apply and trickle down to subprojects, but are not modifiable by child projects. That allows for child projects to leverage data sources in profiles targetted at those child projects.

I suspect data sources will be more nuanced that this, so we may want to make this configurable in the future.

[evankanderson]

Not sure if you want to leave a note that these are case-insensitive and must be unique in a project and all parents

[JAORMX]

You are right, I'll add a comment here.

[evankanderson]

Do you want to do a message-level CEL expression to allow body only for POST/PUT/PATCH requests?

[evankanderson]

(Later is a fine answer)

[JAORMX]

How does one do that?

[evankanderson]

https://github.com/bufbuild/protovalidate/blob/main/docs/custom-constraints.md#message-level-constraints

Note, I don't consider this required, but I'm looking to seed the knowledge of this capability.

[JAORMX]

That sounds handy, though I'm still not quite sure how to use it. Mind adding this in a subsequent PR?

[blkt]

question: is embedding these structs a good idea? I remember @evankanderson having concerns on this approach.

[JAORMX]

We can stop embedding these. But it's highly specific to the Rest driver, and would end up looking like RestFallback

[blkt]

It's ok for me, just raising awareness.

[evankanderson]

Embedding the structs has a few consequences on the naming:

1. In go, the struct will be called RestDataSource_Fallback, which is slighly long. We have some worse offenders that take up about half a line...
2. If we end up with other places where a fallback should be used, we end up needing to duplicate the message, its validation, etc. I think there are a couple places where we ended up making second messages because the first was embedded in an unrelated message.

I don't think it's a major problems, so I didn't bother to raise it here.

Embedding enums inside message definitions or nested message definitions also tends to lead to unreadably-long names in Go.

[evankanderson]

LGTM, a couple comments, but nothing that needs to block.

I would prefer that the server be able to ignore the type field rather than requiring all clients to set it to a static value, but I don't feel super strongly.

[evankanderson]

Not sure if you want to leave a note that these are case-insensitive and must be unique in a project and all parents

[evankanderson]

https://github.com/bufbuild/protovalidate/blob/main/docs/custom-constraints.md#message-level-constraints

Note, I don't consider this required, but I'm looking to seed the knowledge of this capability.

[evankanderson]

Embedding the structs has a few consequences on the naming:

1. In go, the struct will be called RestDataSource_Fallback, which is slighly long. We have some worse offenders that take up about half a line...
2. If we end up with other places where a fallback should be used, we end up needing to duplicate the message, its validation, etc. I think there are a couple places where we ended up making second messages because the first was embedded in an unrelated message.

I don't think it's a major problems, so I didn't bother to raise it here.

Embedding enums inside message definitions or nested message definitions also tends to lead to unreadably-long names in Go.

[evankanderson]

You mean our YAML-to-foo apply methods?

Hmm -- I'd almost prefer that those dropped the type field after parsing, particularly if type must be set by all clients.

For rule types, Charlie Egan pointed me to Rego metadata yesterday, which feels like it might enhance the rego rule authoring experience by allowing you to use e.g. Rego mode in VSCode.