Contracts as a separate package

[sdebruyn]

The documentation has some basic starting points on the dbt artifacts and the information they contain. dbt users have started to work with these artifacts to perform various tasks.

You can replicate that structure in your own code, but a lot easier is to use the contracts package inside dbt-core and use mashumaro - maybe with some json helper code - to get that information back as instances of the contracts classes instead of plain dictionaries.

The disadvantage now might be that you have to install dbt-core into the project using this, update it when dbt-core updates, along with a lot of dependencies you might not need if you're just using the contracts. That's why I think the contracts should reside in their own package - published on PyPi - and be updated separately, only when there are actual changes in the contracts.

This would also formalize them being actual "contracts", offered as a public API and come with some stricter constraints in regards to versioning.

[jtcohen6]

@sdebruyn We're making some progress in this direction, and in a more language-agnostic way, by defining more of these internal objects via proto schemas — especially the ones that show up in metadata interfaces (JSON artifacts + structured logs). I don't think we have an answer yet as far as where / how we'd want to host those contracts, if somewhere outside of the dbt-core codebase.