[ADAP-426] [Feature] Add Flag to the Profile to Redact User Data from Logs

[HRusby]

Is this your first time submitting a feature request?

• I have read the expectations for open source contributors
• I have searched the existing issues, and I could not find an existing issue for this feature
• I am requesting a straightforward extension of existing dbt-snowflake functionality, rather than a Big Idea better suited to a discussion

Describe the feature

In PR #378 a change was made to redact data from the dbt logs on a failed model e.g. due to duplicate data during a dbt run. While this behavior is useful, it ought to be configurable such that developers can disable the redaction whilst investigating a failed model. The end user can also then decide whether data redaction is suitable for them or whether it grants more benefit to show data in the logs.

I would suggest that if the new config option isn't present then data ought to be redacted by default, it should be an explicit choice from the user to not redact data.

Describe alternatives you've considered

No response

Who will this benefit?

Developers investigating failing models
Users that do not require data redaction

Are you interested in contributing this feature?

Yes, I've begun to take a look at the code, though I am currently struggling to get the functional tests to run on a fresh checkout I suspect due to authorisation with my Snowflake Account.

Anything else?

Relevant section of code is at https://github.com/dbt-labs/dbt-snowflake/blob/main/dbt/adapters/snowflake/connections.py#L263

[Fleid]

I'm not against it, as long as it's an optional flag that default to false.
Thanks for looking into it!

[dbeatty10]

@Fleid currently the log redaction is a Snowflake-only feature. Should the redaction feature + the proposed optional config be promoted to dbt-core?

[Fleid]

Good point @dbeatty10. We should have a clear heuristics to make those decisions...

Anyway, here it looks like it's a connections.py thing, which makes it a local problem? So the question may be less about making it a core level ask, and more about replicating it across other adapters?