-
Notifications
You must be signed in to change notification settings - Fork 64
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Activity snapshots: Normalize queries for "filter_query_sample = normalize" #462
Conversation
memo: https://pganalyze.com/docs/collector/settings#pii-filtering-settings
|
…alize" For historic reasons, we were only normalizing pg_stat_activity query texts when "filter_query_sample" was set to "all", not "normalize", which has a similiar intent. Whilst we could also control the filtering of pg_stat_activity query texts with the "filter_query_text" setting (which today controls whether pg_stat_statements query texts are re-normalized), this is intentionally not done here. The main motivation to not do this for now are performance implications of running normalize every 10 seconds for many active queries, and we may adjust this in a future commit based on more benchmarks.
f321cf2
to
ecca5fc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for adjusting, I think it's good to do this way first and we can revisit if it's okay to change more widely in the future.
Minor note: it might be nice to have some comment for why we're passing "unparseable" to util.NormalizeQuery
. I don't think the behavior will be much different here if we pass it or server.Config.FilterQueryText
so I'm gonna approve anyways, but it would be nice if the reason is mentioned well. Maybe you're doing so so that we can control the behavior when the error happens during the normalization? (and make sure not to use the original query with the error case?)
Good point, that's a bit confusing - adding a comment. |
For historic reasons, we were only normalizing pg_stat_activity query texts when
filter_query_sample
was set toall
, notnormalize
, which has a similar intent.Whilst we could also control the filtering of pg_stat_activity query texts with the
filter_query_text
setting (which today controls whether pg_stat_statements query texts are re-normalized, and is enabled by default), this is intentionallynot done here. The main motivation to not do this for now are performance implications of running normalize every 10 seconds for many active queries, and we may adjust this in a future commit based on more benchmarks.