-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix: Do not redact password for conn pool connection uri while connecting to DB #2203
base: main
Are you sure you want to change the base?
Conversation
c4b1b2b
to
2f05de5
Compare
2f05de5
to
a83acb2
Compare
@@ -243,8 +243,12 @@ func (yb *TargetYugabyteDB) InitConnPool() error { | |||
SessionInitScript: getYBSessionInitScript(yb.tconf), | |||
} | |||
yb.connPool = NewConnectionPool(params) | |||
redactedParams := params | |||
redactedParams.ConnUriList = utils.GetRedactedURLs(redactedParams.ConnUriList) | |||
redactedParams := ConnectionParams{ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We're logging the full struct below, so this will mean we'll have to come back to keep updating this copy here whenever we add a new field.
Let's go with redactedParams := *params
?
@@ -85,7 +85,7 @@ jobs: | |||
docker run -d --name yugabytedb \ | |||
-p7000:7000 -p9000:9000 -p15433:15433 -p5433:5433 -p9042:9042 \ | |||
yugabytedb/yugabyte:${{ matrix.version }} \ | |||
bin/yugabyted start --background=false --ui=false | |||
bin/yugabyted start --tserver_flags="ysql_hba_conf_csv={host all yugabyte all trust,host all all all md5}" --background=false --ui=false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shall we just have host all all all md5
? Just want to avoid "trust" altogether 😅
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I started with that only, but somehow yugabyted wasn't coming up with that conf as yugabyte is a default user maybe trust is required for that, I can take that up later after figuring out how to do that
yb-voyager/src/tgtdb/yugabytedb.go
Outdated
@@ -243,8 +243,7 @@ func (yb *TargetYugabyteDB) InitConnPool() error { | |||
SessionInitScript: getYBSessionInitScript(yb.tconf), | |||
} | |||
yb.connPool = NewConnectionPool(params) | |||
redactedParams := params | |||
redactedParams.ConnUriList = utils.GetRedactedURLs(redactedParams.ConnUriList) | |||
redactedParams := *params |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should not do this.
https://go.dev/play/p/NjUBR-pdyd7
This is typical example of shallow copy vs deep copy and can potentially lead to a bug in future again.
I think as a standard practice also, we should always do Deep copy unless there is a usecase for shallow or significant perf implications of deepcopy.
- Either copy each field separately.
- Make use of standard deepcopy libraries available (to address @makalaaneesh's concern)
https://github.com/mohae/deepcopy
https://github.com/barkimedes/go-deepcopy (this seems to be better maintained)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, good point!
+1 to use deep copy by default. (unless there are pointers like mutexes/connections/etc. but then again if the struct has such fields, we wouldn't necessarily want to deep copy)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, missed it thanks for bringing it up @sanyamsinghal .
I made this change but missed one more thing to do redaction of urls back for logging.
I am thinking to not blindly deep copy that to just for logging or for the case where we can also add new fields which don't need to be copied, we can add any new field to logged params as well if required.
I can create the params with redaction for all current fields and log, basically revert the last commit
Sounds good?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah you can copy each field manually also, just the overhead of maintaining the codepath for logging. But since the code is at one place so it will be fine.
btw do you see any issue with simply creating a deep copy and printing it? I believe this is a one time thing only? (i assume these deepcopy libraries will take care of nested pointer fields)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The issue with directly copying the params is that there will still be an overhead of redacting the URLs in ConnectionParams for logging, and this will still be present for any new fields we will add in the future to check if they need redaction.
redactedParams := &ConnectionParams{}
deepcopy.Copy(redactedParams, params)
redactedParams.ConnUriList = utils.GetRedactedURLs(redactedParams.ConnUriList)
So, I think it is better to have the overhead of adding the new field to the logged parameters if required, as it is a sensitive part where we have connection info with passwords.
This reverts commit b54c79a.
Describe the changes in this pull request
Fixing the bug where import data always fails to connect to DB via the connection pool with a password authentication as the connection URLs are getting redacted while passing to the connection pool.
#2204
redaction still happens-
Describe if there are any user-facing changes
Its a fix, no user-facing changes
How was this pull request tested?
automation tests are enough
Does your PR have changes that can cause upgrade issues?