Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#3882 - EDW API - DB User Part 1 #4211

Merged
merged 9 commits into from
Jan 14, 2025
Merged

#3882 - EDW API - DB User Part 1 #4211

merged 9 commits into from
Jan 14, 2025

Conversation

bidyashish
Copy link
Collaborator

@bidyashish bidyashish commented Jan 7, 2025

Technical

Need to create a read only DB user for the EDW API rather than using the admin user.

  • Create new read only DB user with access to all tables

Demo
image

HELM Upgrade

  1. make upgrade NAMESPACE=a6ef19-dev
    image

Crunchy read-only-user setup

Steps to Perform in Master Node of Postgres

  1. Wait around 20 mins to deploy helm chart completely after Helm upgrade.
  2. Run connect-[ENV]-db-superuser for each environment from ~/sources/makefile eg. make connect-dev-db-superuser MASTER_POD=pod_id
  3. For superuser credentials please look into openshift secrets. Secret name: simsdb-pguser-postgres Secret key names: user and password.
  4. Once connected to Database as superuser, run the following commands.
GRANT USAGE ON SCHEMA information_schema TO "read-only-user";
GRANT USAGE ON SCHEMA sims TO "read-only-user";
GRANT SELECT ON ALL TABLES IN SCHEMA sims TO "read-only-user";
ALTER DEFAULT PRIVILEGES IN SCHEMA sims GRANT SELECT ON TABLES TO "read-only-user";

Wiki Updated to https://github.com/bcgov/SIMS/wiki/DevOps-and-Running-the-Application#crunchy-backup-and-restore

Need to update User to
EDW Repo
https://github.com/bcgov/aest-api/blob/bdb09cefe7fbf761624109cac0d193072cc70de1/openshift/sims_dc.yaml#L101C29-L101C60

@bidyashish bidyashish self-assigned this Jan 7, 2025
@bidyashish bidyashish added Devops Devops Dev & Architecture Development and Architecture labels Jan 7, 2025
@bidyashish bidyashish marked this pull request as draft January 9, 2025 00:09
@bidyashish bidyashish force-pushed the feature/#3882-edw-db-user branch from 92788a3 to 8d5873a Compare January 9, 2025 00:12
@bidyashish bidyashish changed the title #3882 - EDW API - DB User #3882 - EDW API - DB User Part 1 Jan 9, 2025
@bidyashish bidyashish marked this pull request as ready for review January 9, 2025 21:57
@guru-aot
Copy link
Collaborator

Please apply the command upgrade 'make upgrade NAMESPACE={{dev-namespace}}', which is working and confirm if the commands above is enough to access the schema and its objects.

I think you may not need
GRANT CONNECT ON DATABASE simsdb TO "read-only-user";

Please confirm. Also provide proof of upgrade in dev as a screenshot.

Thanks

@bidyashish
Copy link
Collaborator Author

Please apply the command upgrade 'make upgrade NAMESPACE={{dev-namespace}}', which is working and confirm if the commands above is enough to access the schema and its objects.

I think you may not need GRANT CONNECT ON DATABASE simsdb TO "read-only-user";

Please confirm. Also provide proof of upgrade in dev as a screenshot.

Thanks

Thanks @guru-aot
Nice catch, CONNECT permission removed from Wiki and PR info, I tested it without CONNECT and it seems to work fine

HELM Upgrade screenshot added.

@bidyashish
Copy link
Collaborator Author

Note:
Permission GRANT USAGE ON SCHEMA information_schema TO "read-only-user"; is added in Wiki and PR.

Above permission is needed to view all table in SIMS schema for DB Beaver(not tested) and Datagrip(tested).

@dheepak-aot dheepak-aot self-requested a review January 10, 2025 16:38
sources/Makefile Outdated
@@ -262,3 +262,23 @@ connect-prod-db:
@echo "Use your prefered DB Manager (e.g. pgAdmin or DBeaver) to connect to the Postgres database using the local port 5757."
@oc project 0c27fb-prod
@oc port-forward services/simsdb-pgbouncer 5757:5432
connect-dev-db-superuser:
@echo "Forward local port 5454 to Openshift master pod default port."
@echo "Use your prefered DB Manager (e.g. pgAdmin or DBeaver) to connect to the Postgres database using the local port 5454."
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor spelling not right on prefered preferred

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @dheepak-aot ,
All words of prefered is changed to preferred .

Copy link

Backend Unit Tests Coverage Report

Totals Coverage
Statements: 22.49% ( 3850 / 17122 )
Methods: 10.37% ( 224 / 2161 )
Lines: 25.88% ( 3322 / 12837 )
Branches: 14.31% ( 304 / 2124 )

Copy link

E2E Workflow Workers Coverage Report

Totals Coverage
Statements: 65.59% ( 589 / 898 )
Methods: 59.63% ( 65 / 109 )
Lines: 68.72% ( 468 / 681 )
Branches: 51.85% ( 56 / 108 )

Copy link

E2E Queue Consumers Coverage Report

Totals Coverage
Statements: 86.14% ( 1249 / 1450 )
Methods: 82.42% ( 136 / 165 )
Lines: 88.51% ( 1032 / 1166 )
Branches: 68.07% ( 81 / 119 )

Copy link
Collaborator

@dheepak-aot dheepak-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing the changes @bidyashish . Looks good 👍

Please update the wiki(if not already updated) with the SQL scripts for read-only-user.

And also add release instructions to execute the SQL pertaining to read-only-user https://app.zenhub.com/workspaces/student-information-management-system-5fce9df5aa1b45000e937014/issues/zh/276

Copy link

E2E SIMS API Coverage Report

Totals Coverage
Statements: 67.77% ( 5992 / 8842 )
Methods: 65.54% ( 738 / 1126 )
Lines: 71.62% ( 4695 / 6555 )
Branches: 48.15% ( 559 / 1161 )

sources/Makefile Outdated Show resolved Hide resolved
Copy link
Collaborator

@guru-aot guru-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice work @bidyashish

Copy link
Collaborator

@andrewsignori-aot andrewsignori-aot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making the changes, looks good 👍

@bidyashish bidyashish added this pull request to the merge queue Jan 14, 2025
Merged via the queue into main with commit 2380a68 Jan 14, 2025
8 of 14 checks passed
@bidyashish bidyashish deleted the feature/#3882-edw-db-user branch January 14, 2025 17:04
@ninosamson ninosamson added Business Items under Business Consideration and removed Dev & Architecture Development and Architecture labels Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Business Items under Business Consideration Devops Devops
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants