Skip to content

Commit

Permalink
Merge pull request #1000 from bcgov/test
Browse files Browse the repository at this point in the history
Create Latest Release
  • Loading branch information
ikethecoder authored Mar 1, 2024
2 parents faec87e + 859778b commit 3147351
Show file tree
Hide file tree
Showing 108 changed files with 5,133 additions and 4,097 deletions.
8 changes: 5 additions & 3 deletions .env.local
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
LOG_LEVEL=info
DISABLE_LOGGING='true'
AUTH_STRATEGY=Oauth2Proxy
KNEX_HOST=kong-db
KNEX_PORT=5432
Expand All @@ -9,18 +11,18 @@ MONGO_URL=mongodb://mongodb:27017/keystonedb4
MONGO_USER=
MONGO_PASSWORD=
KONG_URL=http://kong.localtest.me:8001
JWKS_URL=http://keycloak.localtest.me:9080/auth/realms/master/protocol/openid-connect/certs
JWKS_URL=http://keycloak.localtest.me:9081/auth/realms/master/protocol/openid-connect/certs
FEEDER_URL=http://feeder.localtest.me:6000
NEXT_PUBLIC_API_ROOT=http://oauth2proxy.localtest.me:4180
GWA_API_URL=http://gwa-api.localtest.me:2000
GWA_PROD_ENV_SLUG=E0000000
GWA_RES_SVR_CLIENT_ID=gwa-api
GWA_RES_SVR_CLIENT_SECRET=18900468-3db1-43f7-a8af-e75f079eb742
KEYCLOAK_AUTH_URL=http://keycloak.localtest.me:9080/auth
KEYCLOAK_AUTH_URL=http://keycloak.localtest.me:9081/auth
KEYCLOAK_REALM=master
EMAIL_ENABLED=false
EXTERNAL_URL=http://oauth2proxy.localtest.me:4180
OIDC_ISSUER=http://keycloak.localtest.me:9080/auth/realms/master
OIDC_ISSUER=http://keycloak.localtest.me:9081/auth/realms/master
LOCAL_ENV=true
WORKING_PATH=/tmp
DESTINATION_URL=
Expand Down
58 changes: 29 additions & 29 deletions .github/workflows/aps-cypress-e2e.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
name: Build and Deploy Cypress and Execute Tests
name: Cypress and Execute Tests

on:
workflow_dispatch: {}
push:
branches: ['test', 'cypress*']

Expand All @@ -27,36 +28,35 @@ jobs:
- name: Checkout Portal
uses: actions/checkout@v2

# - name: Determine Download file name
# id: set_variable
# run: |
# echo ${{ runner.arch }}
# if [ "${{ runner.arch }}" == "X64" ]; then
# echo "::set-output name=my_variable::gwa_Linux_x86_64.tgz"
# elif [ "${{ runner.arch }}" == "ARM64" ]; then
# echo "::set-output name=my_variable::gwa_Linux_arm64.tgz"
# else
# echo "unsupported architecture"
# fi

- name: Determine Download file name
id: set_variable
run: |
echo ${{ runner.arch }}
if [ "${{ runner.arch }}" == "X64" ]; then
echo "::set-output name=my_variable::gwa_Linux_x86_64.tgz"
elif [ "${{ runner.arch }}" == "ARM64" ]; then
echo "::set-output name=my_variable::gwa_Linux_arm64.tgz"
else
echo "unsupported architecture"
fi
- name: Download Binar
uses: robinraju/[email protected]
with:
repository: "bcgov/gwa-cli"
latest: true
fileName: ${{ steps.set_variable.outputs.my_variable }}
out-file-path: "${{ github.workspace }}/e2e"
# - name: Download Binary
# uses: robinraju/[email protected]
# with:
# repository: "bcgov/gwa-cli"
# latest: true
# fileName: ${{ steps.set_variable.outputs.my_variable }}
# out-file-path: "${{ github.workspace }}/e2e"

# - name: Unzip file
# run: |
# cd ${{ github.workspace }}/e2e
# tar xvzf ${{ steps.set_variable.outputs.my_variable }}

- name: Unzip file
run: |
cd ${{ github.workspace }}/e2e
tar xvzf ${{ steps.set_variable.outputs.my_variable }}
- name: Build Docker Images
run: |
docker-compose build
docker compose --profile testsuite build
- name: Spin up API Services Portal and Run E2E Tests
run: |
export CY_EXECUTION_ENV=${{ env.EXECUTION_ENV }}
Expand All @@ -68,14 +68,14 @@ jobs:
export CY_COMMIT_MESSAGE="${{ env.GIT_COMMIT_MESSAGE }}"
export CY_REPO_URL=${{ env.GIT_REPO_URL }}
export CY_COMMIT_AUTHOR_EMAIL=$(git --no-pager show -s --format='%ae' ${{ env.GIT_COMMIT_SHA }})
docker-compose up -d
docker compose --profile testsuite up -d
- name: Execute Tests & Clean Up
run: |
while true; do
if [ "$(docker ps -aq -f status=exited -f name=cypress-e2e)" ]; then
# cleanup
docker-compose down
docker compose down
break
else
echo "Waiting for Cypress to Complete E2E Tests....."
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci-build-deploy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -264,9 +264,9 @@ jobs:
NEXT_PUBLIC_HELP_API_DOCS_URL:
value: '/ds/api/v2/console/'
NEXT_PUBLIC_HELP_SUPPORT_URL:
value: 'https://bcgov.github.io/aps-infra-platform/'
value: 'https://mvp.developer.gov.bc.ca/docs/default/component/aps-infra-platform-docs/'
NEXT_PUBLIC_HELP_RELEASE_URL:
value: 'https://bcgov.github.io/aps-infra-platform/releases/'
value: 'https://mvp.developer.gov.bc.ca/docs/default/component/aps-infra-platform-docs/reference/releases/'
NEXT_PUBLIC_HELP_STATUS_URL:
value: 'https://uptime.com/s/bcgov-dss'
NEXT_PUBLIC_DEVELOPER_IDS:
Expand Down
28 changes: 28 additions & 0 deletions .github/workflows/ci-feat-url.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
name: Add URL to Feature PRs

on:
pull_request:
branches:
- dev

jobs:
comment:
runs-on: ubuntu-latest
steps:
- name: Set KEBAB_CASE_BRANCH
run: |
# Convert github.head_ref to kebab case
kebab_case=$(echo "${{ github.head_ref }}" | sed 's/_/-/g; s/\//-/g')
echo "::set-output name=KEBAB_CASE_BRANCH::${kebab_case}"
id: set-branch-id

- name: Check the KEBAB_CASE_BRANCH output
run: echo "The KEBAB_CASE_BRANCH is ${{ steps.set-branch-id.outputs.KEBAB_CASE_BRANCH }}"

- name: PR Description
if: startsWith(github.head_ref, 'feature/') == true
uses: bcgov-nr/[email protected]
with:
add_markdown: |
---
🚀 Feature branch deployment: https://api-services-portal-${{ steps.set-branch-id.outputs.KEBAB_CASE_BRANCH }}.apps.silver.devops.gov.bc.ca
114 changes: 67 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,73 +6,92 @@
![GitHub](https://img.shields.io/github/license/bcgov/aps-portal?style=for-the-badge)
![GitHub tag (latest by date)](https://img.shields.io/github/v/tag/bcgov/aps-portal?label=release&style=for-the-badge)


## Introduction


The `API Services Portal` is a frontend for API Providers to manage the lifecycle of their APIs and for Developers to discover and access these APIs. It works in combination with the Kong Community Edition Gateway and Keycloak IAM solution.

## Running the Project.

### Installation
## Local Deployment

#### 1. Manual

To run this project first run `npm install`.
The repo is setup to create a local deployment of the Portal along with required support services (Postgres, Keycloak, OAuth2-proxy, Feeder and Kong Gateway) using `docker compose`.

This application requires to have an Authentication proxy in front of it. Go to [oauth2-proxy](oauth2-proxy) for instructions on starting the proxy locally.
1. Clone and build the [Gateway Admin API](https://github.com/bcgov/gwa-api) (gwa-api)

You can then run `npm run dev` to start the application on port 3000. The proxy runs on port 4180.
```
git clone https://github.com/bcgov/gwa-api
cd ./microservices/gatewayApi
docker build -t gwa-api:e2e .
```
```
hostip=$(ifconfig en0 | awk '$1 == "inet" {print $2}')
1. Build: Back in `api-services-portal`, run `docker compose --profile testsuite build`.
1. Run: `docker compose up`. Wait for startup to complete - look for `Swagger UI registered`.
1. The Portal is now live at http://oauth2proxy.localtest.me:4180
1. To login, use username `local` and password `local`, or username `janis@idir` and password `awsummer`.
1. If you have made any changes to the app code, update images by running `docker compose build` then `docker compose up`.
1. Clean up: `docker compose down` removes all the hosted services
### Cypress testing
export AUTH_STRATEGY=Oauth2Proxy
export ADAPTER=knex
export KNEX_HOST=$hostip
export KNEX_DATABASE=keystonejs
export KNEX_USER=""
export KNEX_PASSWORD=""
export MONGO_URL=mongodb://$hostip:17017/keystonedb3
export MONGO_USER=""
export MONGO_PASSWORD=""
To run the Cypress test automation suite, run `docker compose --profile testsuite up`.
export FEEDER_URL=http://localhost:6000
### gwa CLI configuration
export KONG_URL=""
export OIDC_ISSUER=""
export JWKS_URL=${OIDC_ISSUER}/protocol/openid-connect/certs
To use the `gwa` command line interace, configure it with:
export NEXT_PUBLIC_API_ROOT=http://localhost:4180
export SSR_API_ROOT=http://localhost:4180
export EXTERNAL_URL="http://localhost:4180"
```
gwa config set host oauth2proxy.localtest.me:4180
gwa config set scheme http
```
export GWA_API_URL=http://localhost:2000
Run this command to test logging in and creating a namespace:
npm run dev
```
gwa login
gwa namespace create --name gw-12345
```
### Keycloak configuration
Once running, the `api services portal` application is reachable via `localhost:4180`.
Keycloak is initialized with `master` realm. The realm configuration is saved in `local/keycloak/master-realm.json`. It also creates a realm user `local` with admin privileges.
#### 2. Docker
### Development
##### Steps
Use the following configuration to run the Portal locally (outside of Docker) against the support components deployed with `docker compose`. Changes to the Portal code will live update instead of requiring `docker build`.
1. Create a `.env` from `.env.local` file
2. Create a `.env` from `.env.local` file under `feeds` directory
3. Remove cypress from docker-compose file (L106-129 & L217-229)
4. Run build steps [here](https://github.com/bcgov/api-services-portal/tree/dev/e2e#build-gateway-api-image)
5. Run `docker-compose build`
5. Run `docker-compose up` to spin up a local development environment with services (Postgres, Keycloak, OAuth2-proxy, APS-Portal, Feeder and Kong Gateway)
6. Go to: http://oauth2proxy.localtest.me:4180
7. To login, use username `local` and password `local`, or username `janis@idir` and password `awsummer`
8. `docker-compose down` : Removes all the hosted services
1. Follow [local deployment instructions](#local-deployment) and run `docker compose up`.
1. In `/src` run `npm install`.
1. If using Node version > 17, run `npm install --legacy-peer-deps`
##### Note:
1. Turn off the docker compose Portal: `docker stop apsportal`
1. Configure the `oauth2-proxy` that is running in Docker:
1. Update `upstreams` in `oauth2-proxy/oauth2-proxy-local.cfg` to include the IP address of your local machine, e.g. `upstreams=["http://172.100.100.01:3000"]`
<br>You can obtain the IP address using `hostname -I`.
1. Restart the oauth2-proxy: `docker compose restart oauth2-proxy`
1. Start the Portal locally:
```sh
cd src
set -o allexport
source ../.env.local
LOG_LEVEL=debug
KNEX_HOST=kong-db.localtest.me
NEXT_PUBLIC_MOCKS=off
set +o allexport
npm run dev
```
1. The Portal is now live at http://oauth2proxy.localtest.me:4180 and should auto-update on code changes.
- Please wait until keycloak service starts and is initialized with `master` realm. The realm configuration is saved in `./keycloak/master-realm.json`. It also creates a realm user `local` with admin privileges.
- You may want to run `docker-compose build` if there are new changes that are not reflected in the last time you built the container images
## Design
The `API Services Portal` is a React application using the Chakra UI component library, and using two frameworks: KeystoneJS V5, and NextJS.
The application is divided up into the following six components:
Expand Down Expand Up @@ -147,9 +166,11 @@ Currently support feeders:
Source: `feeds`
## Development
#### TypeScript
### TypeScript
The client-side Next.js application uses TypeScript, and because it plays nicely with GraphQL types, uses a codegen to generate the API types.
In `development` mode once the API server has started the types are automatically generated, but will need to be regenerated if you make changes to the
Expand All @@ -176,7 +197,7 @@ const Component = () => {

All Typescript paths alias `src/nextapp` to `@/`.

#### Storybook
### Storybook

[Chakra UI](https://chakra-ui.com) was chosen for the UI framework due to its utility and flexibility. A theme has been created which follows the [BC Government Web Design System](https://developer.gov.bc.ca/Design-System) alongside custom components written for the portal.

Expand All @@ -196,7 +217,7 @@ import { Button } from 'chakra-ui/react';

All the core components stories are located in `src/stories`. For custom components add the story in the component folder, ie `src/nextapp/components/card/card.stories.tsx`.

#### Mock Server
### Mock Server

For convenience a mock server is available to fake data via the GraphQL api. Run by opening a new shell window after running `$ npm run dev` and run the following:

Expand All @@ -214,15 +235,15 @@ GWA_API_URL=http://localhost:4000

It should be noted that a 1-to-1 replication of the production API is not the goal of the mock server. It's simply to replicate requests and confirm the content returned will behave in an expected way.

###### Updating mock server schemas
#### Updating mock server schemas

When Keystone-level types are updated, there is a manual step required for the mock server in order to keep the mock data structure in sync with the production server. It is definitely manual at the moment, but fairly easy and quick to do.

1. After the Keystone dev server has started (`$ npm run dev`), open [http://localhost:3000/admin/graphiql](http://localhost:3000/admin/graphiql)
2. The far right of the graphiql interface are 2 tabs, `DOCS` and `SCHEMAS`. You can either download and copy or copy the contents of the `SCHEMAS` tab and paste it in `src/test/mock-server/schemas.js` inside the string literal.
3. Delete any instances of a `@deprecated(reason: "Use `path` instead")` string. These messages break the graphql-tools

#### Coding Style
### Coding Style

There isn't a strict, repo-wide coding style per se, but we use Prettier and ESLint to maintain a consistent code style. Both libraries are included locally as part of the node_modules, so it is recommended to configure your editor to run off local versions instead of global so any API changes between versions don't collide.

Expand Down Expand Up @@ -255,4 +276,3 @@ select 'drop table "' || tablename || '" cascade;' from pg_tables where schemana
```

In the mean time, it is possible to drop the tables and re-run the `init-aps-portal-keystonejs-batch-job`.

Loading

0 comments on commit 3147351

Please sign in to comment.