Skip to content

Commit

Permalink
feat: extend load tests to run from a pod (#361)
Browse files Browse the repository at this point in the history
* chore: reduce dev and test backup frequency to save space

* chore: update load tests, add checks

* feat: extend functionality of load tests to run from pod

* chore: remove redundant readmes

* chore: make make comands
  • Loading branch information
thegentlemanphysicist authored Jul 4, 2024
1 parent 42fc007 commit b470b52
Show file tree
Hide file tree
Showing 15 changed files with 160 additions and 15 deletions.
45 changes: 41 additions & 4 deletions k6/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,20 +26,57 @@ from the [podman-grapher](./local_setup/podman-grapher/) directory. This will la
This test requires a client with a service account to run. E.g if using the default `admin-cli` client of the master realm locally, make sure the following are configured for it:

- In the client settings, set the **Access Type** to confidential, and then toggle on **Service accounts enabled**.
- Make sure that the clientID and clientSecret in [env.js](./env.js) match that client's credentials.
- Make sure that the clientID and clientSecret in [config.json](./k6-runner/src/config/config.json) match that client's credentials.

### Testing the Quarkus release:

Make certain that 'Client authentication' is toggled on and select 'Direct access grants' and 'Service accounts roles' from Authentication Flow.

**Do not do this in a production environment**. In the master realm, go to `Authentication->Direct grant` and disable "Condition- user configured" and "OTP".

If testing a live application, pick an appropriate client to use with a confidential service account.

- Copy `k6-runner/src/config/config.example.json` to `k6-runner/src/config/config.json`. Provide credentials for an account with permissions to create realms and users. If you are setting up locally, use the baseURL `http://localhost:8080/auth`, and you can use the admin-cli client ID with the admin admin credentials for username and password.

If testing a live application, pick an appropriate client to use with a confidential service account.
#### Running the test locally

These tests are adapted from the Ministry of Education's Student Online Access Module (SOAM), [load testing framework](https://github.com/bcgov/EDUC-KEYCLOAK-SOAM/blob/refs%2Fheads%2Fmaster/testing%2Fk6%2FREADME.md). The tests can be un locally by using the `docker-compose.yml` file in the `k6-runner` directory. From the `k6-runner` directory, run:

```
docker-compose run k6 run -e CONFIG=/config/config.json /scripts/constantRateAllFlows.js
```

#### Running the test remotely

The remote test runs on the script `/k6/k6-runner/openshift/k6/start.sh`. To change the test, point the run command at a different file.

To run the test from a kubernetes pod you will need to do the following:

Create a docker image a push it to be hosted in the bcgov ghcr repos. **Note: do not build any secrets into the image, it is a public repo**. From the `ku-runner` directiory run:
```
docker build . -t ghcr.io/bcgov/sso-k6:latest
docker push ghcr.io/bcgov/sso-k6:latest
```
This will create the image and host it. Each time you change the test code of config, this image will need to be rebuilt and pushed.

Next create the k6-config file in the namespace from which you want to run the tests:
```
oc create -n <NAMESPACE> configmap k6-config --from-file=./src/config/config.json
```

Lastly deploy the config from the `sso-keycloak/k6/k6-runner/openshift/k6` directory.
```
oc -n <NAMESPACE> process -f dc.yaml | oc -n <NAMESPACE> apply -f -
```

Be sure to delete the job when done to prevent the load test from re-running in the cluster.

#### Running the test locally without docker.

The tests can also be run without docker, by running:
`k6 run -e CONFIG=../config/config.json ./tests/constantRateAllFlows.js`
from the `sso-keycloak/k6/k6-runner/src` directory.

- Copy `env.example.js` to `env.js`. Provide credentials for an account with permissions to create realms and users. If you are setting up locally, use the baseURL `http://localhost:8080/auth`, and you can use the admin-cli client ID with the admin admin credentials for username and password.
- Run tests with `k6 run <js file>`

## Tests

Expand Down
6 changes: 0 additions & 6 deletions k6/env.example.js

This file was deleted.

8 changes: 8 additions & 0 deletions k6/k6-runner/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM loadimpact/k6:latest

WORKDIR /var/opt

COPY /src/tests /var/opt/scripts
COPY /openshift/k6/start.sh /var/opt

ENTRYPOINT [ "sh", "/var/opt/start.sh" ]
16 changes: 16 additions & 0 deletions k6/k6-runner/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
SHELL := /usr/bin/env bash

# make <comand> NAMESPACE="<namespace>"

.PHONY: config
config:
oc create -n $(NAMESPACE) configmap k6-config --from-file=./src/config/config.json

.PHONY: run_job
run_job:
oc -n $(NAMESPACE) process -f ./openshift/k6/dc.yaml | oc -n $(NAMESPACE) apply -f -

.PHONY: delete
delete:
oc -n $(NAMESPACE) delete job sso-k6
oc -n $(NAMESPACE) delete configmap k6-config
15 changes: 15 additions & 0 deletions k6/k6-runner/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: '3.0'

networks:
k6:

services:
k6:
image: loadimpact/k6:latest
networks:
- k6
ports:
- "6565:6565"
volumes:
- ./src/tests:/scripts
- ./src/config:/config
44 changes: 44 additions & 0 deletions k6/k6-runner/openshift/k6/dc.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
kind: Template
apiVersion: v1
objects:
- apiVersion: batch/v1
kind: Job
spec:
backoffLimit: 0
template:
metadata:
creationTimestamp: null
spec:
volumes:
- name: k6-config
configMap:
name: k6-config
containers:
- image: ${IMAGE_REPOSITORY}:${IMAGE_TAG}
name: ${NAME}
resources:
limits:
cpu: 2
memory: 2Gi
requests:
cpu: 500m
memory: 500Mi
volumeMounts:
- name: k6-config
mountPath: /var/opt/config
restartPolicy: Never
metadata:
name: ${NAME}
labels:
app: ${NAME}
component: ${NAME}-job
parameters:
- name: NAME
value: sso-k6
- name: IMAGE_TAG
value: latest
- name: IMAGE_REPOSITORY
value: ghcr.io/bcgov/sso-k6
- name: HTTP_DEBUG
description: enable http debug logging in the k6 pod
value: "false"
1 change: 1 addition & 0 deletions k6/k6-runner/openshift/k6/start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
k6 run -e CONFIG=/var/opt/config/config.json /var/opt/scripts/constantRateAllFlows.js
1 change: 1 addition & 0 deletions k6/k6-runner/src/config/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
config.json
10 changes: 10 additions & 0 deletions k6/k6-runner/src/config/config.example.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

{
"kcLoadTest": {
"password": "",
"clientId": "",
"baseUrl": "",
"clientSecret": "",
"username": ""
}
}
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import { sleep } from 'k6';
import { createRealm, deleteRealm, createUser, generateRealms, getAccessToken } from './helpers.js';
import { user } from './constants.js';
import { username, password, clientId } from './env.js';

let config = JSON.parse(open(__ENV.CONFIG));

const username = config.kcLoadTest.username;
const password = config.kcLoadTest.password;
const clientId = config.kcLoadTest.clientId;
// Alter configuration to run separate tests. See this test in the readme for configuration details.
const CONCURRENT_LOOPS = 5;
const ITERATIONS_PER_LOOP = 50;
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
import { sleep } from 'k6';
import { createRealm, deleteRealm, createUser, generateRealms, getAccessToken, hitIntrospectionRoute, hitUserInfoRoute, createClient } from './helpers.js';
import { user, client } from './constants.js';
import { username, password, clientId } from './env.js';

let config = JSON.parse(open(__ENV.CONFIG));
const username = config.kcLoadTest.username;
const password = config.kcLoadTest.password;
const clientId = config.kcLoadTest.clientId;
// Alter configuration to run separate tests. See this test in the readme for configuration details.
const TOTAL_REALMS = 1;
// This essentially just means no dropped requests allowed since we dont get to 10000 on the peak profile.
Expand Down
File renamed without changes.
6 changes: 5 additions & 1 deletion k6/helpers.js → k6/k6-runner/src/tests/helpers.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import http, { head } from 'k6/http';
import { baseUrl, clientId, clientSecret } from './env.js';
import { realm, client } from './constants.js';
import encoding from 'k6/encoding';

let config = JSON.parse(open(__ENV.CONFIG));

const baseUrl = config.kcLoadTest.baseUrl;
const clientSecret = config.kcLoadTest.clientSecret;

const getHeaders = (accessToken) => ({
Authorization: `Bearer ${accessToken}`,
'Content-Type': 'application/json',
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
import { sleep } from 'k6';
import { hitIntrospectionRoute, getAccessToken, createClient, deleteClient } from './helpers.js';
import { username, password, clientId } from './env.js';
import { client } from './constants.js';

let config = JSON.parse(open(__ENV.CONFIG));
const username = config.kcLoadTest.username;
const password = config.kcLoadTest.password;
const clientId = config.kcLoadTest.clientId;

const CONCURRENT_LOOPS = 1;
const ITERATIONS_PER_LOOP = 10;
const LOOP_DELAY = 0.01;
Expand Down
6 changes: 5 additions & 1 deletion k6/userInfo.js → k6/k6-runner/src/tests/userInfo.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
import { sleep } from 'k6';
import { getAccessToken, hitUserInfoRoute } from './helpers.js';
import { username, password, clientId } from './env.js';

let config = JSON.parse(open(__ENV.CONFIG));
const username = config.kcLoadTest.username;
const password = config.kcLoadTest.password;
const clientId = config.kcLoadTest.clientId;

const CONCURRENT_LOOPS = 1;
const ITERATIONS_PER_LOOP = 100;
Expand Down

0 comments on commit b470b52

Please sign in to comment.