Skip to content

Integration test details

Jump to bottom
justinbastress edited this page Feb 13, 2018 · 1 revision

A module's integration tests consist of three scripts in its module directory that are run in sequence:

  1. integration_tests/modulename/setup.sh
  2. integration_tests/modulename/test.sh
  3. integration_tests/modulename/cleanup.sh

Setup

The setup script does any necessary steps to ensure that the test script can run. This will typically involve creating a docker image and launching docker containers running the target service.

The setup script should be idempotent, so that if it is already set up, it can quickly exit and continue to the test script.

A typical pattern is something like this:

  1. If the docker image is not in the cache, build the docker image (perhaps based on the zgrab2_service_base image, which has an up-to-date apt cache)
  2. If the docker container is not running, launch the docker container.

If the setup script exits with a nonzero exit code, it fails / stops the integration test without cleaning up (to allow debugging).

Test

The test script runs the end-to-end tests of the module against the containers. Any output it drops in ../../zgrab-output/modulename/* will be validated against the schema as modulename and checked for a SCAN_STATUS_SUCCESS status.

The latest zgrab2 code can be run from its docker container via the docker-runner/docker-run.sh script:

CONTAINER_NAME=$TARGET_CONTAINER_NAME $ZGRAB_ROOT/docker-runner/docker-run.sh modulename --flag1 --flag2 > ../../zgrab-output/output.json

The script sets up a link between the container referenced in the CONTAINER_NAME environment variable and the zgrab2 container. The rest of the arguments are passed to zgrab2.

A typical pattern is something like this:

  1. mkdir -p $ZGRAB_ROOT/zgrab-output/modulename
  2. CONTAINER_NAME=$TARGET_CONTAINER_NAME $ZGRAB_ROOT/docker-runner/docker-run.sh modulename --flag1 --flag2 > $ZGRAB_ROOT/zgrab-output/output.json
  3. Dump container logs to stdout (e.g. docker logs --tail all $TARGET_CONTAINER_NAME) for debugging purposes

More advanced tests -- e.g. testing cases where the status is not SCAN_STATUS_SUCCESS -- can be performed in the same way, but without piping the output to zgrab-output.

If the test script exits with a nonzero status code, the integration-test fails without cleaning up (to allow debugging).

Cleanup

The cleanup script undoes the actions of the setup script (though it can leave docker images around to save time on future runs, since the service images are unlikely to change frequently if at all).

Cleanup scripts are considered to be best-effort only and typically log/ignore errors; so if it does nonetheless exit with a nonzero status code, it will fail the integration-test task immediately just as the setup and test scripts do.

Clone this wiki locally