This repository is built using the following tools: make, git, curl, jq, and docker.
Services are built using make command and a set of targets; see:
Within each directory is a set of files to control the build process:
Makefile- build configuration and control formakecommand.travis.yml- process automation configuration and control for TravisDockerfile- a cross-architecture container definitionbuild.json- Docker container configurationservice.json- service configuration templateuserinput.json- variables template for use in testing servicepattern.json- [optional] pattern configuration template (seePATTERN.mdfor more information).
The Makefile is shared across all services; it is a symbolic link to a common, shared, file in the root of the repository. The top-level makefile invokes targets across service directories.
The Travis process automation system for continuous-integration enables the execution of the build process and tools in a cloud environment. Please see TRAVIS.md for more information.
The Dockerfile controls the container build process. A critical component of that process is the FROM directive, which indicates the container from which to build.
The Dockerfile also includes information for LABEL container information, for example:
LABEL \
org.label-schema.schema-version="1.0" \
org.label-schema.build-date="${BUILD_DATE}" \
org.label-schema.build-arch="${BUILD_ARCH}" \
org.label-schema.name="cpu" \
org.label-schema.description="base alpine container" \
org.label-schema.vcs-url="http://github.com/dcmartin/open-horizon/master/cpu/" \
org.label-schema.vcs-ref="${BUILD_REF}" \
org.label-schema.version="${BUILD_VERSION}" \
org.label-schema.vendor="David C Martin <[email protected]>"
The build.json configuration file provides a mapping for each architecture the service supports. For example, the Alpine-based LINUX base-alpine configuration:
{
"squash": false,
"build_from": {
"amd64": "alpine:3.8",
"arm": "arm32v6/alpine:3.8",
"arm64": "arm64v8/alpine:3.8",
"386": null,
"ppc64": null,
"ppc64le": null,
"mips64": null,
"mips64le": null,
"s390x": null,
"mips": null,
"mipsle": null
},
"args": {}
}
This example indicates three (3) supported architectures with values: arm64, amd64, and arm, and corresponding Docker container tags. The architecture values must be common across the build automation process and configuration files; it also effects the building and naming of container images and services. However, values may be specified as necessary to ensure uniqueness.
NOTE: Version attributions for the BUILD_FROM target is drawn from version of the parent service, e.g. version in the yolo/service.json service configuration template value of 0.0.7; see below:
{
"build_from": {
"amd64": "ubuntu:bionic",
"arm": "arm32v7/ubuntu:bionic",
"arm64": "arm64v8/ubuntu:bionic"
}
}
{
"build_from": {
"amd64": "${DOCKER_REPOSITORY}/amd64_com.github.dcmartin.open-horizon.base-ubuntu:0.0.2",
"arm": "${DOCKER_REPOSITORY}/arm_com.github.dcmartin.open-horizon.base-ubuntu:0.0.2",
"arm64": "${DOCKER_REPOSITORY}/arm64_com.github.dcmartin.open-horizon.base-ubuntu:0.0.2"
}
}
{
"build_from": {
"amd64": "${DOCKER_REPOSITORY}/amd64_com.github.dcmartin.open-horizon.yolo:0.0.7",
"arm": "${DOCKER_REPOSITORY}/arm_com.github.dcmartin.open-horizon.yolo:0.0.7",
"arm64": "${DOCKER_REPOSITORY}/arm64_com.github.dcmartin.open-horizon.yolo:0.0.7"
},
}
The service.json configuration template provides standard Open Horizon service metadata and state information, including:
org- organization in exchangeurl- unique identifier for service in organizationversion- semantic version of service [state]arch-nullin template; derived frombuild.json
In addition to the standard service configuration semantics and structure, there are three additional modifications:
label- this value is now used as a token (non-breaking-string), aka slug, identifier for the service and patternports- a mapping of service ports to host ports during local executiontmpfs- [optional] specification for a temporary, in-memory, file-system for use on IoT devices
Ports may be specified for mapping both TCP and UDP, for example:
"ports": {
"8082/tcp": 8082,
"8080/tcp": 8080,
"8081/tcp": 8081
}
The tmpfs specifies whether a temporary file-system should be created in RAM and it's size in bytes, destination directory (default "/tmpfs"), and mode permissions (default 0177).
"tmpfs": { "size": 2048000, "destination": "/tmpfs", "mode": "0177" }
The userinput.json provides configuration for variables defined the service in the service.json; this is service dependent. Variables, including secrets, may also be defined as files with JSON content. For example, the yolo2msghub service's required Kafka API key variable -- YOLO2MSGHUB_APIKEY -- would be created from an IBM MessageHub Kafka API key JSON file:
% jq '.api_key' {kafka-apiKey-file} > YOLO2MSGHUB_APIKEY
In addition, there are a set of build support scripts that provide required services for the automated build process; as with the Makefile, all scripts are shared across services.
docker-run.sh- standardized local execution of Docker containers perservice.jsonconfiguration templatemkdepend.sh- utilizeshznCLI to create build artefactscheckvars.sh- process service variables foruserinput.jsontest-service.sh- test service output (note: linked astest-<service>.sh)test.sh- test harness for processing output fromtest-service.shexchange-test.sh- test exchange for service or pattern pre-requisites; (note: linked as{pattern,service}-test.sh)fixpattern.sh- process pattern configuration template (seePATTERN.md)
Run the service locally as configured in the template and include all variables, port definitions, temporary file-system, and privilege. Other parameters currently available for service configuration are not available.
Utilize hzn CLI to create temporary build directory and process configuration template for service.
Gather variable(s) values as specified in service configuration template, userinput.json, or corresponding files in directory.
Perform a test of the service to support the test-harness (test.sh) for any service; the script name is dependent on the service label. All services share a common 'test-service.sh script, symbolically linked using the service label, e.g. test-yolo2msghub.sh.
One script with three names for interrogating the exchange. When invoked as service-test.sh, which is symbolically linked to exchange-test.sh, the service is tested to determine if all required services are up-to-date with respect to organization, architecture, and semantic version number. Out-of-date service configurations will fail with an error message.
Process the pattern configuration template with any additional TAG information. See MAKE.md for more information on TAG.
One script with three names for interrogating the exchange. When invoked as pattern-test.sh, which is symbolically linked to exchange-test.sh, the pattern is tested to determine if all services are up-to-date with respect to organization, architecture, and semantic version number. Out-of-date pattern configurations will fail with an error message.
Releases are based on Semantic Versioning, and use the format
of MAJOR.MINOR.PATCH. In a nutshell, the version will be incremented
based on the following:
MAJOR: Incompatible or major changes.MINOR: Backwards-compatible new features and enhancements.PATCH: Backwards-compatible bugfixes and package updates.
David C Martin ([email protected])