STASIS consolidates the steps required to build, test, and deploy calibration pipelines and other software suites maintained by engineers at the Space Telescope Science Institute.

Requirements

• Linux, or MacOS (Darwin)
• cmake
• libcurl
• libxml2
• rsync

Installation

Download the source code

git clone https://github.com/spacetelescope/stasis.git
cd stasis

Create and enter the build directory

mkdir build
cd build

Run cmake

cmake .. -DCMAKE_INSTALL_PREFIX="$HOME/programs/stasis"

Compile and install stasis

make
make install
export PATH="$HOME/programs/stasis/bin:$PATH"

Quickstart

Step 1: Create a mission

Missions definitions live in STASIS's etc/mission directory. Let's create a new one specifically geared toward generic data analysis tools. You may override the path to the etc directory by setting the STASIS_SYSCONFDIR environment variable to different location.

mkdir $HOME/programs/stasis/etc/missions/mymission
touch $HOME/programs/stasis/etc/missions/mymission/mymission.ini

Now populate the new data analysis mission configuration. Refer to the release formatters section to see a list of what each % formatter does.

[meta]
release_fmt = %n-%v-%r-py%p-%o-%a
; e.g. mydelivery-1.2.3-1-py312-linux-x86_64

build_name_fmt = %n-%v
; e.g. mydelivery-1.2.3

build_number_fmt = %v.%r
; e.g. 1.2.3.1

Text files containing STASIS template strings can be stored at the same level as the mission configuration, and rendered to anywhere inside the output directory. This will can you time if you plan to release for multiple platforms and architectures.

[template:readme.md.in]
destination = {{ storage.delivery_dir }}/README-{{ info.release_name }}.md
; e.g [..]/output/delivery/README-mydelivery-1.2.3-1-py312-linux-x86_64.md

[template:Dockerfile.in]
destination = {{ storage.build_docker_dir }}/Dockerfile
; e.g [..]/build/docker/Dockerfile

Step 2: Create a delivery configuration

STASIS's configuration parser does not distinguish input files by extension, so mydelivery.ini, mydelivery.cfg, mydelivery.txt, and abc123.zyx987 are all perfectly valid file names.

All deliveries require a [meta] section. Here global metadata such as the delivery's name, version, and any programs/dependencies that make up the deliverable.

[meta]
mission = mymission
name = mydelivery
version = 2024.3.1
rc = 1
python = 3.12

The [conda] section instructs STASIS how to obtain the conda installer of your choice, and defines the packages to be installed into the delivery's release environment.

[conda]
; e.g. Download Miniforge3-23.11.0-0 for the current system platform and architecture
installer_name = Miniforge3
installer_version = 23.11.0-0
installer_platform = {{env:STASIS_CONDA_PLATFORM}}
installer_arch = {{env:STASIS_CONDA_ARCH}}
installer_baseurl = https://github.com/conda-forge/miniforge/releases/download/{{conda.installer_version}}

conda_packages =
    qt>=5.0
    
pip_packages =
    our_cool_program
    our_other_cool_program

Create some test cases for packages.

[test:our_cool_program]
version = 1.2.3
repository = https://github.com/org/our_cool_program
script_setup =
    pip install -e '.[test]'
script =
    pytest -fEsx \
        --basetemp="{{ func:basetemp_dir() }}" \
        --junitxml="{{ func:junitxml_file() }}" \
        tests/

[test:our_other_cool_program]
version = 4.5.6
repository = https://github.com/org/our_other_cool_program
script_setup =
    pip install -e '.[test]'
script =
    pytest -fEsx \
        --basetemp="{{ func:basetemp_dir() }}" \
        --junitxml="{{ func:junitxml_file() }}" \
        tests/

Step 3: Run STASIS

stasis mydelivery.ini

Configuration

Command Line Options

Long Option	Short Option	Purpose
--help	-h	Display usage statement
--version	-V	Display program version
--continue-on-error	-C	Allow tests to fail
--config ARG	-c ARG	Read STASIS configuration file
--cpu-limit ARG	-l ARG	Number of processes to spawn concurrently (default: cpus - 1)
--pool-status-interval ARG	n/a	Report task status every n seconds (default: 30)
--python ARG	-p ARG	Override version of Python in configuration
--verbose	-v	Increase output verbosity
--unbuffered	-U	Disable line buffering
--update-base	n/a	Update conda installation prior to STATIS environment creation
--fail-fast	n/a	On test error, terminate all tasks
--overwrite	n/a	Overwrite an existing release
--no-docker	n/a	Do not build docker images
--no-artifactory	n/a	Do not upload artifacts to Artifactory
--no-testing	n/a	Do not execute test scripts
--no-parallel	n/a	Do not execute tests in parallel
--no-rewrite	n/a	Do not rewrite paths and URLs in output files
DELIVERY_FILE	n/a	STASIS delivery file

Environment variables

Name	Purpose
TMPDIR	Change default path to store temporary data
STASIS_ROOT	Change default path to write STASIS's data
STASIS_SYSCONFDIR	Change default path to search for configuration files
STASIS_CPU_COUNT	Number of available CPUs
STASIS_GH_TOKEN	GitHub API token (Scope: "Contents" repository permissions (write))
STASIS_JF_ARTIFACTORY_URL	Artifactory service URL (ending in /artifactory)
STASIS_JF_ACCESS_TOKEN	Artifactory Access Token
STASIS_JF_USER	Artifactory username
STASIS_JF_PASSWORD	Artifactory password
STASIS_JF_SSH_KEY_PATH	Path to SSH public key file
STASIS_JF_SSH_PASSPHRASE	Password associated with SSH public key file
STASIS_JF_CLIENT_CERT_CERT_PATH	Path to OpenSSL cert files
STASIS_JF_CLIENT_CERT_KEY_PATH	OpenSSL key file (in cert path)
STASIS_JF_REPO	Artifactory "generic" repository to write to

Main configuration (stasis.ini)

The default path to the configuration file is [CMAKE_INSTALL_PREFIX]/etc/stasis/stasis.ini. You may override this by setting the STASIS_SYSCONFDIR environment variable to a path that points elsewhere.

Sections

default

Name	Type	Purpose
continue_on_error	Boolean	Keep going even if a test fails
always_update_base_environment	Boolean	Update all packages in the base to the latest release
conda_fresh_start	Boolean	Remove conda installation during initialization
conda_install_prefix	String	Install conda in a custom prefix path
conda_packages	List	Conda packages to be installed/overridden in the base environment
pip_packages	List	Python packages to be installed/overridden in the base environment
conda_staging_url	String	URL to conda channel

jfrog_cli_download

Name	Type	Purpose
url	String	Base URL of JFrog CLI release server
project	String	Product identifier (i.e. jfrog-cli)
version_series	String	Product version series (i.e. v2-jf)
version	String	Product version to install. [RELEASE] downloads the latest version.
filename	String	Product file name (i.e. jf)

deploy:artifactory

Name	Type	Purpose
url	String	Set artifactory service URL (ending in /artifactory)
repo	String	Set artifactory repository

Delivery configuration

Sections

All configuration section names and keys are case-sensitive.

meta

Key	Type	Purpose	Required
mission	String	Mission rule directory to use	Y
name	String	Name of the delivery	Y
version	String	Version of delivery	Y
codename	String	Release code name (HST-only)	N
rc	Integer	Release candidate level (i.e. delivery iteration)	Y
final	Boolean	Toggle final delivery logic (possibly deprecated in the future)	N
based_on	String	A conda environment YAML file/URL to use as a baseline	N
python	String	Version of Python to base delivery on	Y

conda

Key	Type	Purpose	Required
installer_name	String	Conda distribution name	Y
installer_version	String	Conda distribution version	Y
installer_platform	String	Conda target platform	Y
installer_arch	String	Conda target architecture	Y
installer_baseurl	String	Conda installer URL	Y
conda_packages	List	Conda packages to install	N
pip_packages	List	Pypi packages to install	N

runtime

Environment variables exported are global to all programs executed by stasis. There is no limit to the number of environment variables that can be set.

Key	Type	Purpose	Required
arbitrary name	String	Export environment variables to build and test environment	N
artitrary_name ... N	-	-	-

test:name

Sections starting with test: will be used during the testing phase of the stasis pipeline. Where the value of name following the colon is an arbitrary value, and only used for reporting which test-run is executing. Section names must be unique.

Key	Type	Purpose	Required
disable	Boolean	Disable script execution (script_setup always executes)	N
parallel	Boolean	Execute test block in parallel (default) or sequentially	N
build_recipe	String	Git repository path to package's conda recipe	N
repository	String	Git repository path or URL to clone	Y
version	String	Git commit or tag to check out	Y
runtime	List	Export environment variables specific to test context	Y
script_setup	List	Body of a shell script that will install dependencies	N
script	List	Body of a shell script that will execute the tests	Y

deploy:artifactory:name

Sections starting with deploy:artifactory: will define the upload behavior of build and test artifacts to Artifactory. Where the value of name is an arbitrary value, and only used for reporting. Section names must be unique.

Key	Type	Purpose	Required
files	List	jf-compatible wildcard path	Y
dest	String	Remote artifactory path to store files	Y

deploy:docker

The deploy:docker section controls how Docker images are created, when a Dockerfile is present in the build_docker_dir.

Key	Type	Purpose	Required
registry	String	Docker registry to use	Y
image_compression	String	Compression program (with arguments)	N
build_args	List	Values passed to docker build --build-args	N
tags	List	Docker image tag(s)	Y

Variable expansion

Template strings

Template strings can be accessed using the {{ subject.key }} notation in any STASIS configuration file.

Name	Purpose
meta.name	Delivery name
meta.version	Delivery version
meta.codename	Delivery codename
meta.mission	Delivery mission
meta.python	Python version (e.g. 3.11)
meta.python_compact	Python (e.g. 311)
info.time_str_epoch	UNIX Epoch timestamp
info.release_name	Rendered delivery release name
info.build_name	Rendered delivery build name
info.build_number	Rendered delivery build number
storage.tmpdir	Ohymcal temp directory
storage.delivery_dir	STASIS delivery output directory
storage.results_dir	STASIS test results directory
storage.conda_artifact_dir	STASIS conda package directory
storage.wheel_artifact_dir	STASIS wheel package directory
storage.build_sources_dir	STASIS sources directory
storage.build_docker_dir	STASIS docker directory
conda.installer_name	Conda distribution name
conda.installer_version	Conda distribution version
conda.installer_platform	Conda target platform
conda.installer_arch	Conda target architecture
conda.installer_baseurl	Conda installer URL
system.arch	System CPU Architecture
system.platform	System Platform (OS)
deploy.docker.registry	Docker registry
deploy.jfrog.repo	Artifactory destination repository
workaround.conda_reactivate	Reinitialize the conda runtime environment. Use this after calling conda install from within a [test:*].script.

The template engine also provides an interface to environment variables using the {{ env:VARIABLE_NAME }} notation.

[meta]
name = {{ env:MY_DYNAMIC_DELIVERY_NAME }}
version = {{ env:MY_DYNAMIC_DELIVERY_VERSION }}
python = {{ env:MY_DYNAMIC_PYTHON_VERSION }}

Template Functions

Template functions can be accessed using the {{ func:NAME(ARG,...) }} notation.

Name	Arguments	Purpose
get_github_release_notes_auto	n/a	Generate release notes for all test contexts
basetemp_dir	n/a	Generate directory path to test block's temporary data directory
junitxml_file	n/a	Generate directory path and file name for test result file

Mission files

Mission rules are defined in the STASIS_SYCONFDIR/mission directory. Each mission configuration file shares the same name as the directory. To create a new mission, example, the directory structure will be as follows:

STASIS_SYSCONFDIR/
    mission/
        example/
            example.ini
            supporting_text.md.in

Sections

meta

Key	Type	Purpose	Required
release_fmt	String	A string containing one or more release format specifiers	Y
build_name_fmt	String	-	Y
build_number_fmt	String	-	Y

Release formatters

Formatter	Purpose
%n	Delivery name
%c	Delivery codename
%r	Delivery RC level
%R	Delivery RC level (final-aware)
%v	Delivery version
%m	Mission name
%P	Python version (i.e. 3.11)
%p	Compact Python version (i.e. 311
%a	System CPU Architecture
%o	System Platform (OS)
%t	UNIX Epoch timestamp

template:filename

Sections starting with template: will make use of the built-in template engine, replacing any strings in filename, relative to the mission directory, with expanded data values. The output will be written to destination.

# example.ini
[template:supporting_text.md.in]
destination = {{ storage.delivery_dir }}/output_filename.md