update readme

ajuvercr · Jul 12, 2022 · 041120e · 041120e
1 parent f061c5e
commit 041120e
Showing 1 changed file with 185 additions and 17 deletions.
diff --git a/README.md b/README.md
@@ -1,22 +1,24 @@
-# nautirust
+# Nautirust
 
-An orchestrator for the connector architecture.
-This architecture defines processors that take input.
-Input can be many things, mostly configuration stuff, streamReaders and streamWriters.
+Nautirust is a cli-utility program designed to assist the user in configuring and starting a pipeline of processes (called steps).
+Nautirust does not restrict on programming language or startup sequence.
 
-Readers and writers are connectors that use a channel to forward messages in some serialization (json, turtle, xml, plain).
-What channel is used, usually doesn't really matter, they all have the same interface. 
+## Intro
 
-Nautirust is primariy used to configure pipelines that consist of steps.
-Each step corresponds to a processor and is configured with a step configuration file denoting the expected runner, runner specific arguments (source file, source function) and arguments to be configured.
-Nautirust configures the parameters used for the processors per step. To do this it mostly asks the user the actual _implementations_ of the arguments.
-Nautirust understands that readers and writers have to be linked up to function (the same channel configuration), this way it can guide the user in creating the correct pipeline.
+Nautirust consists of three concepts, a step, a runner and a channel. A pipeline is a sequence of steps.
 
-After configuring the steps of a pipeline a pipeline config is generated. This config contains all the actual arguments (including channel configuration).
+Runners are designed to start a step and can require steps to define some additional properties. 
+For example a runner that executes a step written in javascript requires the location of the main source file and the name of the function that should be executed.
+
+Each step is programming language independent, but has to specify what runner must be used to start the step. 
+A step also specifies what arguments should be provided to that step, including the type.
+
+The type of these arguments is loose but there are two special cases: stream reader and stream writer. 
+Readers and writers connect two processes together with _a_ channel. Nautirust helps connecting these channels.
+
+The goal of the runner is not only to start up a step, but also to abstract away the underlying channel. This eases the implementation of steps.
+Runners specify what channels they support. Not all runners can support all channels, nautirust uses this information to only suggest plausible channels.
 
-When Nautirust executes a configured pipeline, it executes a specific _runner_ for a specific step.
-For example, if a step consists of a JS function, then a JSRunner is used to actually execute this function.
-Ideally the runner starts up the channel and provides the step with a instance of a reader or writer and the configured arguments.
 
 ## Usage
 
@@ -40,13 +42,13 @@ Example step file (for more details see later)
 
 Generate a plan and save it to plan.json:
 ```
-cargo run -- generate -o plan.json [...steps]
+nautirust generate -o plan.json [...steps]
 ```
 
 
 Execute the plan:
 ```
-cargo run -- run plan.json
+nautirust run plan.json
 ```
 
 
@@ -61,6 +63,7 @@ runners = "configs/runners/*/runner.json"
 This configuration file will look for any channel defined inside `configs/channels` and will
 look for runners defined in `configs/runners` that have a `runner.json` file.
 
+
 ### Channel configuration
 
 Example channel configuration:
@@ -80,7 +83,8 @@ Example channel configuration:
 ```
 
 Here a channel is defined called `file` and specifies two fields are required: `path` and `onReplace`.
-It also defines some options for the orchestrator and user to choose from.
+It also defines some options for the nautirust. The user can choose between these options.
+
 
 ### Runner configuration
 
@@ -136,3 +140,167 @@ Example step configuration:
 A step is a processor. The processor specifies that it can be executed with the `JsRunner` runner.
 It also specifies what arguments have to be defined before being able to execute.
 
+
+## Functionality
+
+```sh
+$ nautirust -h
+nautirust 0.1.1
+
+USAGE:
+    nautirust [OPTIONS] <SUBCOMMAND>
+
+OPTIONS:
+    -c, --channels <CHANNELS>    Glob to indicate channels locations
+        --config <CONFIG>        Location of a config file
+    -h, --help                   Print help information
+    -r, --runners <RUNNERS>      Glob to indicate runners locations
+    -V, --version                Print version information
+
+SUBCOMMANDS:
+    docker      Create a docker-compose file from a nautirust pipeline
+    generate    Generate a pipeline of steps
+    help        Print this message or the help of the given subcommand(s)
+    prepare     Prepares the execution pipeline by starting the required channels/runner
+    run         Run a configured pipeline
+    stop        Gracefully stop the runners and channels specified in the config
+    validate    Validate configureations for runners and channels
+```
+
+You can specify (by glob) where to find channels and runners with `--channels` and `--runners` respectively.
+Nautirust also supports toml file that specifies these properties (`--config`).
+
+
+### generate
+
+```sh
+$ nautirust generate -h
+nautirust-generate 
+Generate a pipeline of steps
+
+USAGE:
+    nautirust generate [OPTIONS] [STEPS]...
+
+ARGS:
+    <STEPS>...    Steps to include in the pipeline (ordered)
+
+OPTIONS:
+    -a, --automatic          Try infer basic configurations details
+    -h, --help               Print help information
+    -o, --output <OUTPUT>    Output location of the generated pipeline file
+```
+
+Nautirust takes multiple steps to create a pipeline configuration file.
+Interactively the user is asked questions about the required arguments, taking special care of stream readers and stream writers. 
+
+These questions can add up, use the `-a` flag to let nautirust infer some basic configuration consisting of:
+- automatic linking of stream readers and writers with the same name
+- automatically choosing a channel configuration when the channel type is specified
+
+`-o` takes a filename to store the generated configuration (default is stdout).
+
+### run
+```sh
+$ nautirust run -h
+nautirust-run 
+Run a configured pipeline
+
+USAGE:
+    nautirust run [OPTIONS] <FILE>
+
+ARGS:
+    <FILE>    Config file
+
+OPTIONS:
+    -h, --help                 Print help information
+    -t, --tmp-dir <TMP_DIR>    temporary directory to put step configuration files
+```
+
+Nautirust runs a generated configuration file.
+
+Each runners takes in a configuration file that specifies the steps that should be executed, with `-t` you can specify the location of these configuration files.
+
+
+### prepare
+```sh
+$ nautirust prepare -h
+nautirust-prepare 
+Prepares the execution pipeline by starting the required channels/runner
+
+USAGE:
+    nautirust prepare <FILE>
+
+ARGS:
+    <FILE>    Config file
+
+OPTIONS:
+    -h, --help    Print help information
+```
+
+Nautirust takes a generated configuration file, and prepares the used steps, runners and channels.
+This can be used to run a build script, start a docker-compose instance, ...
+
+
+### stop
+```sh
+$ nautirust stop -h
+nautirust-stop 
+Gracefully stop the runners and channels specified in the config
+
+USAGE:
+    nautirust stop <FILE>
+
+ARGS:
+    <FILE>    Config file
+
+OPTIONS:
+    -h, --help    Print help information
+```
+
+Same as prepare, but in reverse.
+
+
+### docker
+
+**EXPERIMENTAL**
+```sh
+$ nautirust docker -h
+nautirust-docker 
+Create a docker-compose file from a nautirust pipeline
+
+USAGE:
+    nautirust docker [OPTIONS] <FILE>
+
+ARGS:
+    <FILE>    Config file
+
+OPTIONS:
+    -h, --help                 Print help information
+    -o, --output               
+    -t, --tmp-dir <TMP_DIR>    temporary directory to put step configuration files
+```
+
+Does the same as `nautirust run` but generates a docker-compose file that when executed start the pipeline.
+This only works when the runners have specified a docker command.
+This docker command is expected to generate a Dockerfile somewhere that executes the step.
+This command also prints to stdout the required contents for this step (at least specify the generated Dockerfile and set up the correct docker context).
+
+The used channels also return a part of the docker-compose file (if anything).
+
+
+### validate
+
+```sh
+$ nautirust validate -h
+nautirust-validate 
+Validate configureations for runners and channels
+
+USAGE:
+    nautirust validate
+
+OPTIONS:
+    -h, --help    Print help information
+```
+
+Validates the specified channels and runners.
+