Skip to content

Commit

Permalink
more work on getting started & workflow docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Tony Worm committed May 26, 2024
1 parent 86fe204 commit 2be9007
Show file tree
Hide file tree
Showing 12 changed files with 313 additions and 15 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ __`hof` is a CLI tool you *add* to your workflow.__
| Core Features | |
|:--- |:-- |
| __code generation__ | Data + templates = _ (anything), technology agnostic |
| __data model__ | Define and manage data models, life-cycle features |
| __data modeling__ | Define, checkpoint, and diff data models |
| __task engine__ | Extensible task and DAG workflow engine |
| __CUE cmds__ | Core def, eval, export, and vet commands |
| __creators__ | bootstraping and starter kits from any repo |
Expand Down
32 changes: 32 additions & 0 deletions docs/code/cmd-help/def
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
print consolidated CUE definitions

Usage:
hof def [flags]

Flags:
-a, --attributes diplay field attributes
-C, --comments include comments in output
-e, --expression stringArray evaluate these expressions only
-h, --help help for def
--inline-imports expand references to non-core imports
--list concatenate multiple objects into a list
--out string output data format, when detection does not work
-o, --outfile string filename or - for stdout with optional file prefix
--simplify simplify CUE statements where possible

Global Flags:
-E, --all-errors print all available errors
-i, --ignore-errors turn off output and assume defaults at prompts
-D, --include-data auto include all data files found with cue files
-V, --inject-env inject all ENV VARs as default tag vars
-I, --input stringArray extra data to unify into the root value
-p, --package string the Cue package context to use during execution
-l, --path stringArray CUE expression for single path component when placing data files
-q, --quiet turn off output and assume defaults at prompts
-d, --schema stringArray expression to select schema to apply to data files
--stats print generator statistics
-0, --stdin-empty A flag that ensure stdin is zero and does not block
-t, --tags stringArray @tags() to be injected into CUE code
-U, --user-files stringArray file globs to embed into the root value (<cue-path>=<file-glob>), use % as slash to trim before
-v, --verbosity int set the verbosity of output
--with-context add extra context for data files, usable in the -l/path flag
7 changes: 7 additions & 0 deletions docs/code/cmd-help/dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
unknown command "dev" for "hof"

Did you mean this?
gen
def
vet

41 changes: 41 additions & 0 deletions docs/code/cmd-help/eval
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
evaluate and print CUE configuration

Usage:
hof eval [flags]

Flags:
-a, --all show optional and hidden fields
-A, --attributes display field attributes
-C, --comments include comments in output
-c, --concrete require the evaluation to be concrete
--defaults use default values if not set
-S, --definitions display defintions (default true)
--escape use HTLM escaping
-e, --expression stringArray evaluate these expressions only
--final finalize the value (default true)
-h, --help help for eval
-H, --hidden display hidden fields
--inline-imports expand references to non-core imports
--list concatenate multiple objects into a list
-O, --optional display optional fields
--out string output data format, when detection does not work
-o, --outfile string filename or - for stdout with optional file prefix
--resolve resolve references in value
--simplify simplify CUE statements where possible

Global Flags:
-E, --all-errors print all available errors
-i, --ignore-errors turn off output and assume defaults at prompts
-D, --include-data auto include all data files found with cue files
-V, --inject-env inject all ENV VARs as default tag vars
-I, --input stringArray extra data to unify into the root value
-p, --package string the Cue package context to use during execution
-l, --path stringArray CUE expression for single path component when placing data files
-q, --quiet turn off output and assume defaults at prompts
-d, --schema stringArray expression to select schema to apply to data files
--stats print generator statistics
-0, --stdin-empty A flag that ensure stdin is zero and does not block
-t, --tags stringArray @tags() to be injected into CUE code
-U, --user-files stringArray file globs to embed into the root value (<cue-path>=<file-glob>), use % as slash to trim before
-v, --verbosity int set the verbosity of output
--with-context add extra context for data files, usable in the -l/path flag
31 changes: 31 additions & 0 deletions docs/code/cmd-help/export
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
output data in a standard format

Usage:
hof export [flags]

Flags:
-C, --comments include comments in output
--escape use HTLM escaping
-e, --expression stringArray evaluate these expressions only
-h, --help help for export
--list concatenate multiple objects into a list
--out string output data format, when detection does not work
-o, --outfile string filename or - for stdout with optional file prefix
--simplify simplify CUE statements where possible

Global Flags:
-E, --all-errors print all available errors
-i, --ignore-errors turn off output and assume defaults at prompts
-D, --include-data auto include all data files found with cue files
-V, --inject-env inject all ENV VARs as default tag vars
-I, --input stringArray extra data to unify into the root value
-p, --package string the Cue package context to use during execution
-l, --path stringArray CUE expression for single path component when placing data files
-q, --quiet turn off output and assume defaults at prompts
-d, --schema stringArray expression to select schema to apply to data files
--stats print generator statistics
-0, --stdin-empty A flag that ensure stdin is zero and does not block
-t, --tags stringArray @tags() to be injected into CUE code
-U, --user-files stringArray file globs to embed into the root value (<cue-path>=<file-glob>), use % as slash to trim before
-v, --verbosity int set the verbosity of output
--with-context add extra context for data files, usable in the -l/path flag
35 changes: 35 additions & 0 deletions docs/code/cmd-help/vet
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
validate data with CUE

Usage:
hof vet [flags]

Flags:
-A, --attributes display field attributes
-C, --comments include comments in output
-c, --concrete require the evaluation to be concrete
-S, --definitions display defintions (default true)
-e, --expression stringArray evaluate these expressions only
-h, --help help for vet
-H, --hidden display hidden fields
--list concatenate multiple objects into a list
-O, --optional display optional fields
--out string output data format, when detection does not work
-o, --outfile string filename or - for stdout with optional file prefix
--simplify simplify CUE statements where possible

Global Flags:
-E, --all-errors print all available errors
-i, --ignore-errors turn off output and assume defaults at prompts
-D, --include-data auto include all data files found with cue files
-V, --inject-env inject all ENV VARs as default tag vars
-I, --input stringArray extra data to unify into the root value
-p, --package string the Cue package context to use during execution
-l, --path stringArray CUE expression for single path component when placing data files
-q, --quiet turn off output and assume defaults at prompts
-d, --schema stringArray expression to select schema to apply to data files
--stats print generator statistics
-0, --stdin-empty A flag that ensure stdin is zero and does not block
-t, --tags stringArray @tags() to be injected into CUE code
-U, --user-files stringArray file globs to embed into the root value (<cue-path>=<file-glob>), use % as slash to trim before
-v, --verbosity int set the verbosity of output
--with-context add extra context for data files, usable in the -l/path flag
80 changes: 80 additions & 0 deletions docs/code/getting-started/task-engine/vertex.cue
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
package examples

import "strings"

vertex_chat: {
@flow() // define a flow

steps: {

// task: get auth from external command
gcp: {
@task(os.Exec)
cmd: ["gcloud", "auth", "print-access-token"]
stdout: string
key: strings.TrimSpace(stdout)
}

// task: api call via reusable task
call: _gemini & {
apikey: gcp.key

msg: "What is the CUE language?"

resp: body: _
}

// task: print text to std output
out: {
@task(os.Stdout)
text: call.final.text
}

}
}

// reusable task
_gemini: {
@task(api.Call)

model: string | *"gemini-1.0-pro-002:generateContent"

msg: string
apikey: string
prompt: string | *"You are a model which is direct and concise when responding."

req: {
host: "https://us-central1-aiplatform.googleapis.com"
path: "/v1/projects/hof-io--develop/locations/us-central1/publishers/google/models/\(model)"
headers: {
"Content-Type": "application/json"
Authorization: "Bearer \(apikey)"
}
data: {
systemInstruction: {
role: "MODEL"
parts: [{
text: prompt
}]
}

contents: [{
role: "USER"
parts: [{
text: msg
}]
}]
}
method: "POST"
}

resp: {
body: _
}

// task-local ETL
final: {
cand: resp.body.candidates[0]
text: cand.content.parts[0].text
}
}
75 changes: 65 additions & 10 deletions docs/content/getting-started/command-help.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: "Command Help"
description: "Help text for hof's main commands"
brief: "of the hof CLI"
brief: "Help text for hof's main commands"

weight: 100
---
Expand All @@ -20,29 +20,31 @@ Top level commands and help message
</details>


### hof / datamodel
### hof / gen

Used for data model management (dm for short)
Declarative code generation for directories and files from data, CUE, and templates.
Build with adhoc one-liners or use composable generators to create reusable blueprints.

See the [data modeling section](/data-modeling/) for details.
See [the code generation section](/code-generation/) to learn more

<details>
<summary>hof help datamodel</summary>
{{<codePane file="code/cmd-help/dm" title="$ hof help datamodel" lang="text">}}
<summary>hof help flow</summary>
{{<codePane file="code/cmd-help/gen" title="$ hof help gen" lang="text">}}
</details>

### hof / gen

Create one-liners to generate files with data, CUE, and templates
or use composable generators to build out advanced applications.
### hof / create

See [the code generation section](/code-generation/) to learn more
Generate boilerplate from any git repository using hof.

See [the creators section](/code-generation/creators/) to learn more

<details>
<summary>hof help flow</summary>
{{<codePane file="code/cmd-help/gen" title="$ hof help gen" lang="text">}}
</details>


### hof / flow

Build workflows and scripts with CUE and a DAG engine
Expand All @@ -55,6 +57,18 @@ See [the task engine section](/task-engine/) to learn more
</details>


### hof / datamodel

Used for data model management (dm for short)

See the [data modeling section](/data-modeling/) for details.

<details>
<summary>hof help datamodel</summary>
{{<codePane file="code/cmd-help/dm" title="$ hof help datamodel" lang="text">}}
</details>


### hof / mod

Manage a MVS (Golang) style modules and dependencies.
Expand All @@ -75,3 +89,44 @@ hof mod vendor cue
{{<codePane file="code/cmd-help/mod" title="$ hof help mod" lang="text">}}
</details>


### hof / fmt

Format many languages at once with good defaults.

See the [formatting](/code-generation/formatting) for details.

<details>
<summary>hof help fmt</summary>
{{<codePane file="code/cmd-help/fmt" title="$ hof help fmt" lang="text">}}
</details>


### hof / eval

CUE eval embedded in hof.

<details>
<summary>hof help eval</summary>
{{<codePane file="code/cmd-help/eval" title="$ hof help eval" lang="text">}}
</details>


### hof / export

CUE export embedded in hof.

<details>
<summary>hof help export</summary>
{{<codePane file="code/cmd-help/export" title="$ hof help export" lang="text">}}
</details>


### hof / vet

CUE vet embedded in hof.

<details>
<summary>hof help vet</summary>
{{<codePane file="code/cmd-help/vet" title="$ hof help vet" lang="text">}}
</details>
5 changes: 3 additions & 2 deletions docs/content/getting-started/hof-attributes.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: "#hof & @attributes"
brief: "Special annotations for hof."

weight: 70
---
Expand All @@ -17,10 +18,10 @@ Hof turns these CUE attributes into `#hof` configuration.

- `@gen(<name>)` - the root of a generator
- `@datamodel(<name>)` - the root of a datamodel
- `@flow(<task>)` - the root of a workflow or a task type
- `@flow(<name>)` - the root of a workflow

Datamodels and workflows have a few more attributes
that can be specified under their root.
that can be used under their root.
They are covered in the respective sections on each.

### Schema
Expand Down
5 changes: 4 additions & 1 deletion docs/content/getting-started/task-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,10 @@ Hof's task engine is an extension of cue/flow with
{{</lead>}}


## Overview
### Example


{{<codePane title="hof flow example.cue" file="code/getting-started/task-engine/vertex.html" >}}


### Command
Expand Down
Loading

0 comments on commit 2be9007

Please sign in to comment.