diff --git a/docs/code/data-modeling/by-attribute.cue b/docs/code/data-modeling/by-attribute.cue new file mode 100644 index 000000000..787738fea --- /dev/null +++ b/docs/code/data-modeling/by-attribute.cue @@ -0,0 +1,8 @@ +package main + +MyModels: { + @datamodel() + + foo: "string" + ... +} diff --git a/docs/code/data-modeling/by-hand.cue b/docs/code/data-modeling/by-hand.cue new file mode 100644 index 000000000..ff01e6073 --- /dev/null +++ b/docs/code/data-modeling/by-hand.cue @@ -0,0 +1,8 @@ +package main + +MyModels: { + #hof: datamodel: root: true + + foo: "string" + ... +} diff --git a/docs/code/data-modeling/by-import.cue b/docs/code/data-modeling/by-import.cue new file mode 100644 index 000000000..0d29292fc --- /dev/null +++ b/docs/code/data-modeling/by-import.cue @@ -0,0 +1,8 @@ +package main + +import "github.com/hofstadter-io/hof/schema/dm" + +MyModels: dm.Datamodel & { + foo: "string" + ... +} diff --git a/docs/code/data-modeling/dm-and-code-gen.cue b/docs/code/data-modeling/dm-and-code-gen.cue new file mode 100644 index 000000000..941234ae5 --- /dev/null +++ b/docs/code/data-modeling/dm-and-code-gen.cue @@ -0,0 +1,26 @@ +package main + +import ( + "github.com/hofstadter-io/hof/schema/dm/sql" + "github.com/hofstadter-io/supacode/gen" + "github.com/hofstadter-io/supacode/schema" +) + +MyGen: gen.Generator & { + @gen() + Datamodel: MyModels + App: MyApp +} + +MyModels: sql.Datamodel & { + @datamodel() + Models: { + ... + } +} + +MyApp: schema.App & { + // sometimes a module author will place the datamodel here + // or provide a custom datamodel specific to their generator + ... +} diff --git a/docs/code/data-modeling/providing-structure.cue b/docs/code/data-modeling/providing-structure.cue new file mode 100644 index 000000000..8df4978e3 --- /dev/null +++ b/docs/code/data-modeling/providing-structure.cue @@ -0,0 +1,22 @@ +MyModels: { + @datamodel() + + Models: { + @history() + + User: { + Fields: { + @history() + ... + } + } + + ... + } + + Views: { + @history() + + ... + } +} diff --git a/docs/code/data-modeling/using-enrichers.cue b/docs/code/data-modeling/using-enrichers.cue new file mode 100644 index 000000000..cca0d3a8d --- /dev/null +++ b/docs/code/data-modeling/using-enrichers.cue @@ -0,0 +1,28 @@ +package schema + +import ( + "github.com/hofstadter-io/hof/schema/dm/sql" + "github.com/hofstadter-io/hof/schema/dm/enrichers:go" + "github.com/hofstadter-io/hof/schema/dm/enrichers:py" +) + +Datamodel: sql.Datamodel & { + Models: { + @history() + + // apply to each "model" (CUE pattern constraint) + [string]: { + Fields: { + @history() + + // apply to each "field" (CUE pattern constraint) + [string]: go.FieldEnricher + [string]: py.FieldEnricher + + // These will add GoType and PyType to the "model" "field" + ... + } + } + ... + } +} diff --git a/docs/content/data-modeling/_index.md b/docs/content/data-modeling/_index.md index 9fca36e21..add97d998 100644 --- a/docs/content/data-modeling/_index.md +++ b/docs/content/data-modeling/_index.md @@ -2,44 +2,134 @@ title: Data Modeling weight: 40 -draft: true --- {{}} -Data modeling is core in `hof` because -your data model is core to your applications. -They form the basis for input to code generation and -`hof` has a suite of utilities for working with them. +Data models are core to our applications and architectures. +With `hof`, you specify their types, shapes, and relations using CUE +and will combine, checkpoint, and track history +as your data models and applications evolve. +They become the source of truth when generating or updating +code across yor technology stack or service fleet. {{}} -The goal `hof` started with was to +`hof datamodel` features enable: -- write your data model in one place, as a source of truth -- automate the of utilities +- checkpoints, history tracking, structural diff calculations +- auto generated database migrations +- client/server version negotiation and request/response upgrades +- data lenses and other transformations +- More than CRUD generation from relations and history +## Base Datamodel +The base data model is any CUE Value with +`#hof: datamodel: root: true` set. -enable scaffolding -of boilerplate code across the application stack and technologies. +{{}} +`hof` leaves those details to the user and module authors. +We also provide some common schemas and extras you can use +along with the main data model features `hof` provides. -You will be able to get -- enforce schemas -- checkpoint a history of changes -- database tables & automatic migrations +## Extending Datamodels +With `hof datamodels` powered by CUE, you will be able to: -- command -- history & checkpointing +- enforce schemas and provide defaults +- share them as modules across applications +- merge or unify them like any CUE value +- enrich them conditionally before code gen -## `hof`'s datamodel schema +The first three points are native to CUE, +so you have all the same capabilities when using `hof`. +Modules are currently only available if `hof`. +CUE is working on their own dependency management +and we are involved in that work as well. +Learn more in the [modules section](/modules/). -### `hof datamodel` command +When it comes to enriching, there are a few ways or places this happens. +`hof`'s base datamodel doesn't provide any structure, +in part because there is not one to rule them all, +but also because we want to let users define their own +while still getting the checkpointing, history, and diff features. -### checkpointing & history +### Providing structure ---- +More often than not, you will want to +provide structure to the datamodel. +[hof/schema/dm/sql](https://github.com/hofstadter-io/hof/blob/_dev/schema/dm/sql/dm.cue) is one example of this. +You define the structure by using the `@history()` +on a collection within your datamodel. +This is a tracking and pivot point. +`hof` will manage history and diffs +on a structural level, at each history point. + + +{{}} + +`hof` will manage each CUE field in a value with `@history()`. +In the above code, this would be + +- each model in `Models`, like `User` +- each field in `User.Fields` +- each view in `Views` + +### Enriching values + +Generally, the input `Datamodel` to a `Generator` will use generic field types. +When generating code, it can be helpful to enrich these values to calculate +template output in CUE rather than the templates themselves. +You can apply these by wrapping the datamodel or applying them in your generator inputs. + +Places where this is helpful are: + +- mapping types, especially collections and relations, to the target language or technology. +- adding various string casings or manipulations + +An example of this is mapping `hof`s `schema/dm/fields` like `uuid` +to a package in languages like Go or Python + +{{}} + +## Datamodel Commands + +{{}} +$ hof dm list (print known data models) +NAME TYPE VERSION STATUS ID +Config object - ok Config +MyDatamodel datamodel - ok datamodel-abc123 + +$ hof dm tree (print the structure of the datamodels) + +$ hof dm diff (prints a tree based diff of the datamodel) + +$ hof dm checkpoint -m "a message about this checkpoint" + +$ hof dm log (prints the log of changes from latest to oldest) + +You can also use the -d & -e flags to subselect datamodels and nested values +{{}} + +## Datamodels and Code Generation + +Datamodels form the basis for input to code generation. +Typically, a generator will require a `dm.Datamodel` and +some other inputs specific to that generator. +You can then use them in the templates like any other value. +The [checkpointing & history](/data-modeling/checkpointing-and-history/) page will cover using these during code gen. + +Here is an example snippet that you would use with our +[supacode generator for full stack applications](https://github.com/hofstadter-io/supacode). -{{< childpages >}} +{{}} diff --git a/docs/content/data-modeling/notes.md b/docs/content/data-modeling/notes.md index eb12fce1c..714d47c50 100644 --- a/docs/content/data-modeling/notes.md +++ b/docs/content/data-modeling/notes.md @@ -4,6 +4,10 @@ title: Notes weight: 100 --- +{{}} +These are some notes that haven't found a home yet or have not been written about. +{{}} + Data Lenses: - [Project Cambria](https://www.inkandswitch.com/cambria/) (main inspiration) diff --git a/docs/content/data-modeling/writing-a-datamodel.md b/docs/content/data-modeling/writing-a-datamodel.md deleted file mode 100644 index e69de29bb..000000000