v0.5.0-rc.1

[github-actions[bot]]

v0.5.0-rc.1 - 2024-10-22

Bug Fixes

(ci) Disable sccache when secrets not avail (#874)

• Makes sccache optional so PRs from dependabot and forks can still run
  the test suite.
• Increases sccache allotment to 50g.

---

(cli) Change default installation directory (#873)

• Remplacement PR for fix(cli): default installation directory #843.

• ...

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

---

(docs) Grpc docs (#852)

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(gate) Make __typename returns the variant name on unions (#838)

• Add missing implementation for static injection for parameter
  transformations on the typegate
• Solves
  MET-642:
  Fix the __typename result on union variants: return the variant name
  instead of the parent type name

Migration notes

N/A

Checklist

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(subs) Key collision on redis (#865)

Follow up of #863
When multiple start occurs for redis, some schedules can happen
exactly at the same time resulting into the same identifier (and
leading to an inconsistent state).

This solution simply combines the schedule with the run_id making it
unique instead of using it as is.

mutation AllAtOnce {
  a: start_retry(kwargs: { .. }) # => calls add_schedule( ... date ...)
  b: start_retry(kwargs: { .. })
  c: start_retry(kwargs: { .. })
  d: start_retry(kwargs: { .. }) 
  e: start_retry(kwargs: { .. })
  f: start_retry(kwargs: { .. })
 # ..
}

Migration notes

None

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Fix missing images (#847)

Fix missing images for durable execution blog

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Improve name generation for prisma types (#849)

Solve
MET-657

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Documentation

(blog) Running python with WebAssembly part 1 (#823)

Running python with webassembly (part 1)

Migration notes

None

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Summary by CodeRabbit

Summary by CodeRabbit

• New Features

• Introduced a comprehensive guide for integrating Python runtime with
  WebAssembly (WASI) in the Metatype ecosystem.

• Detailed the advantages of using WebAssembly over Docker for platform
  independence and resource management.

• Provided technical requirements and a refined solution for executing
  Python scripts in a sandboxed environment.

• Expanded vocabulary with new relevant terms for enhanced text
  processing and validation.

• Documentation

• Updated YAML configuration structure in documentation for clarity on
  type gate usage.

---

`/docs/reference/typegraph/client` (#777)

Pre-documentation for the code-first queries feature.

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Features

(dev) Typegraph explorer (#859)

• Add a web version of tree-view which is more interactive
• Enable typegraph serialization without metatype.yml config file

(gate) Empty object as custom scalar (#876)

• Allow empty object on the output without any change
• Just like Int, String, and such, rightfully refer the constant
  {} as a scalar
• Any empty object will now be refered as EmptyObject scalar

Migration notes

None

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(mdk) Overridable templates (#842)

Solve
MET-630

• Make templates in the static sections overridable
  • mdk_rust
  • mdk_python
  • mdk_typescript
• Add a CLI tool to generate extract the default template

Migration notes

No changes needed.

---

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(metagen) Union/either for clients (#857)

• Add union support for the client_xx metagen implementations.

There are still some edge cases especially around variant identification
in the client languages. I tried many things but our hands are tied by
serde. Basically, users will have to be careful when designing their
union types to avoid ambiguity cases. Hopefully,
674
will help there.

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Summary by CodeRabbit

• New Features

• Introduced new methods for rendering union types in both TypeScript
  and Python.

• Enhanced GraphQL query generation with support for multiple union
  types.

• Added a new variants property to the NodeMeta type for improved
  selection handling.

• Bug Fixes

  • Improved error handling for node selections and argument processing.

• Tests

• Updated test cases to reflect schema changes and added new tests for
  client functionality.

(subs) Redis backend (#855)

• Redis Backend base logic port + some improvements
• Moved SUBSTANTIAL_POLL_INTERVAL_SEC and
  SUBSTANTIAL_LEASE_LIFESPAN_SEC to config

Migration notes

• Renamed Backend.fs() and Backend.memory() to Backend.dev_fs()
  and Backend.dev_memory()
• Removed SUBSTANTIAL_RELAUNCH_MS as it was relevant only for purely
  worker-based runs, which rendered the new
  SUBSTANTIAL_POLL_INTERVAL_SEC redundant when an interrupt hits.

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Summary by CodeRabbit

Release Notes

• New Features

• Introduced Redis as a new backend option for enhanced data management.

  • Added Docker Compose configuration for a Redis service.

• Implemented comprehensive testing for Redis functionality and backend
  integration.

• Bug Fixes

  • Improved error handling during backend initialization.

• Documentation

• Updated type definitions for backend configurations to streamline
  Redis integration.

• Chores

• Refactored test cases for clarity and consistency across different
  backend types.

(subs) Retry + timeout on save (#863)

Port and improve retry/timeout.

Migration notes

N/A

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(subs) Child workflows + docs (#867)

Support for child workflows.

Solves MET-689 and MET-668.

Migration notes

Previously

    sub = SubstantialRuntime(backend)
    hello = sub.deno(file="workflow.ts", name="sayHello", deps=[])

    g.expose(
      # each function start, stop, result, ... holds a copy of the workflow data
       start_hello = hello.start(...),
       stop_hello = hello.stop()
    )

This approach relied on workflow files being referenced in each
materializer, but the constructs were too restrictive to support
something like mutation { results(name: "nameManuallyGiven") }.

We now have instead

    file = (
       WorkflowFile
           .deno(file="workflow.ts", deps=[])
           .import_(["sayHello"])
           .build()
    )

    # workflow data are refered only once
    sub = SubstantialRuntime(backend, [file])
    g.expose(
      start_hello = sub.start(...).reduce({ "name": "sayHello" }),
      stop = sub.stop()
    )

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(subs,gate) Substantial integration (#844)

(subs,gate) Port agent concept (#845)

Continuation of #844

Well-defined type comparison semantics (#846)

Solve
MET-655

• Document the type comparison semantics
• Improve the implementation (EnsureSubtypeOf trait)
• Add more test cases for type comparisons

Migration notes

No change is needed.

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Summary by CodeRabbit

Release Notes

• New Features

• Introduced comprehensive type comparison rules and semantics for
  scalar types, optionals, lists, objects, and unions.

• Added support for enumerated types in the type system, allowing for
  more precise type definitions.

• Enhanced parent injection mechanism documentation to clarify type
  compatibility requirements.

• Implemented a new suite of type comparison tests and validation
  mechanisms.

• Bug Fixes

• Improved error reporting and handling in the type validation process.

• Documentation

• Updated and expanded documentation for type comparisons, enumerations,
  and parent injections.

• Tests

  • Added new tests for type comparison and validation scenarios.

Grpc runtime (#819)

Migration notes

...

• The change comes with new or modified tests
• End-user documentation is updated to reflect the change
• Hard-to-understand functions have explanatory comments

---

Python hostcall (#860)

Dead lock on python worker

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Miscellaneous Tasks

Checks/validation on t.either (#868) - BREAKING: Checks/validation on t.either (#868)

Emit a warning or an error when a variant is a subtype of another one.

Migration notes

BREAKING CHANGE: Previously valid typegraph might fail validation.
You will need to fix your types to add some consistency in
t.either/t.union types.

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Update prisma + deno + rust deps (#869)

• Bump deno to 1.46.3
• Update prisma-engines to 5.20
• Update other rust deps.

Closes MET-669 and MET-622 and MET-680.

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Refactor

(gate) Add err msg for missing env vars (#827)

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(gate) Use stream during artifact upload to s3 (#841)

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(gen, doc) Rename mdk to fdk (#851)

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

(sdk) Remove index based names (#848) - BREAKING: remove index based names (#848)

• Replace index based names for types by one that relies on type context
  in graph
• Tests for type deduplication

Migration notes

TODO

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Move as_id out of `TypeNode` (#866) - BREAKING: Move as_id out of `TypeNode` (#866)

Solve
MET-684
and
MET-471

• common/typegraph
• Store the id field in ObjectTypeData instead of in the target
  type (as_id)
  • Add id() method on t.integer and t.string
• typegraph/core
  • Store as_id, injection and policy in TypeRef::attribute
  • Add support for direct and link target in TypeRef
  • Only allow name registration for TypeDef
• Semantics
• Use property name instead of type name in from_parent injection
  source

Migration notes

BREAKING CHANGE
from_parent injections source shall be changed to the key in the
parent t.struct instead of the type name.

Checklist

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

Move injection data to `t.func` (#871)

MET-682

• Move all injection data in ObjectTypeData (i.e. t.func)

MET-656

• Translate reduce to injection specification on t.func

MET-94

• Remove runtime field from TypeNode (#858)

MET-683

• Move runtime-related configs to MaterializerData or
  RuntimeData
• Misc.
• Enable random ports for the typegate (when TG_PORT=0); this will
  work with meta dev with embedded typegate if you set the --gate
  option with port 0.

Migration notes

...

• The change comes with new or modified tests
• Hard-to-understand functions have explanatory comments
• End-user documentation is updated to reflect the change

---

This discussion was created from the release v0.5.0-rc.1.