Async server bug fixes and general enhancements (#406)

Bug fixes and general enhancements to Engine and Subscription

- Update read position before shutting down subscriptions
- Stop server processing on exceptions
- Allow overriding exception handling in the `handle_error` method in Command and Event handlers
- Associate `exit_code` with engine run
- Add comprehensive test cases
- Add documentation and comments

This PR also has changes to:
- [Add option to protean shell to control traversing domain directory](9cd428c)
- [Make value objects immutable](8b50c84)
- [Add validations to disallow unique or identifier fields in VO](e469313)
- [Improve field's repr and str](e4cd078)
- [Fix bug with custom validators in Value Objects](11a21a9)

docs/guides/cli/shell.md

@@ -13,10 +13,11 @@ protean shell [OPTIONS]
 
 ## Options
 
-| Option      | Description                               | Default |
-|-------------|-------------------------------------------|---------|
-| `--domain`  | Sets the domain context for the shell.    | `.`     |
-| `--help`    | Shows the help message and exits.         |         |
+| Option        | Description                               | Default |
+|---------------|-------------------------------------------|---------|
+| `--domain`    | Sets the domain context for the shell.    | `.`     |
+| `--traverse`  | Auto-traverse domain elements             | `False` |
+| `--help`      | Shows the help message and exits.         |         |
 
 ## Launching the Shell
 
@@ -37,4 +38,14 @@ protean shell --domain auth
 
 This command will initiate the shell in the context of `auth` domain, allowing
 you to perform domain-specific operations more conveniently. Read [Domain 
-Discovery](discovery.md) for options to specify the domain.
+Discovery](discovery.md) for options to specify the domain.
+
+### Traversing subdirectories
+
+By default, only the domain and elments in the specified module will be loaded
+into the shell context. If you want traverse files in the folder and its
+subdirectories, you can specify the `--traverse` option.
+
+```shell
+protean shell --domain auth --traverse
+```

docs/guides/compose-a-domain/activate-domain.md

@@ -4,18 +4,77 @@ A `Domain` in protean is always associated with a domain context, which can be
 used to bind an domain object implicitly to the current thread or greenlet. We
 refer to the act of binding the domain object as **activating the domain**.
 
-<!-- FIXME Insert link to fullblown DomainContext documentation -->
-A `DomainContext` helps manage the active domain object for the duration of a
-thread's execution. It also provides a namespace for storing data for the
-duration a domain context is active. 
+## Domain Context
 
-You activate a domain by pushing up its context to the top of the domain stack. 
+A Protean Domain object has attributes, such as config, that are useful to
+access within domain elements. However, importing the domain instance within
+the modules in your project is prone to circular import issues.
 
-## Using a Context Manager
+Protean solves this issue with the domain context. Rather than passing the
+domain around to each method, or referring to a domain directly, you can use
+the `current_domain` proxy instead. The `current_domain` proxy,
+which points to the domain handling the current activity. 
+
+The `DomainContext` helps manage the active domain object for the duration of a
+thread's execution. The domain context keeps track of the domain-level data
+during the lifetime of a domain object, and is used while processing handlers,
+CLI commands, or other activities. 
+
+## Storing Data
+
+The domain context also provides a `g` object for storing data. It is a simple
+namespace object that has the same lifetime as an domain context.
+
+!!! note
+    The `g` name stands for "global", but that is referring to the data
+    being global within a context. The data on `g` is lost after the context
+    ends, and it is not an appropriate place to store data between domain
+    calls. Use a session or a database to store data across domain model calls.
+
+A common use for g is to manage resources during a domain call.
+
+1. `get_X()` creates resource X if it does not exist, caching it as g.X.
+
+2. `teardown_X()` closes or otherwise deallocates the resource if it exists.
+It is registered as a `teardown_domain_context()` handler.
+
+Using this pattern, you can, for example, manage a file connection for the
+lifetime of a domain call:
+
+```python
+from protean.globals import g
+
+def get_log():
+    if 'log' not in g:
+        g.log = open_log_file()
+
+    return g.log
+
+@domain.teardown_appcontext
+def teardown_log_file(exception):
+    file_obj = g.pop('log', None)
+
+    if not file_obj.closed:
+        file_obj.close()
+```
+
+Now, every call to `get_log()` during the domain call will return the same file
+object, and it will be closed automatically at the end of processing.
+
+## Pushing up the Domain Context
+
+A Protean domain is activated close to the application's entrypoint, like an
+API request. In many other cases, like Protean's server processing commands and
+events, or the CLI accessing the domain, Protean automatically activates a
+domain context for the duration of the task.
+
+You activate a domain by pushing up its context to the top of the domain stack:
+
+### With Context Manager
 Protean provides a helpful context manager to nest the domain operations
 under.
 
-```Python hl_lines="18-21"
+```python hl_lines="18-21"
 {! docs_src/guides/composing-a-domain/018.py !}
 ```
 
@@ -27,7 +86,7 @@ This is a convenient pattern to use in conjunction with most API frameworks.
 The domain’s context is pushed up at the beginning of a request and popped out 
 once the request is processed.
 
-## Without the Context Manager
+### Manually
 
 You can also activate the context manually by using the `push` and `pop`
 methods of the domain context:

docs/guides/compose-a-domain/element-decorators.md

@@ -6,7 +6,7 @@ Each element is explored in detail in its own section.
 
 ## `Domain.aggregate`
 
-```Python hl_lines="7-11"
+```python hl_lines="7-11"
 {! docs_src/guides/composing-a-domain/002.py !}
 ```
 
@@ -15,7 +15,7 @@ Read more at Aggregates.
 
 ## `Domain.entity`
 
-```Python hl_lines="14-17"
+```python hl_lines="14-17"
 {! docs_src/guides/composing-a-domain/003.py !}
 ```
 
@@ -24,7 +24,7 @@ Read more at Entities.
 
 ## `Domain.value_object`
 
-```Python hl_lines="7-15 23"
+```python hl_lines="7-15 23"
 {! docs_src/guides/composing-a-domain/004.py !}
 ```
 
@@ -33,7 +33,7 @@ Read more at Value Objects.
 
 ## `Domain.domain_service`
 
-```Python hl_lines="33-37"
+```python hl_lines="33-37"
 {! docs_src/guides/composing-a-domain/005.py !}
 ```
 
@@ -42,7 +42,7 @@ Read more at Domain Services.
 
 ## `Domain.event_sourced_aggregate`
 
-```Python hl_lines="7-10"
+```python hl_lines="7-10"
 {! docs_src/guides/composing-a-domain/006.py !}
 ```
 
@@ -51,7 +51,7 @@ Read more at Event Sourced Aggregates.
 
 ## `Domain.command`
 
-```Python hl_lines="18-23"
+```python hl_lines="18-23"
 {! docs_src/guides/composing-a-domain/007.py !}
 ```
 
@@ -60,7 +60,7 @@ Read more at Commands.
 
 ## `Domain.command_handler`
 
-```Python hl_lines="26-34"
+```python hl_lines="26-34"
 {! docs_src/guides/composing-a-domain/008.py !}
 ```
 
@@ -69,7 +69,7 @@ Read more at Command Handlers.
 
 ## `Domain.event`
 
-```Python hl_lines="18-23"
+```python hl_lines="18-23"
 {! docs_src/guides/composing-a-domain/009.py !}
 ```
 
@@ -78,7 +78,7 @@ Read more at Events.
 
 ## `Domain.event_handler`
 
-```Python hl_lines="28-32"
+```python hl_lines="28-32"
 {! docs_src/guides/composing-a-domain/010.py !}
 ```
 
@@ -87,7 +87,7 @@ Read more at Event Handlers.
 
 ## `Domain.model`
 
-```Python hl_lines="18-25"
+```python hl_lines="18-25"
 {! docs_src/guides/composing-a-domain/011.py !}
 ```
 
@@ -96,7 +96,7 @@ Read more at Models.
 
 ## `Domain.repository`
 
-```Python hl_lines="17-22"
+```python hl_lines="17-22"
 {! docs_src/guides/composing-a-domain/012.py !}
 ```
 
@@ -105,7 +105,7 @@ Read more at Repositories.
 
 ## `Domain.view`
 
-```Python hl_lines="20-24"
+```python hl_lines="20-24"
 {! docs_src/guides/composing-a-domain/013.py !}
 ```
 

docs/guides/compose-a-domain/initialize-domain.md

@@ -27,7 +27,7 @@ with Protean explicitly.
 Protean constructs a graph of all elements registered with a domain and
 exposes them in a registry.
 
-```Python hl_lines="28-35"
+```python hl_lines="28-35"
 {! docs_src/guides/composing-a-domain/016.py !}
 ```
 
@@ -41,7 +41,7 @@ a database that actually persists data.
 Calling `domain.init()` establishes connectivity with the underlying infra,
 testing access, and making them available for use by the rest of the system. 
 
-```Python hl_lines="5-11"
+```python hl_lines="5-11"
 {! docs_src/guides/composing-a-domain/017.py !}
 ```
 

docs/guides/compose-a-domain/object-model.md

@@ -17,14 +17,14 @@ Additional options can be passed to a domain element in two ways:
 
 You can specify options within a nested inner class called `Meta`:
 
-```Python hl_lines="13-14"
+```python hl_lines="13-14"
 {! docs_src/guides/composing-a-domain/020.py !}
 ```
 
 - **Decorator Parameters**
 
 You can also pass options as parameters to the decorator:
 
-```Python hl_lines="7"
+```python hl_lines="7"
 {! docs_src/guides/composing-a-domain/021.py !}
 ```

docs/guides/compose-a-domain/register-elements.md

@@ -5,7 +5,7 @@ the domain.
 
 ## With decorators
 
-```Python hl_lines="7-11"
+```python hl_lines="7-11"
 {! docs_src/guides/composing-a-domain/002.py !}
 ```
 
@@ -16,7 +16,7 @@ A full list of domain decorators along with examples are available in the
 
 You can also choose to register elements manually.
 
-```Python hl_lines="7-13"
+```python hl_lines="7-13"
 {! docs_src/guides/composing-a-domain/014.py !}
 ```
 
@@ -31,7 +31,7 @@ of element in Protean has a distinct base class of its own.
 There might be additional options you will pass in a `Meta` inner class,
 depending upon the element being registered.
 
-```Python hl_lines="12-13"
+```python hl_lines="12-13"
 {! docs_src/guides/composing-a-domain/015.py !}
 ```
 

docs/guides/compose-a-domain/when-to-compose.md

@@ -16,6 +16,6 @@ You would compose the domain along with the app object, and activate it (push
 up the context) before processing a request.
 
 
-```Python hl_lines="29 33 35 38"
+```python hl_lines="29 33 35 38"
 {! docs_src/guides/composing-a-domain/019.py !}
 ```

src/protean/cli/__init__.py

@@ -15,6 +15,7 @@
   Also see (1) from http://click.pocoo.org/5/setuptools/#setuptools-integration
 """
 
+import logging
 import subprocess
 
 from enum import Enum
@@ -32,6 +33,8 @@
 from protean.exceptions import NoDomainException
 from protean.utils.domain_discovery import derive_domain
 
+logger = logging.getLogger(__name__)
+
 # Create the Typer app
 #   `no_args_is_help=True` will show the help message when no arguments are passed
 app = typer.Typer(no_args_is_help=True)
@@ -125,20 +128,26 @@ def test(
 
 @app.command()
 def server(
-    domain: Annotated[str, typer.Option("--domain")] = ".",
+    domain: Annotated[str, typer.Option()] = ".",
     test_mode: Annotated[Optional[bool], typer.Option()] = False,
+    debug: Annotated[Optional[bool], typer.Option()] = False,
 ):
     """Run Async Background Server"""
     # FIXME Accept MAX_WORKERS as command-line input as well
-    from protean.server import Engine
-
-    domain = derive_domain(domain)
-    if not domain:
-        raise NoDomainException(
+    try:
+        domain = derive_domain(domain)
+    except NoDomainException:
+        logger.error(
             "Could not locate a Protean domain. You should provide a domain in"
             '"PROTEAN_DOMAIN" environment variable or pass a domain file in options '
             'and a "domain.py" module was not found in the current directory.'
         )
+        raise typer.Abort()
 
-    engine = Engine(domain, test_mode=test_mode)
+    from protean.server import Engine
+
+    engine = Engine(domain, test_mode=test_mode, debug=debug)
     engine.run()
+
+    if engine.exit_code != 0:
+        raise typer.Exit(code=engine.exit_code)

src/protean/cli/generate.py

@@ -1,10 +1,14 @@
+import logging
+
 import typer
 
 from typing_extensions import Annotated
 
 from protean.exceptions import NoDomainException
 from protean.utils.domain_discovery import derive_domain
 
+logger = logging.getLogger(__name__)
+
 app = typer.Typer(no_args_is_help=True)
 
 
@@ -28,13 +32,16 @@ def docker_compose(
     domain: Annotated[str, typer.Option()] = ".",
 ):
     """Generate a `docker-compose.yml` from Domain config"""
-    print(f"Generating docker-compose.yml for domain at {domain}")
-    domain_instance = derive_domain(domain)
-    if not domain_instance:
-        raise NoDomainException(
+    try:
+        domain_instance = derive_domain(domain)
+    except NoDomainException:
+        logger.error(
             "Could not locate a Protean domain. You should provide a domain in"
             '"PROTEAN_DOMAIN" environment variable or pass a domain file in options'
         )
+        raise typer.Abort()
+
+    print(f"Generating docker-compose.yml for domain at {domain}")
 
     with domain_instance.domain_context():
         domain_instance.init()