Check for database provider access on init (#443)

If a provider's configuration in invalid, we want to throw
an exception as early as possible. This commit validates that
a connection is indeed possible during `Domain.init` and throws
a `ConfigurationError` exception on error.

docs/adapters/database/elasticsearch.md

@@ -1,5 +1,7 @@
 # Elasticsearch
 
+## Configuration
+
 To use Elasticsearch as a database provider, use the below configuration setting:
 
 ```toml
@@ -14,18 +16,18 @@ settings = "{'number_of_shards': 3}"
 
 Additional options for finer control:
 
-### NAMESPACE_PREFIX
+### namespace_prefix
 
 Elasticsearch instance are prefixed with the specified string. For example, if
 the namespace prefix is `prod`, the index for aggregate `Person` will be
 `prod-person`.
 
-### NAMESPACE_SEPARATOR
+### namespace_separator
 
 Custom character to join namespace_prefix =n ${Default} yphen(`-`). For example, with `NAMESPACE_SEPARATOR` as `_` and namespace
 prefix as `prod`, the index of aggregate `Person` will be `prod_person`.
 
-### SETTINGS
+### settings
 
 Index settings passed as-is to Elasticsearch instance.
 

docs/adapters/database/postgresql.md

@@ -1,2 +1,40 @@
 # PostgreSQL
 
+The PostgreSQL adapter uses (`SQLAlchemy`)[https://www.sqlalchemy.org/] under
+the covers as the ORM to communicate with the database.
+
+## Configuration
+
+```toml
+[databases.default]
+provider = "postgresql"
+database_uri = "postgresql://postgres:postgres@localhost:5432/postgres"
+```
+
+## Options
+
+### provider
+
+`postgresql` is the provider for PostgreSQL.
+
+### database_uri
+
+Connection string that specifies how to connect to a PostgreSQL database.
+
+### schema
+
+Specifies the database schema to use in the database.
+
+## SQLAlchemy Model
+
+You can supply a custom SQLAlchemy Model in place of the one that Protean
+generates internally, allowing you full customization.
+
+```python hl_lines="8-11 20-23"
+{! docs_src/adapters/database/postgresql/001.py !}
+```
+
+!!!note
+    The column names specified in the model should exactly match the attribute
+    names of the Aggregate or Entity it represents.
+

docs/adapters/internals.md

@@ -0,0 +1,126 @@
+# Internals
+
+## Database
+
+A database adapter has four components that work together to make database
+communication possible. Each adapter will override the corresponding base
+classes for each of these components with its own implementation.
+
+!!!note
+    An exception to this rule is the `Session` class. It may be preferable to
+    use the `Session` structures provided by the database technology as-is. For
+    example, `PostgreSQL` adapter that is powered by `SQLAlchemy` simply uses
+    (and returns) the [sqlalchemy.orm.Session](https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session)
+    object provided by `SQLAlchemy`.
+
+### Provider
+
+**Base Class**: `protean.port.database.BaseProvider`
+
+The Provider is the main component responsible for interfacing with the
+specific database technology. It contains the configuration and setup needed
+to connect to the database, and it provides methods for interacting with the
+database at a high level.
+
+The Provider acts as the bridge between the application and the database,
+ensuring that the necessary connections are established and maintained.
+It also handles any database-specific nuances, such as connection pooling,
+transaction management, and query optimization.
+
+To add a new provider, subclass from the Provider Base class and implement
+methods for your database technology.
+
+### Data Access Object (DAO)
+
+**Base Class**: `protean.port.database.BaseDAO`
+
+The Data Access Object (DAO) is responsible for encapsulating the details of
+the database interactions. It provides an abstract interface to the database,
+hiding the complexity of the underlying database operations from the rest of
+the application.
+
+The DAO defines methods for CRUD (Create, Read, Update, Delete) operations
+and other database queries, translating them into the appropriate SQL or
+database-specific commands.
+
+By using DAOs, the application code remains clean and decoupled from the
+database logic. DAOs also work in conjunction in [lookups](#lookups) to
+establish a query API that works across various adapters.
+
+To add a new DAO for your provider, subclass from the DAO Base class and
+implement methods for your database technology.
+
+### Session
+
+The Session component manages the lifecycle of database transactions. It is
+responsible for opening and closing connections, beginning and committing
+transactions, and ensuring data consistency.
+
+The Session provides a context for performing database operations,
+ensuring that all database interactions within a transaction are properly
+managed and that resources are released appropriately. This component is
+crucial for maintaining transactional integrity and for handling rollback
+scenarios in case of errors.
+
+The Session object is usually constructed and provided by the database orm
+or technology package. For example, `PostgreSQL` adapter depends on the
+[sqlalchemy.orm.Session](https://docs.sqlalchemy.org/en/20/orm/session_api.html#sqlalchemy.orm.Session)
+object provided by `SQLAlchemy`.
+
+### Model
+
+**Base Class**: `protean.core.model.BaseModel`
+
+The Model represents the domain entities that are persisted in the database.
+It defines the structure of the data, including fields, relationships, and
+constraints. It is also a high-level abstraction for working with database
+records, allowing you to interact with the data using Python objects rather
+than raw queries.
+
+Models are typically defined using a schema or an ORM
+(Object-Relational Mapping) framework that maps the database tables to
+Python objects.
+
+Implementing a model for your database technology can be slightly involved,
+as ORM packages can heavily depend upon interal structures. Every
+database package is structured differently. Consult existing models for
+PostgreSQL
+([SQLAlchemy](https://github.com/proteanhq/protean/blob/main/src/protean/adapters/repository/sqlalchemy.py#L139))
+and Elasticsearch
+([Elasticsearch](https://github.com/proteanhq/protean/blob/main/src/protean/adapters/repository/elasticsearch.py#L43))
+for examples on constructing model classes.
+
+Once you define a base model for your provider, Protean auto-generates the model
+class for aggregates or entities when needed. You can control this behavior
+by supplying an explicit hand-crafted model class for your entity.
+
+<!-- FIXME Add link to customizing models -->
+
+### Lookups
+
+Lookups in Protean are mechanisms used to query the database based on certain
+criteria. They are customized implementations of different types of filters
+that help filter and retrieve data from the database, making it easier to
+perform complex queries without writing raw SQL. Lookups are typically used in
+the DAO layer to fetch records that match specific conditions.
+
+Refer to the section on
+[querying](../guides/change-state/retrieve-aggregates.md#advanced-filtering-criteria)
+aggregates for examples of lookups. Consult the documentation on your specific
+chosen adapter for more information. The adapter may support specialized
+lookups for efficient or technology-specific queries.
+
+## Initialization
+
+Adapters are initialized as part of Domain initialization. Protean creates
+the provider objects in the adapter and establishes connections with the
+underlying infrastructure.
+
+If a connection cannot be established for whatever reason, the Protean
+initialization procedure immediately halts and exits with an error message:
+
+```python hl_lines="8"
+{! docs_src/adapters/001.py !}
+```
+
+1. `foobar` database does not exist on port 5444

docs/start-here.md

@@ -11,7 +11,7 @@ Welcome to Protean! Whether you're a seasoned developer or new to the framework,
 Ready to dive in? The first step is to get Protean up and running on your system. The [Installation guide](./guides/getting-started/installation.md) will take you through a brief tour to setup Protean locally.
 
 ## Core Concepts
-Time to get acquainted with the building blocks of Protean. Get to know the driving principles and core ideas that shape this framework in [Introduction](./core-concepts/index.md). In [Building Blocks](./core-concepts/building-blocks/index.md) explore key DDD concepts like Aggregates, Repositories, Events, and more, to understand the core structures of Protean.
+Get to know the driving principles and core ideas that shape this framework in [Introduction](./core-concepts/index.md). In [Building Blocks](./core-concepts/building-blocks/index.md), explore key DDD concepts like Aggregates, Repositories, Events, and more, to understand the core structures of Protean.
 
 ## Building with Protean
 

docs_src/adapters/001.py

@@ -0,0 +1,16 @@
+from protean import Domain
+
+domain = Domain(__file__, load_toml=False)
+
+# Non-existent database
+domain.config["databases"]["default"] = {
+    "provider": "postgresql",
+    "database_uri": "postgresql://postgres:postgres@localhost:5444/foobar",  # (1)
+}
+
+domain.init(traverse=False)
+
+# Output
+#
+# protean.exceptions.ConfigurationError:
+# Could not connect to database at postgresql://postgres:postgres@localhost:5444/foobar

docs_src/adapters/database/postgresql/001.py

@@ -0,0 +1,29 @@
+import sqlalchemy as sa
+
+from protean import Domain
+from protean.adapters.repository.sqlalchemy import SqlalchemyModel
+from protean.fields import Integer, String
+
+domain = Domain(__file__, load_toml=False)
+domain.config["databases"]["default"] = {
+    "provider": "postgresql",
+    "database_uri": "postgresql://postgres:postgres@localhost:5432/postgres",
+}
+
+
+@domain.aggregate
+class Provider:
+    name = String()
+    age = Integer()
+
+
+@domain.model(part_of=Provider)
+class ProviderCustomModel:
+    name = sa.Column(sa.Text)
+    age = sa.Column(sa.Integer)
+
+
+domain.init()
+with domain.domain_context():
+    model_cls = domain.repository_for(Provider)._model
+    assert issubclass(model_cls, SqlalchemyModel)

mkdocs.yml

@@ -216,6 +216,7 @@ nav:
   # - faq.md
   - Adapters:
     - adapters/index.md
+    - adapters/internals.md
     - Database:
       - adapters/database/index.md
       - adapters/database/postgresql.md

src/protean/adapters/repository/__init__.py

@@ -109,6 +109,13 @@ def _initialize(self):
                 )
                 provider = provider_cls(provider_name, self.domain, conn_info)
 
+                # Initialize a connection to check if everything is ok
+                conn = provider.is_alive()
+                if not conn:
+                    raise ConfigurationError(
+                        f"Could not connect to database at {conn_info['database_uri']}"
+                    )
+
                 provider_objects[provider_name] = provider
 
         self._providers = provider_objects

src/protean/adapters/repository/elasticsearch.py

@@ -376,6 +376,11 @@ def get_connection(self):
             verify_certs=self.conn_info.get("VERIFY_CERTS", False),
         )
 
+    def is_alive(self) -> bool:
+        """Check if the connection is alive"""
+        conn = self.get_connection()
+        return conn.ping()
+
     def get_dao(self, entity_cls, model_cls):
         """Return a DAO object configured with a live connection"""
         return ElasticsearchDAO(self.domain, self, entity_cls, model_cls)

src/protean/adapters/repository/memory.py

@@ -119,6 +119,10 @@ def get_connection(self, session_cls=None):
         """Return the dictionary database object"""
         return MemorySession(self, new_connection=True)
 
+    def is_alive(self) -> bool:
+        """Check if the connection is alive"""
+        return True
+
     def _data_reset(self):
         """Reset data"""
         self._databases = defaultdict(dict)

src/protean/adapters/repository/sqlalchemy.py

@@ -619,6 +619,15 @@ def get_connection(self, session_cls=None):
 
         return conn
 
+    def is_alive(self) -> bool:
+        """Check if the connection to the database is alive"""
+        try:
+            conn = self.get_connection()
+            conn.execute(text("SELECT 1"))
+            return True
+        except DatabaseError:
+            return False
+
     def _data_reset(self):
         conn = self._engine.connect()