This commit allows List fields to manage an array of Value Objects.

Specific changes:
- Update `List` field to support Value Object content type
- Allow ValueObjects to output dict even when outside an entity or aggregate
- Support serialization of Value Objects in SQLAlchemy with a custom JSON serializer

This will come handy when embedding List of Value Objects in containers like Events,
which need to enclose all information related to the event within themselves.

docs/guides/domain-definition/fields/container-fields.md

@@ -1,5 +1,34 @@
 # Container Fields
 
+## `ValueObject`
+
+Represents a field that holds a value object. This field is used to embed a
+Value Object within an entity.
+
+**Arguments**
+
+- **`value_object_cls`**: The class of the value object to be embedded.
+
+```python hl_lines="7-15 20"
+{! docs_src/guides/domain-definition/fields/container-fields/003.py !}
+```
+
+You can provide an instance of the Value Object as input to the value object
+field:
+
+```shell hl_lines="2 8"
+In [1]: account = Account(
+   ...:     balance=Balance(currency="USD", amount=100.0),
+   ...:     name="Checking"
+   ...: )
+
+In [2]: account.to_dict()
+Out[2]: 
+{'balance': {'currency': 'USD', 'amount': 100.0},
+ 'name': 'Checking',
+ 'id': '513b8a78-e00f-45ce-bb6f-11ef0cccbec6'}
+```
+
 ## `List`
 
 A field that represents a list of values.
@@ -18,8 +47,8 @@ Accepted field types are `Boolean`, `Date`, `DateTime`, `Float`, `Identifier`,
     specifying `pickled=True`. Databases that don’t support lists simply store
     the field as a python object.
 
-```python hl_lines="9"
-{! docs_src/guides/domain-definition/fields/simple-fields/001.py !}
+```python hl_lines="10"
+{! docs_src/guides/domain-definition/fields/container-fields/001.py !}
 ```
 
 The value is provided as a `list`, and the values in the `list` are validated
@@ -40,6 +69,86 @@ ERROR: Error during initialization: {'roles': ['Invalid value [1, 2]']}
 ValidationError: {'roles': ['Invalid value [1, 2]']}
 ```
 
+### List of Value Objects
+
+A `List` field can even hold a list of `ValueObject` instances. The content of
+the `List` will be persisted as a list of dicts, so the field will behave
+essentially like `List(Dict())` when it comes to persistence. However, it will
+have the added benefit of a validation structure of content within the List.
+
+```python hl_lines="7-12 19"
+{! docs_src/guides/domain-definition/fields/container-fields/004.py !}
+```
+
+```shell
+In [1]: order = Order(
+   ...:         customer=Customer(
+   ...:             name="John Doe",
+   ...:             email="[email protected]",
+   ...:             addresses=[
+   ...:                 Address(street="123 Main St", city="Anytown", state="CA", country="USA"),
+   ...:                 Address(street="321 Side St", city="Anytown", state="CA", country="USA"),
+   ...:             ],
+   ...:         )
+   ...:     )
+
+In [2]: order.to_dict()
+Out[2]: 
+{'customer': {'name': 'John Doe',
+  'email': '[email protected]',
+  'addresses': [{'street': '123 Main St',
+    'city': 'Anytown',
+    'state': 'CA',
+    'country': 'USA'},
+   {'street': '321 Side St',
+    'city': 'Anytown',
+    'state': 'CA',
+    'country': 'USA'}],
+  'id': 'f5c5a750-e9fe-47db-877e-44b7c0ca1dfc'},
+ 'id': '4a9538bf-1eb1-4621-8ced-86bcc4362a51'}
+
+In [3]: domain.repository_for(Order).add(order)
+Out[3]: <Order: Order object (id: 4a9538bf-1eb1-4621-8ced-86bcc4362a51)>
+
+In [4]: retrieved_order = domain.repository_for(Order).get(order.id)
+
+In [5]: len(retrieved_order.customer.addresses)
+Out[5]: 2
+```
+
+Note that unlike `HasMany` fields, you have to supply a new entire list of
+Value Objects if you want to update the field. Appendind to the list will not
+work.
+
+```shell
+In [6]: retrieved_order.customer.addresses.append(
+   ...:     Address(street="456 Side St", city="Anytown", state="CA", country="USA")
+   ...: )
+
+In [7]: domain.repository_for(Order).add(retrieved_order)
+Out[7]: <Order: Order object (id: 4a9538bf-1eb1-4621-8ced-86bcc4362a51)>
+
+In [8]: updated_order = domain.repository_for(Order).get(order.id)
+
+In [9]: len(updated_order.customer.addresses)
+Out[9]: 2
+# This did not work!
+In [10]: updated_order.customer.addresses = [
+    ...:     Address(street="123 Main St", city="Anytown", state="CA", country="USA"),
+    ...:     Address(street="321 Side St", city="Anytown", state="CA", country="USA"),
+    ...:     Address(street="456 Side St", city="Anytown", state="CA", country="USA"),
+    ...: ]
+
+In [11]: domain.repository_for(Order).add(updated_order)
+Out[11]: <Order: Order object (id: 4a9538bf-1eb1-4621-8ced-86bcc4362a51)>
+
+In [12]: refreshed_order = domain.repository_for(Order).get(order.id)
+
+In [13]: len(refreshed_order.customer.addresses)
+Out[13]: 3
+# This worked!
+```
+
 ## `Dict`
 
 A field that represents a dictionary.
@@ -74,32 +183,3 @@ Out[2]:
     by default. You can force it to store the pickled value as a Python object
     by specifying pickled=True. Databases that don’t support lists simply store
     the field as a python object.
-
-## `ValueObject`
-
-Represents a field that holds a value object. This field is used to embed a
-Value Object within an entity.
-
-**Arguments**
-
-- **`value_object_cls`**: The class of the value object to be embedded.
-
-```python hl_lines="7-15 20"
-{! docs_src/guides/domain-definition/fields/container-fields/003.py !}
-```
-
-You can provide an instance of the Value Object as input to the value object
-field:
-
-```shell hl_lines="2 8"
-In [1]: account = Account(
-   ...:     balance=Balance(currency="USD", amount=100.0),
-   ...:     name="Checking"
-   ...: )
-
-In [2]: account.to_dict()
-Out[2]: 
-{'balance': {'currency': 'USD', 'amount': 100.0},
- 'name': 'Checking',
- 'id': '513b8a78-e00f-45ce-bb6f-11ef0cccbec6'}
-```

docs_src/guides/domain-definition/fields/container-fields/004.py

@@ -0,0 +1,24 @@
+from protean import Domain
+from protean.fields import HasOne, String, List, ValueObject
+
+domain = Domain(__file__, load_toml=False)
+
+
+@domain.value_object
+class Address:
+    street = String(max_length=100)
+    city = String(max_length=25)
+    state = String(max_length=25)
+    country = String(max_length=25)
+
+
+@domain.entity(part_of="Order")
+class Customer:
+    name = String(max_length=50, required=True)
+    email = String(max_length=254, required=True)
+    addresses = List(content_type=ValueObject(Address))
+
+
+@domain.aggregate
+class Order:
+    customer = HasOne(Customer)

src/protean/adapters/repository/sqlalchemy.py

@@ -1,6 +1,7 @@
 """Module with repository implementation for SQLAlchemy"""
 
 import copy
+import json
 import logging
 import uuid
 
@@ -18,6 +19,7 @@
 from sqlalchemy.ext.declarative import as_declarative, declared_attr
 from sqlalchemy.types import CHAR, TypeDecorator
 
+from protean.core.value_object import BaseValueObject
 from protean.core.model import BaseModel
 from protean.exceptions import (
     ConfigurationError,
@@ -37,6 +39,7 @@
     List,
     String,
     Text,
+    ValueObject,
 )
 from protean.fields.association import Reference, _ReferenceField
 from protean.fields.embedded import _ShadowField
@@ -112,6 +115,25 @@ def _get_identity_type():
         return sa_types.String
 
 
+def _default(value):
+    """A function that gets called for objects that can’t otherwise be serialized.
+    We handle the special case of Value Objects here.
+
+    `TypeError` is raised for unknown types.
+    """
+    if isinstance(value, BaseValueObject):
+        return value.to_dict()
+    raise TypeError()
+
+
+def _custom_json_dumps(value):
+    """Custom JSON Serializer method to handle the special case of ValueObject deserialization.
+
+    This method is passed into sqlalchemy as a value for param `json_serializer` in the call to `create_engine`.
+    """
+    return json.dumps(value, default=_default)
+
+
 class DeclarativeMeta(sa_dec.DeclarativeMeta, ABCMeta):
     """Metaclass for the Sqlalchemy declarative schema"""
 
@@ -128,6 +150,7 @@ def __init__(cls, classname, bases, dict_):  # noqa: C901
             String: sa_types.String,
             Text: sa_types.Text,
             _ReferenceField: _get_identity_type(),
+            ValueObject: sa_types.PickleType,
         }
 
         def field_mapping_for(field_obj: Field):
@@ -171,9 +194,21 @@ def field_mapping_for(field_obj: Field):
 
                             # Associate Content Type
                             if field_obj.content_type:
-                                type_args.append(
-                                    field_mapping.get(field_obj.content_type)
-                                )
+                                # Treat `ValueObject` differently because it is a field object instance,
+                                #   not a field type class
+                                #
+                                # `ValueObject` instances are essentially treated as `Dict`. If not pickled,
+                                #   they are persisted as JSON.
+                                if isinstance(field_obj.content_type, ValueObject):
+                                    if not field_obj.pickled:
+                                        field_mapping_type = psql.JSON
+                                    else:
+                                        field_mapping_type = sa_types.PickleType
+                                else:
+                                    field_mapping_type = field_mapping.get(
+                                        field_obj.content_type
+                                    )
+                                type_args.append(field_mapping_type)
                             else:
                                 type_args.append(sa_types.Text)
 
@@ -526,7 +561,11 @@ def __init__(self, *args, **kwargs):
 
         kwargs = self._get_database_specific_engine_args()
 
-        self._engine = create_engine(make_url(self.conn_info["database_uri"]), **kwargs)
+        self._engine = create_engine(
+            make_url(self.conn_info["database_uri"]),
+            json_serializer=_custom_json_dumps,
+            **kwargs,
+        )
 
         if self.conn_info["database"] == self.databases.postgresql.value:
             # Nest database tables under a schema, so that we have complete control

src/protean/fields/basic.py

@@ -10,6 +10,7 @@
 
 from protean.exceptions import InvalidOperationError, OutOfContextError, ValidationError
 from protean.fields import Field, validators
+from protean.fields.embedded import ValueObject
 from protean.globals import current_domain
 from protean.utils import IdentityType
 
@@ -233,7 +234,7 @@ def __init__(self, content_type=String, pickled=False, **kwargs):
             String,
             Text,
             Dict,
-        ]:
+        ] and not isinstance(content_type, ValueObject):
             raise ValidationError({"content_type": ["Content type not supported"]})
         self.content_type = content_type
         self.pickled = pickled
@@ -252,18 +253,24 @@ def _cast_to_type(self, value):
         new_value = []
         try:
             for item in value:
-                new_value.append(self.content_type()._load(item))
+                if isinstance(self.content_type, ValueObject):
+                    new_value.append(self.content_type._load(item))
+                else:
+                    new_value.append(self.content_type()._load(item))
         except ValidationError:
             self.fail("invalid_content", value=value)
 
-        if new_value != value:
-            self.fail("invalid_content", value=value)
-
-        return value
+        return new_value
 
     def as_dict(self, value):
         """Return JSON-compatible value of self"""
-        return value
+        new_value = []
+        for item in value:
+            if isinstance(self.content_type, ValueObject):
+                new_value.append(self.content_type.as_dict(item))
+            else:
+                new_value.append(self.content_type().as_dict(item))
+        return new_value
 
 
 class Dict(Field):

src/protean/fields/embedded.py

@@ -100,14 +100,19 @@ def _construct_embedded_fields(self):
                 referenced_as=field_obj.referenced_as,
             )
 
-        # Refresh underlying embedded field names
         for embedded_field in self.embedded_fields.values():
             if embedded_field.referenced_as:
                 embedded_field.attribute_name = embedded_field.referenced_as
             else:
-                embedded_field.attribute_name = (
-                    self.field_name + "_" + embedded_field.field_name
-                )
+                # VO is associated with an aggregate/entity
+                if self.field_name is not None:
+                    # Refresh underlying embedded field names
+                    embedded_field.attribute_name = (
+                        self.field_name + "_" + embedded_field.field_name
+                    )
+                else:
+                    # VO is being used standalone
+                    embedded_field.attribute_name = embedded_field.field_name
 
     def __set_name__(self, entity_cls, name):
         super().__set_name__(entity_cls, name)

tests/adapters/model/sqlalchemy_model/postgresql/test_array_datatype.py

@@ -112,9 +112,10 @@ def test_array_content_type_validation(test_domain):
         {"email": "[email protected]", "roles": [1.0, 2.0]},
         {"email": "[email protected]", "roles": [datetime.now(UTC)]},
     ]:
-        with pytest.raises(ValidationError) as exception:
+        try:
             ArrayUser(**kwargs)
-        assert exception.value.messages["roles"][0].startswith("Invalid value")
+        except ValidationError:
+            pytest.fail("Failed to convert integers into strings in List field type")
 
     model_cls = test_domain.repository_for(IntegerArrayUser)._model
     user = IntegerArrayUser(email="[email protected]", roles=[1, 2])
@@ -126,7 +127,6 @@ def test_array_content_type_validation(test_domain):
 
     for kwargs in [
         {"email": "[email protected]", "roles": ["ADMIN", "USER"]},
-        {"email": "[email protected]", "roles": ["1", "2"]},
         {"email": "[email protected]", "roles": [datetime.now(UTC)]},
     ]:
         with pytest.raises(ValidationError) as exception: