Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Async Factory Support #803

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CREDITS
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ The project has received contributions from (in alphabetical order):
* Martin Bächtold <[email protected]> (https://github.com/mbaechtold)
* Michael Joseph <[email protected]>
* Mikhail Korobov <[email protected]>
* Nadege Michel <[email protected]> (https://github.com/nadege)
* Oleg Pidsadnyi <[email protected]>
* Omer <[email protected]>
* Pauly Fenwar <[email protected]>
Expand Down
1 change: 1 addition & 0 deletions docs/changelog.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ ChangeLog
- Add support for Django 4.2
- Add support for Django 5.0
- Add support for Python 3.12
- Add support for asynchronous factories

*Bugfix:*

Expand Down
37 changes: 37 additions & 0 deletions docs/introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,7 @@ All factories support two built-in strategies:

* ``build`` provides a local object
* ``create`` instantiates a local object, and saves it to the database.
* ``create_async`` similar to ``create`` but can run asynchronous code.

.. note:: For 1.X versions, the ``create`` will actually call ``AssociatedClass.objects.create``,
as for a Django model.
Expand Down Expand Up @@ -370,3 +371,39 @@ Calling a :class:`~factory.Factory` subclass will provide an object through the


The default strategy can be changed by setting the ``class Meta`` :attr:`~factory.FactoryOptions.strategy` attribute.


Asynchronous Factories
----------------------

You need to override the asynchronous method :meth:`factory.Factory._create_model_async`
to define how your objects are created and saved to the database.

Then, you can then either:

* use :meth:`factory.Factory.create_async`
* inherit from :class:`factory.AsyncFactory` instead of :class:`~factory.Factory`
nadege marked this conversation as resolved.
Show resolved Hide resolved
to make :attr:`enums.ASYNC_CREATE_STRATEGY` the default strategy and then call the factory.

.. code-block:: python

class MyFactory(factory.AsyncFactory):
# ...

@classmethod
async def _create_model_async(cls, model_class, *args, **kwargs):
await model_class.write_in_db(*args, **kwargs)

.. code-block:: pycon

>>> MyFactory.create()
<MyClass: X (saved)>

>>> MyFactory.create_async()
<MyClass: X (saved using async code)>

>>> MyFactory.build()
<MyClass: X (unsaved)>

>>> MyFactory() # equivalent to MyFactory.create_async()
<MyClass: X (saved using async code)>
53 changes: 51 additions & 2 deletions docs/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ Attributes and methods

.. classmethod:: build_batch(cls, size, **kwargs)

Provides a list of :obj:`size` instances from the :class:`Factory`,
Provides a list of ``size`` instances from the :class:`Factory`,
through the 'build' strategy.


Expand All @@ -213,10 +213,20 @@ Attributes and methods

.. classmethod:: create_batch(cls, size, **kwargs)

Provides a list of :obj:`size` instances from the :class:`Factory`,
Provides a list of ``size`` instances from the :class:`Factory`,
through the 'create' strategy.


.. classmethod:: create_async(cls, **kwargs)

Provides a new object, using the ``create_async`` strategy.

.. classmethod:: create_async_batch(cls, size, **kwargs)

Asynchronously provides a list of ``size`` instances from the :class:`Factory`,
through the ``create_async`` strategy.


.. classmethod:: stub(cls, **kwargs)

Provides a new stub
Expand Down Expand Up @@ -319,6 +329,32 @@ Attributes and methods

.. OHAI_VIM*


.. classmethod:: _create_model_async(cls, model_class, *args, **kwargs)

.. OHAI_VIM*

The :meth:`_create_model_async` method is called whenever an instance needs to be
created asynchronously.
It receives the same arguments as :meth:`_build` and :meth:`_create`.

Subclasses may override this for specific persistence backends:

.. code-block:: python

class BaseBackendFactory(factory.Factory):
class Meta:
abstract = True # Optional

@classmethod
async def _create_model_async(cls, model_class, *args, **kwargs):
obj = model_class(*args, **kwargs)
await obj.async_save()
return obj

.. OHAI_VIM*


.. classmethod:: _after_postgeneration(cls, obj, create, results=None)

:arg object obj: The object just generated
Expand Down Expand Up @@ -377,6 +413,11 @@ Attributes and methods
factory in the chain.


.. class:: AsyncFactory

Similar to the :class:`Factory` class but with ``create_async`` as default strategy.


.. _parameters:

Parameters
Expand Down Expand Up @@ -588,6 +629,14 @@ factory_boy supports two main strategies for generating instances, plus stubs.
:class:`Factory` wasn't overridden.


.. data:: ASYNC_CREATE_STRATEGY

The ``create_async`` strategy is similar to the ``create`` strategy but asynchronous.

This is the default strategy for subclasses of :class:`AsyncFactory`.

Default behavior is to call :meth:`~Factory._create`, this can be overridden in :meth:`Factory._create_model_async`.

.. function:: use_strategy(strategy)

.. deprecated:: 3.2
Expand Down
5 changes: 4 additions & 1 deletion factory/__init__.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Copyright: See the LICENSE file.

from .base import (
AsyncFactory,
BaseDictFactory,
BaseListFactory,
DictFactory,
Expand Down Expand Up @@ -28,14 +29,16 @@
Trait,
Transformer,
)
from .enums import BUILD_STRATEGY, CREATE_STRATEGY, STUB_STRATEGY
from .enums import ASYNC_CREATE_STRATEGY, BUILD_STRATEGY, CREATE_STRATEGY, STUB_STRATEGY
from .errors import FactoryError
from .faker import Faker
from .helpers import (
build,
build_batch,
container_attribute,
create,
create_async,
create_async_batch,
create_batch,
debug,
generate,
Expand Down
29 changes: 29 additions & 0 deletions factory/alchemy.py
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,32 @@ def _save(cls, model_class, session, args, kwargs):
elif session_persistence == SESSION_PERSISTENCE_COMMIT:
session.commit()
return obj


class SQLAlchemyModelAsyncFactory(SQLAlchemyModelFactory, base.AsyncFactory):
"""Async Factory for SQLAlchemy models. """

class Meta:
abstract = True

@classmethod
async def _create_model_async(cls, model_class, *args, **kwargs):
session = cls._meta.sqlalchemy_session

if session is None:
raise RuntimeError("No session provided.")

async with session.begin():
return await cls._save(model_class, session, args, kwargs)

@classmethod
async def _save(cls, model_class, session, args, kwargs):
session_persistence = cls._meta.sqlalchemy_session_persistence

obj = model_class(*args, **kwargs)
session.add(obj)
if session_persistence == SESSION_PERSISTENCE_FLUSH:
await session.flush()
elif session_persistence == SESSION_PERSISTENCE_COMMIT:
await session.commit()
return obj
92 changes: 87 additions & 5 deletions factory/base.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
# Copyright: See the LICENSE file.


import asyncio
import collections
import inspect
import logging
import warnings

Expand Down Expand Up @@ -38,6 +39,8 @@ def __call__(cls, **kwargs):
return cls.build(**kwargs)
elif cls._meta.strategy == enums.CREATE_STRATEGY:
return cls.create(**kwargs)
elif cls._meta.strategy == enums.ASYNC_CREATE_STRATEGY:
return cls.create_async(**kwargs)
elif cls._meta.strategy == enums.STUB_STRATEGY:
return cls.stub(**kwargs)
else:
Expand Down Expand Up @@ -315,14 +318,16 @@ def instantiate(self, step, args, kwargs):
return self.factory._build(model, *args, **kwargs)
elif step.builder.strategy == enums.CREATE_STRATEGY:
return self.factory._create(model, *args, **kwargs)
elif step.builder.strategy == enums.ASYNC_CREATE_STRATEGY:
return self.factory._create_async(model, *args, **kwargs)
else:
assert step.builder.strategy == enums.STUB_STRATEGY
return StubObject(**kwargs)

def use_postgeneration_results(self, step, instance, results):
self.factory._after_postgeneration(
instance,
create=step.builder.strategy == enums.CREATE_STRATEGY,
create=step.builder.strategy in (enums.CREATE_STRATEGY, enums.ASYNC_CREATE_STRATEGY),
results=results,
)

Expand Down Expand Up @@ -505,6 +510,38 @@ def _create(cls, model_class, *args, **kwargs):
"""
return model_class(*args, **kwargs)

@classmethod
def _create_async(cls, model_class, *args, **kwargs):
"""Actually create a Task that will create an instance of the
model_class when awaited.

Args:
model_class (type): the class for which an instance should be
created
args (tuple): arguments to use when creating the class
kwargs (dict): keyword arguments to use when creating the class
"""

async def maker_coroutine():
for key, value in kwargs.items():
# When using SubFactory, you'll have a Task in the corresponding kwarg
# Await tasks to pass model instances instead, not the Task
if inspect.isawaitable(value):
kwargs[key] = await value

return await cls._create_model_async(model_class, *args, **kwargs)

# A Task can be awaited multiple times, unlike a coroutine.
# Useful when a factory and a subfactory must share the same object.
return asyncio.create_task(maker_coroutine())

@classmethod
async def _create_model_async(cls, model_class, *args, **kwargs):
"""
By default just run the synchronous create function
"""
return cls._create(model_class, *args, **kwargs)

@classmethod
def build(cls, **kwargs):
"""Build an instance of the associated class, with overridden attrs."""
Expand Down Expand Up @@ -539,6 +576,25 @@ def create_batch(cls, size, **kwargs):
"""
return [cls.create(**kwargs) for _ in range(size)]

@classmethod
def create_async(cls, **kwargs):
"""Create a Task that will return an instance of the associated class,
with overridden attrs, when awaited."""
return cls._generate(enums.ASYNC_CREATE_STRATEGY, kwargs)

@classmethod
async def create_async_batch(cls, size, **kwargs):
"""Create a batch of instances of the given class, with overridden attrs,
using async creation.

Args:
size (int): the number of instances to create

Returns:
A list containing the created instances
"""
return [await cls.create_async(**kwargs) for _ in range(size)]

@classmethod
def stub(cls, **kwargs):
"""Retrieve a stub of the associated class, with overridden attrs.
Expand Down Expand Up @@ -573,7 +629,9 @@ def generate(cls, strategy, **kwargs):
Returns:
object: the generated instance
"""
assert strategy in (enums.STUB_STRATEGY, enums.BUILD_STRATEGY, enums.CREATE_STRATEGY)
assert strategy in (
enums.STUB_STRATEGY, enums.BUILD_STRATEGY, enums.CREATE_STRATEGY, enums.ASYNC_CREATE_STRATEGY,
)
action = getattr(cls, strategy)
return action(**kwargs)

Expand All @@ -582,7 +640,7 @@ def generate_batch(cls, strategy, size, **kwargs):
"""Generate a batch of instances.

The instances will be created with the given strategy (one of
BUILD_STRATEGY, CREATE_STRATEGY, STUB_STRATEGY).
BUILD_STRATEGY, CREATE_STRATEGY, STUB_STRATEGY, ASYNC_CREATE_STRATEGY).

Args:
strategy (str): the strategy to use for generating the instance.
Expand All @@ -591,7 +649,9 @@ def generate_batch(cls, strategy, size, **kwargs):
Returns:
object list: the generated instances
"""
assert strategy in (enums.STUB_STRATEGY, enums.BUILD_STRATEGY, enums.CREATE_STRATEGY)
assert strategy in (
enums.STUB_STRATEGY, enums.BUILD_STRATEGY, enums.CREATE_STRATEGY, enums.ASYNC_CREATE_STRATEGY
)
batch_action = getattr(cls, '%s_batch' % strategy)
return batch_action(size, **kwargs)

Expand Down Expand Up @@ -642,6 +702,16 @@ class Meta(BaseMeta):
Factory.AssociatedClassError = errors.AssociatedClassError


class AsyncFactory(Factory):
"""Same as Factory but creation is async by default

ex: await MyAsyncFactory()
"""

class Meta(BaseMeta):
strategy = enums.ASYNC_CREATE_STRATEGY


class StubObject:
"""A generic container."""
def __init__(self, **kwargs):
Expand All @@ -663,6 +733,10 @@ def build(cls, **kwargs):
def create(cls, **kwargs):
raise errors.UnsupportedStrategy()

@classmethod
def create_async(cls, **kwargs):
raise errors.UnsupportedStrategy()


class BaseDictFactory(Factory):
"""Factory for dictionary-like classes."""
Expand All @@ -680,6 +754,10 @@ def _build(cls, model_class, *args, **kwargs):
def _create(cls, model_class, *args, **kwargs):
return cls._build(model_class, *args, **kwargs)

@classmethod
async def _create_async(cls, model_class, *args, **kwargs):
return cls._build(model_class, *args, **kwargs)


class DictFactory(BaseDictFactory):
class Meta:
Expand All @@ -706,6 +784,10 @@ def _build(cls, model_class, *args, **kwargs):
def _create(cls, model_class, *args, **kwargs):
return cls._build(model_class, *args, **kwargs)

@classmethod
async def _create_async(cls, model_class, *args, **kwargs):
return cls._build(model_class, *args, **kwargs)


class ListFactory(BaseListFactory):
class Meta:
Expand Down
Loading