Skip to content

Commit

Permalink
Add notes on redis connection pool to greenlet doc
Browse files Browse the repository at this point in the history
Fixes #809
  • Loading branch information
coleifer committed Aug 6, 2024
1 parent 8208d64 commit 0dfc0a3
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions docs/consumer.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,8 @@ their default values.
If you have a CPU-intensive workload, you may want to increase the number
of workers to the number of CPU cores (or 2x CPU cores). Lastly, if you are
using the ``greenlet`` worker type, you can easily run tens or hundreds of
workers as they are extremely lightweight.
workers as they are extremely lightweight (however make sure, when using
Redis, that you create a large enough pool for all these connections).

``-k``, ``--worker-type``
Choose the worker type, ``thread``, ``process`` or ``greenlet``. The
Expand All @@ -91,7 +92,8 @@ their default values.
cheap in terms of memory, you can easily run a large number of workers.
Note that all code that does **not** consist in waiting for a socket will
be blocking and cannot be pre-empted. Understand the tradeoffs before
jumping to use greenlets.
jumping to use greenlets. When using with Redis, ensure that your
connection pool is large enough to provide connections for each greenlet.
* Anything else: use "thread". You get the benefits of pre-emptive
multi-tasking without the overhead of multiple processes. A safe choice
and the default.
Expand Down Expand Up @@ -214,7 +216,9 @@ different mechanisms to achieve this concurrency.
reason, gevent is a good choice for tasks that perform lots of socket I/O,
but may give worse performance for tasks that are CPU-bound (e.g., parsing
large files, manipulating images, generating reports, etc). Understand the
tradeoff thoroughly before using this worker type.
tradeoff thoroughly before using this worker type. When using Redis, ensure
that your connection pool is large enough for each greenlet to have its own
connection.

When in doubt, the default setting (``thread``) is a safe choice.

Expand Down

0 comments on commit 0dfc0a3

Please sign in to comment.