diff --git a/docs/releases/0.10.rst b/docs/releases/0.10.rst index 9827802..f029a1e 100644 --- a/docs/releases/0.10.rst +++ b/docs/releases/0.10.rst @@ -37,10 +37,10 @@ Minor changes also include: * Python 3.11 is now the minimum version required; this will not affect you at all if you run Takahē via our docker image, as is recommended. -A remote post pruning system, to shrink the database of old data that was no -longer needed, was in the development version but has been removed in this -release due to the extra database load it caused. An alternative approach to -this will hopefully land in a future release. +An automatic remote post pruning system, to shrink the database of old data +that was no longer needed, was in the development version but has been switched +to a set of manual commands as of 0.10.1 - you can read more below or in +:doc:`/tuning`. If you'd like to help with code, design, or other areas, see :doc:`/contributing` to see how to get in touch. @@ -62,6 +62,11 @@ This is a bugfix and small feature addition release: pruning (deletion of old content) of Posts and Identities respectively. You can read more about them in :doc:`/tuning`. +* Stator's default concurrency levels have been significantly reduced as it's + now way more efficient at using individual database connections, but as a + result it places way more load on them. You can read more about tuning this + in :doc:`/tuning`. + Upgrade Notes ------------- @@ -79,3 +84,13 @@ We recommend: * Temporarily stopping all instances of the webserver and Stator * Applying the migration (should be less than a few minutes on most installs) * Restarting the instances of webserver and Stator + +Stator +~~~~~~ + +Stator's new internal architecture allocates a worker thread and a database +connection up to its concurrency value; this means it is a _lot_ more efficient +for a given "concurrency" number than the old system and also uses a lot more +database connections. We recommend you reduce your configuration values for +these by 5-10x; if you didn't set them manually, then don't worry, we've +reduced the default values by a similar amount. diff --git a/docs/tuning.rst b/docs/tuning.rst index 244e9f4..73cc616 100644 --- a/docs/tuning.rst +++ b/docs/tuning.rst @@ -56,10 +56,13 @@ Stator (Task Processing) Takahē's background task processing system is called Stator, and it uses asynchronous Python to pack loads of tasks at once time into a single process. -By default, it will try to run up to 100 tasks at once, with a maximum of 40 -from any single model (FanOut will usually be the one it's doing most of). -You can tweak these with the ``TAKAHE_STATOR_CONCURRENCY`` and -``TAKAHE_STATOR_CONCURRENCY_PER_MODEL`` environment variables. +By default, it will try to run up to 20 tasks at once, with a maximum of 4 from +any single model (FanOut will usually be the one it's doing most of). You can +tweak these with the ``TAKAHE_STATOR_CONCURRENCY`` and +``TAKAHE_STATOR_CONCURRENCY_PER_MODEL`` environment variables; for every extra +element of concurrency you add, however, it will use an additional database +connection in a new worker thread. Be wary of hitting your database's +connection limits. The only real limits Stator can hit are CPU and memory usage; if you see your Stator (worker) containers not using anywhere near all of their CPU or memory,