diff --git a/docs/limits.rst b/docs/limits.rst index ec94b46..7975b70 100644 --- a/docs/limits.rst +++ b/docs/limits.rst @@ -3,22 +3,14 @@ Limits ------ -Do not use if: -.............. +High rate of database modifications +................................... -- Your project runs on several servers **and** each server is connected to - the same database **and** each server uses a different cache **and** - each server cannot have access to all other caches. **However if each server - can have access to all other caches**, simply specify all other caches as - non-default in the ``CACHES`` setting. This way, django-cachalot will - automatically invalidate all other caches when something changes in the - database. Otherwise, django-cachalot has no way to know on a given server - that another server triggered a database change, and it will therefore serve - stale data because the cache of the given server was not invalidated. -- Your project has more than 50 database modifications per second on most of - its tables. There will be no problem, but django-cachalot will become - inefficient and will end up slowing your project instead of speeding it. - Read :ref:`the introduction ` for more details. +Do not use django-cachalot if your project has more than 50 database +modifications per second on most of its tables. There will be no problem, +but django-cachalot will become inefficient and will end up slowing +your project instead of speeding it. +Read :ref:`the introduction ` for more details. Redis ..... @@ -136,6 +128,12 @@ to have their clocks as synchronised as possible. Otherwise, invalidations will happen with a latency from one server to another. A difference of even a few seconds can be harmful, so double check this! +To get a rough idea of the clock synchronisation of two servers, simply run +``python -c 'import time; print(time.time())'`` on both servers at the same +time. This will give you a number of seconds, and it should be almost the same, +with a difference inferior to 1 second. This number is independent +of the time zone. + To keep your clocks synchronised, use the `Network Time Protocol `_. @@ -164,3 +162,45 @@ Use :ref:`the signal ` and :meth:`cachalot.api.invalidate` this way: def invalidate_replica(sender, **kwargs): if kwargs['db_alias'] == 'default': invalidate(sender, db_alias='replica') + +Multiple cache servers for the same database +............................................ + +On large projects, we often end up having multiple Django servers on several +physical machines. For performance reasons, we generally decide to have a cache +per server, while the database stays on a single server. But the problem with +django-cachalot is that it only invalidates the cache configured using +``CACHALOT_CACHE``. So all caches end up serving stale data. + +To avoid this, you need inside each Django server to be able to communicate +with the rest of the servers in order to invalidate other caches when +an invalidation occurs. If this is not possible in your situation, you must not +use django-cachalot. But if you can, each Django server must also have all +other caches in the ``CACHES`` setting. Then, you need to manually invalidate +all other caches when an invalidation occurs. Add this to a `models.py` file +of an installed application: + +.. code:: python + + import threading + + from cachalot.api import invalidate + from cachalot.signals import post_invalidation + from django.dispatch import receiver + from django.conf import settings + + SIGNAL_INFO = threading.local() + + @receiver(post_invalidation) + def invalidate_other_caches(sender, **kwargs): + if getattr(SIGNAL_INFO, 'was_called', False): + return + db_alias = kwargs['db_alias'] + for cache_alias in settings.CACHES: + if cache_alias == settings.CACHALOT_CACHE: + continue + SIGNAL_INFO.was_called = True + try: + invalidate(sender, db_alias=db_alias, cache_alias=cache_alias) + finally: + SIGNAL_INFO.was_called = False diff --git a/docs/quickstart.rst b/docs/quickstart.rst index d1fcdcc..7e437c9 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -19,7 +19,7 @@ Requirements - PostgreSQL - SQLite - - MySQL (but on older versions like 5.5, django-cachalot has no effect, + - MySQL (but on older versions like MySQL 5.5, django-cachalot has no effect, see :ref:`MySQL limits `) Usage @@ -30,11 +30,11 @@ Usage #. If you use multiple servers with a common cache server, :ref:`double check their clock synchronisation ` #. If you modify data outside Django - – typically after restoring a SQL database –, run - ``./manage.py invalidate_cachalot`` + – typically after restoring a SQL database –, + use the :ref:`manage.py command ` #. Be aware of :ref:`the few other limits ` #. If you use - `django-debug-toolbar `_, + `django-debug-toolbar `_, you can add ``'cachalot.panels.CachalotPanel',`` to your ``DEBUG_TOOLBAR_PANELS`` #. Enjoy! @@ -50,13 +50,21 @@ Settings :Default: ``True`` :Description: If set to ``False``, disables SQL caching but keeps invalidating - to avoid stale cache + to avoid stale cache. ``CACHALOT_CACHE`` ~~~~~~~~~~~~~~~~~~ :Default: ``'default'`` -:Description: Alias of the cache from |CACHES|_ used by django-cachalot +:Description: + Alias of the cache from |CACHES|_ used by django-cachalot. + + .. warning:: + After modifying this setting, you should invalidate the cache + :ref:`using the manage.py command ` or :ref:`the API `. + Indeed, only the cache configured using this setting is automatically + invalidated by django-cachalot – for optimisation reasons. So when you + change this setting, you end up on a cache that may contain stale data. .. |CACHES| replace:: ``CACHES`` .. _CACHES: https://docs.djangoproject.com/en/1.7/ref/settings/#std:setting-CACHES @@ -83,7 +91,7 @@ Settings :Default: ``False`` :Description: If set to ``True``, caches random queries - (those with ``order_by('?')``) + (those with ``order_by('?')``). .. _CACHALOT_INVALIDATE_RAW: @@ -91,8 +99,9 @@ Settings ~~~~~~~~~~~~~~~~~~~~~~~~~~~ :Default: ``True`` -:Description: If set to ``False``, disables automatic invalidation on raw - SQL queries – read :ref:`raw queries limits ` for more info +:Description: + If set to ``False``, disables automatic invalidation on raw + SQL queries – read :ref:`raw queries limits ` for more info. ``CACHALOT_ONLY_CACHABLE_TABLES`` @@ -142,6 +151,7 @@ Settings Clear your cache after changing this setting (it’s not enough to use ``./manage.py invalidate_cachalot``). + .. _Dynamic overriding: Dynamic overriding @@ -166,6 +176,29 @@ For example: settings.CACHALOT_ENABLED = False +.. _Command: + +``manage.py`` command +..................... + +``manage.py invalidate_cachalot`` is available to invalidate all the cache keys +set by django-cachalot. If you run it without any argument, it invalidates all +models on all caches and all databases. But you can specify what applications +or models are invalidated, and on which cache or database. + +Examples: + +``./manage.py invalidate_cachalot auth`` + Invalidates all models from the 'auth' application. +``./manage.py invalidate_cachalot your_app auth.User`` + Invalidates all models from the 'your_app' application, but also + the ``User`` model from the 'auth' application. +``./manage.py invalidate_cachalot -c redis -p postgresql`` + Invalidates all models, + but only for the database configured with the 'postgresql' alias, + and only for the cache configured with the 'redis' alias. + + .. _Template utils: Template utils @@ -217,7 +250,6 @@ Example of a quite heavy nested loop with a lot of SQL queries are also available (see :meth:`cachalot.api.get_last_invalidation`). - Jinja2 statement and function ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~