diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 2731842..1eae44a 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -13,7 +13,7 @@ jobs: services: postgres: - image: postgres:11 + image: postgres:12 env: POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres diff --git a/CHANGELOG.md b/CHANGELOG.md index 05c25c0..0b62cd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,18 +1,30 @@ # Changes +#### Improvements + +- feat: Add `--before-date` option to `auditlogflush` to support retention windows ([#365](https://github.com/jazzband/django-auditlog/pull/365)) +- feat: Add db_index to the `LogEntry.timestamp` column ([#364](https://github.com/jazzband/django-auditlog/pull/364)) +- feat: Add register model from settings ([#368](https://github.com/jazzband/django-auditlog/pull/368)) +- Context manager set_actor() for use in Celery tasks ([#262](https://github.com/jazzband/django-auditlog/pull/262)) +- Tracking of changes in many-to-many fields ([#309](https://github.com/jazzband/django-auditlog/pull/309)) + #### Fixes - Fix inconsistent changes with JSONField ([#355](https://github.com/jazzband/django-auditlog/pull/355)) +- Disable `add` button in admin ui ([#378](https://github.com/jazzband/django-auditlog/pull/378)) +- Fix n+1 query problem([#381](https://github.com/jazzband/django-auditlog/pull/381)) ## 2.0.0 (2022-05-09) #### Improvements + - feat: enable use of replica database (delegating the choice to `DATABASES_ROUTER`) ([#359](https://github.com/jazzband/django-auditlog/pull/359)) - Add `mask_fields` argument in `register` to mask sensitive information when logging ([#310](https://github.com/jazzband/django-auditlog/pull/310)) - Django: Drop 2.2 support. `django_jsonfield_backport` is not required anymore ([#370](https://github.com/jazzband/django-auditlog/pull/370)) - Remove `default_app_config` configuration ([#372](https://github.com/jazzband/django-auditlog/pull/372)) #### Important notes + - LogEntry no longer save to same database instance is using ## 1.0.0 (2022-01-24) @@ -46,7 +58,6 @@ - Support Django's save method `update_fields` kwarg ([#336](https://github.com/jazzband/django-auditlog/pull/336)) - Fix invalid escape sequence on Python 3.7 - ### Alpha 1 (1.0a1, 2020-09-07) #### Improvements @@ -61,14 +72,12 @@ - Fix field choices diff - Allow higher versions of python-dateutil than 2.6.0 - ## 0.4.8 (2019-11-12) ### Improvements - Add support for PostgreSQL 10 - ## 0.4.7 (2019-12-19) ### Improvements @@ -77,7 +86,6 @@ - Django: add 2.1 and 2.2 support, drop < 1.11 versions - Python: add 3.7 support - ## 0.4.6 (2018-09-18) ### Features @@ -94,14 +102,12 @@ - Fix the rendering of the `msg` field with Django 2.0 ([#166](https://github.com/jazzband/django-auditlog/pull/166)) - Mark `LogEntryAdminMixin` methods output as safe where required ([#167](https://github.com/jazzband/django-auditlog/pull/167)) - ## 0.4.5 (2018-01-12) ### Improvements Added support for Django 2.0, along with a number of bug fixes. - ## 0.4.4 (2017-11-17) ### Improvements @@ -118,14 +124,12 @@ Added support for Django 2.0, along with a number of bug fixes. - Add management commands package to setup.py ([#130](https://github.com/jazzband/django-auditlog/pull/130)) - Add `changes_display_dict` property to `LogEntry` model to display diff in a more human readable format ([#94](https://github.com/jazzband/django-auditlog/pull/94)) - ## 0.4.3 (2017-02-16) ### Fixes - Fixes cricital bug in admin mixin making the library only usable on Django 1.11 - ## 0.4.2 (2017-02-16) _As it turns out, haste is never good. Due to the focus on quickly releasing this version a nasty bug was not spotted, which makes this version only usable with Django 1.11 and above. Upgrading to 0.4.3 is not only encouraged but most likely necessary. Apologies for the inconvenience and lacking quality control._ @@ -139,7 +143,6 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - A lot, yes, [_really_ a lot](https://github.com/jjkester/django-auditlog/milestone/8?closed=1), of fixes for the admin integration - Flush command fixed for Django 1.10 - ## 0.4.1 (2016-12-27) ### Improvements @@ -150,7 +153,6 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - Fixed multithreading issue where the wrong user was written to the log - ## 0.4.0 (2016-08-17) ### Breaking changes @@ -171,7 +173,6 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - Solved migration error for MySQL users - ## 0.3.3 (2016-01-23) ### Fixes @@ -184,7 +185,6 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - The `object_pk` field is now limited to 255 chars - ## 0.3.2 (2015-10-19) ### New functionality @@ -195,14 +195,12 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - Enhanced performance for non-integer primary key lookups - ## 0.3.1 (2015-07-29) ### Fixes - Auditlog data is now correctly stored in the thread. - ## 0.3.0 (2015-07-22) ### Breaking changes @@ -223,14 +221,12 @@ _As it turns out, haste is never good. Due to the focus on quickly releasing thi - Better documentation - Compatibility with [django-polymorphic](https://pypi.org/project/django-polymorphic/) - ## 0.2.1 (2014-07-08) ### New functionality - South compatibility for `AuditlogHistoryField` - ## 0.2.0 (2014-03-08) Although this release contains mostly bugfixes, the improvements were significant enough to justify a higher version number. @@ -241,7 +237,6 @@ Although this release contains mostly bugfixes, the improvements were significan - Model diffs use unicode strings instead of regular strings - Tests on middleware - ## 0.1.1 (2013-12-12) ### New functionality @@ -253,7 +248,6 @@ Although this release contains mostly bugfixes, the improvements were significan - Only save a new log entry if there are actual changes - Better way of loading the user model in the middleware - ## 0.1.0 (2013-10-21) First beta release of Auditlog. diff --git a/auditlog/admin.py b/auditlog/admin.py index f50bbcc..c8db263 100644 --- a/auditlog/admin.py +++ b/auditlog/admin.py @@ -52,5 +52,9 @@ class LogEntryAdmin(admin.ModelAdmin, LogEntryAdminMixin): show_full_result_count = False paginator = TimeLimitedPaginator + def has_add_permission(self, request): + # As audit admin doesn't allow log creation from admin + return False + admin.site.register(LogEntry, LogEntryAdmin) diff --git a/auditlog/apps.py b/auditlog/apps.py index 097558f..0e9266e 100644 --- a/auditlog/apps.py +++ b/auditlog/apps.py @@ -5,3 +5,8 @@ class AuditlogConfig(AppConfig): name = "auditlog" verbose_name = "Audit log" default_auto_field = "django.db.models.AutoField" + + def ready(self): + from auditlog.registry import auditlog + + auditlog.register_from_settings() diff --git a/auditlog/conf.py b/auditlog/conf.py new file mode 100644 index 0000000..fdb685c --- /dev/null +++ b/auditlog/conf.py @@ -0,0 +1,17 @@ +from django.conf import settings + +# Register all models when set to True +settings.AUDITLOG_INCLUDE_ALL_MODELS = getattr( + settings, "AUDITLOG_INCLUDE_ALL_MODELS", False +) + +# Exclude models in registration process +# It will be considered when `AUDITLOG_INCLUDE_ALL_MODELS` is True +settings.AUDITLOG_EXCLUDE_TRACKING_MODELS = getattr( + settings, "AUDITLOG_EXCLUDE_TRACKING_MODELS", () +) + +# Register models and define their logging behaviour +settings.AUDITLOG_INCLUDE_TRACKING_MODELS = getattr( + settings, "AUDITLOG_INCLUDE_TRACKING_MODELS", () +) diff --git a/auditlog/context.py b/auditlog/context.py index adac85d..6e9513d 100644 --- a/auditlog/context.py +++ b/auditlog/context.py @@ -33,7 +33,6 @@ def set_actor(actor, remote_addr=None): try: yield - finally: try: auditlog = threadlocal.auditlog diff --git a/auditlog/management/commands/auditlogflush.py b/auditlog/management/commands/auditlogflush.py index a2becd4..e57ef2c 100644 --- a/auditlog/management/commands/auditlogflush.py +++ b/auditlog/management/commands/auditlogflush.py @@ -1,3 +1,5 @@ +import datetime + from django.core.management.base import BaseCommand from auditlog.models import LogEntry @@ -8,27 +10,43 @@ class Command(BaseCommand): def add_arguments(self, parser): parser.add_argument( - "-y, --yes", + "-y", + "--yes", action="store_true", default=None, help="Continue without asking confirmation.", dest="yes", ) + parser.add_argument( + "-b", + "--before-date", + default=None, + help="Flush all entries with a timestamp before a given date (ISO 8601).", + dest="before_date", + type=datetime.date.fromisoformat, + ) def handle(self, *args, **options): answer = options["yes"] + before = options["before_date"] if answer is None: - self.stdout.write( + warning_message = ( "This action will clear all log entries from the database." ) + if before is not None: + warning_message = f"This action will clear all log entries before {before} from the database." + self.stdout.write(warning_message) response = ( input("Are you sure you want to continue? [y/N]: ").lower().strip() ) answer = response == "y" if answer: - count, _ = LogEntry.objects.all().delete() + entries = LogEntry.objects.all() + if before is not None: + entries = entries.filter(timestamp__date__lt=before) + count, _ = entries.delete() self.stdout.write("Deleted %d objects." % count) else: self.stdout.write("Aborted.") diff --git a/auditlog/middleware.py b/auditlog/middleware.py index 6ce6ebb..65e6611 100644 --- a/auditlog/middleware.py +++ b/auditlog/middleware.py @@ -3,13 +3,7 @@ import contextlib from auditlog.context import set_actor -@contextlib.contextmanager -def nullcontext(): - """Equivalent to contextlib.nullcontext(None) from Python 3.7.""" - yield - - -class AuditlogMiddleware(object): +class AuditlogMiddleware: """ Middleware to couple the request's user to log items. This is accomplished by currying the signal receiver with the user from the request (or None if the user is not authenticated). @@ -29,7 +23,7 @@ class AuditlogMiddleware(object): if hasattr(request, "user") and request.user.is_authenticated: context = set_actor(actor=request.user, remote_addr=remote_addr) else: - context = nullcontext() + context = contextlib.nullcontext() with context: return self.get_response(request) diff --git a/auditlog/migrations/0012_alter_logentry_timestamp.py b/auditlog/migrations/0012_alter_logentry_timestamp.py new file mode 100644 index 0000000..8768ec1 --- /dev/null +++ b/auditlog/migrations/0012_alter_logentry_timestamp.py @@ -0,0 +1,20 @@ +# Generated by Django 4.0.3 on 2022-03-11 23:16 + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ("auditlog", "0011_alter_logentry_additional_data"), + ] + + operations = [ + migrations.AlterField( + model_name="logentry", + name="timestamp", + field=models.DateTimeField( + auto_now_add=True, db_index=True, verbose_name="timestamp" + ), + ), + ] diff --git a/auditlog/mixins.py b/auditlog/mixins.py index 3cf8e60..e9b6158 100644 --- a/auditlog/mixins.py +++ b/auditlog/mixins.py @@ -99,14 +99,14 @@ class LogEntryAdminMixin: msg.append(self._format_header("#", "Relationship", "Action", "Objects")) for i, (field, change) in enumerate(sorted(m2m_changes.items()), 1): change_html = format_html_join( - mark_safe("
"), + mark_safe("
"), "{}", [(value,) for value in change["objects"]], ) msg.append( format_html( - "{}{}{}{}", + "{}{}{}{}", i, field, change["operation"], diff --git a/auditlog/models.py b/auditlog/models.py index c5a88f2..1256fc0 100644 --- a/auditlog/models.py +++ b/auditlog/models.py @@ -74,10 +74,14 @@ class LogEntryManager(models.Manager): ): """Create a new "changed" log entry from m2m record. + :param changed_queryset: The added or removed related objects. + :type changed_queryset: QuerySet :param instance: The model instance to log a change for. :type instance: Model :param operation: "add" or "delete". :type action: str + :param field_name: The name of the changed m2m field. + :type field_name: str :param kwargs: Field overrides for the :py:class:`LogEntry` object. :return: The new log entry or `None` if there were no changes. :rtype: LogEntry @@ -109,12 +113,7 @@ class LogEntryManager(models.Manager): } } ) - db = instance._state.db - return ( - self.create(**kwargs) - if db is None or db == "" - else self.using(db).create(**kwargs) - ) + return self.create(**kwargs) return None @@ -269,7 +268,9 @@ class LogEntry(models.Model): remote_addr = models.GenericIPAddressField( blank=True, null=True, verbose_name=_("remote address") ) - timestamp = models.DateTimeField(auto_now_add=True, verbose_name=_("timestamp")) + timestamp = models.DateTimeField( + db_index=True, auto_now_add=True, verbose_name=_("timestamp") + ) additional_data = models.JSONField( blank=True, null=True, verbose_name=_("additional data") ) diff --git a/auditlog/registry.py b/auditlog/registry.py index 50ebce7..06f5f0a 100644 --- a/auditlog/registry.py +++ b/auditlog/registry.py @@ -1,6 +1,18 @@ +import copy from collections import defaultdict -from typing import Callable, Collection, Dict, List, Optional, Tuple +from typing import ( + Any, + Callable, + Collection, + Dict, + Iterable, + List, + Optional, + Tuple, + Union, +) +from django.apps import apps from django.db.models import Model from django.db.models.base import ModelBase from django.db.models.signals import ( @@ -11,6 +23,8 @@ from django.db.models.signals import ( pre_save, ) +from auditlog.conf import settings + DispatchUID = Tuple[int, int, int] @@ -19,6 +33,8 @@ class AuditlogModelRegistry: A registry that keeps track of the models that use Auditlog to track changes. """ + DEFAULT_EXCLUDE_MODELS = ("auditlog.LogEntry", "admin.LogEntry") + def __init__( self, create: bool = True, @@ -61,7 +77,7 @@ class AuditlogModelRegistry: :param exclude_fields: The fields to exclude. Overrides the fields to include. :param mapping_fields: Mapping from field names to strings in diff. :param mask_fields: The fields to mask for sensitive info. - :param m2m_fields: The fields to map as many to many. + :param m2m_fields: The fields to handle as many to many. """ @@ -184,5 +200,92 @@ class AuditlogModelRegistry: """Generate a dispatch_uid which is unique for a combination of self, signal, and receiver.""" return id(self), id(signal), id(receiver) + def _get_model_classes(self, app_model: str) -> List[ModelBase]: + try: + try: + app_label, model_name = app_model.split(".") + return [apps.get_model(app_label, model_name)] + except ValueError: + return apps.get_app_config(app_model).get_models() + except LookupError: + return [] + + def _get_exclude_models( + self, exclude_tracking_models: Iterable[str] + ) -> List[ModelBase]: + exclude_models = [ + model + for app_model in exclude_tracking_models + self.DEFAULT_EXCLUDE_MODELS + for model in self._get_model_classes(app_model) + ] + return exclude_models + + def _register_models(self, models: Iterable[Union[str, Dict[str, Any]]]) -> None: + models = copy.deepcopy(models) + for model in models: + if isinstance(model, str): + for model_class in self._get_model_classes(model): + self.unregister(model_class) + self.register(model_class) + elif isinstance(model, dict): + model["model"] = self._get_model_classes(model["model"])[0] + self.unregister(model["model"]) + self.register(**model) + + def register_from_settings(self): + """ + Register models from settings variables + """ + if not isinstance(settings.AUDITLOG_INCLUDE_ALL_MODELS, bool): + raise TypeError("Setting 'AUDITLOG_INCLUDE_ALL_MODELS' must be a boolean") + + if not isinstance(settings.AUDITLOG_EXCLUDE_TRACKING_MODELS, (list, tuple)): + raise TypeError( + "Setting 'AUDITLOG_EXCLUDE_TRACKING_MODELS' must be a list or tuple" + ) + + if ( + not settings.AUDITLOG_INCLUDE_ALL_MODELS + and settings.AUDITLOG_EXCLUDE_TRACKING_MODELS + ): + raise ValueError( + "In order to use setting 'AUDITLOG_EXCLUDE_TRACKING_MODELS', " + "setting 'AUDITLOG_INCLUDE_ALL_MODELS' must set to 'True'" + ) + + if not isinstance(settings.AUDITLOG_INCLUDE_TRACKING_MODELS, (list, tuple)): + raise TypeError( + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' must be a list or tuple" + ) + + for item in settings.AUDITLOG_INCLUDE_TRACKING_MODELS: + if not isinstance(item, (str, dict)): + raise TypeError( + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' items must be str or dict" + ) + + if isinstance(item, dict): + if "model" not in item: + raise ValueError( + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' dict items must contain 'model' key" + ) + if "." not in item["model"]: + raise ValueError( + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' model must be in the format ." + ) + + if settings.AUDITLOG_INCLUDE_ALL_MODELS: + exclude_models = self._get_exclude_models( + settings.AUDITLOG_EXCLUDE_TRACKING_MODELS + ) + models = apps.get_models() + + for model in models: + if model in exclude_models: + continue + self.register(model) + + self._register_models(settings.AUDITLOG_INCLUDE_TRACKING_MODELS) + auditlog = AuditlogModelRegistry() diff --git a/auditlog_tests/models.py b/auditlog_tests/models.py index ded4eff..68d04d5 100644 --- a/auditlog_tests/models.py +++ b/auditlog_tests/models.py @@ -62,39 +62,43 @@ class ProxyModel(SimpleModel): proxy = True -class RelatedModel(models.Model): +class RelatedModelParent(models.Model): + """ + Use multi table inheritance to make a OneToOneRel field + """ + + +class RelatedModel(RelatedModelParent): """ A model with a foreign key. """ - related = models.ForeignKey(to="self", on_delete=models.CASCADE) + related = models.ForeignKey(to="SimpleModel", on_delete=models.CASCADE) + one_to_one = models.OneToOneField( + to="SimpleModel", on_delete=models.CASCADE, related_name="reverse_one_to_one" + ) history = AuditlogHistoryField() class ManyRelatedModel(models.Model): """ - A model with a many to many relation. + A model with many-to-many relations. """ - related = models.ManyToManyField("self") + recursive = models.ManyToManyField("self") + related = models.ManyToManyField("ManyRelatedOtherModel", related_name="related") history = AuditlogHistoryField() + def get_additional_data(self): + related = self.related.first() + return {"related_model_id": related.id if related else None} -class FirstManyRelatedModel(models.Model): + +class ManyRelatedOtherModel(models.Model): """ - A model with a many to many relation to another model similar. - """ - - related = models.ManyToManyField("OtherManyRelatedModel", related_name="related") - - history = AuditlogHistoryField() - - -class OtherManyRelatedModel(models.Model): - """ - A model that 'receives' the other side of the many to many relation from 'FirstManyRelatedModel'. + A model related to ManyRelatedModel as many-to-many. """ history = AuditlogHistoryField() @@ -261,10 +265,8 @@ auditlog.register(UUIDPrimaryKeyModel) auditlog.register(ProxyModel) auditlog.register(RelatedModel) auditlog.register(ManyRelatedModel) -auditlog.register(ManyRelatedModel.related.through) -m2m_only_auditlog.register( - FirstManyRelatedModel, include_fields=["pk", "history"], m2m_fields={"related"} -) +auditlog.register(ManyRelatedModel.recursive.through) +m2m_only_auditlog.register(ManyRelatedModel, m2m_fields={"related"}) auditlog.register(SimpleExcludeModel, exclude_fields=["text"]) auditlog.register(SimpleMappingModel, mapping_fields={"sku": "Product No."}) auditlog.register(AdditionalDataIncludedModel) diff --git a/auditlog_tests/test_commands.py b/auditlog_tests/test_commands.py new file mode 100644 index 0000000..c7ec41b --- /dev/null +++ b/auditlog_tests/test_commands.py @@ -0,0 +1,109 @@ +"""Tests for auditlog.management.commands""" + +import datetime +from io import StringIO +from unittest import mock + +import freezegun +from django.core.management import call_command +from django.test import TestCase + +from auditlog_tests.models import SimpleModel + + +class AuditlogFlushTest(TestCase): + def setUp(self): + input_patcher = mock.patch("builtins.input") + self.mock_input = input_patcher.start() + self.addCleanup(input_patcher.stop) + + def make_object(self): + return SimpleModel.objects.create(text="I am a simple model.") + + def call_command(self, *args, **kwargs): + outbuf = StringIO() + errbuf = StringIO() + call_command("auditlogflush", *args, stdout=outbuf, stderr=errbuf, **kwargs) + return outbuf.getvalue().strip(), errbuf.getvalue().strip() + + def test_flush_yes(self): + obj = self.make_object() + self.assertEqual(obj.history.count(), 1, msg="There is one log entry.") + + out, err = self.call_command("--yes") + + self.assertEqual(obj.history.count(), 0, msg="There are no log entries.") + self.assertEqual( + out, "Deleted 1 objects.", msg="Output shows deleted 1 object." + ) + self.assertEqual(err, "", msg="No stderr") + + def test_flush_no(self): + obj = self.make_object() + self.assertEqual(obj.history.count(), 1, msg="There is one log entry.") + + self.mock_input.return_value = "N\n" + out, err = self.call_command() + + self.assertEqual(obj.history.count(), 1, msg="There is still one log entry.") + self.assertEqual( + out, + "This action will clear all log entries from the database.\nAborted.", + msg="Output shows warning and aborted.", + ) + self.assertEqual(err, "", msg="No stderr") + + def test_flush_input_yes(self): + obj = self.make_object() + self.assertEqual(obj.history.count(), 1, msg="There is one log entry.") + + self.mock_input.return_value = "Y\n" + out, err = self.call_command() + + self.assertEqual(obj.history.count(), 0, msg="There are no log entries.") + self.assertEqual( + out, + "This action will clear all log entries from the database.\nDeleted 1 objects.", + msg="Output shows warning and deleted 1 object.", + ) + self.assertEqual(err, "", msg="No stderr") + + def test_before_date_input(self): + self.mock_input.return_value = "N\n" + out, err = self.call_command("--before-date=2000-01-01") + self.assertEqual( + out, + "This action will clear all log entries before 2000-01-01 from the database.\nAborted.", + msg="Output shows warning with date and then aborted.", + ) + self.assertEqual(err, "", msg="No stderr") + + def test_before_date(self): + with freezegun.freeze_time("1999-12-31"): + obj = self.make_object() + + with freezegun.freeze_time("2000-01-02"): + obj.text = "I have new text" + obj.save() + + self.assertEqual( + {v["timestamp"] for v in obj.history.values("timestamp")}, + { + datetime.datetime(1999, 12, 31, tzinfo=datetime.timezone.utc), + datetime.datetime(2000, 1, 2, tzinfo=datetime.timezone.utc), + }, + msg="Entries exist for 1999-12-31 and 2000-01-02", + ) + + out, err = self.call_command("--yes", "--before-date=2000-01-01") + self.assertEqual( + {v["timestamp"] for v in obj.history.values("timestamp")}, + { + datetime.datetime(2000, 1, 2, tzinfo=datetime.timezone.utc), + }, + msg="An entry exists only for 2000-01-02", + ) + self.assertEqual( + out, "Deleted 1 objects.", msg="Output shows deleted 1 object." + ) + self.assertEqual(err, "", msg="No stderr") diff --git a/auditlog_tests/tests.py b/auditlog_tests/tests.py index 95344d4..b79f1a6 100644 --- a/auditlog_tests/tests.py +++ b/auditlog_tests/tests.py @@ -4,30 +4,32 @@ import json from unittest import mock from dateutil.tz import gettz +from django.apps import apps from django.conf import settings +from django.contrib.admin.sites import AdminSite from django.contrib.auth import get_user_model from django.contrib.auth.models import AnonymousUser, User from django.contrib.contenttypes.models import ContentType from django.db.models.signals import pre_save -from django.test import RequestFactory, TestCase +from django.test import RequestFactory, TestCase, override_settings from django.utils import dateformat, formats, timezone +from auditlog.admin import LogEntryAdmin from auditlog.context import set_actor from auditlog.diff import model_instance_diff from auditlog.middleware import AuditlogMiddleware from auditlog.models import LogEntry -from auditlog.registry import auditlog +from auditlog.registry import AuditlogModelRegistry, auditlog from auditlog_tests.models import ( AdditionalDataIncludedModel, AltPrimaryKeyModel, CharfieldTextfieldModel, ChoicesFieldModel, DateTimeFieldModel, - FirstManyRelatedModel, JSONModel, ManyRelatedModel, + ManyRelatedOtherModel, NoDeleteHistoryModel, - OtherManyRelatedModel, PostgresArrayFieldModel, ProxyModel, RelatedModel, @@ -300,77 +302,66 @@ class ProxyModelWithActorTest(WithActorMixin, ProxyModelBase): class ManyRelatedModelTest(TestCase): """ - Test the behaviour of a default many-to-many relationship. + Test the behaviour of many-to-many relationships. """ def setUp(self): self.obj = ManyRelatedModel.objects.create() - self.rel_obj = ManyRelatedModel.objects.create() - self.obj.related.add(self.rel_obj) + self.recursive = ManyRelatedModel.objects.create() + self.related = ManyRelatedOtherModel.objects.create() + self.base_log_entry_count = ( + LogEntry.objects.count() + ) # created by the create() calls above - def test_related(self): + def test_recursive(self): + self.obj.recursive.add(self.recursive) self.assertEqual( - LogEntry.objects.get_for_objects(self.obj.related.all()).count(), - self.rel_obj.history.count(), + LogEntry.objects.get_for_objects(self.obj.recursive.all()).first(), + self.recursive.history.first(), ) - self.assertEqual( - LogEntry.objects.get_for_objects(self.obj.related.all()).first(), - self.rel_obj.history.first(), - ) - - -class FirstManyRelatedModelTest(TestCase): - """ - Test the behaviour of a many-to-many relationship. - """ - - def setUp(self): - self.obj = FirstManyRelatedModel.objects.create() - self.rel_obj = OtherManyRelatedModel.objects.create() def test_related_add_from_first_side(self): - self.obj.related.add(self.rel_obj) - self.assertEqual( - LogEntry.objects.get_for_objects(self.obj.related.all()).count(), - self.rel_obj.history.count(), - ) + self.obj.related.add(self.related) self.assertEqual( LogEntry.objects.get_for_objects(self.obj.related.all()).first(), - self.rel_obj.history.first(), + self.related.history.first(), ) - self.assertEqual(LogEntry.objects.count(), 1) + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 1) def test_related_add_from_other_side(self): - self.rel_obj.related.add(self.obj) - self.assertEqual( - LogEntry.objects.get_for_objects(self.obj.related.all()).count(), - self.rel_obj.history.count(), - ) + self.related.related.add(self.obj) self.assertEqual( LogEntry.objects.get_for_objects(self.obj.related.all()).first(), - self.rel_obj.history.first(), + self.related.history.first(), ) - self.assertEqual(LogEntry.objects.count(), 1) + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 1) def test_related_remove_from_first_side(self): - self.obj.related.add(self.rel_obj) - self.obj.related.remove(self.rel_obj) - self.assertEqual(LogEntry.objects.count(), 2) + self.obj.related.add(self.related) + self.obj.related.remove(self.related) + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 2) def test_related_remove_from_other_side(self): - self.rel_obj.related.add(self.obj) - self.rel_obj.related.remove(self.obj) - self.assertEqual(LogEntry.objects.count(), 2) + self.related.related.add(self.obj) + self.related.related.remove(self.obj) + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 2) def test_related_clear_from_first_side(self): - self.obj.related.add(self.rel_obj) + self.obj.related.add(self.related) self.obj.related.clear() - self.assertEqual(LogEntry.objects.count(), 2) + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 2) def test_related_clear_from_other_side(self): - self.rel_obj.related.add(self.obj) - self.rel_obj.related.clear() - self.assertEqual(LogEntry.objects.count(), 2) + self.related.related.add(self.obj) + self.related.related.clear() + self.assertEqual(LogEntry.objects.count(), self.base_log_entry_count + 2) + + def test_additional_data(self): + self.obj.related.add(self.related) + log_entry = self.obj.history.first() + self.assertEqual( + log_entry.additional_data, {"related_model_id": self.related.id} + ) class MiddlewareTest(TestCase): @@ -929,6 +920,170 @@ class UnregisterTest(TestCase): self.assertEqual(LogEntry.objects.count(), 0, msg="There are no log entries") +class RegisterModelSettingsTest(TestCase): + def setUp(self): + self.test_auditlog = AuditlogModelRegistry() + + def tearDown(self): + for model in self.test_auditlog.get_models(): + self.test_auditlog.unregister(model) + + def test_get_model_classes(self): + self.assertEqual( + len(list(self.test_auditlog._get_model_classes("auditlog"))), + len(list(apps.get_app_config("auditlog").get_models())), + ) + self.assertEqual([], self.test_auditlog._get_model_classes("fake_model")) + + def test_get_exclude_models(self): + # By default it returns DEFAULT_EXCLUDE_MODELS + self.assertEqual(len(self.test_auditlog._get_exclude_models(())), 2) + + # Exclude just one model + self.assertTrue( + SimpleExcludeModel + in self.test_auditlog._get_exclude_models( + ("auditlog_tests.SimpleExcludeModel",) + ) + ) + + # Exclude all model of an app + self.assertTrue( + SimpleExcludeModel + in self.test_auditlog._get_exclude_models(("auditlog_tests",)) + ) + + def test_register_models_no_models(self): + self.test_auditlog._register_models(()) + + self.assertEqual(self.test_auditlog._registry, {}) + + def test_register_models_register_single_model(self): + self.test_auditlog._register_models(("auditlog_tests.SimpleExcludeModel",)) + + self.assertTrue(self.test_auditlog.contains(SimpleExcludeModel)) + self.assertEqual(len(self.test_auditlog._registry), 1) + + def test_register_models_register_app(self): + self.test_auditlog._register_models(("auditlog_tests",)) + + self.assertTrue(self.test_auditlog.contains(SimpleExcludeModel)) + self.assertTrue(self.test_auditlog.contains(ChoicesFieldModel)) + self.assertEqual(len(self.test_auditlog.get_models()), 19) + + def test_register_models_register_model_with_attrs(self): + self.test_auditlog._register_models( + ( + { + "model": "auditlog_tests.SimpleExcludeModel", + "include_fields": ["label"], + "exclude_fields": [ + "text", + ], + }, + ) + ) + + self.assertTrue(self.test_auditlog.contains(SimpleExcludeModel)) + fields = self.test_auditlog.get_model_fields(SimpleExcludeModel) + self.assertEqual(fields["include_fields"], ["label"]) + self.assertEqual(fields["exclude_fields"], ["text"]) + + def test_register_models_register_model_with_m2m_fields(self): + self.test_auditlog._register_models( + ( + { + "model": "auditlog_tests.ManyRelatedModel", + "m2m_fields": {"related"}, + }, + ) + ) + + self.assertTrue(self.test_auditlog.contains(ManyRelatedModel)) + self.assertEqual( + self.test_auditlog._registry[ManyRelatedModel]["m2m_fields"], {"related"} + ) + + def test_register_from_settings_invalid_settings(self): + with override_settings(AUDITLOG_INCLUDE_ALL_MODELS="str"): + with self.assertRaisesMessage( + TypeError, "Setting 'AUDITLOG_INCLUDE_ALL_MODELS' must be a boolean" + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_EXCLUDE_TRACKING_MODELS="str"): + with self.assertRaisesMessage( + TypeError, + "Setting 'AUDITLOG_EXCLUDE_TRACKING_MODELS' must be a list or tuple", + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_EXCLUDE_TRACKING_MODELS=("app1.model1",)): + with self.assertRaisesMessage( + ValueError, + "In order to use setting 'AUDITLOG_EXCLUDE_TRACKING_MODELS', " + "setting 'AUDITLOG_INCLUDE_ALL_MODELS' must set to 'True'", + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_INCLUDE_TRACKING_MODELS="str"): + with self.assertRaisesMessage( + TypeError, + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' must be a list or tuple", + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_INCLUDE_TRACKING_MODELS=(1, 2)): + with self.assertRaisesMessage( + TypeError, + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' items must be str or dict", + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_INCLUDE_TRACKING_MODELS=({"test": "test"},)): + with self.assertRaisesMessage( + ValueError, + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' dict items must contain 'model' key", + ): + self.test_auditlog.register_from_settings() + + with override_settings(AUDITLOG_INCLUDE_TRACKING_MODELS=({"model": "test"},)): + with self.assertRaisesMessage( + ValueError, + "Setting 'AUDITLOG_INCLUDE_TRACKING_MODELS' model must be in the format .", + ): + self.test_auditlog.register_from_settings() + + @override_settings( + AUDITLOG_INCLUDE_ALL_MODELS=True, + AUDITLOG_EXCLUDE_TRACKING_MODELS=("auditlog_tests.SimpleExcludeModel",), + ) + def test_register_from_settings_register_all_models_with_exclude_models(self): + self.test_auditlog.register_from_settings() + + self.assertFalse(self.test_auditlog.contains(SimpleExcludeModel)) + self.assertTrue(self.test_auditlog.contains(ChoicesFieldModel)) + + @override_settings( + AUDITLOG_INCLUDE_TRACKING_MODELS=( + { + "model": "auditlog_tests.SimpleExcludeModel", + "include_fields": ["label"], + "exclude_fields": [ + "text", + ], + }, + ) + ) + def test_register_from_settings_register_models(self): + self.test_auditlog.register_from_settings() + + self.assertTrue(self.test_auditlog.contains(SimpleExcludeModel)) + fields = self.test_auditlog.get_model_fields(SimpleExcludeModel) + self.assertEqual(fields["include_fields"], ["label"]) + self.assertEqual(fields["exclude_fields"], ["text"]) + + class ChoicesFieldModelTest(TestCase): def setUp(self): self.obj = ChoicesFieldModel.objects.create( @@ -1070,7 +1225,7 @@ class AdminPanelTest(TestCase): res = self.client.get("/admin/auditlog/logentry/") assert res.status_code == 200 res = self.client.get("/admin/auditlog/logentry/add/") - assert res.status_code == 200 + assert res.status_code == 403 res = self.client.get(f"/admin/auditlog/logentry/{log_pk}/", follow=True) assert res.status_code == 200 res = self.client.get(f"/admin/auditlog/logentry/{log_pk}/delete/") @@ -1079,6 +1234,87 @@ class AdminPanelTest(TestCase): assert res.status_code == 200 +class DiffMsgTest(TestCase): + def setUp(self): + super().setUp() + self.site = AdminSite() + self.admin = LogEntryAdmin(LogEntry, self.site) + + def _create_log_entry(self, action, changes): + return LogEntry.objects.log_create( + SimpleModel.objects.create(), # doesn't affect anything + action=action, + changes=json.dumps(changes), + ) + + def test_changes_msg__delete(self): + log_entry = self._create_log_entry(LogEntry.Action.DELETE, {}) + + self.assertEqual(self.admin.msg(log_entry), "") + + def test_changes_msg__create(self): + log_entry = self._create_log_entry( + LogEntry.Action.CREATE, + { + "field two": [None, 11], + "field one": [None, "a value"], + }, + ) + + self.assertEqual( + self.admin.msg(log_entry), + ( + "" + "" + "" + "" + "
#FieldFromTo
1field oneNonea value
2field twoNone11
" + ), + ) + + def test_changes_msg__update(self): + log_entry = self._create_log_entry( + LogEntry.Action.UPDATE, + { + "field two": [11, 42], + "field one": ["old value of field one", "new value of field one"], + }, + ) + + self.assertEqual( + self.admin.msg(log_entry), + ( + "" + "" + "" + "" + "
#FieldFromTo
1field oneold value of field onenew value of field one
2field two1142
" + ), + ) + + def test_changes_msg__m2m(self): + log_entry = self._create_log_entry( + LogEntry.Action.UPDATE, + { # mimicking the format used by log_m2m_changes + "some_m2m_field": { + "type": "m2m", + "operation": "add", + "objects": ["Example User (user 1)", "Illustration (user 42)"], + }, + }, + ) + + self.assertEqual( + self.admin.msg(log_entry), + ( + "" + "" + "" + "
#RelationshipActionObjects
1some_m2m_fieldaddExample User (user 1)
Illustration (user 42)
" + ), + ) + + class NoDeleteHistoryTest(TestCase): def test_delete_related(self): instance = SimpleModel.objects.create(integer=1) @@ -1178,7 +1414,39 @@ class JSONModelTest(TestCase): class ModelInstanceDiffTest(TestCase): - def test_when_field_doesnt_exit(self): + def test_diff_models_with_related_fields(self): + """No error is raised when comparing models with related fields.""" + + # This tests that track_field() does indeed ignore related fields. + + # a model without reverse relations + simple1 = SimpleModel() + simple1.save() + + # a model with reverse relations + simple2 = SimpleModel() + simple2.save() + related = RelatedModel(related=simple2, one_to_one=simple2) + related.save() + + # Demonstrate that simple1 can have DoesNotExist on reverse + # OneToOne relation. + with self.assertRaises( + SimpleModel.reverse_one_to_one.RelatedObjectDoesNotExist + ): + simple1.reverse_one_to_one # equals None + + # accessing relatedmodel_set won't trigger DoesNotExist. + self.assertEqual(simple1.relatedmodel_set.count(), 0) + + # simple2 DOES have these relations + self.assertEqual(simple2.reverse_one_to_one, related) + self.assertEqual(simple2.relatedmodel_set.count(), 1) + + model_instance_diff(simple2, simple1) + model_instance_diff(simple1, simple2) + + def test_when_field_doesnt_exist(self): """No error is raised and the default is returned.""" first = SimpleModel(boolean=True) second = SimpleModel() diff --git a/docs/source/conf.py b/docs/source/conf.py index 2a4c7b4..4dbcd08 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -9,7 +9,8 @@ import os import sys from datetime import date -from importlib.metadata import version + +from pkg_resources import get_distribution # -- Path setup -------------------------------------------------------------- @@ -32,7 +33,7 @@ project = "django-auditlog" author = "Jan-Jelle Kester and contributors" copyright = f"2013-{date.today().year}, {author}" -release = version("django-auditlog") +release = get_distribution("django-auditlog").version # for example take major/minor version = ".".join(release.split(".")[:2]) diff --git a/docs/source/usage.rst b/docs/source/usage.rst index f5c3f99..05154f6 100644 --- a/docs/source/usage.rst +++ b/docs/source/usage.rst @@ -11,6 +11,8 @@ even more convenience, :py:class:`LogEntryManager` provides a number of methods See :doc:`internals` for all details. +.. _Automatically logging changes: + Automatically logging changes ----------------------------- @@ -56,15 +58,15 @@ If you have field names on your models that aren't intuitive or user friendly yo during the `register()` call. .. code-block:: python - + class MyModel(modelsModel): sku = models.CharField(max_length=20) version = models.CharField(max_length=5) product = models.CharField(max_length=50, verbose_name='Product Name') history = AuditlogHistoryField() - + auditlog.register(MyModel, mapping_fields={'sku': 'Product No.', 'version': 'Product Revision'}) - + .. code-block:: python log = MyModel.objects.first().history.latest() @@ -91,9 +93,80 @@ For example, to mask the field ``address``, use:: Masking fields +**Many-to-many fields** + +Changes to many-to-many fields are not tracked by default. If you want to enable tracking of a many-to-many field on a model, pass ``m2m_fields`` to the ``register`` method: + +.. code-block:: python + + auditlog.register(MyModel, m2m_fields={"tags", "contacts"}) + +This functionality is based on the ``m2m_changed`` signal sent by the ``through`` model of the relationship. + +Note that when the user changes multiple many-to-many fields on the same object through the admin, both adding and removing some objects from each, this code will generate multiple log entries: each log entry will represent a single operation (add or delete) of a single field, e.g. if you both add and delete values from 2 fields on the same form in the same request, you'll get 4 log entries. + +.. versionadded:: 2.1.0 + +Settings +-------- + +**AUDITLOG_INCLUDE_ALL_MODELS** + +You can use this setting to register all your models: + +.. code-block:: python + + AUDITLOG_INCLUDE_ALL_MODELS=True + +.. versionadded:: 2.1.0 + +**AUDITLOG_EXCLUDE_TRACKING_MODELS** + +You can use this setting to exclude models in registration process. +It will be considered when ``AUDITLOG_INCLUDE_ALL_MODELS`` is `True`. + +.. code-block:: python + + AUDITLOG_EXCLUDE_TRACKING_MODELS = ( + "", + "." + ) + +.. versionadded:: 2.1.0 + +**AUDITLOG_INCLUDE_TRACKING_MODELS** + +You can use this setting to configure your models registration and other behaviours. +It must be a list or tuple. Each item in this setting can be a: + +* ``str``: To register a model. +* ``dict``: To register a model and define its logging behaviour. e.g. include_fields, exclude_fields. + +.. code-block:: python + + AUDITLOG_INCLUDE_TRACKING_MODELS = ( + ".", + { + "model": ".", + "include_fields": ["field1", "field2"], + "exclude_fields": ["field3", "field4"], + "mapping_fields": { + "field1": "FIELD", + }, + "mask_fields": ["field5", "field6"], + "m2m_fields": ["field7", "field8"], + }, + ".", + ) + +.. versionadded:: 2.1.0 + Actors ------ +Middleware +********** + When using automatic logging, the actor is empty by default. However, auditlog can set the actor from the current request automatically. This does not need any custom code, adding a middleware class is enough. When an actor is logged the remote address of that actor will be logged as well. @@ -115,6 +188,22 @@ It is recommended to keep all middleware that alters the request loaded before A user as actor. To only have some object changes to be logged with the current request's user as actor manual logging is required. +Context manager +*************** + +.. versionadded:: 2.1.0 + +To enable the automatic logging of the actors outside of request context (e.g. in a Celery task), you can use a context +manager:: + + from auditlog.context import set_actor + + def do_stuff(actor_id: int): + actor = get_user(actor_id) + with set_actor(actor): + # if your code here leads to creation of LogEntry instances, these will have the actor set + ... + Object history -------------- @@ -177,10 +266,9 @@ Many-to-many relationships .. versionadded:: 0.3.0 -.. warning:: +.. note:: - To-many relations are not officially supported. However, this section shows a workaround which can be used for now. - In the future, this workaround may be used in an official API or a completly different strategy might be chosen. + This section shows a workaround which can be used to track many-to-many relationships on older versions of django-auditlog. For versions 2.1.0 and onwards, please see the many-to-many fields section of :ref:`Automatically logging changes`. **Do not rely on the workaround here to be stable across releases.** By default, many-to-many relationships are not tracked by Auditlog. @@ -206,12 +294,17 @@ Management commands Auditlog provides the ``auditlogflush`` management command to clear all log entries from the database. -By default, the command asks for confirmation. It is possible to run the command with the `-y` or `--yes` flag to skip +By default, the command asks for confirmation. It is possible to run the command with the ``-y`` or ``--yes`` flag to skip confirmation and immediately delete all entries. +You may also specify a date using the ``-b`` or ``--before-date`` option in ISO 8601 format (YYYY-mm-dd) to delete all +log entries prior to a given date. This may be used to implement time based retention windows. + +.. versionadded:: 2.1.0 + .. warning:: - Using the ``auditlogflush`` command deletes all log entries permanently and irreversibly from the database. + Using the ``auditlogflush`` command deletes log entries permanently and irreversibly from the database. Django Admin integration ------------------------ diff --git a/pyproject.toml b/pyproject.toml index b9bf80c..be27ad0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,7 +1,3 @@ -[build-system] -requires = ["setuptools>=45", "wheel", "setuptools_scm>=6.2"] -build-backend = "setuptools.build_meta" - [tool.black] target-version = ["py37"] diff --git a/tox.ini b/tox.ini index fe34d2a..e4d3627 100644 --- a/tox.ini +++ b/tox.ini @@ -2,7 +2,7 @@ envlist = {py37,py38,py39,py310}-django32 {py38,py39,py310}-django{40,main} - py38-docs + py37-docs py38-lint [testenv] @@ -19,6 +19,7 @@ deps = coverage codecov django-multiselectfield + freezegun psycopg2-binary==2.8.6 passenv= TEST_DB_HOST @@ -33,7 +34,7 @@ basepython = py38: python3.8 py37: python3.7 -[testenv:py38-docs] +[testenv:py37-docs] changedir = docs/source deps = -rdocs/requirements.txt commands = sphinx-build -W -b html -d {envtmpdir}/doctrees . {envtmpdir}/html