diff --git a/.gitignore b/.gitignore index f013ec3..7380ca5 100644 --- a/.gitignore +++ b/.gitignore @@ -7,4 +7,5 @@ MANIFEST build dist -/tests/media +/tests/media/* +!/tests/media/lenna.png diff --git a/.travis.yml b/.travis.yml index 5a5e2e7..80c5fd1 100644 --- a/.travis.yml +++ b/.travis.yml @@ -2,6 +2,6 @@ language: python python: - 2.7 install: pip install tox --use-mirrors -script: tox -e py27-django13,py27-django12,py26-django13,py27-django12 +script: tox notifications: irc: "irc.freenode.org#imagekit" diff --git a/AUTHORS b/AUTHORS index 24968f0..d91cb93 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,9 +1,10 @@ ImageKit was originally written by `Justin Driscoll`_. -The field-based API was written by the bright minds at HZDG_. +The field-based API and other post-1.0 stuff was written by the bright people at +HZDG_. Maintainers -~~~~~~~~~~~ +----------- * `Bryan Veloso`_ * `Matthew Tretter`_ @@ -11,7 +12,7 @@ Maintainers * `Greg Newman`_ Contributors -~~~~~~~~~~~~ +------------ * `Josh Ourisman`_ * `Jonathan Slenders`_ diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst new file mode 100644 index 0000000..875ec2c --- /dev/null +++ b/CONTRIBUTING.rst @@ -0,0 +1,24 @@ +Contributing +------------ + +We love contributions! These guidelines will help make sure we can get your +contributions merged as quickly as possible: + +1. Write `good commit messages`__! +2. If you want to add a new feature, talk to us on the `mailing list`__ or + `IRC`__ first. We might already have plans, or be able to offer some advice. +3. Make sure your code passes the tests that ImageKit already has. To run the + tests, use ``make test``. This will let you know about any errors or style + issues. +4. While we're talking about tests, creating new ones for your code makes it + much easier for us to merge your code quickly. ImageKit uses nose_, so + writing tests is painless. Check out `ours`__ for examples. +5. It's a good idea to do your work in a branch; that way, you can work on more + than one contribution at a time without making them interdependent. + + +__ http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html +__ https://groups.google.com/forum/#!forum/django-imagekit +__ irc://irc.freenode.net/imagekit +.. _nose: https://nose.readthedocs.org/en/latest/ +__ https://github.com/jdriscoll/django-imagekit/tree/develop/tests diff --git a/README.rst b/README.rst index 041d02d..4263e8c 100644 --- a/README.rst +++ b/README.rst @@ -1,6 +1,7 @@ -ImageKit is a Django app that helps you to add variations of uploaded images -to your models. These variations are called "specs" and can include things -like different sizes (e.g. thumbnails) and black and white versions. +ImageKit is a Django app for processing images. Need a thumbnail? A +black-and-white version of a user-uploaded image? ImageKit will make them for +you. If you need to programatically generate one image from another, you need +ImageKit. **For the complete documentation on the latest stable version of ImageKit, see** `ImageKit on RTD`_. Our `changelog is also available`_. @@ -10,10 +11,10 @@ like different sizes (e.g. thumbnails) and black and white versions. Installation ------------- +============ -1. Install `PIL`_ or `Pillow`_. If you're using an ``ImageField`` in Django, - you should have already done this. +1. Install `PIL`_ or `Pillow`_. (If you're using an ``ImageField`` in Django, + you should have already done this.) 2. ``pip install django-imagekit`` (or clone the source and put the imagekit module on your path) 3. Add ``'imagekit'`` to your ``INSTALLED_APPS`` list in your project's settings.py @@ -27,85 +28,317 @@ Installation .. _`Pillow`: http://pypi.python.org/pypi/Pillow -Adding Specs to a Model ------------------------ +Usage Overview +============== -Much like ``django.db.models.ImageField``, Specs are defined as properties -of a model class: + +Specs +----- + +You have one image and you want to do something to it to create another image. +But how do you tell ImageKit what to do? By defining an image spec. + +An **image spec** is a type of **image generator** that generates a new image +from a source image. + + +Defining Specs In Models +^^^^^^^^^^^^^^^^^^^^^^^^ + +The easiest way to use define an image spec is by using an ImageSpecField on +your model class: .. code-block:: python from django.db import models from imagekit.models import ImageSpecField + from imagekit.processors import ResizeToFill - class Photo(models.Model): - original_image = models.ImageField(upload_to='photos') - formatted_image = ImageSpecField(image_field='original_image', format='JPEG', - options={'quality': 90}) + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(source='avatar', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) -Accessing the spec through a model instance will create the image and return -an ImageFile-like object (just like with a normal -``django.db.models.ImageField``): + profile = Profile.objects.all()[0] + print profile.avatar_thumbnail.url # > /media/CACHE/images/982d5af84cddddfd0fbf70892b4431e4.jpg + print profile.avatar_thumbnail.width # > 100 -.. code-block:: python +As you can probably tell, ImageSpecFields work a lot like Django's +ImageFields. The difference is that they're automatically generated by +ImageKit based on the instructions you give. In the example above, the avatar +thumbnail is a resized version of the avatar image, saved as a JPEG with a +quality of 60. - photo = Photo.objects.all()[0] - photo.original_image.url # > '/media/photos/birthday.tiff' - photo.formatted_image.url # > '/media/cache/photos/birthday_formatted_image.jpeg' - -Check out ``imagekit.models.ImageSpecField`` for more information. - -If you only want to save the processed image (without maintaining the original), -you can use a ``ProcessedImageField``: +Sometimes, however, you don't need to keep the original image (the avatar in +the above example); when the user uploads an image, you just want to process it +and save the result. In those cases, you can use the ``ProcessedImageField`` +class: .. code-block:: python from django.db import models - from imagekit.models.fields import ProcessedImageField + from imagekit.models import ProcessedImageField - class Photo(models.Model): - processed_image = ProcessedImageField(format='JPEG', options={'quality': 90}) + class Profile(models.Model): + avatar_thumbnail = ProcessedImageField(upload_to='avatars', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) -See the class documentation for details. + profile = Profile.objects.all()[0] + print profile.avatar_thumbnail.url # > /media/avatars/MY-avatar.jpg + print profile.avatar_thumbnail.width # > 100 + +This is pretty similar to our previous example. We don't need to specify a +"source" any more since we're not processing another image field, but we do need +to pass an "upload_to" argument. This behaves exactly as it does for Django +ImageFields. + +.. note:: + + You might be wondering why we didn't need an "upload_to" argument for our + ImageSpecField. The reason is that ProcessedImageFields really are just like + ImageFields—they save the file path in the database and you need to run + syncdb (or create a migration) when you add one to your model. + + ImageSpecFields, on the other hand, are virtual—they add no fields to your + database and don't require a database. This is handy for a lot of reasons, + but it means that the path to the image file needs to be programmatically + constructed based on the source image and the spec. + + +Defining Specs Outside of Models +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Defining specs as models fields is one very convenient way to process images, +but it isn't the only way. Sometimes you can't (or don't want to) add fields to +your models, and that's okay. You can define image spec classes and use them +directly. This can be especially useful for doing image processing in views— +particularly when the processing being done depends on user input. + +.. code-block:: python + + from imagekit import ImageSpec + from imagekit.processors import ResizeToFill + + class Thumbnail(ImageSpec): + processors = [ResizeToFill(100, 50)] + format = 'JPEG' + options = {'quality': 60} + +It's probaby not surprising that this class is capable of processing an image +in the exact same way as our ImageSpecField above. However, unlike with the +image spec model field, this class doesn't define what source the spec is acting +on, or what should be done with the result; that's up to you: + +.. code-block:: python + + source_file = open('/path/to/myimage.jpg') + image_generator = Thumbnail(source=source_file) + result = image_generator.generate() + +The result of calling ``generate()`` on an image spec is a file-like object +containing our resized image, with which you can do whatever you want. For +example, if you wanted to save it to disk: + +.. code-block:: python + + dest = open('/path/to/dest.jpg', 'w') + dest.write(result.read()) + dest.close() + + +Using Specs In Templates +^^^^^^^^^^^^^^^^^^^^^^^^ + +If you have a model with an ImageSpecField or ProcessedImageField, you can +easily use those processed image just as you would a normal image field: + +.. code-block:: html + + + +(This is assuming you have a view that's setting a context variable named +"profile" to an instance of our Profile model.) + +But you can also generate processed image files directly in your template—from +any image—without adding anything to your model. In order to do this, you'll +first have to define an image generator class (remember, specs are a type of +generator) in your app somewhere, just as we did in the last section. You'll +also need a way of referring to the generator in your template, so you'll need +to register it. + +.. code-block:: python + + from imagekit import ImageSpec + from imagekit.processors import ResizeToFill + + class Thumbnail(ImageSpec): + processors = [ResizeToFill(100, 50)] + format = 'JPEG' + options = {'quality': 60} + + register.generator('myapp:thumbnail', Thumbnail) + +.. note:: + + You can register your generator with any id you want, but choose wisely! + If you pick something too generic, you could have a conflict with another + third-party app you're using. For this reason, it's a good idea to prefix + your generator ids with the name of your app. Also, ImageKit recognizes + colons as separators when doing pattern matching (e.g. in the generateimages + management command), so it's a good idea to use those too! + +.. warning:: + + This code can go in any file you want—but you need to make sure it's loaded! + In order to keep things simple, ImageKit will automatically try to load an + module named "imagegenerators" in each of your installed apps. So why don't + you just save yourself the headache and put your image specs in there? + +Now that we've created an image generator class and registered it with ImageKit, +we can use it in our templates! + + +generateimage +""""""""""""" + +The most generic template tag that ImageKit gives you is called "generateimage". +It requires at least one argument: the id of a registered image generator. +Additional keyword-style arguments are passed to the registered generator class. +As we saw above, image spec constructors expect a source keyword argument, so +that's what we need to pass to use our thumbnail spec: + +.. code-block:: html + + {% load imagekit %} + + {% generateimage 'myapp:thumbnail' source=source_image %} + +This will output the following HTML: + +.. code-block:: html + + + +You can also add additional HTML attributes; just separate them from your +keyword args using two dashes: + +.. code-block:: html + + {% load imagekit %} + + {% generateimage 'myapp:thumbnail' source=source_image -- alt="A picture of Me" id="mypicture" %} + +Not generating HTML image tags? No problem. The tag also functions as an +assignment tag, providing access to the underlying file object: + +.. code-block:: html + + {% load imagekit %} + + {% generateimage 'myapp:thumbnail' source=source_image as th %} + Click to download a cool {{ th.width }} x {{ th.height }} image! + + +thumbnail +""""""""" + +Because it's such a common use case, ImageKit also provides a "thumbnail" +template tag: + +.. code-block:: html + + {% load imagekit %} + + {% thumbnail '100x50' source_image %} + +Like the generateimage tag, the thumbnail tag outputs an tag: + +.. code-block:: html + + + +Comparing this syntax to the generateimage tag above, you'll notice a few +differences. + +First, we didn't have to specify an image generator id; unless we tell it +otherwise, thumbnail tag uses the generator registered with the id +"imagekit:thumbnail". **It's important to note that this tag is *not* using the +Thumbnail spec class we defined earlier**; it's using the generator registered +with the id "imagekit:thumbnail" which, by default, is +``imagekit.generatorlibrary.Thumbnail``. + +Second, we're passing two positional arguments (the dimensions and the source +image) as opposed to the keyword arguments we used with the generateimage tag. + +Like with the generatethumbnail tag, you can also specify additional HTML +attributes for the thumbnail tag, or use it as an assignment tag: + +.. code-block:: html + + {% load imagekit %} + + {% thumbnail '100x50' source_image -- alt="A picture of Me" id="mypicture" %} + {% thumbnail '100x50' source_image as th %} + + +Using Specs in Forms +^^^^^^^^^^^^^^^^^^^^ + +In addition to the model field above, there's also a form field version of the +``ProcessedImageField`` class. The functionality is basically the same (it +processes an image once and saves the result), but it's used in a form class: + +.. code-block:: python + + from django import forms + from imagekit.forms import ProcessedImageField + from imagekit.processors import ResizeToFill + + class ProfileForm(forms.Form): + avatar_thumbnail = ProcessedImageField(spec_id='myapp:profile:avatar_thumbnail', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) + +The benefit of using ``imagekit.forms.ProcessedImageField`` (as opposed to +``imagekit.models.ProcessedImageField`` above) is that it keeps the logic for +creating the image outside of your model (in which you would use a normal Django +ImageField). You can even create multiple forms, each with their own +ProcessedImageField, that all store their results in the same image field. Processors ---------- -The real power of ImageKit comes from processors. Processors take an image, do -something to it, and return the result. By providing a list of processors to -your spec, you can expose different versions of the original image: +So far, we've only seen one processor: ``imagekit.processors.ResizeToFill``. But +ImageKit is capable of far more than just resizing images, and that power comes +from its processors. + +Processors take a PIL image object, do something to it, and return a new one. +A spec can make use of as many processors as you'd like, which will all be run +in order. .. code-block:: python - from django.db import models - from imagekit.models import ImageSpecField - from imagekit.processors import ResizeToFill, Adjust + from imagekit import ImageSpec + from imagekit.processors import TrimBorderColor, Adjust - class Photo(models.Model): - original_image = models.ImageField(upload_to='photos') - thumbnail = ImageSpecField([Adjust(contrast=1.2, sharpness=1.1), - ResizeToFill(50, 50)], image_field='original_image', - format='JPEG', options={'quality': 90}) - -The ``thumbnail`` property will now return a cropped image: - -.. code-block:: python - - photo = Photo.objects.all()[0] - photo.thumbnail.url # > '/media/cache/photos/birthday_thumbnail.jpeg' - photo.thumbnail.width # > 50 - photo.original_image.width # > 1000 - -The original image is not modified; ``thumbnail`` is a new file that is the -result of running the ``imagekit.processors.ResizeToFill`` processor on the -original. (If you only need to save the processed image, and not the original, -pass processors to a ``ProcessedImageField`` instead of an ``ImageSpecField``.) + class MySpec(ImageSpec): + processors = [ + TrimBorderColor(), + Adjust(contrast=1.2, sharpness=1.1), + ] + format = 'JPEG' + options = {'quality': 60} The ``imagekit.processors`` module contains processors for many common image manipulations, like resizing, rotating, and color adjustments. However, if they aren't up to the task, you can create your own. All you have to do is -implement a ``process()`` method: +define a class that implements a ``process()`` method: .. code-block:: python @@ -114,10 +347,23 @@ implement a ``process()`` method: # Code for adding the watermark goes here. return image - class Photo(models.Model): - original_image = models.ImageField(upload_to='photos') - watermarked_image = ImageSpecField([Watermark()], image_field='original_image', - format='JPEG', options={'quality': 90}) +That's all there is to it! To use your fancy new custom processor, just include +it in your spec's ``processors`` list: + +.. code-block:: python + + from imagekit import ImageSpec + from imagekit.processors import TrimBorderColor, Adjust + from myapp.processors import Watermark + + class MySpec(ImageSpec): + processors = [ + TrimBorderColor(), + Adjust(contrast=1.2, sharpness=1.1), + Watermark(), + ] + format = 'JPEG' + options = {'quality': 60} Admin @@ -134,12 +380,10 @@ Django admin classes: from imagekit.admin import AdminThumbnail from .models import Photo - class PhotoAdmin(admin.ModelAdmin): list_display = ('__str__', 'admin_thumbnail') admin_thumbnail = AdminThumbnail(image_field='thumbnail') - admin.site.register(Photo, PhotoAdmin) AdminThumbnail can even use a custom template. For more information, see @@ -148,42 +392,16 @@ AdminThumbnail can even use a custom template. For more information, see .. _`Django admin change list`: https://docs.djangoproject.com/en/dev/intro/tutorial02/#customize-the-admin-change-list -Image Cache Backends --------------------- +Managment Commands +------------------ -Whenever you access properties like ``url``, ``width`` and ``height`` of an -``ImageSpecField``, its cached image is validated; whenever you save a new image -to the ``ImageField`` your spec uses as a source, the spec image is invalidated. -The default way to validate a cache image is to check to see if the file exists -and, if not, generate a new one; the default way to invalidate the cache is to -delete the image. This is a very simple and straightforward way to handle cache -validation, but it has its drawbacks—for example, checking to see if the image -exists means frequently hitting the storage backend. - -Because of this, ImageKit allows you to define custom image cache backends. To -be a valid image cache backend, a class must implement three methods: -``validate``, ``invalidate``, and ``clear`` (which is called when the image is -no longer needed in any form, i.e. the model is deleted). Each of these methods -must accept a file object, but the internals are up to you. For example, you -could store the state (valid, invalid) of the cache in a database to avoid -filesystem access. You can then specify your image cache backend on a per-field -basis: - -.. code-block:: python - - class Photo(models.Model): - ... - thumbnail = ImageSpecField(..., image_cache_backend=MyImageCacheBackend()) - -Or in your ``settings.py`` file if you want to use it as the default: - -.. code-block:: python - - IMAGEKIT_DEFAULT_IMAGE_CACHE_BACKEND = 'path.to.MyImageCacheBackend' +ImageKit has one management command—`generateimages`—which will generate cache +files for all of your registered image generators. You can also pass it a list +of generator ids in order to generate images selectively. Community ---------- +========= Please use `the GitHub issue tracker `_ to report bugs with django-imagekit. `A mailing list `_ @@ -192,7 +410,7 @@ also exists to discuss the project and ask questions, as well as the official Contributing ------------- +============ We love contributions! And you don't have to be an expert with the library—or even Django—to contribute either: ImageKit's processors are standalone classes @@ -200,6 +418,12 @@ that are completely separate from the more intimidating internals of Django's ORM. If you've written a processor that you think might be useful to other people, open a pull request so we can take a look! -ImageKit's image cache backends are also fairly isolated from the ImageKit guts. -If you've fine-tuned one to work perfectly for a popular file storage backend, -let us take a look! Maybe other people could use it. +You can also check out our list of `open, contributor-friendly issues`__ for +ideas. + +Check out our `contributing guidelines`__ for more information about pitching in +with ImageKit. + + +__ https://github.com/jdriscoll/django-imagekit/issues?labels=contributor-friendly&state=open +__ https://github.com/jdriscoll/django-imagekit/blob/master/CONTRIBUTING.rst diff --git a/docs/advanced_usage.rst b/docs/advanced_usage.rst new file mode 100644 index 0000000..917b642 --- /dev/null +++ b/docs/advanced_usage.rst @@ -0,0 +1,284 @@ +Advanced Usage +************** + + +Models +====== + + +The ``ImageSpecField`` Shorthand Syntax +--------------------------------------- + +If you've read the README, you already know what an ``ImageSpecField`` is and +the basics of defining one: + +.. code-block:: python + + from django.db import models + from imagekit.models import ImageSpecField + from imagekit.processors import ResizeToFill + + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(source='avatar', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) + +This will create an ``avatar_thumbnail`` field which is a resized version of the +image stored in the ``avatar`` image field. But this is actually just shorthand +for creating an ``ImageSpec``, registering it, and associating it with an +``ImageSpecField``: + +.. code-block:: python + + from django.db import models + from imagekit import ImageSpec, register + from imagekit.models import ImageSpecField + from imagekit.processors import ResizeToFill + + class AvatarThumbnail(ImageSpec): + processors = [ResizeToFill(100, 50)] + format = 'JPEG' + options = {'quality': 60} + + register.generator('myapp:profile:avatar_thumbnail', AvatarThumbnail) + + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(source='avatar', + spec_id='myapp:profile:avatar_thumbnail') + +Obviously, the shorthand version is a lot, well…shorter. So why would you ever +want to go through the trouble of using the long form? The answer is that the +long form—creating an image spec class and registering it—gives you a lot more +power over the generated image. + + +.. _dynamic-specs: + +Specs That Change +----------------- + +As you'll remember from the README, an image spec is just a type of image +generator that generates a new image from a source image. How does the image +spec get access to the source image? Simple! It's passed to the constructor as +a keyword argument and stored as an attribute of the spec. Normally, we don't +have to concern ourselves with this; the ``ImageSpec`` knows what to do with the +source image and we're happy to let it do its thing. However, having access to +the source image in our spec class can be very useful… + +Often, when using an ``ImageSpecField``, you may want the spec to vary based on +properties of a model. (For example, you might want to store image dimensions on +the model and then use them to generate your thumbnail.) Now that we know how to +access the source image from our spec, it's a simple matter to extract its model +and use it to create our processors list. In fact, ImageKit includes a utility +for getting this information. + +.. code-block:: python + :emphasize-lines: 11-14 + + from django.db import models + from imagekit import ImageSpec, register + from imagekit.models import ImageSpecField + from imagekit.processors import ResizeToFill + from imagekit.utils import get_field_info + + class AvatarThumbnail(ImageSpec): + format = 'JPEG' + options = {'quality': 60} + + @property + def processors(self): + model, field_name = get_field_info(self.source) + return [ResizeToFill(model.thumbnail_width, thumbnail.avatar_height)] + + register.generator('myapp:profile:avatar_thumbnail', AvatarThumbnail) + + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(source='avatar', + spec_id='myapp:profile:avatar_thumbnail') + thumbnail_width = models.PositiveIntegerField() + thumbnail_height = models.PositiveIntegerField() + +Now each avatar thumbnail will be resized according to the dimensions stored on +the model! + +Of course, processors aren't the only thing that can vary based on the model of +the source image; spec behavior can change in any way you want. + + +Optimizing +========== + +Unlike Django's ImageFields, ImageKit's ImageSpecFields and template tags don't +persist any data in the database. Therefore, in order to know whether an image +file needs to be generated, ImageKit needs to check if the file already exists +(using the appropriate file storage object`__). The object responsible for +performing these checks is called a *cache file backend*. + + +Cache! +------ + +By default, ImageKit checks for the existence of a cache file every time you +attempt to use the file and, if it doesn't exist, creates it synchronously. This +is a very safe behavior because it ensures that your ImageKit-generated images +are always available. However, that's a lot of checking with storage and those +kinds of operations can be slow—especially if you're using a remote storage—so +you'll want to try to avoid them as much as possible. + +Luckily, the default cache file backend makes use of Django's caching +abilities to mitigate the number of checks it actually has to do; it will use +the cache specified by the ``IMAGEKIT_CACHE_BACKEND`` to save the state of the +generated file. If your Django project is running in debug mode +(``settings.DEBUG`` is true), this will be a dummy cache by default. Otherwise, +it will use your project's default cache. + +In normal operation, your cache files will never be deleted; once they're +created, they'll stay created. So the simplest optimization you can make is to +set your ``IMAGEKIT_CACHE_BACKEND`` to a cache with a very long, or infinite, +timeout. + + +Even More Advanced +------------------ + +For many applications—particularly those using local storage for generated image +files—a cache with a long timeout is all the optimization you'll need. However, +there may be times when that simply doesn't cut it. In these cases, you'll want +to change when the generation is actually done. + +The objects responsible for specifying when cache files are created are +called *cache file strategies*. The default strategy can be set using the +``IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY`` setting, and its default value is +`'imagekit.cachefiles.strategies.JustInTime'`. As we've already seen above, +the "just in time" strategy determines whether a file needs to be generated each +time it's accessed and, if it does, generates it synchronously (that is, as part +of the request-response cycle). + +Another strategy is to simply assume the file exists. This requires the fewest +number of checks (zero!), so we don't have to worry about expensive IO. The +strategy that takes this approach is +``imagekit.cachefiles.strategies.Optimistic``. In order to use this +strategy, either set the ``IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY`` setting or, +to use it on a per-generator basis, set the ``cachefile_strategy`` attribute +of your spec or generator. Avoiding checking for file existence can be a real +boon to performance, but it also means that ImageKit has no way to know when a +file needs to be generated—well, at least not all the time. + +With image specs, we can know at least some of the times that a new file needs +to be generated: whenever the source image is created or changed. For this +reason, the optimistic strategy defines callbacks for these events. Every +source registered with ImageKit will automatically cause its specs' files to be +generated when it is created or changed. + +.. note:: + + In order to understand source registration, read :ref:`source-groups` + +If you have specs that :ref:`change based on attributes of the source +`, that's not going to cut it, though; the file will also need to +be generated when those attributes change. Likewise, image generators that don't +have sources (i.e. generators that aren't specs) won't cause files to be +generated automatically when using the optimistic strategy. (ImageKit can't know +when those need to be generated, if not on access.) In both cases, you'll have +to trigger the file generation yourself—either by generating the file in code +when necessary, or by periodically running the ``generateimages`` management +command. Luckily, ImageKit makes this pretty easy: + +.. code-block:: python + + from imagekit.cachefiles import LazyGeneratedImageFile + + file = LazyGeneratedImageFile('myapp:profile:avatar_thumbnail', source=source_file) + file.generate() + +One final situation in which images won't be generated automatically when using +the optimistic strategy is when you use a spec with a source that hasn't been +registered with it. Unlike the previous two examples, this situation cannot be +rectified by running the ``generateimages`` management command, for the simple +reason that the command has no way of knowing it needs to generate a file for +that spec from that source. Typically, this situation would arise when using the +template tags. Unlike ImageSpecFields, which automatically register all the +possible source images with the spec you define, the template tags +("generateimage" and "thumbnail") let you use any spec with any source. +Therefore, in order to generate the appropriate files using the +``generateimages`` management command, you'll need to first register a source +group that represents all of the sources you wish to use with the corresponding +specs. See :ref:`source-groups` for more information. + + +.. _source-groups: + +Source Groups +============= + +When you run the ``generateimages`` management command, how does ImageKit know +which source images to use with which specs? Obviously, when you define an +ImageSpecField, the source image is being connected to a spec, but what's going +on underneath the hood? + +The answer is that, when you define an ImageSpecField, ImageKit automatically +creates and registers an object called a *source group*. Source groups are +responsible for two things: + +1. They dispatch signals when a source is created, changed, or deleted, and +2. They expose a generator method that enumerates source files. + +When these objects are registered (using ``imagekit.register.source_group()``), +their signals will trigger callbacks on the cache file strategies associated +with image specs that use the source. (So, for example, you can chose to +generate a file every time the source image changes.) In addition, the generator +method is used (indirectly) to create the list of files to generate with the +``generateimages`` management command. + +Currently, there is only one source group class bundled with ImageKit—the one +used by ImageSpecFields. This source group +(``imagekit.specs.sourcegroups.ImageFieldSourceGroup``) represents an ImageField +on every instance of a particular model. In terms of the above description, the +instance ``ImageFieldSourceGroup(Profile, 'avatar')`` 1) dispatches a signal +every time the image in Profile's avatar ImageField changes, and 2) exposes a +generator method that iterates over every Profile's "avatar" image. + +Chances are, this is the only source group you will ever need to use, however, +ImageKit lets you define and register custom source groups easily. This may be +useful, for example, if you're using the template tags "generateimage" and +"thumbnail" and the optimistic cache file strategy. Again, the purpose is +to tell ImageKit which specs are used with which sources (so the +"generateimages" management command can generate those files) and when the +source image has been created or changed (so that the strategy has the +opportunity to act on it). + +A simple example of a custom source group class is as follows: + +.. code-block:: python + + import glob + import os + + class JpegsInADirectory(object): + def __init__(self, dir): + self.dir = dir + + def files(self): + os.chdir(self.dir) + for name in glob.glob('*.jpg'): + yield open(name) + +Instances of this class could then be registered with one or more spec id: + +.. code-block:: python + + from imagekit import register + + register.source_group('myapp:profile:avatar_thumbnail', JpegsInADirectory('/path/to/some/pics')) + +Running the "generateimages" management command would now cause thumbnails to be +generated (using the "myapp:profile:avatar_thumbnail" spec) for each of the +JPEGs in `/path/to/some/pics`. + +Note that, since this source group doesnt send the `source_created` or +`source_changed` signals, the corresponding cache file strategy callbacks +would not be called for them. diff --git a/docs/apireference.rst b/docs/apireference.rst deleted file mode 100644 index d4a2ed8..0000000 --- a/docs/apireference.rst +++ /dev/null @@ -1,29 +0,0 @@ -API Reference -============= - - -:mod:`models` Module --------------------- - -.. automodule:: imagekit.models.fields - :members: - - -:mod:`processors` Module ------------------------- - -.. automodule:: imagekit.processors - :members: - -.. automodule:: imagekit.processors.resize - :members: - -.. automodule:: imagekit.processors.crop - :members: - - -:mod:`admin` Module --------------------- - -.. automodule:: imagekit.admin - :members: diff --git a/docs/conf.py b/docs/conf.py index e0913b9..35f184d 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -11,7 +11,7 @@ # All configuration values have a default; values that are commented out # serve to show the default. -import sys, os +import re, sys, os # If extensions (or modules to document with autodoc) are in another directory, # add these directories to sys.path here. If the directory is relative to the @@ -45,14 +45,18 @@ master_doc = 'index' project = u'ImageKit' copyright = u'2011, Justin Driscoll, Bryan Veloso, Greg Newman, Chris Drackett & Matthew Tretter' +pkgmeta = {} +execfile(os.path.join(os.path.dirname(__file__), '..', 'imagekit', + 'pkgmeta.py'), pkgmeta) + # The version info for the project you're documenting, acts as replacement for # |version| and |release|, also used in various other places throughout the # built documents. # # The short X.Y version. -version = '2.0.2' +version = re.match('\d+\.\d+', pkgmeta['__version__']).group() # The full version, including alpha/beta/rc tags. -release = '2.0.2' +release = pkgmeta['__version__'] # The language for content autogenerated by Sphinx. Refer to documentation # for a list of supported languages. diff --git a/docs/configuration.rst b/docs/configuration.rst new file mode 100644 index 0000000..6216158 --- /dev/null +++ b/docs/configuration.rst @@ -0,0 +1,74 @@ +.. _settings: + +Configuration +============= + + +Settings +-------- + +.. currentmodule:: django.conf.settings + + +.. attribute:: IMAGEKIT_CACHEFILE_DIR + + :default: ``'CACHE/images'`` + + The directory to which image files will be cached. + + +.. attribute:: IMAGEKIT_DEFAULT_FILE_STORAGE + + :default: ``None`` + + The qualified class name of a Django storage backend to use to save the + cached images. If no value is provided for ``IMAGEKIT_DEFAULT_FILE_STORAGE``, + and none is specified by the spec definition, the storage of the source file + will be used. + + +.. attribute:: IMAGEKIT_DEFAULT_CACHEFILE_BACKEND + + :default: ``'imagekit.cachefiles.backends.Simple'`` + + Specifies the class that will be used to validate cached image files. + + +.. attribute:: IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY + + :default: ``'imagekit.cachefiles.strategies.JustInTime'`` + + The class responsible for specifying how and when cache files are + generated. + + +.. attribute:: IMAGEKIT_CACHE_BACKEND + + :default: If ``DEBUG`` is ``True``, ``'django.core.cache.backends.dummy.DummyCache'``. + Otherwise, ``'default'``. + + The Django cache backend to be used to store information like the state of + cached images (i.e. validated or not). + + +.. attribute:: IMAGEKIT_CACHE_PREFIX + + :default: ``'imagekit:'`` + + A cache prefix to be used when values are stored in ``IMAGEKIT_CACHE_BACKEND`` + + +.. attribute:: IMAGEKIT_CACHEFILE_NAMER + + :default: ``'imagekit.cachefiles.namers.hash'`` + + A function responsible for generating file names for non-spec cache files. + + +.. attribute:: IMAGEKIT_SPEC_CACHEFILE_NAMER + + :default: ``'imagekit.cachefiles.namers.source_name_as_path'`` + + A function responsible for generating file names for cache files that + correspond to image specs. Since you will likely want to base the name of + your cache files on the name of the source, this extra setting is provided. diff --git a/docs/index.rst b/docs/index.rst index 2c06385..04eee0f 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,43 +1,24 @@ -Getting Started -=============== - .. include:: ../README.rst -Commands --------- - -.. automodule:: imagekit.management.commands.ikcacheinvalidate - -.. automodule:: imagekit.management.commands.ikcachevalidate - - Authors -------- +======= .. include:: ../AUTHORS -Community ---------- - -The official Freenode channel for ImageKit is `#imagekit `_. -You should always find some fine people to answer your questions -about ImageKit there. - - -Digging Deeper --------------- - -.. toctree:: - - apireference - changelog - - Indices and tables ================== * :ref:`genindex` * :ref:`modindex` * :ref:`search` + +.. toctree:: + :glob: + :maxdepth: 2 + + configuration + advanced_usage + changelog + upgrading diff --git a/docs/upgrading.rst b/docs/upgrading.rst new file mode 100644 index 0000000..4efe7c0 --- /dev/null +++ b/docs/upgrading.rst @@ -0,0 +1,114 @@ +Upgrading from 2.x +================== + +ImageKit 3.0 introduces new APIs and tools that augment, improve, and in some +cases entirely replace old IK workflows. Below, you'll find some useful guides +for migrating your ImageKit 2.0 apps over to the shiny new IK3. + + +Model Specs +----------- + +IK3 is chock full of new features and better tools for even the most +sophisticated use cases. Despite this, not too much has changed when it +comes to the most common of use cases: processing an ``ImageField`` on a model. + +In IK2, you may have used an ``ImageSpecField`` on a model to process an +existing ``ImageField``: + +.. code-block:: python + + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(image_field='avatar', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) + +In IK3, things look much the same: + +.. code-block:: python + + class Profile(models.Model): + avatar = models.ImageField(upload_to='avatars') + avatar_thumbnail = ImageSpecField(source='avatar', + processors=[ResizeToFill(100, 50)], + format='JPEG', + options={'quality': 60}) + +The major difference is that ``ImageSpecField`` no longer takes an +``image_field`` kwarg. Instead, you define a ``source``. + + +Image Cache Backends +-------------------- + +In IK2, you could gain some control over how your cached images were generated +by providing an ``image_cache_backend``: + +.. code-block:: python + + class Photo(models.Model): + ... + thumbnail = ImageSpecField(..., image_cache_backend=MyImageCacheBackend()) + +This gave you great control over *how* your images are generated and stored, +but it could be difficult to control *when* they were generated and stored. + +IK3 retains the image cache backend concept (now called cache file backends), +but separates the 'when' control out to cache file strategies: + +.. code-block:: python + + class Photo(models.Model): + ... + thumbnail = ImageSpecField(..., + cachefile_backend=MyCacheFileBackend(), + cachefile_strategy=MyCacheFileStrategy()) + +If you are using the IK2 default image cache backend setting: + +.. code-block:: python + + IMAGEKIT_DEFAULT_IMAGE_CACHE_BACKEND = 'path.to.MyImageCacheBackend' + +IK3 provides analogous settings for cache file backends and strategies: + +.. code-block:: python + + IMAGEKIT_DEFAULT_CACHEFILE_BACKEND = 'path.to.MyCacheFileBackend' + IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY = 'path.to.MyCacheFileStrategy' + +See the documentation on `cache file backends`_ and `cache file strategies`_ +for more details. + +.. _`cache file backends`: +.. _`cache file strategies`: + + +Conditional model ``processors`` +-------------------------------- + +In IK2, an ``ImageSpecField`` could take a ``processors`` callable instead of +an iterable, which allowed processing decisions to made based on other +properties of the model. IK3 does away with this feature for consistency's sake +(if one kwarg could be callable, why not all?), but provides a much more robust +solution: the custom ``spec``. See the `advanced usage`_ documentation for more. + +.. _`advanced usage`: + + +Conditonal ``cache_to`` file names +---------------------------------- + +IK2 provided a means of specifying custom cache file names for your +image specs by passing a ``cache_to`` callable to an ``ImageSpecField``. +IK3 does away with this feature, again, for consistency. + +There is a way to achieve custom file names by overriding your spec's +``cachefile_name``, but it is not recommended, as the spec's default +behavior is to hash the combination of ``source``, ``processors``, ``format``, +and other spec options to ensure that changes to the spec always result in +unique file names. See the documentation on `specs`_ for more. + +.. _`specs`: diff --git a/imagekit/__init__.py b/imagekit/__init__.py index 9dd4ffe..e65c270 100644 --- a/imagekit/__init__.py +++ b/imagekit/__init__.py @@ -1,34 +1,7 @@ -__title__ = 'django-imagekit' -__author__ = 'Justin Driscoll, Bryan Veloso, Greg Newman, Chris Drackett, Matthew Tretter, Eric Eldredge' -__version__ = (2, 0, 2, 'final', 0) -__license__ = 'BSD' +# flake8: noqa - -def get_version(version=None): - """Derives a PEP386-compliant version number from VERSION.""" - if version is None: - version = __version__ - assert len(version) == 5 - assert version[3] in ('alpha', 'beta', 'rc', 'final') - - # Now build the two parts of the version number: - # main = X.Y[.Z] - # sub = .devN - for pre-alpha releases - # | {a|b|c}N - for alpha, beta and rc releases - - parts = 2 if version[2] == 0 else 3 - main = '.'.join(str(x) for x in version[:parts]) - - sub = '' - if version[3] == 'alpha' and version[4] == 0: - # At the toplevel, this would cause an import loop. - from django.utils.version import get_svn_revision - svn_revision = get_svn_revision()[4:] - if svn_revision != 'unknown': - sub = '.dev%s' % svn_revision - - elif version[3] != 'final': - mapping = {'alpha': 'a', 'beta': 'b', 'rc': 'c'} - sub = mapping[version[3]] + str(version[4]) - - return main + sub +from . import conf +from . import generatorlibrary +from .specs import ImageSpec +from .pkgmeta import * +from .registry import register, unregister diff --git a/imagekit/admin.py b/imagekit/admin.py index 4466e6e..6b37a4c 100644 --- a/imagekit/admin.py +++ b/imagekit/admin.py @@ -27,10 +27,10 @@ class AdminThumbnail(object): try: thumbnail = getattr(obj, self.image_field) except AttributeError: - raise Exception('The property %s is not defined on %s.' % \ + raise Exception('The property %s is not defined on %s.' % (self.image_field, obj.__class__.__name__)) - original_image = getattr(thumbnail, 'source_file', None) or thumbnail + original_image = getattr(thumbnail, 'source', None) or thumbnail template = self.template or 'imagekit/admin/thumbnail.html' return render_to_string(template, { diff --git a/imagekit/cachefiles/__init__.py b/imagekit/cachefiles/__init__.py new file mode 100644 index 0000000..25c822f --- /dev/null +++ b/imagekit/cachefiles/__init__.py @@ -0,0 +1,95 @@ +from django.conf import settings +from django.core.files.images import ImageFile +from django.utils.functional import LazyObject +from ..files import BaseIKFile +from ..registry import generator_registry +from ..signals import before_access +from ..utils import get_logger, get_singleton, generate, get_by_qname + + +class GeneratedImageFile(BaseIKFile, ImageFile): + """ + A file that represents the result of a generator. Creating an instance of + this class is not enough to trigger the generation of the file. In fact, + one of the main points of this class is to allow the creation of the file + to be deferred until the time that the cache file strategy requires it. + + """ + def __init__(self, generator, name=None, storage=None, cachefile_backend=None): + """ + :param generator: The object responsible for generating a new image. + :param name: The filename + :param storage: A Django storage object that will be used to save the + file. + :param cachefile_backend: The object responsible for managing the + state of the file. + + """ + self.generator = generator + + name = name or getattr(generator, 'cachefile_name', None) + if not name: + fn = get_by_qname(settings.IMAGEKIT_CACHEFILE_NAMER, 'namer') + name = fn(generator) + self.name = name + + storage = storage or getattr(generator, 'cachefile_storage', + None) or get_singleton(settings.IMAGEKIT_DEFAULT_FILE_STORAGE, + 'file storage backend') + self.cachefile_backend = cachefile_backend or getattr(generator, + 'cachefile_backend', None) + + super(GeneratedImageFile, self).__init__(storage=storage) + + def _require_file(self): + before_access.send(sender=self, file=self) + return super(GeneratedImageFile, self)._require_file() + + def generate(self, force=False): + if force: + self._generate() + else: + self.cachefile_backend.ensure_exists(self) + + def _generate(self): + # Generate the file + content = generate(self.generator) + + actual_name = self.storage.save(self.name, content) + + if actual_name != self.name: + get_logger().warning('The storage backend %s did not save the file' + ' with the requested name ("%s") and instead used' + ' "%s". This may be because a file already existed with' + ' the requested name. If so, you may have meant to call' + ' ensure_exists() instead of generate(), or there may be a' + ' race condition in the file backend %s. The saved file' + ' will not be used.' % (self.storage, + self.name, actual_name, + self.cachefile_backend)) + + +class LazyGeneratedImageFile(LazyObject): + def __init__(self, generator_id, *args, **kwargs): + super(LazyGeneratedImageFile, self).__init__() + + def setup(): + generator = generator_registry.get(generator_id, *args, **kwargs) + self._wrapped = GeneratedImageFile(generator) + + self.__dict__['_setup'] = setup + + def __repr__(self): + if self._wrapped is None: + self._setup() + return '<%s: %s>' % (self.__class__.__name__, self or 'None') + + def __str__(self): + if self._wrapped is None: + self._setup() + return str(self._wrapped) + + def __unicode__(self): + if self._wrapped is None: + self._setup() + return unicode(self._wrapped) diff --git a/imagekit/cachefiles/actions.py b/imagekit/cachefiles/actions.py new file mode 100644 index 0000000..634bcbe --- /dev/null +++ b/imagekit/cachefiles/actions.py @@ -0,0 +1,22 @@ +def generate(file): + file.generate() + + +try: + from celery.task import task +except ImportError: + pass +else: + generate_task = task(generate) + + +def generate_deferred(file): + try: + import celery # NOQA + except: + raise ImportError("Deferred validation requires the the 'celery' library") + generate_task.delay(file) + + +def clear_now(file): + file.clear() diff --git a/imagekit/cachefiles/backends.py b/imagekit/cachefiles/backends.py new file mode 100644 index 0000000..15813be --- /dev/null +++ b/imagekit/cachefiles/backends.py @@ -0,0 +1,64 @@ +from ..utils import get_singleton +from django.core.cache import get_cache +from django.core.exceptions import ImproperlyConfigured + + +def get_default_cachefile_backend(): + """ + Get the default file backend. + + """ + from django.conf import settings + return get_singleton(settings.IMAGEKIT_DEFAULT_CACHEFILE_BACKEND, + 'file backend') + + +class InvalidFileBackendError(ImproperlyConfigured): + pass + + +class CachedFileBackend(object): + @property + def cache(self): + if not getattr(self, '_cache', None): + from django.conf import settings + self._cache = get_cache(settings.IMAGEKIT_CACHE_BACKEND) + return self._cache + + def get_key(self, file): + from django.conf import settings + return '%s%s-exists' % (settings.IMAGEKIT_CACHE_PREFIX, file.name) + + def file_exists(self, file): + key = self.get_key(file) + exists = self.cache.get(key) + if exists is None: + exists = self._file_exists(file) + self.cache.set(key, exists) + return exists + + def ensure_exists(self, file): + if self.file_exists(file): + self.create(file) + self.cache.set(self.get_key(file), True) + + +class Simple(CachedFileBackend): + """ + The most basic file backend. The storage is consulted to see if the file + exists. + + """ + + def _file_exists(self, file): + if not getattr(file, '_file', None): + # No file on object. Have to check storage. + return not file.storage.exists(file.name) + return False + + def create(self, file): + """ + Generates a new image by running the processors on the source file. + + """ + file.generate(force=True) diff --git a/imagekit/cachefiles/namers.py b/imagekit/cachefiles/namers.py new file mode 100644 index 0000000..d6bc95a --- /dev/null +++ b/imagekit/cachefiles/namers.py @@ -0,0 +1,91 @@ +""" +Functions responsible for returning filenames for the given image generator. +Users are free to define their own functions; these are just some some sensible +choices. + +""" + +from django.conf import settings +import os +from ..utils import format_to_extension, suggest_extension + + +def source_name_as_path(generator): + """ + A namer that, given the following source file name:: + + photos/thumbnails/bulldog.jpg + + will generate a name like this:: + + /path/to/generated/images/photos/thumbnails/bulldog/5ff3233527c5ac3e4b596343b440ff67.jpg + + where "/path/to/generated/images/" is the value specified by the + ``IMAGEKIT_CACHEFILE_DIR`` setting. + + """ + source_filename = getattr(generator.source, 'name', None) + + if source_filename is None or os.path.isabs(source_filename): + # Generally, we put the file right in the cache file directory. + dir = settings.IMAGEKIT_CACHEFILE_DIR + else: + # For source files with relative names (like Django media files), + # use the source's name to create the new filename. + dir = os.path.join(settings.IMAGEKIT_CACHEFILE_DIR, + os.path.splitext(source_filename)[0]) + + ext = suggest_extension(source_filename or '', generator.format) + return os.path.normpath(os.path.join(dir, + '%s%s' % (generator.get_hash(), ext))) + + +def source_name_dot_hash(generator): + """ + A namer that, given the following source file name:: + + photos/thumbnails/bulldog.jpg + + will generate a name like this:: + + /path/to/generated/images/photos/thumbnails/bulldog.5ff3233527c5.jpg + + where "/path/to/generated/images/" is the value specified by the + ``IMAGEKIT_CACHEFILE_DIR`` setting. + + """ + source_filename = getattr(generator.source, 'name', None) + + if source_filename is None or os.path.isabs(source_filename): + # Generally, we put the file right in the cache file directory. + dir = settings.IMAGEKIT_CACHEFILE_DIR + else: + # For source files with relative names (like Django media files), + # use the source's name to create the new filename. + dir = os.path.join(settings.IMAGEKIT_CACHEFILE_DIR, + os.path.dirname(source_filename)) + + ext = suggest_extension(source_filename or '', generator.format) + basename = os.path.basename(source_filename) + return os.path.normpath(os.path.join(dir, '%s.%s%s' % ( + os.path.splitext(basename)[0], generator.get_hash()[:12], ext))) + + +def hash(generator): + """ + A namer that, given the following source file name:: + + photos/thumbnails/bulldog.jpg + + will generate a name like this:: + + /path/to/generated/images/5ff3233527c5ac3e4b596343b440ff67.jpg + + where "/path/to/generated/images/" is the value specified by the + ``IMAGEKIT_CACHEFILE_DIR`` setting. + + """ + format = getattr(generator, 'format', None) + ext = format_to_extension(format) if format else '' + return os.path.normpath(os.path.join(settings.IMAGEKIT_CACHEFILE_DIR, + '%s%s' % (generator.get_hash(), ext))) diff --git a/imagekit/cachefiles/strategies.py b/imagekit/cachefiles/strategies.py new file mode 100644 index 0000000..1104de6 --- /dev/null +++ b/imagekit/cachefiles/strategies.py @@ -0,0 +1,56 @@ +from django.utils.functional import LazyObject +from ..utils import get_singleton + + +class JustInTime(object): + """ + A strategy that ensures the file exists right before it's needed. + + """ + + def before_access(self, file): + file.generate() + + +class Optimistic(object): + """ + A strategy that acts immediately when the source file changes and assumes + that the cache files will not be removed (i.e. it doesn't ensure the + cache file exists when it's accessed). + + """ + + def on_source_created(self, file): + file.generate() + + def on_source_changed(self, file): + file.generate() + + +class DictStrategy(object): + def __init__(self, callbacks): + for k, v in callbacks.items(): + setattr(self, k, v) + + +class StrategyWrapper(LazyObject): + def __init__(self, strategy): + if isinstance(strategy, basestring): + strategy = get_singleton(strategy, 'cache file strategy') + elif isinstance(strategy, dict): + strategy = DictStrategy(strategy) + elif callable(strategy): + strategy = strategy() + self._wrapped = strategy + + def __getstate__(self): + return {'_wrapped': self._wrapped} + + def __setstate__(self, state): + self._wrapped = state['_wrapped'] + + def __unicode__(self): + return unicode(self._wrapped) + + def __str__(self): + return str(self._wrapped) diff --git a/imagekit/conf.py b/imagekit/conf.py index b429ff2..4c2f2e7 100644 --- a/imagekit/conf.py +++ b/imagekit/conf.py @@ -1,6 +1,23 @@ from appconf import AppConf +from django.conf import settings class ImageKitConf(AppConf): - DEFAULT_IMAGE_CACHE_BACKEND = 'imagekit.imagecache.PessimisticImageCacheBackend' - DEFAULT_FILE_STORAGE = None + CACHEFILE_NAMER = 'imagekit.cachefiles.namers.hash' + SPEC_CACHEFILE_NAMER = 'imagekit.cachefiles.namers.source_name_as_path' + CACHEFILE_DIR = 'CACHE/images' + DEFAULT_CACHEFILE_BACKEND = 'imagekit.cachefiles.backends.Simple' + DEFAULT_CACHEFILE_STRATEGY = 'imagekit.cachefiles.strategies.JustInTime' + + DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage' + + CACHE_BACKEND = None + CACHE_PREFIX = 'imagekit:' + + def configure_cache_backend(self, value): + if value is None: + if getattr(settings, 'CACHES', None): + value = 'django.core.cache.backends.dummy.DummyCache' if settings.DEBUG else 'default' + else: + value = 'dummy://' if settings.DEBUG else settings.CACHE_BACKEND + return value diff --git a/imagekit/exceptions.py b/imagekit/exceptions.py new file mode 100644 index 0000000..308c0a1 --- /dev/null +++ b/imagekit/exceptions.py @@ -0,0 +1,18 @@ +class AlreadyRegistered(Exception): + pass + + +class NotRegistered(Exception): + pass + + +class UnknownExtensionError(Exception): + pass + + +class UnknownFormatError(Exception): + pass + + +class MissingGeneratorId(Exception): + pass diff --git a/imagekit/files.py b/imagekit/files.py new file mode 100644 index 0000000..fb4375c --- /dev/null +++ b/imagekit/files.py @@ -0,0 +1,95 @@ +from django.core.files.base import File, ContentFile +from django.utils.encoding import smart_str, smart_unicode +import os +from .utils import format_to_mimetype, extension_to_mimetype + + +class BaseIKFile(File): + """ + This class contains all of the methods we need from + django.db.models.fields.files.FieldFile, but with the model stuff ripped + out. It's only extended by one class, but we keep it separate for + organizational reasons. + + """ + + def __init__(self, storage): + self.storage = storage + + def _require_file(self): + if not self: + raise ValueError() + + def _get_file(self): + self._require_file() + if not hasattr(self, '_file') or self._file is None: + self._file = self.storage.open(self.name, 'rb') + return self._file + + def _set_file(self, file): + self._file = file + + def _del_file(self): + del self._file + + file = property(_get_file, _set_file, _del_file) + + def _get_path(self): + self._require_file() + return self.storage.path(self.name) + path = property(_get_path) + + def _get_url(self): + self._require_file() + return self.storage.url(self.name) + url = property(_get_url) + + def _get_size(self): + self._require_file() + if not self._committed: + return self.file.size + return self.storage.size(self.name) + size = property(_get_size) + + def open(self, mode='rb'): + self._require_file() + self.file.open(mode) + + def _get_closed(self): + file = getattr(self, '_file', None) + return file is None or file.closed + closed = property(_get_closed) + + def close(self): + file = getattr(self, '_file', None) + if file is not None: + file.close() + + +class IKContentFile(ContentFile): + """ + Wraps a ContentFile in a file-like object with a filename and a + content_type. A PIL image format can be optionally be provided as a content + type hint. + + """ + def __init__(self, filename, content, format=None): + self.file = ContentFile(content) + self.file.name = filename + mimetype = getattr(self.file, 'content_type', None) + if format and not mimetype: + mimetype = format_to_mimetype(format) + if not mimetype: + ext = os.path.splitext(filename or '')[1] + mimetype = extension_to_mimetype(ext) + self.file.content_type = mimetype + + @property + def name(self): + return self.file.name + + def __str__(self): + return smart_str(self.file.name or '') + + def __unicode__(self): + return smart_unicode(self.file.name or u'') diff --git a/imagekit/forms/__init__.py b/imagekit/forms/__init__.py new file mode 100644 index 0000000..f7310d1 --- /dev/null +++ b/imagekit/forms/__init__.py @@ -0,0 +1,3 @@ +# flake8: noqa + +from .fields import ProcessedImageField diff --git a/imagekit/forms/fields.py b/imagekit/forms/fields.py new file mode 100644 index 0000000..903f6ae --- /dev/null +++ b/imagekit/forms/fields.py @@ -0,0 +1,29 @@ +from django.forms import ImageField +from ..specs import SpecHost +from ..utils import generate + + +class ProcessedImageField(ImageField, SpecHost): + + def __init__(self, processors=None, format=None, options=None, + autoconvert=True, spec_id=None, spec=None, *args, **kwargs): + + if spec_id is None: + # Unlike model fields, form fields are never told their field name. + # (Model fields are done so via `contribute_to_class()`.) Therefore + # we can't really generate a good spec id automatically. + raise TypeError('You must provide a spec_id') + + SpecHost.__init__(self, processors=processors, format=format, + options=options, autoconvert=autoconvert, spec=spec, + spec_id=spec_id) + super(ProcessedImageField, self).__init__(*args, **kwargs) + + def clean(self, data, initial=None): + data = super(ProcessedImageField, self).clean(data, initial) + + if data: + spec = self.get_spec(source=data) + data = generate(spec) + + return data diff --git a/imagekit/generatorlibrary.py b/imagekit/generatorlibrary.py new file mode 100644 index 0000000..bc5a0f8 --- /dev/null +++ b/imagekit/generatorlibrary.py @@ -0,0 +1,13 @@ +from .registry import register +from .processors import Thumbnail as ThumbnailProcessor +from .specs import ImageSpec + + +class Thumbnail(ImageSpec): + def __init__(self, width=None, height=None, anchor=None, crop=None, **kwargs): + self.processors = [ThumbnailProcessor(width, height, anchor=anchor, + crop=crop)] + super(Thumbnail, self).__init__(**kwargs) + + +register.generator('imagekit:thumbnail', Thumbnail) diff --git a/imagekit/generators.py b/imagekit/generators.py deleted file mode 100644 index 4cdceaa..0000000 --- a/imagekit/generators.py +++ /dev/null @@ -1,67 +0,0 @@ -import os -from .lib import StringIO -from .processors import ProcessorPipeline -from .utils import (img_to_fobj, open_image, IKContentFile, extension_to_format, - UnknownExtensionError, get_default_file_storage) - - -class SpecFileGenerator(object): - def __init__(self, processors=None, format=None, options=None, - autoconvert=True, storage=None): - self.processors = processors - self.format = format - self.options = options or {} - self.autoconvert = autoconvert - self.storage = storage - - def process_content(self, content, filename=None, source_file=None): - img = open_image(content) - original_format = img.format - - # Run the processors - processors = self.processors - if callable(processors): - processors = processors(source_file) - img = ProcessorPipeline(processors or []).process(img) - - options = dict(self.options or {}) - - # Determine the format. - format = self.format - if filename and not format: - # Try to guess the format from the extension. - extension = os.path.splitext(filename)[1].lower() - if extension: - try: - format = extension_to_format(extension) - except UnknownExtensionError: - pass - format = format or img.format or original_format or 'JPEG' - - imgfile = img_to_fobj(img, format, **options) - content = IKContentFile(filename, imgfile.read(), format=format) - return img, content - - def generate_file(self, filename, source_file, save=True): - """ - Generates a new image file by processing the source file and returns - the content of the result, ready for saving. - - """ - if source_file: # TODO: Should we error here or something if the source_file doesn't exist? - # Process the original image file. - - try: - fp = source_file.storage.open(source_file.name) - except IOError: - return - fp.seek(0) - fp = StringIO(fp.read()) - - img, content = self.process_content(fp, filename, source_file) - - if save: - storage = self.storage or get_default_file_storage() or source_file.storage - storage.save(filename, content) - - return content diff --git a/imagekit/imagecache/__init__.py b/imagekit/imagecache/__init__.py deleted file mode 100644 index cf98a9d..0000000 --- a/imagekit/imagecache/__init__.py +++ /dev/null @@ -1,34 +0,0 @@ -from django.core.exceptions import ImproperlyConfigured -from django.utils.importlib import import_module - -from imagekit.imagecache.base import InvalidImageCacheBackendError, PessimisticImageCacheBackend, NonValidatingImageCacheBackend - -_default_image_cache_backend = None - - -def get_default_image_cache_backend(): - """ - Get the default image cache backend. Uses the same method as - django.core.file.storage.get_storage_class - - """ - global _default_image_cache_backend - if not _default_image_cache_backend: - from django.conf import settings - import_path = settings.IMAGEKIT_DEFAULT_IMAGE_CACHE_BACKEND - try: - dot = import_path.rindex('.') - except ValueError: - raise ImproperlyConfigured("%s isn't an image cache backend module." % \ - import_path) - module, classname = import_path[:dot], import_path[dot + 1:] - try: - mod = import_module(module) - except ImportError, e: - raise ImproperlyConfigured('Error importing image cache backend module %s: "%s"' % (module, e)) - try: - cls = getattr(mod, classname) - _default_image_cache_backend = cls() - except AttributeError: - raise ImproperlyConfigured('Image cache backend module "%s" does not define a "%s" class.' % (module, classname)) - return _default_image_cache_backend diff --git a/imagekit/imagecache/base.py b/imagekit/imagecache/base.py deleted file mode 100644 index f06c9b5..0000000 --- a/imagekit/imagecache/base.py +++ /dev/null @@ -1,60 +0,0 @@ -from django.core.exceptions import ImproperlyConfigured - - -class InvalidImageCacheBackendError(ImproperlyConfigured): - pass - - -class PessimisticImageCacheBackend(object): - """ - A very safe image cache backend. Guarantees that files will always be - available, but at the cost of hitting the storage backend. - - """ - - def is_invalid(self, file): - if not getattr(file, '_file', None): - # No file on object. Have to check storage. - return not file.storage.exists(file.name) - return False - - def validate(self, file): - """ - Generates a new image by running the processors on the source file. - - """ - if self.is_invalid(file): - file.generate(save=True) - - def invalidate(self, file): - file.delete(save=False) - - def clear(self, file): - file.delete(save=False) - - -class NonValidatingImageCacheBackend(object): - """ - A backend that is super optimistic about the existence of spec files. It - will hit your file storage much less frequently than the pessimistic - backend, but it is technically possible for a cache file to be missing - after validation. - - """ - - def validate(self, file): - """ - NonValidatingImageCacheBackend has faith, so validate's a no-op. - - """ - pass - - def invalidate(self, file): - """ - Immediately generate a new spec file upon invalidation. - - """ - file.generate(save=True) - - def clear(self, file): - file.delete(save=False) diff --git a/imagekit/imagecache/celery.py b/imagekit/imagecache/celery.py deleted file mode 100644 index 9dee5ca..0000000 --- a/imagekit/imagecache/celery.py +++ /dev/null @@ -1,43 +0,0 @@ -# -*- coding: utf-8 -*- -from __future__ import absolute_import - -from imagekit.imagecache import PessimisticImageCacheBackend, InvalidImageCacheBackendError - - -def generate(model, pk, attr): - try: - instance = model._default_manager.get(pk=pk) - except model.DoesNotExist: - pass # The model was deleted since the task was scheduled. NEVER MIND! - else: - field_file = getattr(instance, attr) - field_file.delete(save=False) - field_file.generate(save=True) - - -class CeleryImageCacheBackend(PessimisticImageCacheBackend): - """ - A pessimistic cache state backend that uses celery to generate its spec - images. Like PessimisticCacheStateBackend, this one checks to see if the - file exists on validation, so the storage is hit fairly frequently, but an - image is guaranteed to exist. However, while validation guarantees the - existence of *an* image, it does not necessarily guarantee that you will get - the correct image, as the spec may be pending regeneration. In other words, - while there are `generate` tasks in the queue, it is possible to get a - stale spec image. The tradeoff is that calling `invalidate()` won't block - to interact with file storage. - - """ - def __init__(self): - try: - from celery.task import task - except: - raise InvalidImageCacheBackendError("Celery image cache backend requires the 'celery' library") - if not getattr(CeleryImageCacheBackend, '_task', None): - CeleryImageCacheBackend._task = task(generate) - - def invalidate(self, file): - self._task.delay(file.instance.__class__, file.instance.pk, file.attname) - - def clear(self, file): - file.delete(save=False) diff --git a/imagekit/lib.py b/imagekit/lib.py index 574e587..7137e08 100644 --- a/imagekit/lib.py +++ b/imagekit/lib.py @@ -1,3 +1,5 @@ +# flake8: noqa + # Required PIL classes may or may not be available from the root namespace # depending on the installation method used. try: diff --git a/imagekit/management/commands/generateimages.py b/imagekit/management/commands/generateimages.py new file mode 100644 index 0000000..099fe3d --- /dev/null +++ b/imagekit/management/commands/generateimages.py @@ -0,0 +1,47 @@ +from django.core.management.base import BaseCommand +import re +from ...registry import generator_registry, cachefile_registry + + +class Command(BaseCommand): + help = ("""Generate files for the specified image generators (or all of them if +none was provided). Simple, glob-like wildcards are allowed, with * +matching all characters within a segment, and ** matching across +segments. (Segments are separated with colons.) So, for example, +"a:*:c" will match "a:b:c", but not "a:b:x:c", whereas "a:**:c" will +match both. Subsegments are always matched, so "a" will match "a" as +well as "a:b" and "a:b:c".""") + args = '[generator_ids]' + + def handle(self, *args, **options): + generators = generator_registry.get_ids() + + if args: + patterns = self.compile_patterns(args) + generators = (id for id in generators if any(p.match(id) for p in patterns)) + + for generator_id in generators: + self.stdout.write('Validating generator: %s\n' % generator_id) + for file in cachefile_registry.get(generator_id): + self.stdout.write(' %s\n' % file) + try: + # TODO: Allow other validation actions through command option + file.generate() + except Exception, err: + # TODO: How should we handle failures? Don't want to error, but should call it out more than this. + self.stdout.write(' FAILED: %s\n' % err) + + def compile_patterns(self, generator_ids): + return [self.compile_pattern(id) for id in generator_ids] + + def compile_pattern(self, generator_id): + parts = re.split(r'(\*{1,2})', generator_id) + pattern = '' + for part in parts: + if part == '*': + pattern += '[^:]*' + elif part == '**': + pattern += '.*' + else: + pattern += re.escape(part) + return re.compile('^%s(:.*)?$' % pattern) diff --git a/imagekit/management/commands/ikcacheinvalidate.py b/imagekit/management/commands/ikcacheinvalidate.py deleted file mode 100644 index 2b6e915..0000000 --- a/imagekit/management/commands/ikcacheinvalidate.py +++ /dev/null @@ -1,14 +0,0 @@ -from django.core.management.base import BaseCommand -from django.db.models.loading import cache -from ...utils import invalidate_app_cache - - -class Command(BaseCommand): - help = ('Invalidates the image cache for a list of apps.') - args = '[apps]' - requires_model_validation = True - can_import_settings = True - - def handle(self, *args, **options): - apps = args or cache.app_models.keys() - invalidate_app_cache(apps) diff --git a/imagekit/management/commands/ikcachevalidate.py b/imagekit/management/commands/ikcachevalidate.py deleted file mode 100644 index 8e9fc6c..0000000 --- a/imagekit/management/commands/ikcachevalidate.py +++ /dev/null @@ -1,30 +0,0 @@ -from optparse import make_option -from django.core.management.base import BaseCommand -from django.db.models.loading import cache -from ...utils import validate_app_cache - - -class Command(BaseCommand): - help = ('Validates the image cache for a list of apps.') - args = '[apps]' - requires_model_validation = True - can_import_settings = True - - option_list = BaseCommand.option_list + ( - make_option('--force-revalidation', - dest='force_revalidation', - action='store_true', - default=False, - help='Invalidate each image file before validating it, thereby' - ' ensuring its revalidation. This is very similar to' - ' running ikcacheinvalidate and then running' - ' ikcachevalidate; the difference being that this option' - ' causes files to be invalidated and validated' - ' one-at-a-time, whereas running the two commands in series' - ' would invalidate all images before validating any.' - ), - ) - - def handle(self, *args, **options): - apps = args or cache.app_models.keys() - validate_app_cache(apps, options['force_revalidation']) diff --git a/imagekit/models/__init__.py b/imagekit/models/__init__.py index 4207987..b13b38f 100644 --- a/imagekit/models/__init__.py +++ b/imagekit/models/__init__.py @@ -1,11 +1,4 @@ +# flake8: noqa + from .. import conf from .fields import ImageSpecField, ProcessedImageField -import warnings - - -class ImageSpec(ImageSpecField): - def __init__(self, *args, **kwargs): - warnings.warn('ImageSpec has been moved to' - ' imagekit.models.ImageSpecField. Please use that instead.', - DeprecationWarning) - super(ImageSpec, self).__init__(*args, **kwargs) diff --git a/imagekit/models/fields/__init__.py b/imagekit/models/fields/__init__.py index 3e11e69..292f410 100644 --- a/imagekit/models/fields/__init__.py +++ b/imagekit/models/fields/__init__.py @@ -1,107 +1,55 @@ -import os - from django.db import models - -from ...imagecache import get_default_image_cache_backend -from ...generators import SpecFileGenerator -from .files import ImageSpecFieldFile, ProcessedImageFieldFile -from ..receivers import configure_receivers -from .utils import ImageSpecFileDescriptor, ImageKitMeta, BoundImageKitMeta -from ...utils import suggest_extension +from .files import ProcessedImageFieldFile +from .utils import ImageSpecFileDescriptor +from ...specs import SpecHost +from ...specs.sourcegroups import ImageFieldSourceGroup +from ...registry import register -configure_receivers() +class SpecHostField(SpecHost): + def set_spec_id(self, cls, name): + # Generate a spec_id to register the spec with. The default spec id is + # ":_" + if not getattr(self, 'spec_id', None): + spec_id = (u'%s:%s:%s' % (cls._meta.app_label, + cls._meta.object_name, name)).lower() + + # Register the spec with the id. This allows specs to be overridden + # later, from outside of the model definition. + super(SpecHostField, self).set_spec_id(spec_id) -class ImageSpecField(object): +class ImageSpecField(SpecHostField): """ The heart and soul of the ImageKit library, ImageSpecField allows you to add variants of uploaded images to your models. """ def __init__(self, processors=None, format=None, options=None, - image_field=None, pre_cache=None, storage=None, cache_to=None, - autoconvert=True, image_cache_backend=None): - """ - :param processors: A list of processors to run on the original image. - :param format: The format of the output file. If not provided, - ImageSpecField will try to guess the appropriate format based on the - extension of the filename and the format of the input image. - :param options: A dictionary that will be passed to PIL's - ``Image.save()`` method as keyword arguments. Valid options vary - between formats, but some examples include ``quality``, - ``optimize``, and ``progressive`` for JPEGs. See the PIL - documentation for others. - :param image_field: The name of the model property that contains the - original image. - :param storage: A Django storage system to use to save the generated - image. - :param cache_to: Specifies the filename to use when saving the image - cache file. This is modeled after ImageField's ``upload_to`` and - can be either a string (that specifies a directory) or a - callable (that returns a filepath). Callable values should - accept the following arguments: + source=None, cachefile_storage=None, autoconvert=None, + cachefile_backend=None, cachefile_strategy=None, spec=None, + id=None): - - instance -- The model instance this spec belongs to - - path -- The path of the original image - - specname -- the property name that the spec is bound to on - the model instance - - extension -- A recommended extension. If the format of the - spec is set explicitly, this suggestion will be - based on that format. if not, the extension of the - original file will be passed. You do not have to use - this extension, it's only a recommendation. - :param autoconvert: Specifies whether automatic conversion using - ``prepare_image()`` should be performed prior to saving. - :param image_cache_backend: An object responsible for managing the state - of cached files. Defaults to an instance of - IMAGEKIT_DEFAULT_IMAGE_CACHE_BACKEND + SpecHost.__init__(self, processors=processors, format=format, + options=options, cachefile_storage=cachefile_storage, + autoconvert=autoconvert, + cachefile_backend=cachefile_backend, + cachefile_strategy=cachefile_strategy, spec=spec, + spec_id=id) - """ - - if pre_cache is not None: - raise Exception('The pre_cache argument has been removed in favor' - ' of cache state backends.') - - # The generator accepts a callable value for processors, but it - # takes different arguments than the callable that ImageSpecField - # expects, so we create a partial application and pass that instead. - # TODO: Should we change the signatures to match? Even if `instance` is not part of the signature, it's accessible through the source file object's instance property. - p = lambda file: processors(instance=file.instance, file=file) if \ - callable(processors) else processors - - self.generator = SpecFileGenerator(p, format=format, options=options, - autoconvert=autoconvert, storage=storage) - self.image_field = image_field - self.storage = storage - self.cache_to = cache_to - self.image_cache_backend = image_cache_backend or \ - get_default_image_cache_backend() + # TODO: Allow callable for source. See https://github.com/jdriscoll/django-imagekit/issues/158#issuecomment-10921664 + self.source = source def contribute_to_class(self, cls, name): setattr(cls, name, ImageSpecFileDescriptor(self, name)) - try: - # Make sure we don't modify an inherited ImageKitMeta instance - ik = cls.__dict__['ik'] - except KeyError: - try: - base = getattr(cls, '_ik') - except AttributeError: - ik = ImageKitMeta() - else: - # Inherit all the spec fields. - ik = ImageKitMeta(base.spec_fields) - setattr(cls, '_ik', ik) - ik.spec_fields.append(name) + self.set_spec_id(cls, name) - # Register the field with the image_cache_backend - try: - self.image_cache_backend.register_field(cls, self, name) - except AttributeError: - pass + # Add the model and field as a source for this spec id + register.source_group(self.spec_id, + ImageFieldSourceGroup(cls, self.source)) -class ProcessedImageField(models.ImageField): +class ProcessedImageField(models.ImageField, SpecHostField): """ ProcessedImageField is an ImageField that runs processors on the uploaded image *before* saving it to storage. This is in contrast to specs, which @@ -112,8 +60,8 @@ class ProcessedImageField(models.ImageField): attr_class = ProcessedImageFieldFile def __init__(self, processors=None, format=None, options=None, - verbose_name=None, name=None, width_field=None, height_field=None, - autoconvert=True, **kwargs): + verbose_name=None, name=None, width_field=None, height_field=None, + autoconvert=True, spec=None, spec_id=None, **kwargs): """ The ProcessedImageField constructor accepts all of the arguments that the :class:`django.db.models.ImageField` constructor accepts, as well @@ -121,21 +69,15 @@ class ProcessedImageField(models.ImageField): :class:`imagekit.models.ImageSpecField`. """ - if 'quality' in kwargs: - raise Exception('The "quality" keyword argument has been' - """ deprecated. Use `options={'quality': %s}` instead.""" \ - % kwargs['quality']) + SpecHost.__init__(self, processors=processors, format=format, + options=options, autoconvert=autoconvert, spec=spec, + spec_id=spec_id) models.ImageField.__init__(self, verbose_name, name, width_field, height_field, **kwargs) - self.generator = SpecFileGenerator(processors, format=format, - options=options, autoconvert=autoconvert) - def get_filename(self, filename): - filename = os.path.normpath(self.storage.get_valid_name( - os.path.basename(filename))) - name, ext = os.path.splitext(filename) - ext = suggest_extension(filename, self.generator.format) - return u'%s%s' % (name, ext) + def contribute_to_class(self, cls, name): + self.set_spec_id(cls, name) + return super(ProcessedImageField, self).contribute_to_class(cls, name) try: diff --git a/imagekit/models/fields/files.py b/imagekit/models/fields/files.py index 9ae2734..0fbbad6 100644 --- a/imagekit/models/fields/files.py +++ b/imagekit/models/fields/files.py @@ -1,174 +1,13 @@ +from django.db.models.fields.files import ImageFieldFile import os -import datetime - -from django.db.models.fields.files import ImageField, ImageFieldFile -from django.utils.encoding import force_unicode, smart_str - -from ...utils import suggest_extension, get_default_file_storage - - -class ImageSpecFieldFile(ImageFieldFile): - def __init__(self, instance, field, attname): - super(ImageSpecFieldFile, self).__init__(instance, field, None) - self.attname = attname - - @property - def source_file(self): - field_name = getattr(self.field, 'image_field', None) - if field_name: - field_file = getattr(self.instance, field_name) - else: - image_fields = [getattr(self.instance, f.attname) for f in \ - self.instance.__class__._meta.fields if \ - isinstance(f, ImageField)] - if len(image_fields) == 0: - raise Exception('%s does not define any ImageFields, so your' \ - ' %s ImageSpecField has no image to act on.' % \ - (self.instance.__class__.__name__, self.attname)) - elif len(image_fields) > 1: - raise Exception('%s defines multiple ImageFields, but you' \ - ' have not specified an image_field for your %s' \ - ' ImageSpecField.' % (self.instance.__class__.__name__, - self.attname)) - else: - field_file = image_fields[0] - return field_file - - def _require_file(self): - if not self.source_file: - raise ValueError("The '%s' attribute's image_field has no file associated with it." % self.attname) - else: - self.validate() - - def clear(self): - return self.field.image_cache_backend.clear(self) - - def invalidate(self): - return self.field.image_cache_backend.invalidate(self) - - def validate(self): - return self.field.image_cache_backend.validate(self) - - def generate(self, save=True): - """ - Generates a new image file by processing the source file and returns - the content of the result, ready for saving. - - """ - return self.field.generator.generate_file(self.name, self.source_file, - save) - - def delete(self, save=False): - """ - Pulled almost verbatim from ``ImageFieldFile.delete()`` and - ``FieldFile.delete()`` but with the attempts to reset the instance - property removed. - - """ - # Clear the image dimensions cache - if hasattr(self, '_dimensions_cache'): - del self._dimensions_cache - - # Only close the file if it's already open, which we know by the - # presence of self._file. - if hasattr(self, '_file'): - self.close() - del self.file - - if self.name and self.storage.exists(self.name): - try: - self.storage.delete(self.name) - except NotImplementedError: - pass - - # Delete the filesize cache. - if hasattr(self, '_size'): - del self._size - self._committed = False - - if save: - self.instance.save() - - def _default_cache_to(self, instance, path, specname, extension): - """ - Determines the filename to use for the transformed image. Can be - overridden on a per-spec basis by setting the cache_to property on - the spec. - - """ - filepath, basename = os.path.split(path) - filename = os.path.splitext(basename)[0] - new_name = '%s_%s%s' % (filename, specname, extension) - return os.path.join('cache', filepath, new_name) - - @property - def name(self): - """ - Specifies the filename that the cached image will use. The user can - control this by providing a `cache_to` method to the ImageSpecField. - - """ - name = getattr(self, '_name', None) - if not name: - filename = self.source_file.name - new_filename = None - if filename: - cache_to = self.field.cache_to or self._default_cache_to - - if not cache_to: - raise Exception('No cache_to or default_cache_to value' - ' specified') - if callable(cache_to): - suggested_extension = suggest_extension( - self.source_file.name, self.field.generator.format) - new_filename = force_unicode( - datetime.datetime.now().strftime( - smart_str(cache_to(self.instance, - self.source_file.name, self.attname, - suggested_extension)))) - else: - dir_name = os.path.normpath( - force_unicode(datetime.datetime.now().strftime( - smart_str(cache_to)))) - filename = os.path.normpath(os.path.basename(filename)) - new_filename = os.path.join(dir_name, filename) - - self._name = new_filename - return self._name - - @name.setter - def name(self, value): - # TODO: Figure out a better way to handle this. We really don't want - # to allow anybody to set the name, but ``File.__init__`` (which is - # called by ``ImageSpecFieldFile.__init__``) does, so we have to allow - # it at least that one time. - pass - - @property - def storage(self): - if not getattr(self, '_storage', None): - self._storage = self.field.storage or get_default_file_storage() or self.source_file.storage - return self._storage - - @storage.setter - def storage(self, storage): - self._storage = storage - - def __getstate__(self): - return dict( - attname=self.attname, - instance=self.instance, - ) - - def __setstate__(self, state): - self.attname = state['attname'] - self.instance = state['instance'] - self.field = getattr(self.instance.__class__, self.attname) +from ...utils import suggest_extension, generate class ProcessedImageFieldFile(ImageFieldFile): def save(self, name, content, save=True): - new_filename = self.field.generate_filename(self.instance, name) - img, content = self.field.generator.process_content(content, - new_filename, self) - return super(ProcessedImageFieldFile, self).save(name, content, save) + filename, ext = os.path.splitext(name) + spec = self.field.get_spec(source=content) + ext = suggest_extension(name, spec.format) + new_name = '%s%s' % (filename, ext) + content = generate(spec) + return super(ProcessedImageFieldFile, self).save(new_name, content, save) diff --git a/imagekit/models/fields/utils.py b/imagekit/models/fields/utils.py index 1b3ccaa..bfb6d89 100644 --- a/imagekit/models/fields/utils.py +++ b/imagekit/models/fields/utils.py @@ -1,27 +1,5 @@ -from .files import ImageSpecFieldFile - - -class BoundImageKitMeta(object): - def __init__(self, instance, spec_fields): - self.instance = instance - self.spec_fields = spec_fields - - @property - def spec_files(self): - return [getattr(self.instance, n) for n in self.spec_fields] - - -class ImageKitMeta(object): - def __init__(self, spec_fields=None): - self.spec_fields = list(spec_fields) if spec_fields else [] - - def __get__(self, instance, owner): - if instance is None: - return self - else: - ik = BoundImageKitMeta(instance, self.spec_fields) - setattr(instance, '_ik', ik) - return ik +from ...cachefiles import GeneratedImageFile +from django.db.models.fields.files import ImageField class ImageSpecFileDescriptor(object): @@ -33,10 +11,28 @@ class ImageSpecFileDescriptor(object): if instance is None: return self.field else: - img_spec_file = ImageSpecFieldFile(instance, self.field, - self.attname) - instance.__dict__[self.attname] = img_spec_file - return img_spec_file + field_name = getattr(self.field, 'source', None) + if field_name: + source = getattr(instance, field_name) + else: + image_fields = [getattr(instance, f.attname) for f in + instance.__class__._meta.fields if + isinstance(f, ImageField)] + if len(image_fields) == 0: + raise Exception('%s does not define any ImageFields, so your' + ' %s ImageSpecField has no image to act on.' % + (instance.__class__.__name__, self.attname)) + elif len(image_fields) > 1: + raise Exception('%s defines multiple ImageFields, but you' + ' have not specified a source for your %s' + ' ImageSpecField.' % (instance.__class__.__name__, + self.attname)) + else: + source = image_fields[0] + spec = self.field.get_spec(source=source) + file = GeneratedImageFile(spec) + instance.__dict__[self.attname] = file + return file def __set__(self, instance, value): instance.__dict__[self.attname] = value diff --git a/imagekit/models/receivers.py b/imagekit/models/receivers.py deleted file mode 100644 index da93a69..0000000 --- a/imagekit/models/receivers.py +++ /dev/null @@ -1,48 +0,0 @@ -from django.db.models.signals import post_init, post_save, post_delete -from ..utils import ik_model_receiver - - -def update_source_hashes(instance): - """ - Stores hashes of the source image files so that they can be compared - later to see whether the source image has changed (and therefore whether - the spec file needs to be regenerated). - - """ - instance._ik._source_hashes = dict((f.attname, hash(f.source_file)) \ - for f in instance._ik.spec_files) - return instance._ik._source_hashes - - -@ik_model_receiver -def post_save_receiver(sender, instance=None, created=False, raw=False, **kwargs): - if not raw: - old_hashes = instance._ik._source_hashes.copy() - new_hashes = update_source_hashes(instance) - for attname in instance._ik.spec_fields: - if old_hashes[attname] != new_hashes[attname]: - getattr(instance, attname).invalidate() - - -@ik_model_receiver -def post_delete_receiver(sender, instance=None, **kwargs): - for spec_file in instance._ik.spec_files: - spec_file.clear() - - -@ik_model_receiver -def post_init_receiver(sender, instance, **kwargs): - update_source_hashes(instance) - - -def configure_receivers(): - # Connect the signals. We have to listen to every model (not just those - # with IK fields) and filter in our receivers because of a Django issue with - # abstract base models. - # Related: - # https://github.com/jdriscoll/django-imagekit/issues/126 - # https://code.djangoproject.com/ticket/9318 - uid = 'ik_spec_field_receivers' - post_init.connect(post_init_receiver, dispatch_uid=uid) - post_save.connect(post_save_receiver, dispatch_uid=uid) - post_delete.connect(post_delete_receiver, dispatch_uid=uid) diff --git a/imagekit/pkgmeta.py b/imagekit/pkgmeta.py new file mode 100644 index 0000000..20e62db --- /dev/null +++ b/imagekit/pkgmeta.py @@ -0,0 +1,5 @@ +__title__ = 'django-imagekit' +__author__ = 'Justin Driscoll, Bryan Veloso, Greg Newman, Chris Drackett, Matthew Tretter, Eric Eldredge' +__version__ = '3.0a1' +__license__ = 'BSD' +__all__ = ['__title__', '__author__', '__version__', '__license__'] diff --git a/imagekit/processors/__init__.py b/imagekit/processors/__init__.py index c2c9320..6c8d0b7 100644 --- a/imagekit/processors/__init__.py +++ b/imagekit/processors/__init__.py @@ -1,3 +1,5 @@ +# flake8: noqa + """ Imagekit image processors. diff --git a/imagekit/processors/crop.py b/imagekit/processors/crop.py index da5c0fb..b039d30 100644 --- a/imagekit/processors/crop.py +++ b/imagekit/processors/crop.py @@ -1,4 +1,4 @@ -from .base import Anchor +from .base import Anchor # noqa from .utils import histogram_entropy from ..lib import Image, ImageChops, ImageDraw, ImageStat diff --git a/imagekit/processors/resize.py b/imagekit/processors/resize.py index e4b747a..0bae122 100644 --- a/imagekit/processors/resize.py +++ b/imagekit/processors/resize.py @@ -1,5 +1,4 @@ from imagekit.lib import Image -import warnings from .base import Anchor @@ -211,10 +210,51 @@ class ResizeToFit(object): ratio = float(self.width) / cur_width new_dimensions = (int(round(cur_width * ratio)), int(round(cur_height * ratio))) - if (cur_width > new_dimensions[0] or cur_height > new_dimensions[1]) or \ - self.upscale: - img = Resize(new_dimensions[0], - new_dimensions[1]).process(img) - if self.mat_color: + if (cur_width > new_dimensions[0] or cur_height > new_dimensions[1]) or self.upscale: + img = Resize(new_dimensions[0], new_dimensions[1]).process(img) + if self.mat_color is not None: img = ResizeCanvas(self.width, self.height, self.mat_color, anchor=self.anchor).process(img) return img + + +class Thumbnail(object): + """ + Resize the image for use as a thumbnail. Wraps ``ResizeToFill``, + ``ResizeToFit``, and ``SmartResize``. + + Note: while it doesn't currently, in the future this processor may also + sharpen based on the amount of reduction. + + """ + + def __init__(self, width=None, height=None, anchor=None, crop=None): + self.width = width + self.height = height + if anchor: + if crop is False: + raise Exception("You can't specify an anchor point if crop is False.") + else: + crop = True + elif crop is None: + # Assume we are cropping if both a width and height are provided. If + # only one is, we must be resizing to fit. + crop = width is not None and height is not None + + # A default anchor if cropping. + if crop and anchor is None: + anchor = 'auto' + self.crop = crop + self.anchor = anchor + + def process(self, img): + if self.crop: + if not self.width or not self.height: + raise Exception('You must provide both a width and height when' + ' cropping.') + if self.anchor == 'auto': + processor = SmartResize(self.width, self.height) + else: + processor = ResizeToFill(self.width, self.height, self.anchor) + else: + processor = ResizeToFit(self.width, self.height) + return processor.process(img) diff --git a/imagekit/registry.py b/imagekit/registry.py new file mode 100644 index 0000000..2523f9b --- /dev/null +++ b/imagekit/registry.py @@ -0,0 +1,193 @@ +from .exceptions import AlreadyRegistered, NotRegistered +from .signals import before_access, source_created, source_changed, source_deleted +from .utils import call_strategy_method + + +class GeneratorRegistry(object): + """ + An object for registering generators. This registry provides + a convenient way for a distributable app to define default generators + without locking the users of the app into it. + + """ + def __init__(self): + self._generators = {} + before_access.connect(self.before_access_receiver) + + def register(self, id, generator): + if id in self._generators: + raise AlreadyRegistered('The generator with id %s is' + ' already registered' % id) + self._generators[id] = generator + + def unregister(self, id, generator): + # TODO: Either don't require the generator, or--if we do--assert that it's registered with the provided id + try: + del self._generators[id] + except KeyError: + raise NotRegistered('The generator with id %s is not' + ' registered' % id) + + def get(self, id, **kwargs): + try: + generator = self._generators[id] + except KeyError: + raise NotRegistered('The generator with id %s is not' + ' registered' % id) + if callable(generator): + return generator(**kwargs) + else: + return generator + + def get_ids(self): + return self._generators.keys() + + def before_access_receiver(self, sender, file, **kwargs): + generator = file.generator + + # FIXME: I guess this means you can't register functions? + if generator.__class__ in self._generators.values(): + # Only invoke the strategy method for registered generators. + call_strategy_method(generator, 'before_access', file=file) + + +class SourceGroupRegistry(object): + """ + The source group registry is responsible for listening to source_* signals + on source groups, and relaying them to the image generated file strategies + of the appropriate generators. + + In addition, registering a new source group also registers its generated + files with that registry. + + """ + _signals = { + source_created: 'on_source_created', + source_changed: 'on_source_changed', + source_deleted: 'on_source_deleted', + } + + def __init__(self): + self._source_groups = {} + for signal in self._signals.keys(): + signal.connect(self.source_group_receiver) + + def register(self, generator_id, source_group): + from .specs.sourcegroups import SourceGroupFilesGenerator + generator_ids = self._source_groups.setdefault(source_group, set()) + generator_ids.add(generator_id) + cachefile_registry.register(generator_id, + SourceGroupFilesGenerator(source_group, generator_id)) + + def unregister(self, generator_id, source_group): + from .specs.sourcegroups import SourceGroupFilesGenerator + generator_ids = self._source_groups.setdefault(source_group, set()) + if generator_id in generator_ids: + generator_ids.remove(generator_id) + cachefile_registry.unregister(generator_id, + SourceGroupFilesGenerator(source_group, generator_id)) + + def source_group_receiver(self, sender, source, signal, **kwargs): + """ + Relay source group signals to the appropriate spec strategy. + + """ + from .cachefiles import GeneratedImageFile + source_group = sender + + # Ignore signals from unregistered groups. + if source_group not in self._source_groups: + return + + specs = [generator_registry.get(id, source=source) for id in + self._source_groups[source_group]] + callback_name = self._signals[signal] + + for spec in specs: + file = GeneratedImageFile(spec) + call_strategy_method(spec, callback_name, file=file) + + +class CacheFileRegistry(object): + """ + An object for registering generated files with image generators. The two are + associated with each other via a string id. We do this (as opposed to + associating them directly by, for example, putting a ``cachefiles`` + attribute on image generators) so that image generators can be overridden + without losing the associated files. That way, a distributable app can + define its own generators without locking the users of the app into it. + + """ + + def __init__(self): + self._cachefiles = {} + + def register(self, generator_id, cachefiles): + """ + Associates generated files with a generator id + + """ + if cachefiles not in self._cachefiles: + self._cachefiles[cachefiles] = set() + self._cachefiles[cachefiles].add(generator_id) + + def unregister(self, generator_id, cachefiles): + """ + Disassociates generated files with a generator id + + """ + try: + self._cachefiles[cachefiles].remove(generator_id) + except KeyError: + pass + + def get(self, generator_id): + for k, v in self._cachefiles.items(): + if generator_id in v: + for file in k(): + yield file + + +class Register(object): + """ + Register generators and generated files. + + """ + def generator(self, id, generator=None): + if generator is None: + # Return a decorator + def decorator(cls): + self.generator(id, cls) + return cls + return decorator + + generator_registry.register(id, generator) + + # iterable that returns kwargs or callable that returns iterable of kwargs + def cachefiles(self, generator_id, cachefiles): + cachefile_registry.register(generator_id, cachefiles) + + def source_group(self, generator_id, source_group): + source_group_registry.register(generator_id, source_group) + + +class Unregister(object): + """ + Unregister generators and generated files. + + """ + def generator(self, id, generator): + generator_registry.unregister(id, generator) + + def cachefiles(self, generator_id, cachefiles): + cachefile_registry.unregister(generator_id, cachefiles) + + def source_group(self, generator_id, source_group): + source_group_registry.unregister(generator_id, source_group) + + +generator_registry = GeneratorRegistry() +cachefile_registry = CacheFileRegistry() +source_group_registry = SourceGroupRegistry() +register = Register() +unregister = Unregister() diff --git a/imagekit/signals.py b/imagekit/signals.py new file mode 100644 index 0000000..36c915b --- /dev/null +++ b/imagekit/signals.py @@ -0,0 +1,10 @@ +from django.dispatch import Signal + + +# Generated file signals +before_access = Signal() + +# Source group signals +source_created = Signal() +source_changed = Signal() +source_deleted = Signal() diff --git a/imagekit/specs/__init__.py b/imagekit/specs/__init__.py new file mode 100644 index 0000000..969dcf3 --- /dev/null +++ b/imagekit/specs/__init__.py @@ -0,0 +1,203 @@ +from django.conf import settings +from django.db.models.fields.files import ImageFieldFile +from hashlib import md5 +import pickle +from ..cachefiles.backends import get_default_cachefile_backend +from ..cachefiles.strategies import StrategyWrapper +from ..processors import ProcessorPipeline +from ..utils import open_image, img_to_fobj, get_by_qname +from ..registry import generator_registry, register + + +class BaseImageSpec(object): + """ + An object that defines how an new image should be generated from a source + image. + + """ + + cachefile_storage = None + """A Django storage system to use to save a cache file.""" + + cachefile_backend = None + """ + An object responsible for managing the state of cache files. Defaults to + an instance of ``IMAGEKIT_DEFAULT_CACHEFILE_BACKEND`` + + """ + + cachefile_strategy = settings.IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY + """ + A dictionary containing callbacks that allow you to customize how and when + the image file is created. Defaults to + ``IMAGEKIT_DEFAULT_CACHEFILE_STRATEGY``. + + """ + + def __init__(self): + self.cachefile_backend = self.cachefile_backend or get_default_cachefile_backend() + self.cachefile_strategy = StrategyWrapper(self.cachefile_strategy) + + def generate(self): + raise NotImplementedError + + +class ImageSpec(BaseImageSpec): + """ + An object that defines how to generate a new image from a source file using + PIL-based processors. (See :mod:`imagekit.processors`) + + """ + + processors = [] + """A list of processors to run on the original image.""" + + format = None + """ + The format of the output file. If not provided, ImageSpecField will try to + guess the appropriate format based on the extension of the filename and the + format of the input image. + + """ + + options = None + """ + A dictionary that will be passed to PIL's ``Image.save()`` method as keyword + arguments. Valid options vary between formats, but some examples include + ``quality``, ``optimize``, and ``progressive`` for JPEGs. See the PIL + documentation for others. + + """ + + autoconvert = True + """ + Specifies whether automatic conversion using ``prepare_image()`` should be + performed prior to saving. + + """ + + def __init__(self, source): + self.source = source + super(ImageSpec, self).__init__() + + @property + def cachefile_name(self): + fn = get_by_qname(settings.IMAGEKIT_SPEC_CACHEFILE_NAMER, 'namer') + return fn(self) + + def __getstate__(self): + state = self.__dict__ + + # Unpickled ImageFieldFiles won't work (they're missing a storage + # object). Since they're such a common use case, we special case them. + if isinstance(self.source, ImageFieldFile): + field = getattr(self.source, 'field') + state['_field_data'] = { + 'instance': getattr(self.source, 'instance', None), + 'attname': getattr(field, 'name', None), + } + return state + + def __setstate__(self, state): + field_data = state.pop('_field_data', None) + self.__dict__ = state + if field_data: + self.source = getattr(field_data['instance'], field_data['attname']) + + def get_hash(self): + return md5(pickle.dumps([ + self.source.name, + self.processors, + self.format, + self.options, + self.autoconvert, + ])).hexdigest() + + def generate(self): + # TODO: Move into a generator base class + # TODO: Factor out a generate_image function so you can create a generator and only override the PIL.Image creating part. (The tricky part is how to deal with original_format since generator base class won't have one.) + img = open_image(self.source) + original_format = img.format + + # Run the processors + processors = self.processors + img = ProcessorPipeline(processors or []).process(img) + + options = dict(self.options or {}) + format = self.format or img.format or original_format or 'JPEG' + content = img_to_fobj(img, format, **options) + return content + + +def create_spec_class(class_attrs): + + class DynamicSpecBase(ImageSpec): + def __reduce__(self): + try: + getstate = self.__getstate__ + except AttributeError: + state = self.__dict__ + else: + state = getstate() + return (create_spec, (class_attrs, state)) + + return type('DynamicSpec', (DynamicSpecBase,), class_attrs) + + +def create_spec(class_attrs, state): + cls = create_spec_class(class_attrs) + instance = cls.__new__(cls) # Create an instance without calling the __init__ (which may have required args). + try: + setstate = instance.__setstate__ + except AttributeError: + instance.__dict__ = state + else: + setstate(state) + return instance + + +class SpecHost(object): + """ + An object that ostensibly has a spec attribute but really delegates to the + spec registry. + + """ + def __init__(self, spec=None, spec_id=None, **kwargs): + + spec_attrs = dict((k, v) for k, v in kwargs.items() if v is not None) + + if spec_attrs: + if spec: + raise TypeError('You can provide either an image spec or' + ' arguments for the ImageSpec constructor, but not both.') + else: + spec = create_spec_class(spec_attrs) + + self._original_spec = spec + + if spec_id: + self.set_spec_id(spec_id) + + def set_spec_id(self, id): + """ + Sets the spec id for this object. Useful for when the id isn't + known when the instance is constructed (e.g. for ImageSpecFields whose + generated `spec_id`s are only known when they are contributed to a + class). If the object was initialized with a spec, it will be registered + under the provided id. + + """ + self.spec_id = id + register.generator(id, self._original_spec) + + def get_spec(self, source): + """ + Look up the spec by the spec id. We do this (instead of storing the + spec as an attribute) so that users can override apps' specs--without + having to edit model definitions--simply by registering another spec + with the same id. + + """ + if not getattr(self, 'spec_id', None): + raise Exception('Object %s has no spec id.' % self) + return generator_registry.get(self.spec_id, source=source) diff --git a/imagekit/specs/sourcegroups.py b/imagekit/specs/sourcegroups.py new file mode 100644 index 0000000..ec85af8 --- /dev/null +++ b/imagekit/specs/sourcegroups.py @@ -0,0 +1,163 @@ +""" +Source groups are the means by which image spec sources are identified. They +have two responsibilities: + +1. To dispatch ``source_created``, ``source_changed``, and ``source_deleted`` + signals. (These will be relayed to the corresponding specs' cache file + strategies.) +2. To provide the source files that they represent, via a generator method named + ``files()``. (This is used by the generateimages management command for + "pre-caching" image files.) + +""" + +from django.db.models.signals import post_init, post_save, post_delete +from django.utils.functional import wraps +from ..cachefiles import LazyGeneratedImageFile +from ..signals import source_created, source_changed, source_deleted + + +def ik_model_receiver(fn): + """ + A method decorator that filters out signals coming from models that don't + have fields that function as ImageFieldSourceGroup sources. + + """ + @wraps(fn) + def receiver(self, sender, **kwargs): + if sender in (src.model_class for src in self._source_groups): + fn(self, sender=sender, **kwargs) + return receiver + + +class ModelSignalRouter(object): + """ + Normally, ``ImageFieldSourceGroup`` would be directly responsible for + watching for changes on the model field it represents. However, Django does + not dispatch events for abstract base classes. Therefore, we must listen for + the signals on all models and filter out those that aren't represented by + ``ImageFieldSourceGroup``s. This class encapsulates that functionality. + + Related: + https://github.com/jdriscoll/django-imagekit/issues/126 + https://code.djangoproject.com/ticket/9318 + + """ + + def __init__(self): + self._source_groups = [] + uid = 'ik_spec_field_receivers' + post_init.connect(self.post_init_receiver, dispatch_uid=uid) + post_save.connect(self.post_save_receiver, dispatch_uid=uid) + post_delete.connect(self.post_delete_receiver, dispatch_uid=uid) + + def add(self, source_group): + self._source_groups.append(source_group) + + def init_instance(self, instance): + instance._ik = getattr(instance, '_ik', {}) + + def update_source_hashes(self, instance): + """ + Stores hashes of the source image files so that they can be compared + later to see whether the source image has changed (and therefore whether + the spec file needs to be regenerated). + + """ + self.init_instance(instance) + instance._ik['source_hashes'] = dict((attname, hash(file_field)) + for attname, file_field in self.get_field_dict(instance).items()) + return instance._ik['source_hashes'] + + def get_field_dict(self, instance): + """ + Returns the source fields for the given instance, in a dictionary whose + keys are the field names and values are the fields themselves. + + """ + return dict((src.image_field, getattr(instance, src.image_field)) for + src in self._source_groups if src.model_class is instance.__class__) + + @ik_model_receiver + def post_save_receiver(self, sender, instance=None, created=False, raw=False, **kwargs): + if not raw: + self.init_instance(instance) + old_hashes = instance._ik.get('source_hashes', {}).copy() + new_hashes = self.update_source_hashes(instance) + for attname, file in self.get_field_dict(instance).items(): + if created: + self.dispatch_signal(source_created, file, sender, instance, + attname) + elif old_hashes[attname] != new_hashes[attname]: + self.dispatch_signal(source_changed, file, sender, instance, + attname) + + @ik_model_receiver + def post_delete_receiver(self, sender, instance=None, **kwargs): + for attname, file in self.get_field_dict(instance).items(): + self.dispatch_signal(source_deleted, file, sender, instance, attname) + + @ik_model_receiver + def post_init_receiver(self, sender, instance=None, **kwargs): + self.update_source_hashes(instance) + + def dispatch_signal(self, signal, file, model_class, instance, attname): + """ + Dispatch the signal for each of the matching source groups. Note that + more than one source can have the same model and image_field; it's + important that we dispatch the signal for each. + + """ + for source_group in self._source_groups: + if source_group.model_class is model_class and source_group.image_field == attname: + signal.send(sender=source_group, source=file) + + +class ImageFieldSourceGroup(object): + """ + A source group that repesents a particular field across all instances of a + model. + + """ + def __init__(self, model_class, image_field): + self.model_class = model_class + self.image_field = image_field + signal_router.add(self) + + def files(self): + """ + A generator that returns the source files that this source group + represents; in this case, a particular field of every instance of a + particular model. + + """ + for instance in self.model_class.objects.all(): + yield getattr(instance, self.image_field) + + +class SourceGroupFilesGenerator(object): + """ + A Python generator that yields cache file objects for source groups. + + """ + def __init__(self, source_group, generator_id): + self.source_group = source_group + self.generator_id = generator_id + + def __eq__(self, other): + return (isinstance(other, self.__class__) + and self.__dict__ == other.__dict__) + + def __ne__(self, other): + return not self.__eq__(other) + + def __hash__(self): + return hash((self.source_group, self.generator_id)) + + def __call__(self): + for source_file in self.source_group.files(): + yield LazyGeneratedImageFile(self.generator_id, + source=source_file) + + +signal_router = ModelSignalRouter() diff --git a/tests/core/__init__.py b/imagekit/templatetags/__init__.py similarity index 100% rename from tests/core/__init__.py rename to imagekit/templatetags/__init__.py diff --git a/imagekit/templatetags/compat.py b/imagekit/templatetags/compat.py new file mode 100644 index 0000000..f26e8b8 --- /dev/null +++ b/imagekit/templatetags/compat.py @@ -0,0 +1,161 @@ +# flake8: noqa +""" +This module contains code from django.template.base +(sha 90d3af380e8efec0301dd91600c6686232de3943). Bundling this code allows us to +support older versions of Django that did not contain it (< 1.4). + + +Copyright (c) Django Software Foundation and individual contributors. +All rights reserved. + +Redistribution and use in source and binary forms, with or without modification, +are permitted provided that the following conditions are met: + + 1. Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + + 2. Redistributions in binary form must reproduce the above copyright + notice, this list of conditions and the following disclaimer in the + documentation and/or other materials provided with the distribution. + + 3. Neither the name of Django nor the names of its contributors may be used + to endorse or promote products derived from this software without + specific prior written permission. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND +ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED +WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE +DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR +ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES +(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; +LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON +ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS +SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +""" + +from django.template import TemplateSyntaxError +import re + + +# Regex for token keyword arguments +kwarg_re = re.compile(r"(?:(\w+)=)?(.+)") + + +def token_kwargs(bits, parser, support_legacy=False): + """ + A utility method for parsing token keyword arguments. + + :param bits: A list containing remainder of the token (split by spaces) + that is to be checked for arguments. Valid arguments will be removed + from this list. + + :param support_legacy: If set to true ``True``, the legacy format + ``1 as foo`` will be accepted. Otherwise, only the standard ``foo=1`` + format is allowed. + + :returns: A dictionary of the arguments retrieved from the ``bits`` token + list. + + There is no requirement for all remaining token ``bits`` to be keyword + arguments, so the dictionary will be returned as soon as an invalid + argument format is reached. + """ + if not bits: + return {} + match = kwarg_re.match(bits[0]) + kwarg_format = match and match.group(1) + if not kwarg_format: + if not support_legacy: + return {} + if len(bits) < 3 or bits[1] != 'as': + return {} + + kwargs = {} + while bits: + if kwarg_format: + match = kwarg_re.match(bits[0]) + if not match or not match.group(1): + return kwargs + key, value = match.groups() + del bits[:1] + else: + if len(bits) < 3 or bits[1] != 'as': + return kwargs + key, value = bits[2], bits[0] + del bits[:3] + kwargs[key] = parser.compile_filter(value) + if bits and not kwarg_format: + if bits[0] != 'and': + return kwargs + del bits[:1] + return kwargs + + +def parse_bits(parser, bits, params, varargs, varkw, defaults, + takes_context, name): + """ + Parses bits for template tag helpers (simple_tag, include_tag and + assignment_tag), in particular by detecting syntax errors and by + extracting positional and keyword arguments. + """ + if takes_context: + if params[0] == 'context': + params = params[1:] + else: + raise TemplateSyntaxError( + "'%s' is decorated with takes_context=True so it must " + "have a first argument of 'context'" % name) + args = [] + kwargs = {} + unhandled_params = list(params) + for bit in bits: + # First we try to extract a potential kwarg from the bit + kwarg = token_kwargs([bit], parser) + if kwarg: + # The kwarg was successfully extracted + param, value = list(kwarg.items())[0] + if param not in params and varkw is None: + # An unexpected keyword argument was supplied + raise TemplateSyntaxError( + "'%s' received unexpected keyword argument '%s'" % + (name, param)) + elif param in kwargs: + # The keyword argument has already been supplied once + raise TemplateSyntaxError( + "'%s' received multiple values for keyword argument '%s'" % + (name, param)) + else: + # All good, record the keyword argument + kwargs[str(param)] = value + if param in unhandled_params: + # If using the keyword syntax for a positional arg, then + # consume it. + unhandled_params.remove(param) + else: + if kwargs: + raise TemplateSyntaxError( + "'%s' received some positional argument(s) after some " + "keyword argument(s)" % name) + else: + # Record the positional argument + args.append(parser.compile_filter(bit)) + try: + # Consume from the list of expected positional arguments + unhandled_params.pop(0) + except IndexError: + if varargs is None: + raise TemplateSyntaxError( + "'%s' received too many positional arguments" % + name) + if defaults is not None: + # Consider the last n params handled, where n is the + # number of defaults. + unhandled_params = unhandled_params[:-len(defaults)] + if unhandled_params: + # Some positional arguments were not supplied + raise TemplateSyntaxError( + "'%s' did not receive value(s) for the argument(s): %s" % + (name, ", ".join(["'%s'" % p for p in unhandled_params]))) + return args, kwargs diff --git a/imagekit/templatetags/imagekit.py b/imagekit/templatetags/imagekit.py new file mode 100644 index 0000000..c845dcc --- /dev/null +++ b/imagekit/templatetags/imagekit.py @@ -0,0 +1,293 @@ +from django import template +from django.utils.html import escape +from django.utils.safestring import mark_safe +from .compat import parse_bits +from ..cachefiles import GeneratedImageFile +from ..registry import generator_registry + + +register = template.Library() + + +ASSIGNMENT_DELIMETER = 'as' +HTML_ATTRS_DELIMITER = '--' +DEFAULT_THUMBNAIL_GENERATOR = 'imagekit:thumbnail' + + +def get_cachefile(context, generator_id, generator_kwargs, source=None): + generator_id = generator_id.resolve(context) + kwargs = dict((k, v.resolve(context)) for k, v in generator_kwargs.items()) + generator = generator_registry.get(generator_id, **kwargs) + return GeneratedImageFile(generator) + + +def parse_dimensions(dimensions): + """ + Parse the width and height values from a dimension string. Valid values are + '1x1', '1x', and 'x1'. If one of the dimensions is omitted, the parse result + will be None for that value. + + """ + width, height = [d.strip() or None for d in dimensions.split('x')] + return dict(width=width, height=height) + + +class GenerateImageAssignmentNode(template.Node): + + def __init__(self, variable_name, generator_id, generator_kwargs): + self._generator_id = generator_id + self._generator_kwargs = generator_kwargs + self._variable_name = variable_name + + def get_variable_name(self, context): + return unicode(self._variable_name) + + def render(self, context): + from ..utils import autodiscover + autodiscover() + + variable_name = self.get_variable_name(context) + context[variable_name] = get_cachefile(context, self._generator_id, + self._generator_kwargs) + return '' + + +class GenerateImageTagNode(template.Node): + + def __init__(self, generator_id, generator_kwargs, html_attrs): + self._generator_id = generator_id + self._generator_kwargs = generator_kwargs + self._html_attrs = html_attrs + + def render(self, context): + from ..utils import autodiscover + autodiscover() + + file = get_cachefile(context, self._generator_id, + self._generator_kwargs) + attrs = dict((k, v.resolve(context)) for k, v in + self._html_attrs.items()) + + # Only add width and height if neither is specified (to allow for + # proportional in-browser scaling). + if not 'width' in attrs and not 'height' in attrs: + attrs.update(width=file.width, height=file.height) + + attrs['src'] = file.url + attr_str = ' '.join('%s="%s"' % (escape(k), escape(v)) for k, v in + attrs.items()) + return mark_safe(u'' % attr_str) + + +class ThumbnailAssignmentNode(template.Node): + + def __init__(self, variable_name, generator_id, dimensions, source, generator_kwargs): + self._variable_name = variable_name + self._generator_id = generator_id + self._dimensions = dimensions + self._source = source + self._generator_kwargs = generator_kwargs + + def get_variable_name(self, context): + return unicode(self._variable_name) + + def render(self, context): + from ..utils import autodiscover + autodiscover() + + variable_name = self.get_variable_name(context) + + generator_id = self._generator_id.resolve(context) if self._generator_id else DEFAULT_THUMBNAIL_GENERATOR + kwargs = dict((k, v.resolve(context)) for k, v in + self._generator_kwargs.items()) + kwargs['source'] = self._source.resolve(context) + kwargs.update(parse_dimensions(self._dimensions.resolve(context))) + generator = generator_registry.get(generator_id, **kwargs) + + context[variable_name] = GeneratedImageFile(generator) + + return '' + + +class ThumbnailImageTagNode(template.Node): + + def __init__(self, generator_id, dimensions, source, generator_kwargs, html_attrs): + self._generator_id = generator_id + self._dimensions = dimensions + self._source = source + self._generator_kwargs = generator_kwargs + self._html_attrs = html_attrs + + def render(self, context): + from ..utils import autodiscover + autodiscover() + + generator_id = self._generator_id.resolve(context) if self._generator_id else DEFAULT_THUMBNAIL_GENERATOR + dimensions = parse_dimensions(self._dimensions.resolve(context)) + kwargs = dict((k, v.resolve(context)) for k, v in + self._generator_kwargs.items()) + kwargs['source'] = self._source.resolve(context) + kwargs.update(dimensions) + generator = generator_registry.get(generator_id, **kwargs) + + file = GeneratedImageFile(generator) + + attrs = dict((k, v.resolve(context)) for k, v in + self._html_attrs.items()) + + # Only add width and height if neither is specified (to allow for + # proportional in-browser scaling). + if not 'width' in attrs and not 'height' in attrs: + attrs.update(width=file.width, height=file.height) + + attrs['src'] = file.url + attr_str = ' '.join('%s="%s"' % (escape(k), escape(v)) for k, v in + attrs.items()) + return mark_safe(u'' % attr_str) + + +def parse_ik_tag_bits(parser, bits): + """ + Parses the tag name, html attributes and variable name (for assignment tags) + from the provided bits. The preceding bits may vary and are left to be + parsed by specific tags. + + """ + varname = None + html_attrs = {} + tag_name = bits.pop(0) + + if len(bits) >= 2 and bits[-2] == ASSIGNMENT_DELIMETER: + varname = bits[-1] + bits = bits[:-2] + + if HTML_ATTRS_DELIMITER in bits: + + if varname: + raise template.TemplateSyntaxError('Do not specify html attributes' + ' (using "%s") when using the "%s" tag as an assignment' + ' tag.' % (HTML_ATTRS_DELIMITER, tag_name)) + + index = bits.index(HTML_ATTRS_DELIMITER) + html_bits = bits[index + 1:] + bits = bits[:index] + + if not html_bits: + raise template.TemplateSyntaxError('Don\'t use "%s" unless you\'re' + ' setting html attributes.' % HTML_ATTRS_DELIMITER) + + args, html_attrs = parse_bits(parser, html_bits, [], 'args', + 'kwargs', None, False, tag_name) + if len(args): + raise template.TemplateSyntaxError('All "%s" tag arguments after' + ' the "%s" token must be named.' % (tag_name, + HTML_ATTRS_DELIMITER)) + + return (tag_name, bits, html_attrs, varname) + + +#@register.tag +def generateimage(parser, token): + """ + Creates an image based on the provided arguments. + + By default:: + + {% generateimage 'myapp:thumbnail' source=mymodel.profile_image %} + + generates an ```` tag:: + + + + You can add additional attributes to the tag using "--". For example, + this:: + + {% generateimage 'myapp:thumbnail' source=mymodel.profile_image -- alt="Hello!" %} + + will result in the following markup:: + + Hello! + + For more flexibility, ``generateimage`` also works as an assignment tag:: + + {% generateimage 'myapp:thumbnail' source=mymodel.profile_image as th %} + + + """ + bits = token.split_contents() + + tag_name, bits, html_attrs, varname = parse_ik_tag_bits(parser, bits) + + args, kwargs = parse_bits(parser, bits, ['generator_id'], 'args', 'kwargs', + None, False, tag_name) + + if len(args) != 1: + raise template.TemplateSyntaxError('The "%s" tag requires exactly one' + ' unnamed argument (the generator id).' % tag_name) + + generator_id = args[0] + + if varname: + return GenerateImageAssignmentNode(varname, generator_id, kwargs) + else: + return GenerateImageTagNode(generator_id, kwargs, html_attrs) + + +#@register.tag +def thumbnail(parser, token): + """ + A convenient shortcut syntax for generating a thumbnail. The following:: + + {% thumbnail '100x100' mymodel.profile_image %} + + is equivalent to:: + + {% generateimage 'imagekit:thumbnail' source=mymodel.profile_image width=100 height=100 %} + + The thumbnail tag supports the "--" and "as" bits for adding html + attributes and assigning to a variable, respectively. It also accepts the + kwargs "anchor", and "crop". + + To use "smart cropping" (the ``SmartResize`` processor):: + + {% thumbnail '100x100' mymodel.profile_image %} + + To crop, anchoring the image to the top right (the ``ResizeToFill`` + processor):: + + {% thumbnail '100x100' mymodel.profile_image anchor='tr' %} + + To resize without cropping (using the ``ResizeToFit`` processor):: + + {% thumbnail '100x100' mymodel.profile_image crop=0 %} + + """ + bits = token.split_contents() + + tag_name, bits, html_attrs, varname = parse_ik_tag_bits(parser, bits) + + args, kwargs = parse_bits(parser, bits, [], 'args', 'kwargs', + None, False, tag_name) + + if len(args) < 2: + raise template.TemplateSyntaxError('The "%s" tag requires at least two' + ' unnamed arguments: the dimensions and the source image.' + % tag_name) + elif len(args) > 3: + raise template.TemplateSyntaxError('The "%s" tag accepts at most three' + ' unnamed arguments: a generator id, the dimensions, and the' + ' source image.' % tag_name) + + dimensions, source = args[-2:] + generator_id = args[0] if len(args) > 2 else None + + if varname: + return ThumbnailAssignmentNode(varname, generator_id, dimensions, + source, kwargs) + else: + return ThumbnailImageTagNode(generator_id, dimensions, source, kwargs, + html_attrs) + + +generateimage = register.tag(generateimage) +thumbnail = register.tag(thumbnail) diff --git a/imagekit/utils.py b/imagekit/utils.py index ede309b..ff02a50 100644 --- a/imagekit/utils.py +++ b/imagekit/utils.py @@ -1,15 +1,17 @@ +import logging import os import mimetypes import sys +from tempfile import NamedTemporaryFile import types from django.core.exceptions import ImproperlyConfigured -from django.core.files.base import ContentFile +from django.core.files import File from django.db.models.loading import cache from django.utils.functional import wraps -from django.utils.encoding import smart_str, smart_unicode from django.utils.importlib import import_module +from .exceptions import UnknownExtensionError, UnknownFormatError from .lib import Image, ImageFile, StringIO @@ -17,31 +19,6 @@ RGBA_TRANSPARENCY_FORMATS = ['PNG'] PALETTE_TRANSPARENCY_FORMATS = ['PNG', 'GIF'] -class IKContentFile(ContentFile): - """ - Wraps a ContentFile in a file-like object with a filename and a - content_type. A PIL image format can be optionally be provided as a content - type hint. - - """ - def __init__(self, filename, content, format=None): - self.file = ContentFile(content) - self.file.name = filename - mimetype = getattr(self.file, 'content_type', None) - if format and not mimetype: - mimetype = format_to_mimetype(format) - if not mimetype: - ext = os.path.splitext(filename or '')[1] - mimetype = extension_to_mimetype(ext) - self.file.content_type = mimetype - - def __str__(self): - return smart_str(self.file.name or '') - - def __unicode__(self): - return smart_unicode(self.file.name or u'') - - def img_to_fobj(img, format, autoconvert=True, **options): return save_image(img, StringIO(), format, options, autoconvert) @@ -76,14 +53,6 @@ def _wrap_copy(f): return copy -class UnknownExtensionError(Exception): - pass - - -class UnknownFormatError(Exception): - pass - - _pil_init = 0 @@ -179,28 +148,6 @@ def _get_models(apps): return models -def invalidate_app_cache(apps): - for model in _get_models(apps): - print 'Invalidating cache for "%s.%s"' % (model._meta.app_label, model.__name__) - for obj in model._default_manager.order_by('-pk'): - for f in get_spec_files(obj): - f.invalidate() - - -def validate_app_cache(apps, force_revalidation=False): - for model in _get_models(apps): - for obj in model._default_manager.order_by('-pk'): - model_name = '%s.%s' % (model._meta.app_label, model.__name__) - if force_revalidation: - print 'Invalidating & validating cache for "%s"' % model_name - else: - print 'Validating cache for "%s"' % model_name - for f in get_spec_files(obj): - if force_revalidation: - f.invalidate() - f.validate() - - def suggest_extension(name, format): original_extension = os.path.splitext(name)[1] try: @@ -375,45 +322,110 @@ def prepare_image(img, format): return img, save_kwargs -def ik_model_receiver(fn): - @wraps(fn) - def receiver(sender, **kwargs): - if getattr(sender, '_ik', None): - fn(sender, **kwargs) - return receiver +def get_by_qname(path, desc): + try: + dot = path.rindex('.') + except ValueError: + raise ImproperlyConfigured("%s isn't a %s module." % (path, desc)) + module, objname = path[:dot], path[dot + 1:] + try: + mod = import_module(module) + except ImportError, e: + raise ImproperlyConfigured('Error importing %s module %s: "%s"' % + (desc, module, e)) + try: + obj = getattr(mod, objname) + return obj + except AttributeError: + raise ImproperlyConfigured('%s module "%s" does not define "%s"' + % (desc[0].upper() + desc[1:], module, objname)) -_default_file_storage = None +_singletons = {} -# Nasty duplication of get_default_image_cache_backend. Cleaned up in ik3 -def get_default_file_storage(): +def get_singleton(class_path, desc): + global _singletons + cls = get_by_qname(class_path, desc) + instance = _singletons.get(cls) + if not instance: + instance = _singletons[cls] = cls() + return instance + + +def autodiscover(): """ - Get the default storage. Uses the same method as - django.core.file.storage.get_storage_class + Auto-discover INSTALLED_APPS imagegenerators.py modules and fail silently + when not present. This forces an import on them to register any admin bits + they may want. + + Copied from django.contrib.admin + """ + + from django.conf import settings + from django.utils.importlib import import_module + from django.utils.module_loading import module_has_submodule + + for app in settings.INSTALLED_APPS: + mod = import_module(app) + # Attempt to import the app's admin module. + try: + import_module('%s.imagegenerators' % app) + except: + # Decide whether to bubble up this error. If the app just + # doesn't have an imagegenerators module, we can ignore the error + # attempting to import it, otherwise we want it to bubble up. + if module_has_submodule(mod, 'imagegenerators'): + raise + + +def get_logger(logger_name='imagekit', add_null_handler=True): + logger = logging.getLogger(logger_name) + if add_null_handler: + logger.addHandler(logging.NullHandler()) + return logger + + +def get_field_info(field_file): + """ + A utility for easily extracting information about the host model from a + Django FileField (or subclass). This is especially useful for when you want + to alter processors based on a property of the source model. For example:: + + class MySpec(ImageSpec): + def __init__(self, source): + instance, attname = get_field_info(source) + self.processors = [SmartResize(instance.thumbnail_width, + instance.thumbnail_height)] """ - global _default_file_storage - if not _default_file_storage: - from django.conf import settings - import_path = settings.IMAGEKIT_DEFAULT_FILE_STORAGE + return ( + getattr(field_file, 'instance', None), + getattr(getattr(field_file, 'field', None), 'attname', None), + ) - if not import_path: - return None - try: - dot = import_path.rindex('.') - except ValueError: - raise ImproperlyConfigured("%s isn't an storage module." % \ - import_path) - module, classname = import_path[:dot], import_path[dot + 1:] - try: - mod = import_module(module) - except ImportError, e: - raise ImproperlyConfigured('Error importing storage module %s: "%s"' % (module, e)) - try: - cls = getattr(mod, classname) - _default_file_storage = cls() - except AttributeError: - raise ImproperlyConfigured('Storage module "%s" does not define a "%s" class.' % (module, classname)) - return _default_file_storage +def generate(generator): + """ + Calls the ``generate()`` method of a generator instance, and then wraps the + result in a Django File object so Django knows how to save it. + + """ + content = generator.generate() + + # If the file doesn't have a name, Django will raise an Exception while + # trying to save it, so we create a named temporary file. + if not getattr(content, 'name', None): + f = NamedTemporaryFile() + f.write(content.read()) + f.seek(0) + content = f + + return File(content) + + +def call_strategy_method(generator, method_name, *args, **kwargs): + strategy = getattr(generator, 'cachefile_strategy', None) + fn = getattr(strategy, method_name, None) + if fn is not None: + fn(*args, **kwargs) diff --git a/requirements.txt b/requirements.txt deleted file mode 100644 index 5161716..0000000 --- a/requirements.txt +++ /dev/null @@ -1,3 +0,0 @@ -Django>=1.3.1 -django-appconf>=0.5 -PIL>=1.1.7 diff --git a/setup.py b/setup.py index 6f662f9..61cc546 100644 --- a/setup.py +++ b/setup.py @@ -1,22 +1,34 @@ #/usr/bin/env python import codecs import os +from setuptools import setup, find_packages import sys -from setuptools import setup, find_packages + +# Workaround for multiprocessing/nose issue. See http://bugs.python.org/msg170215 +try: + import multiprocessing +except ImportError: + pass + if 'publish' in sys.argv: os.system('python setup.py sdist upload') sys.exit() + read = lambda filepath: codecs.open(filepath, 'r', 'utf-8').read() -# Dynamically calculate the version based on imagekit.VERSION. -version = __import__('imagekit').get_version() + +# Load package meta from the pkgmeta module without loading imagekit. +pkgmeta = {} +execfile(os.path.join(os.path.dirname(__file__), + 'imagekit', 'pkgmeta.py'), pkgmeta) + setup( name='django-imagekit', - version=version, + version=pkgmeta['__version__'], description='Automated image processing for Django models.', long_description=read(os.path.join(os.path.dirname(__file__), 'README.rst')), author='Justin Driscoll', @@ -28,6 +40,14 @@ setup( packages=find_packages(), zip_safe=False, include_package_data=True, + tests_require=[ + 'beautifulsoup4==4.1.3', + 'nose==1.2.1', + 'nose-progressive==1.3', + 'django-nose==1.1', + 'Pillow==1.7.8', + ], + test_suite='testrunner.run_tests', install_requires=[ 'django-appconf>=0.5', ], diff --git a/testrunner.py b/testrunner.py new file mode 100644 index 0000000..e4d27c7 --- /dev/null +++ b/testrunner.py @@ -0,0 +1,19 @@ +# A wrapper for Django's test runner. +# See http://ericholscher.com/blog/2009/jun/29/enable-setuppy-test-your-django-apps/ +# and http://gremu.net/blog/2010/enable-setuppy-test-your-django-apps/ +import os +import sys + +os.environ['DJANGO_SETTINGS_MODULE'] = 'tests.settings' +test_dir = os.path.dirname(__file__) +sys.path.insert(0, test_dir) + +from django.test.utils import get_runner +from django.conf import settings + + +def run_tests(): + cls = get_runner(settings) + runner = cls() + failures = runner.run_tests(['tests']) + sys.exit(failures) diff --git a/tests/core/assets/Lenna.png b/tests/assets/Lenna.png similarity index 100% rename from tests/core/assets/Lenna.png rename to tests/assets/Lenna.png diff --git a/tests/core/assets/lenna-800x600-white-border.jpg b/tests/assets/lenna-800x600-white-border.jpg similarity index 100% rename from tests/core/assets/lenna-800x600-white-border.jpg rename to tests/assets/lenna-800x600-white-border.jpg diff --git a/tests/core/assets/lenna-800x600.jpg b/tests/assets/lenna-800x600.jpg similarity index 100% rename from tests/core/assets/lenna-800x600.jpg rename to tests/assets/lenna-800x600.jpg diff --git a/tests/core/tests.py b/tests/core/tests.py deleted file mode 100644 index a73915e..0000000 --- a/tests/core/tests.py +++ /dev/null @@ -1,87 +0,0 @@ -from __future__ import with_statement - -import os - -from django.test import TestCase - -from imagekit import utils -from .models import (Photo, AbstractImageModel, ConcreteImageModel1, - ConcreteImageModel2) -from .testutils import create_photo, pickleback - - -class IKTest(TestCase): - - def setUp(self): - self.photo = create_photo('test.jpg') - - def test_nodelete(self): - """Don't delete the spec file when the source image hasn't changed. - - """ - filename = self.photo.thumbnail.file.name - self.photo.save() - self.assertTrue(self.photo.thumbnail.storage.exists(filename)) - - def test_save_image(self): - photo = Photo.objects.get(id=self.photo.id) - self.assertTrue(os.path.isfile(photo.original_image.path)) - - def test_setup(self): - self.assertEqual(self.photo.original_image.width, 800) - self.assertEqual(self.photo.original_image.height, 600) - - def test_thumbnail_creation(self): - photo = Photo.objects.get(id=self.photo.id) - self.assertTrue(os.path.isfile(photo.thumbnail.file.name)) - - def test_thumbnail_size(self): - """ Explicit and smart-cropped thumbnail size """ - self.assertEqual(self.photo.thumbnail.width, 50) - self.assertEqual(self.photo.thumbnail.height, 50) - self.assertEqual(self.photo.smartcropped_thumbnail.width, 50) - self.assertEqual(self.photo.smartcropped_thumbnail.height, 50) - - def test_thumbnail_source_file(self): - self.assertEqual( - self.photo.thumbnail.source_file, self.photo.original_image) - - -class IKUtilsTest(TestCase): - def test_extension_to_format(self): - self.assertEqual(utils.extension_to_format('.jpeg'), 'JPEG') - self.assertEqual(utils.extension_to_format('.rgba'), 'SGI') - - self.assertRaises(utils.UnknownExtensionError, - lambda: utils.extension_to_format('.txt')) - - def test_format_to_extension_no_init(self): - self.assertEqual(utils.format_to_extension('PNG'), '.png') - self.assertEqual(utils.format_to_extension('ICO'), '.ico') - - self.assertRaises(utils.UnknownFormatError, - lambda: utils.format_to_extension('TXT')) - - -class PickleTest(TestCase): - def test_model(self): - ph = pickleback(create_photo('pickletest.jpg')) - - # This isn't supposed to error. - ph.thumbnail.source_file - - def test_field(self): - thumbnail = pickleback(create_photo('pickletest2.jpg').thumbnail) - - # This isn't supposed to error. - thumbnail.source_file - - -class InheritanceTest(TestCase): - def test_abstract_base(self): - self.assertEqual(set(AbstractImageModel._ik.spec_fields), - set(['abstract_class_spec'])) - self.assertEqual(set(ConcreteImageModel1._ik.spec_fields), - set(['abstract_class_spec', 'first_spec'])) - self.assertEqual(set(ConcreteImageModel2._ik.spec_fields), - set(['abstract_class_spec', 'second_spec'])) diff --git a/tests/core/testutils.py b/tests/core/testutils.py deleted file mode 100644 index 4acc13c..0000000 --- a/tests/core/testutils.py +++ /dev/null @@ -1,46 +0,0 @@ -import os -import tempfile - -from django.core.files.base import ContentFile - -from imagekit.lib import Image, StringIO -from .models import Photo -import pickle - - -def generate_lenna(): - """ - See also: - - http://en.wikipedia.org/wiki/Lenna - http://sipi.usc.edu/database/database.php?volume=misc&image=12 - - """ - tmp = tempfile.TemporaryFile() - lennapath = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'assets', 'lenna-800x600-white-border.jpg') - with open(lennapath, "r+b") as lennafile: - Image.open(lennafile).save(tmp, 'JPEG') - tmp.seek(0) - return tmp - - -def create_instance(model_class, image_name): - instance = model_class() - img = generate_lenna() - file = ContentFile(img.read()) - instance.original_image = file - instance.original_image.save(image_name, file) - instance.save() - img.close() - return instance - - -def create_photo(name): - return create_instance(Photo, name) - - -def pickleback(obj): - pickled = StringIO() - pickle.dump(obj, pickled) - pickled.seek(0) - return pickle.load(pickled) diff --git a/tests/imagegenerators.py b/tests/imagegenerators.py new file mode 100644 index 0000000..11e87b3 --- /dev/null +++ b/tests/imagegenerators.py @@ -0,0 +1,16 @@ +from imagekit import ImageSpec, register +from imagekit.processors import ResizeToFill + + +class TestSpec(ImageSpec): + pass + + +class ResizeTo1PixelSquare(ImageSpec): + def __init__(self, width=None, height=None, anchor=None, crop=None, **kwargs): + self.processors = [ResizeToFill(1, 1)] + super(ResizeTo1PixelSquare, self).__init__(**kwargs) + + +register.generator('testspec', TestSpec) +register.generator('1pxsq', ResizeTo1PixelSquare) diff --git a/tests/media/lenna.png b/tests/media/lenna.png new file mode 100644 index 0000000..59ef68a Binary files /dev/null and b/tests/media/lenna.png differ diff --git a/tests/core/models.py b/tests/models.py similarity index 59% rename from tests/core/models.py rename to tests/models.py index 2c3d8e4..17e887b 100644 --- a/tests/core/models.py +++ b/tests/models.py @@ -1,23 +1,31 @@ from django.db import models +from imagekit.models import ProcessedImageField from imagekit.models import ImageSpecField -from imagekit.processors import Adjust -from imagekit.processors import ResizeToFill -from imagekit.processors import SmartCrop +from imagekit.processors import Adjust, ResizeToFill, SmartCrop + + +class ImageModel(models.Model): + image = models.ImageField(upload_to='b') class Photo(models.Model): original_image = models.ImageField(upload_to='photos') thumbnail = ImageSpecField([Adjust(contrast=1.2, sharpness=1.1), - ResizeToFill(50, 50)], image_field='original_image', format='JPEG', + ResizeToFill(50, 50)], source='original_image', format='JPEG', options={'quality': 90}) smartcropped_thumbnail = ImageSpecField([Adjust(contrast=1.2, - sharpness=1.1), SmartCrop(50, 50)], image_field='original_image', + sharpness=1.1), SmartCrop(50, 50)], source='original_image', format='JPEG', options={'quality': 90}) +class ProcessedImageFieldModel(models.Model): + processed = ProcessedImageField([SmartCrop(50, 50)], format='JPEG', + options={'quality': 90}, upload_to='p') + + class AbstractImageModel(models.Model): original_image = models.ImageField(upload_to='photos') abstract_class_spec = ImageSpecField() diff --git a/tests/run_tests.sh b/tests/run_tests.sh deleted file mode 100755 index 6d3f37b..0000000 --- a/tests/run_tests.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/bash -PYTHONPATH=$PWD:$PWD/..${PYTHONPATH:+:$PYTHONPATH} -export PYTHONPATH - -echo "Running django-imagekit tests..." -django-admin.py test core --settings=settings diff --git a/tests/settings.py b/tests/settings.py index f034e9c..3272aee 100644 --- a/tests/settings.py +++ b/tests/settings.py @@ -25,7 +25,22 @@ INSTALLED_APPS = [ 'django.contrib.auth', 'django.contrib.contenttypes', 'imagekit', - 'core', + 'tests', + 'django_nose', +] + +TEST_RUNNER = 'django_nose.NoseTestSuiteRunner' +NOSE_ARGS = [ + '-s', + '--with-progressive', + + # When the tests are run --with-coverage, these args configure coverage + # reporting (requires coverage to be installed). + # Without the --with-coverage flag, they have no effect. + '--cover-tests', + '--cover-html', + '--cover-package=imagekit', + '--cover-html-dir=%s' % os.path.join(BASE_PATH, 'cover') ] DEBUG = True diff --git a/tests/test_fields.py b/tests/test_fields.py new file mode 100644 index 0000000..df513ee --- /dev/null +++ b/tests/test_fields.py @@ -0,0 +1,36 @@ +from django import forms +from django.core.files.base import File +from django.core.files.uploadedfile import SimpleUploadedFile +from imagekit import forms as ikforms +from imagekit.processors import SmartCrop +from nose.tools import eq_ +from . import imagegenerators # noqa +from .models import ProcessedImageFieldModel, ImageModel +from .utils import get_image_file + + +def test_model_processedimagefield(): + instance = ProcessedImageFieldModel() + file = File(get_image_file()) + instance.processed.save('whatever.jpeg', file) + instance.save() + + eq_(instance.processed.width, 50) + eq_(instance.processed.height, 50) + + +def test_form_processedimagefield(): + class TestForm(forms.ModelForm): + image = ikforms.ProcessedImageField(spec_id='tests:testform_image', + processors=[SmartCrop(50, 50)], format='JPEG') + + class Meta: + model = ImageModel + + upload_file = get_image_file() + file_dict = {'image': SimpleUploadedFile('abc.jpg', upload_file.read())} + form = TestForm({}, file_dict) + instance = form.save() + + eq_(instance.image.width, 50) + eq_(instance.image.height, 50) diff --git a/tests/test_generateimage_tag.py b/tests/test_generateimage_tag.py new file mode 100644 index 0000000..c39b794 --- /dev/null +++ b/tests/test_generateimage_tag.py @@ -0,0 +1,52 @@ +from django.template import TemplateSyntaxError +from nose.tools import eq_, assert_false, raises, assert_not_equal +from . import imagegenerators # noqa +from .utils import render_tag, get_html_attrs + + +def test_img_tag(): + ttag = r"""{% generateimage 'testspec' source=img %}""" + attrs = get_html_attrs(ttag) + expected_attrs = set(['src', 'width', 'height']) + eq_(set(attrs.keys()), expected_attrs) + for k in expected_attrs: + assert_not_equal(attrs[k].strip(), '') + + +def test_img_tag_attrs(): + ttag = r"""{% generateimage 'testspec' source=img -- alt="Hello" %}""" + attrs = get_html_attrs(ttag) + eq_(attrs.get('alt'), 'Hello') + + +@raises(TemplateSyntaxError) +def test_dangling_html_attrs_delimiter(): + ttag = r"""{% generateimage 'testspec' source=img -- %}""" + render_tag(ttag) + + +@raises(TemplateSyntaxError) +def test_html_attrs_assignment(): + """ + You can either use generateimage as an assigment tag or specify html attrs, + but not both. + + """ + ttag = r"""{% generateimage 'testspec' source=img -- alt="Hello" as th %}""" + render_tag(ttag) + + +def test_single_dimension_attr(): + """ + If you only provide one of width or height, the other should not be added. + + """ + ttag = r"""{% generateimage 'testspec' source=img -- width="50" %}""" + attrs = get_html_attrs(ttag) + assert_false('height' in attrs) + + +def test_assignment_tag(): + ttag = r"""{% generateimage 'testspec' source=img as th %}{{ th.url }}""" + html = render_tag(ttag) + assert_not_equal(html.strip(), '') diff --git a/tests/test_processors.py b/tests/test_processors.py new file mode 100644 index 0000000..316b57f --- /dev/null +++ b/tests/test_processors.py @@ -0,0 +1,31 @@ +from imagekit.lib import Image +from imagekit.processors import ResizeToFill, ResizeToFit, SmartCrop +from nose.tools import eq_ +from .utils import create_image + + +def test_smartcrop(): + img = SmartCrop(100, 100).process(create_image()) + eq_(img.size, (100, 100)) + + +def test_resizetofill(): + img = ResizeToFill(100, 100).process(create_image()) + eq_(img.size, (100, 100)) + + +def test_resizetofit(): + # First create an image with aspect ratio 2:1... + img = Image.new('RGB', (200, 100)) + + # ...then resize it to fit within a 100x100 canvas. + img = ResizeToFit(100, 100).process(img) + + # Assert that the image has maintained the aspect ratio. + eq_(img.size, (100, 50)) + + +def test_resizetofit_mat(): + img = Image.new('RGB', (200, 100)) + img = ResizeToFit(100, 100, mat_color=0x000000).process(img) + eq_(img.size, (100, 100)) diff --git a/tests/test_serialization.py b/tests/test_serialization.py new file mode 100644 index 0000000..cd7b6d7 --- /dev/null +++ b/tests/test_serialization.py @@ -0,0 +1,13 @@ +""" +Make sure that the various IK classes can be successfully serialized and +deserialized. This is important when using IK with Celery. + +""" + +from .utils import create_photo, pickleback + + +def test_imagespecfield(): + instance = create_photo('pickletest2.jpg') + thumbnail = pickleback(instance.thumbnail) + thumbnail.generate() diff --git a/tests/test_thumbnail_tag.py b/tests/test_thumbnail_tag.py new file mode 100644 index 0000000..e31304a --- /dev/null +++ b/tests/test_thumbnail_tag.py @@ -0,0 +1,66 @@ +from django.template import TemplateSyntaxError +from nose.tools import eq_, raises, assert_not_equal +from . import imagegenerators # noqa +from .utils import render_tag, get_html_attrs + + +def test_img_tag(): + ttag = r"""{% thumbnail '100x100' img %}""" + attrs = get_html_attrs(ttag) + expected_attrs = set(['src', 'width', 'height']) + eq_(set(attrs.keys()), expected_attrs) + for k in expected_attrs: + assert_not_equal(attrs[k].strip(), '') + + +def test_img_tag_attrs(): + ttag = r"""{% thumbnail '100x100' img -- alt="Hello" %}""" + attrs = get_html_attrs(ttag) + eq_(attrs.get('alt'), 'Hello') + + +@raises(TemplateSyntaxError) +def test_dangling_html_attrs_delimiter(): + ttag = r"""{% thumbnail '100x100' img -- %}""" + render_tag(ttag) + + +@raises(TemplateSyntaxError) +def test_not_enough_args(): + ttag = r"""{% thumbnail '100x100' %}""" + render_tag(ttag) + + +@raises(TemplateSyntaxError) +def test_too_many_args(): + ttag = r"""{% thumbnail 'generator_id' '100x100' img 'extra' %}""" + render_tag(ttag) + + +@raises(TemplateSyntaxError) +def test_html_attrs_assignment(): + """ + You can either use thumbnail as an assigment tag or specify html attrs, + but not both. + + """ + ttag = r"""{% thumbnail '100x100' img -- alt="Hello" as th %}""" + render_tag(ttag) + + +def test_assignment_tag(): + ttag = r"""{% thumbnail '100x100' img as th %}{{ th.url }}""" + html = render_tag(ttag) + assert_not_equal(html, '') + + +def test_single_dimension(): + ttag = r"""{% thumbnail '100x' img as th %}{{ th.width }}""" + html = render_tag(ttag) + eq_(html, '100') + + +def test_alternate_generator(): + ttag = r"""{% thumbnail '1pxsq' '100x' img as th %}{{ th.width }}""" + html = render_tag(ttag) + eq_(html, '1') diff --git a/tests/test_utils.py b/tests/test_utils.py new file mode 100644 index 0000000..f4c777e --- /dev/null +++ b/tests/test_utils.py @@ -0,0 +1,23 @@ +from imagekit.exceptions import UnknownFormatError, UnknownExtensionError +from imagekit.utils import extension_to_format, format_to_extension +from nose.tools import eq_, raises + + +def test_extension_to_format(): + eq_(extension_to_format('.jpeg'), 'JPEG') + eq_(extension_to_format('.rgba'), 'SGI') + + +def test_format_to_extension_no_init(): + eq_(format_to_extension('PNG'), '.png') + eq_(format_to_extension('ICO'), '.ico') + + +@raises(UnknownFormatError) +def test_unknown_format(): + format_to_extension('TXT') + + +@raises(UnknownExtensionError) +def test_unknown_extension(): + extension_to_format('.txt') diff --git a/tests/utils.py b/tests/utils.py new file mode 100644 index 0000000..2763f8f --- /dev/null +++ b/tests/utils.py @@ -0,0 +1,55 @@ +from bs4 import BeautifulSoup +import os +from django.conf import settings +from django.core.files import File +from django.template import Context, Template +from imagekit.lib import Image, StringIO +import pickle +from .models import Photo + + +def get_image_file(): + """ + See also: + + http://en.wikipedia.org/wiki/Lenna + http://sipi.usc.edu/database/database.php?volume=misc&image=12 + + """ + path = os.path.join(settings.MEDIA_ROOT, 'lenna.png') + return open(path, 'r+b') + + +def create_image(): + return Image.open(get_image_file()) + + +def create_instance(model_class, image_name): + instance = model_class() + img = File(get_image_file()) + instance.original_image.save(image_name, img) + instance.save() + img.close() + return instance + + +def create_photo(name): + return create_instance(Photo, name) + + +def pickleback(obj): + pickled = StringIO() + pickle.dump(obj, pickled) + pickled.seek(0) + return pickle.load(pickled) + + +def render_tag(ttag): + img = get_image_file() + template = Template('{%% load imagekit %%}%s' % ttag) + context = Context({'img': img}) + return template.render(context) + + +def get_html_attrs(ttag): + return BeautifulSoup(render_tag(ttag)).img.attrs diff --git a/tox.ini b/tox.ini index 1ad1957..c7c54cf 100644 --- a/tox.ini +++ b/tox.ini @@ -4,42 +4,34 @@ envlist = py26-django14, py26-django13, py26-django12 [testenv] -changedir = tests -setenv = PYTHONPATH = {toxinidir}/tests -commands = django-admin.py test core --settings=settings +commands = python setup.py test [testenv:py27-django14] basepython = python2.7 deps = Django>=1.4,<1.5 - Pillow [testenv:py27-django13] basepython = python2.7 deps = Django>=1.3,<1.4 - Pillow [testenv:py27-django12] basepython = python2.7 deps = Django>=1.2,<1.3 - Pillow [testenv:py26-django14] basepython = python2.6 deps = Django>=1.4,<1.5 - Pillow [testenv:py26-django13] basepython = python2.6 deps = Django>=1.3,<1.4 - Pillow [testenv:py26-django12] basepython = python2.6 deps = Django>=1.2,<1.3 - Pillow