Refs #25, refs #39, refs #40, refs #42 - Big refactoring in documentation and demo: narrative documentation uses examples from demo project.

This commit is contained in:
Benoît Bryon 2013-10-28 16:58:18 +01:00
parent 413f7a9052
commit 874f3b9b54
69 changed files with 1040 additions and 804 deletions

View file

@ -9,5 +9,7 @@ Original code by `Novapost <http://www.novapost.fr>`_ team:
* Nicolas Tobo <https://github.com/nicolastobo>
* Lauréline Guérin <https://github.com/zebuline>
* Gregory Tappero <https://github.com/coulix>
* Rémy Hubscher <remy.hubscher@novapost.fr>
* Benoît Bryon <benoit@marmelune.net>
* Rémy Hubscher <remy.hubscher@novapost.fr>
Developers: https://github.com/benoitbryon/django-downloadview/graphs/contributors

View file

@ -2,7 +2,7 @@
License
#######
Copyright (c) 2012, Benoît Bryon.
Copyright (c) 2012-2013, Benoît Bryon.
All rights reserved.
Redistribution and use in source and binary forms, with or without

View file

@ -72,10 +72,15 @@ documentation: sphinx
demo: develop
mkdir -p var/media/document
$(BIN_DIR)/demo syncdb --noinput
cp $(ROOT_DIR)/demo/demoproject/download/fixtures/hello-world.txt var/media/document/
$(BIN_DIR)/demo loaddata $(ROOT_DIR)/demo/demoproject/download/fixtures/demo.json
# Install fixtures.
mkdir -p var/media
cp -r $(ROOT_DIR)/demo/demoproject/fixtures var/media/object
cp -r $(ROOT_DIR)/demo/demoproject/fixtures var/media/object-other
cp -r $(ROOT_DIR)/demo/demoproject/fixtures var/media/nginx
$(BIN_DIR)/demo loaddata demo.json
runserver: demo
$(BIN_DIR)/demo runserver

View file

@ -2,8 +2,20 @@
Demo project
############
The :file:`demo/` folder holds a demo project to illustrate django-downloadview
usage.
`Demo folder in project's repository`_ contains a Django project to illustrate
`django-downloadview` usage.
*****************************************
Documentation includes code from the demo
*****************************************
Almost every example in this documentation comes from the demo:
* discover examples in the documentation;
* browse related code and tests in demo project.
Examples in documentation are tested via demo project!
***********************
@ -19,7 +31,7 @@ Deploy the demo
System requirements:
* `Python`_ version 2.6 or 2.7, available as ``python`` command.
* `Python`_ version 2.7, available as ``python`` command.
.. note::
@ -34,7 +46,7 @@ Execute:
git clone git@github.com:benoitbryon/django-downloadview.git
cd django-downloadview/
make demo
make runserver
It installs and runs the demo server on localhost, port 8000. So have a look
at http://localhost:8000/
@ -47,44 +59,6 @@ at http://localhost:8000/
Browse and use :file:`demo/demoproject/` as a sandbox.
*********************************
Base example provided in the demo
*********************************
In the "demoproject" project, there is an application called "download".
:file:`demo/demoproject/settings.py`:
.. literalinclude:: ../demo/demoproject/settings.py
:language: python
:lines: 33-49
:emphasize-lines: 44
This application holds a ``Document`` model.
:file:`demo/demoproject/download/models.py`:
.. literalinclude:: ../demo/demoproject/download/models.py
:language: python
.. note::
The ``storage`` is the default one, i.e. it uses ``settings.MEDIA_ROOT``.
Combined to this ``upload_to`` configuration, files for ``Document`` model
live in :file:`var/media/document/` folder, relative to your
django-downloadview clone root.
There is a download view named "download_document" for this model:
:file:`demo/demoproject/download/urls.py`:
.. literalinclude:: ../demo/demoproject/download/urls.py
:language: python
As is, Django is to serve the files, i.e. load chunks into memory and stream
them.
**********
References
**********

View file

@ -1 +0,0 @@

View file

@ -1,7 +0,0 @@
from django.db import models
class Document(models.Model):
"""A sample model with a FileField."""
slug = models.SlugField(verbose_name='slug')
file = models.FileField(verbose_name='file', upload_to='document')

View file

@ -1,107 +0,0 @@
# coding=utf8
"""Test suite for demoproject.download."""
from os import listdir
from os.path import abspath, dirname, join
from django.core.files import File
from django.core.urlresolvers import reverse
from django.test import TestCase
from django_downloadview.test import temporary_media_root
from demoproject.download.models import Document
app_dir = dirname(abspath(__file__))
fixtures_dir = join(app_dir, 'fixtures')
class DownloadTestCase(TestCase):
"""Base class for download tests."""
def setUp(self):
"""Common setup."""
super(DownloadTestCase, self).setUp()
self.files = {}
for f in listdir(fixtures_dir):
self.files[f] = abspath(join(fixtures_dir, f))
def assertDownloadHelloWorld(self, response, is_attachment=True):
"""Assert response is 'hello-world.txt' download."""
self.assertEquals(response.status_code, 200)
self.assertEquals(response['Content-Type'],
'text/plain; charset=utf-8')
self.assertFalse('ContentEncoding' in response)
if is_attachment:
self.assertEquals(response['Content-Disposition'],
'attachment; filename=hello-world.txt')
else:
self.assertFalse('Content-Disposition' in response)
self.assertEqual(open(self.files['hello-world.txt']).read(),
''.join(response.streaming_content))
class PathDownloadViewTestCase(DownloadTestCase):
"""Test "hello_world" and "hello_world_inline" views."""
def test_download_hello_world(self):
"""hello_world view returns hello-world.txt as attachement."""
download_url = reverse('hello_world')
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)
def test_download_hello_world_inline(self):
"""hello_world view returns hello-world.txt as attachement."""
download_url = reverse('hello_world_inline')
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response, is_attachment=False)
class CustomPathDownloadViewTestCase(DownloadTestCase):
"""Test "fixture_from_path" view."""
def test_download_hello_world(self):
"""fixture_from_path view returns hello-world.txt as attachement."""
download_url = reverse('fixture_from_path', args=['hello-world.txt'])
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)
class StorageDownloadViewTestCase(DownloadTestCase):
"""Test "fixture_from_storage" view."""
def test_download_hello_world(self):
"""fixture_from_storage view returns hello-world.txt as attachement."""
download_url = reverse('fixture_from_storage',
args=['hello-world.txt'])
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)
class ObjectDownloadViewTestCase(DownloadTestCase):
"""Test generic ObjectDownloadView."""
@temporary_media_root()
def test_download_hello_world(self):
"""'download_document' view returns hello-world.txt as attachement."""
slug = 'hello-world'
download_url = reverse('document', kwargs={'slug': slug})
Document.objects.create(slug=slug,
file=File(open(self.files['hello-world.txt'])))
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)
class GeneratedDownloadViewTestCase(DownloadTestCase):
"""Test "generated_hello_world" view."""
def test_download_hello_world(self):
"""generated_hello_world view returns hello-world.txt as attachement.
"""
download_url = reverse('generated_hello_world')
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)
class ProxiedDownloadViewTestCase(DownloadTestCase):
"""Test "http_hello_world" view."""
def test_download_readme(self):
"""http_hello_world view proxies file from URL."""
download_url = reverse('http_hello_world')
response = self.client.get(download_url)
self.assertDownloadHelloWorld(response)

View file

@ -1,34 +0,0 @@
# coding=utf8
"""URL mapping."""
from django.conf.urls import patterns, url
urlpatterns = patterns(
'demoproject.download.views',
# Model-based downloads.
url(r'^document/(?P<slug>[a-zA-Z0-9_-]+)/$',
'download_document',
name='document'),
# Storage-based downloads.
url(r'^storage/(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$',
'download_fixture_from_storage',
name='fixture_from_storage'),
# Path-based downloads.
url(r'^hello-world\.txt$',
'download_hello_world',
name='hello_world'),
url(r'^hello-world-inline\.txt$',
'download_hello_world_inline',
name='hello_world_inline'),
url(r'^path/(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$',
'download_fixture_from_path',
name='fixture_from_path'),
# URL-based downloads.
url(r'^http/readme\.txt$',
'download_http_hello_world',
name='http_hello_world'),
# Generated downloads.
url(r'^generated/hello-world\.txt$',
'download_generated_hello_world',
name='generated_hello_world'),
)

View file

@ -1,90 +0,0 @@
# coding=utf8
"""Demo download views."""
from cStringIO import StringIO
from os.path import abspath, dirname, join
from django.core.files.storage import FileSystemStorage
from django_downloadview.files import VirtualFile
from django_downloadview import views
from demoproject.download.models import Document
# Some initializations.
#: Directory containing code of :py:module:`demoproject.download.views`.
app_dir = dirname(abspath(__file__))
#: Directory containing files fixtures.
fixtures_dir = join(app_dir, 'fixtures')
#: Path to a text file that says 'Hello world!'.
hello_world_path = join(fixtures_dir, 'hello-world.txt')
#: Storage for fixtures.
fixtures_storage = FileSystemStorage(location=fixtures_dir)
# Here are the views.
#: Pre-configured download view for :py:class:`Document` model.
download_document = views.ObjectDownloadView.as_view(model=Document)
#: Same as download_document, but streamed inline, i.e. not as attachments.
download_document_inline = views.ObjectDownloadView.as_view(model=Document,
attachment=False)
#: Pre-configured view using a storage.
download_fixture_from_storage = views.StorageDownloadView.as_view(
storage=fixtures_storage)
#: Direct download of one file, based on an absolute path.
#:
#: You could use this example as a shortcut, inside other views.
download_hello_world = views.PathDownloadView.as_view(path=hello_world_path)
#: Direct download of one file, based on an absolute path, not as attachment.
download_hello_world_inline = views.PathDownloadView.as_view(
path=hello_world_path,
attachment=False)
class CustomPathDownloadView(views.PathDownloadView):
"""Example of customized PathDownloadView."""
def get_path(self):
"""Convert relative path (provided in URL) into absolute path.
Notice that this particularly simple use case is covered by
:py:class:`django_downloadview.views.StorageDownloadView`.
.. warning::
If you are doing such things, make the path secure! Prevent users
to download files anywhere in the filesystem.
"""
path = super(CustomPathDownloadView, self).get_path()
return join(fixtures_dir, path)
#: Pre-configured :py:class:`CustomPathDownloadView`.
download_fixture_from_path = CustomPathDownloadView.as_view()
class StringIODownloadView(views.VirtualDownloadView):
"""Sample download view using StringIO object."""
def get_file(self):
"""Return wrapper on StringIO object."""
file_obj = StringIO(u"Hello world!\n".encode('utf-8'))
return VirtualFile(file_obj, name='hello-world.txt')
#: Pre-configured view that serves "Hello world!" via a StringIO.
download_generated_hello_world = StringIODownloadView.as_view()
download_http_hello_world = views.HTTPDownloadView.as_view(
url=u'http://localhost:8000/download/hello-world.txt',
basename=u'hello-world.txt')

View file

@ -1,7 +1,7 @@
[
{
"pk": 1,
"model": "download.document",
"model": "object.document",
"fields": {
"slug": "hello-world",
"file": "document/hello-world.txt"

View file

@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
"""Demo for :class:`django_downloadview.HTTPDownloadView`.
Code in this package is included in documentation's :doc:`/views/http`.
Make sure to maintain both together.
"""

View file

@ -0,0 +1 @@
"""Required to make a Django application."""

View file

@ -0,0 +1,16 @@
from django.core.urlresolvers import reverse
import django.test
from django_downloadview import assert_download_response
class SimpleURLTestCase(django.test.TestCase):
def test_download_response(self):
"""'simple_url' serves 'hello-world.txt' from Github."""
url = reverse('http:simple_url')
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')

View file

@ -0,0 +1,11 @@
from django.conf.urls import patterns, url
from demoproject.http import views
urlpatterns = patterns(
'',
url(r'^simple_url/$',
views.simple_url,
name='simple_url'),
)

View file

@ -0,0 +1,12 @@
from django_downloadview import HTTPDownloadView
class SimpleURLDownloadView(HTTPDownloadView):
def get_url(self):
"""Return URL of hello-world.txt file on GitHub."""
return 'https://raw.github.com/benoitbryon/django-downloadview' \
'/b7f660c5e3f37d918b106b02c5af7a887acc0111' \
'/demo/demoproject/download/fixtures/hello-world.txt'
simple_url = SimpleURLDownloadView.as_view()

View file

@ -1 +1 @@
"""Nginx optimizations applied to demoproject.download."""
"""Nginx optimizations."""

View file

@ -0,0 +1 @@
"""Required to make a Django application."""

View file

@ -1,62 +1,51 @@
"""Test suite for demoproject.nginx."""
from django.core.files import File
from django.core.urlresolvers import reverse_lazy as reverse
import os
from django.core.files.base import ContentFile
from django.core.urlresolvers import reverse
import django.test
from django_downloadview.nginx import assert_x_accel_redirect
from django_downloadview.test import temporary_media_root
from demoproject.download.models import Document
from demoproject.download.tests import DownloadTestCase
from demoproject.nginx.views import storage, storage_dir
class XAccelRedirectDecoratorTestCase(DownloadTestCase):
@temporary_media_root()
def setup_file():
if not os.path.exists(storage_dir):
os.makedirs(storage_dir)
storage.save('hello-world.txt', ContentFile(u'Hello world!\n'))
class OptimizedByMiddlewareTestCase(django.test.TestCase):
def test_response(self):
"""'download_document_nginx' view returns a valid X-Accel response."""
document = Document.objects.create(
slug='hello-world',
file=File(open(self.files['hello-world.txt'])),
)
download_url = reverse('download_document_nginx',
kwargs={'slug': 'hello-world'})
response = self.client.get(download_url)
self.assertEquals(response.status_code, 200)
# Validation shortcut: assert_x_accel_redirect.
"""'nginx:optimized_by_middleware' returns X-Accel response."""
setup_file()
url = reverse('nginx:optimized_by_middleware')
response = self.client.get(url)
assert_x_accel_redirect(
self,
response,
content_type="text/plain; charset=utf-8",
charset="utf-8",
basename="hello-world.txt",
redirect_url="/download-optimized/document/hello-world.txt",
redirect_url="/nginx-optimized-by-middleware/hello-world.txt",
expires=None,
with_buffering=None,
limit_rate=None)
# Check some more items, because this test is part of
# django-downloadview tests.
self.assertFalse('ContentEncoding' in response)
self.assertEquals(response['Content-Disposition'],
'attachment; filename=hello-world.txt')
class InlineXAccelRedirectTestCase(DownloadTestCase):
@temporary_media_root()
class OptimizedByDecoratorTestCase(django.test.TestCase):
def test_response(self):
"""X-Accel optimization respects ``attachment`` attribute."""
document = Document.objects.create(
slug='hello-world',
file=File(open(self.files['hello-world.txt'])),
)
download_url = reverse('download_document_nginx_inline',
kwargs={'slug': 'hello-world'})
response = self.client.get(download_url)
"""'nginx:optimized_by_decorator' returns X-Accel response."""
setup_file()
url = reverse('nginx:optimized_by_decorator')
response = self.client.get(url)
assert_x_accel_redirect(
self,
response,
content_type="text/plain; charset=utf-8",
charset="utf-8",
attachment=False,
redirect_url="/download-optimized/document/hello-world.txt",
basename="hello-world.txt",
redirect_url="/nginx-optimized-by-decorator/hello-world.txt",
expires=None,
with_buffering=None,
limit_rate=None)

View file

@ -4,9 +4,10 @@ from django.conf.urls import patterns, url
urlpatterns = patterns(
'demoproject.nginx.views',
url(r'^document-nginx/(?P<slug>[a-zA-Z0-9_-]+)/$',
'download_document_nginx', name='download_document_nginx'),
url(r'^document-nginx-inline/(?P<slug>[a-zA-Z0-9_-]+)/$',
'download_document_nginx_inline',
name='download_document_nginx_inline'),
url(r'^optimized-by-middleware/$',
'optimized_by_middleware',
name='optimized_by_middleware'),
url(r'^optimized-by-decorator/$',
'optimized_by_decorator',
name='optimized_by_decorator'),
)

View file

@ -1,18 +1,22 @@
"""Views."""
from django.conf import settings
import os
from django.conf import settings
from django.core.files.storage import FileSystemStorage
from django_downloadview import StorageDownloadView
from django_downloadview.nginx import x_accel_redirect
from demoproject.download import views
storage_dir = os.path.join(settings.MEDIA_ROOT, 'nginx')
storage = FileSystemStorage(location=storage_dir,
base_url=''.join([settings.MEDIA_URL, 'nginx/']))
download_document_nginx = x_accel_redirect(
views.download_document,
source_dir='/var/www/files',
destination_url='/download-optimized')
optimized_by_middleware = StorageDownloadView.as_view(storage=storage,
path='hello-world.txt')
download_document_nginx_inline = x_accel_redirect(
views.download_document_inline,
source_dir=settings.MEDIA_ROOT,
destination_url='/download-optimized')
optimized_by_decorator = x_accel_redirect(
StorageDownloadView.as_view(storage=storage, path='hello-world.txt'),
source_url=storage.base_url,
destination_url='/nginx-optimized-by-decorator/')

View file

@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
"""Demo for :class:`django_downloadview.ObjectDownloadView`.
Code in this package is included in documentation's :doc:`/views/object`.
Make sure to maintain both together.
"""

View file

@ -0,0 +1,8 @@
from django.db import models
class Document(models.Model):
slug = models.SlugField()
file = models.FileField(upload_to='object')
another_file = models.FileField(upload_to='object-other')
basename = models.CharField(max_length=100)

View file

@ -0,0 +1,70 @@
from django.core.files.base import ContentFile
from django.core.urlresolvers import reverse
import django.test
from django_downloadview import temporary_media_root, assert_download_response
from demoproject.object.models import Document
# Fixtures.
slug = 'hello-world'
basename = 'hello-world.txt'
file_name = 'file.txt'
another_name = 'another_file.txt'
file_content = 'Hello world!\n'
another_content = 'Goodbye world!\n'
def setup_document():
document = Document(slug=slug, basename=basename)
document.file.save(file_name,
ContentFile(file_content),
save=False)
document.another_file.save(another_name,
ContentFile(another_content),
save=False)
document.save()
return document
class DefaultFileTestCase(django.test.TestCase):
@temporary_media_root()
def test_download_response(self):
"""'default_file' streams Document.file."""
setup_document()
url = reverse('object:default_file', kwargs={'slug': slug})
response = self.client.get(url)
assert_download_response(self,
response,
content=file_content,
basename=file_name,
mime_type='text/plain')
class AnotherFileTestCase(django.test.TestCase):
@temporary_media_root()
def test_download_response(self):
"""'another_file' streams Document.another_file."""
setup_document()
url = reverse('object:another_file', kwargs={'slug': slug})
response = self.client.get(url)
assert_download_response(self,
response,
content=another_content,
basename=another_name,
mime_type='text/plain')
class DeserializedBasenameTestCase(django.test.TestCase):
@temporary_media_root()
def test_download_response(self):
"'deserialized_basename' streams Document.file with custom basename."
setup_document()
url = reverse('object:deserialized_basename', kwargs={'slug': slug})
response = self.client.get(url)
assert_download_response(self,
response,
content=file_content,
basename=basename,
mime_type='text/plain')

View file

@ -0,0 +1,17 @@
from django.conf.urls import patterns, url
from demoproject.object import views
urlpatterns = patterns(
'',
url(r'^default-file/(?P<slug>[a-zA-Z0-9_-]+)/$',
views.default_file_view,
name='default_file'),
url(r'^another-file/(?P<slug>[a-zA-Z0-9_-]+)/$',
views.another_file_view,
name='another_file'),
url(r'^deserialized_basename/(?P<slug>[a-zA-Z0-9_-]+)/$',
views.deserialized_basename_view,
name='deserialized_basename'),
)

View file

@ -0,0 +1,18 @@
from django_downloadview import ObjectDownloadView
from demoproject.object.models import Document
#: Serve ``file`` attribute of ``Document`` model.
default_file_view = ObjectDownloadView.as_view(model=Document)
#: Serve ``another_file`` attribute of ``Document`` model.
another_file_view = ObjectDownloadView.as_view(
model=Document,
file_field='another_file')
#: Serve ``file`` attribute of ``Document`` model, using client-side filename
#: from model.
deserialized_basename_view = ObjectDownloadView.as_view(
model=Document,
basename_field='basename')

View file

@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
"""Demo for :class:`django_downloadview.PathDownloadView`.
Code in this package is included in documentation's :doc:`/views/path`.
Make sure to maintain both together.
"""

View file

@ -0,0 +1 @@
"""Required to make a Django application."""

View file

@ -0,0 +1,28 @@
from django.core.urlresolvers import reverse
import django.test
from django_downloadview import assert_download_response
class StaticPathTestCase(django.test.TestCase):
def test_download_response(self):
"""'static_path' serves 'fixtures/hello-world.txt'."""
url = reverse('path:static_path')
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')
class DynamicPathTestCase(django.test.TestCase):
def test_download_response(self):
"""'dynamic_path' serves 'fixtures/{path}'."""
url = reverse('path:dynamic_path', kwargs={'path': 'hello-world.txt'})
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')

View file

@ -0,0 +1,14 @@
from django.conf.urls import patterns, url
from demoproject.path import views
urlpatterns = patterns(
'',
url(r'^static-path/$',
views.static_path,
name='static_path'),
url(r'^dynamic-path/(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$',
views.dynamic_path,
name='dynamic_path'),
)

View file

@ -0,0 +1,39 @@
import os
from django_downloadview import PathDownloadView
# Let's initialize some fixtures.
app_dir = os.path.dirname(os.path.abspath(__file__))
project_dir = os.path.dirname(app_dir)
fixtures_dir = os.path.join(project_dir, 'fixtures')
#: Path to a text file that says 'Hello world!'.
hello_world_path = os.path.join(fixtures_dir, 'hello-world.txt')
#: Serve ``fixtures/hello-world.txt`` file.
static_path = PathDownloadView.as_view(path=hello_world_path)
class DynamicPathDownloadView(PathDownloadView):
"""Serve file in ``settings.MEDIA_ROOT``.
.. warning::
Make sure to prevent "../" in path via URL patterns.
.. note::
This particular setup would be easier to perform with
:class:`StorageDownloadView`
"""
def get_path(self):
"""Return path inside fixtures directory."""
# Get path from URL resolvers or as_view kwarg.
relative_path = super(DynamicPathDownloadView, self).get_path()
# Make it absolute.
absolute_path = os.path.join(fixtures_dir, relative_path)
return absolute_path
dynamic_path = DynamicPathDownloadView.as_view()

View file

@ -45,30 +45,34 @@ INSTALLED_APPS = (
'django.contrib.staticfiles',
# The actual django-downloadview demo.
'demoproject',
'demoproject.download', # Sample standard download views.
'demoproject.nginx', # Sample optimizations for Nginx.
'demoproject.object', # Demo around ObjectDownloadView
'demoproject.storage', # Demo around StorageDownloadView
'demoproject.path', # Demo around PathDownloadView
'demoproject.http', # Demo around HTTPDownloadView
'demoproject.virtual', # Demo around VirtualDownloadView
'demoproject.nginx', # Sample optimizations for Nginx X-Accel.
# For test purposes. The demo project is part of django-downloadview
# test suite.
'django_nose',
)
# Default middlewares. You may alter the list later.
# Middlewares.
MIDDLEWARE_CLASSES = [
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django_downloadview.DownloadDispatcherMiddleware'
]
# Uncomment the following lines to enable global Nginx optimizations.
#MIDDLEWARE_CLASSES.append('django_downloadview.DownloadDispatcherMiddleware')
DOWNLOADVIEW_MIDDLEWARES = (
('default', 'django_downloadview.nginx.XAccelRedirectMiddleware',
{'source_dir': MEDIA_ROOT,
'destination_url': '/proxied-download'}),
{'source_url': '/media/nginx/',
'destination_url': '/nginx-optimized-by-middleware/'}),
)

View file

@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
"""Demo for :class:`django_downloadview.StorageDownloadView`.
Code in this package is included in documentation's :doc:`/views/storage`.
Make sure to maintain both together.
"""

View file

@ -0,0 +1 @@
"""Required to make a Django application."""

View file

@ -0,0 +1,4 @@
from django.core.files.storage import FileSystemStorage
storage = FileSystemStorage()

View file

@ -0,0 +1,48 @@
try:
from unittest import mock
except ImportError: # Python 2.x fallback.
import mock
from django.core.files.base import ContentFile
from django.core.urlresolvers import reverse
import django.test
from django_downloadview import temporary_media_root, assert_download_response
from demoproject.storage.views import storage
# Fixtures.
file_content = 'Hello world!\n'
def setup_file(path):
storage.save(path, ContentFile(file_content))
class StaticPathTestCase(django.test.TestCase):
@temporary_media_root()
def test_download_response(self):
"""'static_path' streams file by path."""
setup_file('1.txt')
url = reverse('storage:static_path', kwargs={'path': '1.txt'})
response = self.client.get(url)
assert_download_response(self,
response,
content=file_content,
basename='1.txt',
mime_type='text/plain')
class DynamicPathTestCase(django.test.TestCase):
@temporary_media_root()
def test_download_response(self):
"""'dynamic_path' streams file by generated path."""
setup_file('1.TXT')
url = reverse('storage:dynamic_path', kwargs={'path': '1.txt'})
response = self.client.get(url)
assert_download_response(self,
response,
content=file_content,
basename='1.TXT',
mime_type='text/plain')

View file

@ -0,0 +1,14 @@
from django.conf.urls import patterns, url
from demoproject.storage import views
urlpatterns = patterns(
'',
url(r'^static-path/(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$',
views.static_path,
name='static_path'),
url(r'^dynamic-path/(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$',
views.dynamic_path,
name='dynamic_path'),
)

View file

@ -0,0 +1,20 @@
from django.core.files.storage import FileSystemStorage
from django_downloadview import StorageDownloadView
storage = FileSystemStorage()
#: Serve file using ``path`` argument.
static_path = StorageDownloadView.as_view(storage=storage)
class DynamicStorageDownloadView(StorageDownloadView):
"""Serve file of storage by path.upper()."""
def get_path(self):
"""Return uppercase path."""
return super(DynamicStorageDownloadView, self).get_path().upper()
dynamic_path = DynamicStorageDownloadView.as_view(storage=storage)

View file

@ -10,19 +10,7 @@
<h2>Serving files with Django</h2>
<p>In the following views, Django streams the files, no optimization
has been setup.</p>
<ul>
<li><a href="{% url 'hello_world' %}">PathDownloadView</a></li>
<li><a href="{% url 'fixture_from_path' 'hello-world.txt' %}">
PathDownloadView + argument in URL
</a></li>
<li><a href="{% url 'fixture_from_storage' 'hello-world.txt' %}">
StorageDownloadView + path in URL
</a></li>
<li><a href="{% url 'document' 'hello-world' %}">
ObjectDownloadView
</a></li>
<li><a href="{% url 'http_hello_world' %}">
HTTPDownloadView</a>, a simple HTTP proxy</li>
<ul>
</ul>
<h2>Optimized downloads</h2>
@ -31,9 +19,6 @@
<p>Since nginx and other servers aren't installed on the demo, you
will get raw "X-Sendfile" responses. Look at the headers!</p>
<ul>
<li><a href="{% url 'download_document_nginx' 'hello-world' %}">
ObjectDownloadView (nginx)
</a></li>
</ul>
</body>
</html>

View file

@ -7,10 +7,30 @@ home = TemplateView.as_view(template_name='home.html')
urlpatterns = patterns(
'',
# Standard download views.
url(r'^download/', include('demoproject.download.urls')),
# ObjectDownloadView.
url(r'^object/', include('demoproject.object.urls',
app_name='object',
namespace='object')),
# StorageDownloadView.
url(r'^storage/', include('demoproject.storage.urls',
app_name='storage',
namespace='storage')),
# PathDownloadView.
url(r'^path/', include('demoproject.path.urls',
app_name='path',
namespace='path')),
# HTTPDownloadView.
url(r'^http/', include('demoproject.http.urls',
app_name='http',
namespace='http')),
# VirtualDownloadView.
url(r'^virtual/', include('demoproject.virtual.urls',
app_name='virtual',
namespace='virtual')),
# Nginx optimizations.
url(r'^nginx/', include('demoproject.nginx.urls')),
url(r'^nginx/', include('demoproject.nginx.urls',
app_name='nginx',
namespace='nginx')),
# An informative homepage.
url(r'', home, name='home')
)

View file

@ -0,0 +1,7 @@
# -*- coding: utf-8 -*-
"""Demo for :class:`django_downloadview.VirtualDownloadView`.
Code in this package is included in documentation's :doc:`/views/virtual`.
Make sure to maintain both together.
"""

View file

@ -0,0 +1 @@
"""Required to make a Django application."""

View file

@ -0,0 +1,40 @@
from django.core.urlresolvers import reverse
import django.test
from django_downloadview import assert_download_response
class TextTestCase(django.test.TestCase):
def test_download_response(self):
"""'virtual:text' serves 'hello-world.txt' from unicode."""
url = reverse('virtual:text')
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')
class StringIOTestCase(django.test.TestCase):
def test_download_response(self):
"""'virtual:stringio' serves 'hello-world.txt' from stringio."""
url = reverse('virtual:stringio')
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')
class GeneratedTestCase(django.test.TestCase):
def test_download_response(self):
"""'virtual:generated' serves 'hello-world.txt' from generator."""
url = reverse('virtual:generated')
response = self.client.get(url)
assert_download_response(self,
response,
content='Hello world!\n',
basename='hello-world.txt',
mime_type='text/plain')

View file

@ -0,0 +1,17 @@
from django.conf.urls import patterns, url
from demoproject.virtual import views
urlpatterns = patterns(
'',
url(r'^text/$',
views.TextDownloadView.as_view(),
name='text'),
url(r'^stringio/$',
views.StringIODownloadView.as_view(),
name='stringio'),
url(r'^gerenated/$',
views.GeneratedDownloadView.as_view(),
name='generated'),
)

View file

@ -0,0 +1,33 @@
from StringIO import StringIO
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
from django_downloadview import VirtualFile
from django_downloadview import StringIteratorIO
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!\n", name='hello-world.txt')
class StringIODownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIO`` object."""
file_obj = StringIO(u"Hello world!\n".encode('utf-8'))
return VirtualFile(file_obj, name='hello-world.txt')
def generate_hello():
yield u'Hello '
yield u'world!'
yield u'\n'
class GeneratedDownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIteratorIO`` object."""
file_obj = StringIteratorIO(generate_hello())
return VirtualFile(file_obj, name='hello-world.txt')

View file

@ -3,7 +3,8 @@
from django_downloadview.io import StringIteratorIO # NoQA
from django_downloadview.files import (StorageFile, # NoQA
VirtualFile,
HTTPFile)
HTTPFile,
File)
from django_downloadview.response import (DownloadResponse, # NoQA
ProxiedDownloadResponse)
from django_downloadview.middlewares import (BaseDownloadMiddleware, # NoQA
@ -12,5 +13,8 @@ from django_downloadview.nginx import XAccelRedirectMiddleware # NoQA
from django_downloadview.views import (PathDownloadView, # NoQA
ObjectDownloadView,
StorageDownloadView,
HTTPDownloadView,
VirtualDownloadView)
from django_downloadview.sendfile import sendfile # NoQA
from django_downloadview.test import (assert_download_response, # NoQA
temporary_media_root)

View file

@ -1,13 +1,13 @@
"""View decorators.
See also decorators provided by server-specific modules, such as
:py:func:`django_downloadview.nginx.x_accel_redirect`.
:func:`django_downloadview.nginx.x_accel_redirect`.
"""
class DownloadDecorator(object):
"""View decorator factory to apply middleware to ``view_func`` response.
"""View decorator factory to apply middleware to ``view_func``'s response.
Middleware instance is built from ``middleware_factory`` with ``*args`` and
``**kwargs``. Middleware factory is typically a class, such as some

View file

@ -1,10 +1,43 @@
# -*- coding: utf-8 -*-
"""File wrappers for use as exchange data between views and responses."""
from django.core.files import File
from __future__ import absolute_import
from io import BytesIO
from urlparse import urlparse
import django.core.files
from django.utils.encoding import force_bytes
import requests
class File(django.core.files.File):
"""Patch Django's :meth:`__iter__` implementation.
See https://code.djangoproject.com/ticket/21321
"""
def __iter__(self):
# Iterate over this file-like object by newlines
buffer_ = None
for chunk in self.chunks():
chunk_buffer = BytesIO(force_bytes(chunk))
for line in chunk_buffer:
if buffer_:
line = buffer_ + line
buffer_ = None
# If this is the end of a line, yield
# otherwise, wait for the next round
if line[-1] in ('\n', '\r'):
yield line
else:
buffer_ = line
if buffer_ is not None:
yield buffer_
class StorageFile(File):
"""A file in a Django storage.
@ -186,7 +219,14 @@ class HTTPFile(File):
**kwargs):
self.request_factory = request_factory
self.url = url
self.name = name
if name is None:
parts = urlparse(url)
if parts.path: # Name from path.
self.name = parts.path.strip('/').rsplit('/', 1)[-1]
else: # Name from domain.
self.name = parts.netloc
else:
self.name = name
kwargs['stream'] = True
self.request_kwargs = kwargs

View file

@ -102,6 +102,10 @@ class DownloadDispatcherMiddleware(BaseDownloadMiddleware):
return response
class NoRedirectionMatch(Exception):
"""Response object does not match redirection rules."""
class ProxiedDownloadMiddleware(RealDownloadMiddleware):
"""Base class for middlewares that use optimizations of reverse proxies."""
def __init__(self, source_dir=None, source_url=None, destination_url=None):
@ -114,7 +118,7 @@ class ProxiedDownloadMiddleware(RealDownloadMiddleware):
"""Return redirect URL for file wrapped into response."""
url = None
file_url = ''
if self.source_url is not None:
if self.source_url:
try:
file_url = response.file.url
except AttributeError:
@ -122,9 +126,9 @@ class ProxiedDownloadMiddleware(RealDownloadMiddleware):
else:
if file_url.startswith(self.source_url):
file_url = file_url[len(self.source_url):]
url = file_url
url = file_url
file_name = ''
if url is None and self.source_dir is not None:
if url is None and self.source_dir:
try:
file_name = response.file.name
except AttributeError:
@ -132,7 +136,7 @@ class ProxiedDownloadMiddleware(RealDownloadMiddleware):
else:
if file_name.startswith(self.source_dir):
file_name = os.path.relpath(file_name, self.source_dir)
url = file_name.replace(os.path.sep, '/')
url = file_name.replace(os.path.sep, '/')
if url is None:
message = ("""Couldn't capture/convert file attributes into a """
"""redirection. """
@ -144,5 +148,5 @@ class ProxiedDownloadMiddleware(RealDownloadMiddleware):
'file_url': file_url,
'source_dir': self.source_dir,
'file_name': file_name})
raise Exception(message)
raise NoRedirectionMatch(message)
return '/'.join((self.destination_url.rstrip('/'), url.lstrip('/')))

View file

@ -4,9 +4,13 @@ from django_downloadview.decorators import DownloadDecorator
from django_downloadview.nginx.middlewares import XAccelRedirectMiddleware
#: Apply BaseXAccelRedirectMiddleware to ``view_func`` response.
#:
#: Proxies additional arguments (``*args``, ``**kwargs``) to
#: :py:class:`BaseXAccelRedirectMiddleware` constructor (``expires``,
#: ``with_buffering``, and ``limit_rate``).
x_accel_redirect = DownloadDecorator(XAccelRedirectMiddleware)
def x_accel_redirect(view_func, *args, **kwargs):
"""Apply
:class:`~django_downloadview.nginx.middlewares.XAccelRedirectMiddleware` to
``view_func``.
Proxies (``*args``, ``**kwargs``) to middleware constructor.
"""
decorator = DownloadDecorator(XAccelRedirectMiddleware)
return decorator(view_func, *args, **kwargs)

View file

@ -3,7 +3,8 @@ import warnings
from django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from django_downloadview.middlewares import ProxiedDownloadMiddleware
from django_downloadview.middlewares import (ProxiedDownloadMiddleware,
NoRedirectionMatch)
from django_downloadview.nginx.response import XAccelRedirectResponse
@ -50,7 +51,10 @@ class XAccelRedirectMiddleware(ProxiedDownloadMiddleware):
def process_download_response(self, request, response):
"""Replace DownloadResponse instances by NginxDownloadResponse ones."""
redirect_url = self.get_redirect_url(response)
try:
redirect_url = self.get_redirect_url(response)
except NoRedirectionMatch:
return response
if self.expires:
expires = self.expires
else:

View file

@ -8,44 +8,64 @@ from django.http import HttpResponse, StreamingHttpResponse
class DownloadResponse(StreamingHttpResponse):
"""File download response.
"""File download response (Django serves file, client downloads it).
:py:attr:`content` attribute is supposed to be a file object wrapper, which
makes this response lazy.
This is a specialization of :class:`django.http.StreamingHttpResponse`
where :attr:`~django.http.StreamingHttpResponse.streaming_content` is a
file wrapper.
This is a specialization of :py:class:`django.http.StreamingHttpResponse`.
Constructor differs a bit from :class:`~django.http.response.HttpResponse`:
``file_instance``
A :doc:`file wrapper instance </files>`, such as
:class:`~django.core.files.base.File`.
``attachement``
Boolean. Whether to return the file as attachment or not.
Affects ``Content-Disposition`` header.
``basename``
Unicode. Client-side name of the file to stream.
Only used if ``attachment`` is ``True``.
Affects ``Content-Disposition`` header.
``status``
HTTP status code.
``content_type``
Value for ``Content-Type`` header.
If ``None``, then mime-type and encoding will be populated by the
response (default implementation uses mimetypes, based on file
name).
Here are some highlights to understand internal mechanisms and motivations:
* Let's start by quoting :pep:`3333` (WSGI specification):
For large files, or for specialized uses of HTTP streaming,
applications will usually return an iterator (often a
generator-iterator) that produces the output in a block-by-block
fashion.
* `Django WSGI handler (application implementation) return response object
<https://github.com/django/django/blob/fd1279a44df3b9a837453cd79fd0fbcf81bae39d/django/core/handlers/wsgi.py#L268>`_.
* :class:`django.http.HttpResponse` and subclasses are iterators.
* In :class:`~django.http.StreamingHttpResponse`, the
:meth:`~container.__iter__` implementation proxies to
:attr:`~django.http.StreamingHttpResponse.streaming_content`.
* In :class:`DownloadResponse` and subclasses, :attr:`streaming_content`
is a :doc:`file wrapper </files>`. File wrapper is itself an iterator
over actual file content, and it also encapsulates access to file
attributes (size, name, ...).
"""
def __init__(self, file_instance, attachment=True, basename=None,
status=200, content_type=None):
"""Constructor.
It differs a bit from HttpResponse constructor.
file_instance:
A file wrapper object. Could be a FieldFile.
attachement:
Boolean, whether to return the file as attachment or not. Affects
"Content-Disposition" header.
Defaults to ``True``.
basename:
Unicode. Only used if ``attachment`` is ``True``. Client-side name
of the file to stream. Affects "Content-Disposition" header.
Defaults to basename(``file_instance.name``).
status:
HTTP status code.
Defaults to 200.
content_type:
Value for "Content-Type" header.
If ``None``, then mime-type and encoding will be populated by the
response (default implementation uses mimetypes, based on file name).
Defaults is ``None``.
"""
"""Constructor."""
self.file = file_instance
super(DownloadResponse, self).__init__(streaming_content=self.file,
status=status,

View file

@ -9,7 +9,11 @@ from django_downloadview.middlewares import is_download_response
class temporary_media_root(override_settings):
"""Context manager or decorator to override settings.MEDIA_ROOT.
"""Temporarily override settings.MEDIA_ROOT with a temporary directory.
The temporary directory is automatically created and destroyed.
Use this function as a context manager:
>>> from django_downloadview.test import temporary_media_root
>>> from django.conf import settings
@ -20,6 +24,8 @@ class temporary_media_root(override_settings):
>>> global_media_root == settings.MEDIA_ROOT
True
Or as a decorator:
>>> @temporary_media_root()
... def use_temporary_media_root():
... return settings.MEDIA_ROOT
@ -73,9 +79,10 @@ class DownloadResponseValidator(object):
test_case.assertTrue(is_download_response(response))
def assert_basename(self, test_case, response, value):
test_case.assertEqual(response.basename, value)
test_case.assertTrue('filename={name}'.format(name=response.basename),
value)
"""Implies ``attachement is True``."""
test_case.assertTrue(
response['Content-Disposition'].endswith(
'filename={name}'.format(name=value)))
def assert_content_type(self, test_case, response, value):
test_case.assertEqual(response['Content-Type'], value)
@ -84,7 +91,6 @@ class DownloadResponseValidator(object):
test_case.assertTrue(response['Content-Type'].startswith(value))
def assert_content(self, test_case, response, value):
test_case.assertEqual(response.file.read(), value)
test_case.assertEqual(''.join(response.streaming_content), value)
def assert_attachment(self, test_case, response, value):
@ -93,7 +99,7 @@ class DownloadResponseValidator(object):
def assert_download_response(test_case, response, **assertions):
"""Make ``test_case`` assert that ``response`` is a DownloadResponse.
"""Make ``test_case`` assert that ``response`` meets ``assertions``.
Optional ``assertions`` dictionary can be used to check additional items:
@ -101,9 +107,12 @@ def assert_download_response(test_case, response, **assertions):
* ``content_type``: the value of "Content-Type" header.
* ``charset``: the value of ``X-Accel-Charset`` header.
* ``mime_type``: the MIME type part of "Content-Type" header (without
charset).
* ``content``: the content of the file to be downloaded.
* ``content``: the contents of the file.
* ``attachment``: whether the file is returned as attachment or not.
"""
validator = DownloadResponseValidator()

View file

@ -15,15 +15,27 @@ class HTTPDownloadView(BaseDownloadView):
request_kwargs = {}
def get_request_factory(self):
"""Return request factory to perform actual HTTP request."""
"""Return request factory to perform actual HTTP request.
Default implementation returns :func:`requests.get` callable.
"""
return requests.get
def get_request_kwargs(self):
"""Return keyword arguments for use with request factory."""
"""Return keyword arguments for use with :meth:`get_request_factory`.
Default implementation returns :attr:`request_kwargs`.
"""
return self.request_kwargs
def get_url(self):
"""Return remote file URL (the one we are proxying)."""
"""Return remote file URL (the one we are proxying).
Default implementation returns :attr:`url`.
"""
return self.url
def get_file(self):

View file

@ -6,8 +6,6 @@ from django_downloadview.views.base import BaseDownloadView
class VirtualDownloadView(BaseDownloadView):
"""Serve not-on-disk or generated-on-the-fly file.
Use this class to serve :py:class:`StringIO` files.
Override the :py:meth:`get_file` method to customize file wrapper.
"""

View file

@ -197,6 +197,7 @@ intersphinx_mapping = {
'python': ('http://docs.python.org/2.7', None),
'django': ('http://docs.djangoproject.com/en/1.5/',
'http://docs.djangoproject.com/en/1.5/_objects/'),
'requests': ('http://docs.python-requests.org/en/latest/', None),
}
# -- Options for LaTeX output --------------------------------------------------

View file

@ -44,7 +44,7 @@ Setup a development environment
System requirements:
* `Python`_ version 2.6 or 2.7, available as ``python`` command.
* `Python`_ version 2.7, available as ``python`` command.
.. note::
@ -80,21 +80,6 @@ The :file:`Makefile` is intended to be a live reference for the development
environment.
*************
Documentation
*************
Follow `style guide for Sphinx-based documentations`_ when editing the
documentation.
**************
Test and build
**************
Use `the Makefile`_.
*********************
Demo project included
*********************

View file

@ -11,67 +11,41 @@ proxy:
As a result, you get increased performance: reverse proxies are more efficient
than Django at serving static files.
The setup depends on the reverse proxy:
.. toctree::
:maxdepth: 2
:titlesonly:
nginx
Currently, only `nginx's X-Accel`_ is supported, but `contributions are
welcome`_!
.. note::
Currently, only `nginx's X-Accel`_ is supported, but `contributions are
welcome`_!
*****************
How does it work?
*****************
The feature is inspired by `Django's TemplateResponse`_: the download views
return some :py:class:`django_downloadview.response.DownloadResponse` instance.
Such a response does not contain file data.
View return some :class:`~django_downloadview.response.DownloadResponse`
instance, which itself carries a :doc:`file wrapper </files>`.
By default, at the end of Django's request/response handling, Django iterates
over the ``content`` attribute of the response. In a `DownloadResponse``,
this ``content`` attribute is a file wrapper.
`django-downloadview` provides response middlewares and decorators that are
able to capture :class:`~django_downloadview.response.DownloadResponse`
instances and convert them to
:class:`~django_downloadview.response.ProxiedDownloadResponse`.
It means that decorators and middlewares are given an opportunity to capture
the ``DownloadResponse`` before the content of the file is loaded into memory
As an example, :py:class:`django_downloadview.nginx.XAccelRedirectMiddleware`
replaces ``DownloadResponse`` intance by some
:py:class:`django_downloadview.nginx.XAccelRedirectResponse`.
.. note::
The feature is inspired by :mod:`Django's TemplateResponse
<django.template.response>`
*********
Configure
*********
Add `django_downloadview.DownloadDispatcherMiddleware` to `MIDDLEWARE_CLASSES`
in your Django settings.
Then register as many download middlewares as you wish in
`DOWNLOADVIEW_MIDDLEWARES`.
.. code:: python
DOWNLOADVIEW_MIDDLEWARES = (
('default',
'django_downloadview.nginx.XAccelRedirectMiddleware',
{'source_dir': MEDIA_ROOT, 'destination_url': '/proxied-download'}),
)
The first item is an identifier.
The second item is the import path of some download middleware factory
(typically a class).
The third item is a dictionary of keyword arguments passed to the middleware
factory.
.. rubric:: References
.. rubric:: Notes & references
.. target-notes::
.. _`nginx's X-Accel`: http://wiki.nginx.org/X-accel
.. _`contributions are welcome`:
https://github.com/benoitbryon/django-downloadview/issues?labels=optimizations
.. _`Django's TemplateResponse`:
https://docs.djangoproject.com/en/1.5/ref/template-response/

View file

@ -2,8 +2,8 @@
Nginx
#####
If you serve Django behind Nginx, then you can delegate the file download
service to Nginx and get increased performance:
If you serve Django behind Nginx, then you can delegate the file streaming
to Nginx and get increased performance:
* lower resources used by Python/Django workers ;
* faster download.
@ -11,103 +11,94 @@ service to Nginx and get increased performance:
See `Nginx X-accel documentation`_ for details.
****************************
Configure some download view
****************************
************
Given a view
************
Let's start in the situation described in the :doc:`demo application </demo>`:
Let's consider the following view:
* a project "demoproject"
* an application "demoproject.download"
* a :py:class:`django_downloadview.views.ObjectDownloadView` view serves files
of a "Document" model.
.. literalinclude:: /../demo/demoproject/nginx/views.py
:language: python
:lines: 1-6, 8-16
We are to make it more efficient with Nginx.
What is important here is that the files will have an ``url`` property
implemented by storage. Let's setup an optimization rule based on that URL.
.. note::
Examples below are taken from the :doc:`demo project </demo>`.
It is generally easier to setup rules based on URL rather than based on
name in filesystem. This is because path is generally relative to storage,
whereas URL usually contains some storage identifier, i.e. it is easier to
target a specific location by URL rather than by filesystem name.
***********
Write tests
***********
********************************
Setup XAccelRedirect middlewares
********************************
Use :py:func:`django_downloadview.nginx.assert_x_accel_redirect` function as
a shortcut in your tests.
Make sure ``django_downloadview.DownloadDispatcherMiddleware`` is in
``MIDDLEWARE_CLASSES`` of your `Django` settings.
:file:`demo/demoproject/nginx/tests.py`:
Example:
.. literalinclude:: ../../demo/demoproject/nginx/tests.py
.. literalinclude:: /../demo/demoproject/settings.py
:language: python
:emphasize-lines: 5, 25-34
:lines: 61-68
Right now, this test should fail, since you haven't implemented the view yet.
Then register as many
:class:`~django_downloadview.nginx.middlewares.XAccelRedirectMiddleware`
instances as you wish in ``DOWNLOADVIEW_MIDDLEWARES``.
.. literalinclude:: /../demo/demoproject/settings.py
:language: python
:lines: 72-76
The first item is an identifier.
The second item is the import path of
:class:`~django_downloadview.nginx.middlewares.XAccelRedirectMiddleware` class.
The third item is a dictionary of keyword arguments passed to the middleware
factory. In the example above, we capture responses by ``source_url`` and
convert them to internal redirects to ``destination_url``.
.. autoclass:: django_downloadview.nginx.middlewares.XAccelRedirectMiddleware
:members:
:inherited-members:
:undoc-members:
:show-inheritance:
:member-order: bysource
************
Setup Django
************
**********************************************
Per-view setup with x_accel_redirect decorator
**********************************************
At the end of this setup, the test should pass, but you still have to `setup
Nginx`_!
Middlewares should be enough for most use cases, but you may want per-view
configuration. For `nginx`, there is ``x_accel_redirect``:
You have two options: global setup with a middleware, or per-view setup with
decorators.
.. autofunction:: django_downloadview.nginx.decorators.x_accel_redirect
Global delegation, with XAccelRedirectMiddleware
================================================
As an example:
If you want to delegate all file downloads to Nginx, then use
:py:class:`django_downloadview.nginx.XAccelRedirectMiddleware`.
.. literalinclude:: /../demo/demoproject/nginx/views.py
:language: python
:lines: 1-7, 17-
Register it in your settings:
.. code-block:: python
*******************************************
Test responses with assert_x_accel_redirect
*******************************************
MIDDLEWARE_CLASSES = (
# ...
'django_downloadview.nginx.XAccelRedirectMiddleware',
# ...
)
Use :func:`~django_downloadview.nginx.decorators.assert_x_accel_redirect`
function as a shortcut in your tests.
Setup the middleware:
.. code-block:: python
NGINX_DOWNLOAD_MIDDLEWARE_MEDIA_ROOT = MEDIA_ROOT # Could be elsewhere.
NGINX_DOWNLOAD_MIDDLEWARE_MEDIA_URL = '/proxied-download'
Optionally fine-tune the middleware. Default values are ``None``, which means
"use Nginx's defaults".
.. code-block:: python
NGINX_DOWNLOAD_MIDDLEWARE_EXPIRES = False # Force no expiration.
NGINX_DOWNLOAD_MIDDLEWARE_WITH_BUFFERING = False # Force buffering off.
NGINX_DOWNLOAD_MIDDLEWARE_LIMIT_RATE = False # Force limit rate off.
Local delegation, with x_accel_redirect decorator
=================================================
If you want to delegate file downloads to Nginx on a per-view basis, then use
:py:func:`django_downloadview.nginx.x_accel_redirect` decorator.
:file:`demo/demoproject/nginx/views.py`:
.. literalinclude:: ../../demo/demoproject/nginx/views.py
.. literalinclude:: /../demo/demoproject/nginx/tests.py
:language: python
And use it in som URL conf, as an example in
:file:`demo/demoproject/nginx/urls.py`:
.. autofunction:: django_downloadview.nginx.tests.assert_x_accel_redirect
.. literalinclude:: ../../demo/demoproject/nginx/urls.py
:language: python
.. note::
In real life, you'd certainly want to replace the "download_document" view
instead of registering a new view.
The tests above assert the `Django` part is OK. Now let's configure `nginx`.
***********
@ -136,11 +127,11 @@ Here is what you could have in :file:`/etc/nginx/sites-available/default`:
# like /optimized-download/myfile.tar.gz
#
# See http://wiki.nginx.org/X-accel
# and https://github.com/benoitbryon/django-downloadview
# and https://django-downloadview.readthedocs.org
#
location /proxied-download {
internal;
# Location to files on disk.
# See Django's settings.NGINX_DOWNLOAD_MIDDLEWARE_MEDIA_ROOT
alias /var/www/files/;
}
@ -158,12 +149,17 @@ section.
.. note::
``/proxied-download`` is not available for the client, i.e. users
won't be able to download files via ``/optimized-download/<filename>``.
``/proxied-download`` has the ``internal`` flag, so this location is not
available for the client, i.e. users are not able to download files via
``/optimized-download/<filename>``.
.. warning::
Make sure Nginx can read the files to download! Check permissions.
*********************************************
Assert everything goes fine with healthchecks
*********************************************
:doc:`Healthchecks </healthchecks>` are the best way to check the complete
setup.
*************

View file

@ -18,11 +18,14 @@ Here is an overview of ``django-downloadview``'s answer...
Generic views cover commons patterns
************************************
* :doc:`/views/object` when you have a model with a file field;
* :doc:`/views/storage` when you manage files in a storage;
* :doc:`/views/path` when you have an absolute filename on local filesystem;
* :doc:`/views/http` when you have an URL (the resource is proxied);
* :doc:`/views/virtual` when you generate a file dynamically.
Choose the generic view depending on the file you want to serve:
* :doc:`/views/object`: file field in a model;
* :doc:`/views/storage`: file in a storage;
* :doc:`/views/path`: absolute filename on local filesystem;
* :doc:`/views/http`: URL (the resource is proxied);
* :doc:`/views/virtual`: bytes, text, :class:`~StringIO.StringIO`, generated
file...
*************************************************
@ -39,13 +42,7 @@ Views return DownloadResponse
Views return :py:class:`~django_downloadview.response.DownloadResponse`. It is
a special :py:class:`django.http.StreamingHttpResponse` where content is
encapsulated in a file wrapper. If the response is sent to the client, the file
content content is loaded.
.. note::
Middlewares and decorators are given the opportunity to optimize the
streaming before file content loading.
encapsulated in a file wrapper.
Learn more in :doc:`responses`.
@ -54,11 +51,14 @@ Learn more in :doc:`responses`.
DownloadResponse carry file wrapper
***********************************
A download view instanciates a :doc:`file wrapper </files>` and use it to
initialize :py:class:`~django_downloadview.response.DownloadResponse`.
Views instanciate a :doc:`file wrapper </files>` and use it to initialize
responses.
File wrappers describe files. They carry files properties, but not file
content. They implement loading and iterating over file content.
**File wrappers describe files**: they carry files properties such as name,
size, encoding...
**File wrappers implement loading and iterating over file content**. Whenever
possible, file wrappers do not embed file data, in order to save memory.
Learn more about available file wrappers in :doc:`files`.
@ -67,11 +67,18 @@ Learn more about available file wrappers in :doc:`files`.
Middlewares convert DownloadResponse into ProxiedDownloadResponse
*****************************************************************
Decorators and middlewares may capture
:py:class:`~django_downloadview.response.DownloadResponse` instances in order
to optimize the streaming. A good optimization is to delegate streaming to
reverse proxies such as Nginx via X-Accel redirections or Apache via
X-Sendfile.
Before WSGI application use file wrapper to load file contents, middlewares
(or decorators) are given the opportunity to capture
:class:`~django_downloadview.response.DownloadResponse` instances.
Let's take this opportunity to optimize file loading and streaming!
A good optimization it to delegate streaming to a reverse proxy, such as
`nginx`_ via `X-Accel`_ internal redirects.
`django_downloadview` provides middlewares that convert
:class:`~django_downloadview.response.DownloadResponse` into
:class:`~django_downloadview.response.ProxiedDownloadResponse`.
Learn more in :doc:`optimizations/index`.
@ -80,12 +87,23 @@ Learn more in :doc:`optimizations/index`.
Testing matters
***************
``django-downloadview`` also helps you :doc:`test the views you customized
`django-downloadview` also helps you :doc:`test the views you customized
<testing>`.
You may also :doc:`write healthchecks </healthchecks>` to make sure everything
goes fine in live environments.
************
What's next?
************
Convinced? Let's :doc:`install django-downloadview <install>`.
Let's :doc:`install django-downloadview <install>`.
.. rubric:: Notes & references
.. target-notes::
.. _`nginx`: http://nginx.org
.. _`X-Accel`: http://wiki.nginx.org/X-accel

View file

@ -2,15 +2,31 @@
Responses
#########
.. currentmodule:: django_downloadview.response
********************
``DownloadResponse``
********************
Views return :class:`DownloadResponse`.
Middlewares (and decorators) are given the opportunity to capture responses and
convert them to :class:`ProxiedDownloadResponse`.
****************
DownloadResponse
****************
***************************
``ProxiedDownloadResponse``
***************************
.. autoclass:: DownloadResponse
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
***********************
ProxiedDownloadResponse
***********************
.. autoclass:: ProxiedDownloadResponse
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource

View file

@ -24,16 +24,9 @@ response content such as gzip middleware.
Example:
.. code:: python
MIDDLEWARE_CLASSES = [
'django.middleware.common.CommonMiddleware',
'django.contrib.sessions.middleware.SessionMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
'django.contrib.auth.middleware.AuthenticationMiddleware',
'django.contrib.messages.middleware.MessageMiddleware',
'django_downloadview.DownloadDispatcherMiddleware',
]
.. literalinclude:: /../demo/demoproject/settings.py
:language: python
:lines: 61-68
************************
@ -59,13 +52,14 @@ The list expects items ``(id, path, options)`` such as:
Example:
.. code:: python
DOWNLOADVIEW_MIDDLEWARES = (
('default',
'django_downloadview.nginx.XAccelMiddleware',
{'source_dir': MEDIA_ROOT, 'destination_url': '/proxied-download'}),
)
.. literalinclude:: /../demo/demoproject/settings.py
:language: python
:lines: 72-76
See :doc:`/optimizations/index` for details about middlewares and their
options.
.. note::
You can register several middlewares. It allows you to setup several
conversion rules with distinct source/destination patterns.

View file

@ -2,6 +2,32 @@
Write tests
###########
This project includes shortcuts to simplify testing.
`django_downloadview` embeds test utilities:
See :py:mod:`django_downloadview.test` for details.
* :func:`~django_downloadview.test.assert_download_response`
* :func:`~django_downloadview.test.temporary_media_root`
* :func:`~django_downloadview.nginx.tests.assert_x_accel_redirect`
************************
assert_download_response
************************
.. autofunction:: django_downloadview.test.assert_download_response
********************
temporary_media_root
********************
.. autofunction:: django_downloadview.test.temporary_media_root
*******
Example
*******
Here are the tests related to :doc:`StorageDownloadView demo </views/storage>`:
.. literalinclude:: /../demo/demoproject/storage/tests.py
:language: python

View file

@ -2,6 +2,7 @@
Make your own view
##################
.. currentmodule:: django_downloadview.views.base
*************
DownloadMixin
@ -13,6 +14,12 @@ is a base class which you can inherit of to create custom download views.
``DownloadMixin`` is a base of `BaseDownloadView`_, which itself is a base of
all other django_downloadview's builtin views.
.. autoclass:: DownloadMixin
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
****************
BaseDownloadView
@ -26,3 +33,44 @@ The only thing it does is to implement
:py:meth:`get <django_downloadview.views.BaseDownloadView.get>`: it triggers
:py:meth:`DownloadMixin's render_to_response
<django_downloadview.views.DownloadMixin.render_to_response>`.
.. autoclass:: BaseDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
************************************
Handling http not modified responses
************************************
Sometimes, you know the latest date and time the content was generated at, and
you know a new request would generate exactly the same content. In such a case,
you should implement :py:meth:`~VirtualDownloadView.was_modified_since` in your
view.
.. note::
Default :py:meth:`~VirtualDownloadView.was_modified_since` implementation
trusts file wrapper's ``was_modified_since`` if any. Else (if calling
``was_modified_since()`` raises ``NotImplementedError`` or
``AttributeError``) it returns ``True``, i.e. it assumes the file was
modified.
As an example, the download views above always generate "Hello world!"... so,
if the client already downloaded it, we can safely return some HTTP "304 Not
Modified" response:
.. code:: python
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!", name='hello-world.txt')
def was_modified_since(self, file_instance, since):
return False # Never modified, always u"Hello world!".

View file

@ -22,14 +22,8 @@ Simple example
Setup a view to stream files given URL:
.. code:: python
from django_downloadview import HTTPDownloadView
class TravisStatusView(HTTPDownloadView):
def get_url(self):
"""Return URL of django-downloadview's build status."""
return u'https://travis-ci.org/benoitbryon/django-downloadview.png'
.. literalinclude:: /../demo/demoproject/http/views.py
:language: python
*************

View file

@ -21,39 +21,45 @@ Simple example
Given a model with a :class:`~django.db.models.FileField`:
.. code:: python
.. literalinclude:: /../demo/demoproject/object/models.py
:language: python
:lines: 1-6
from django.db import models
class Document(models.Model):
file = models.FileField(upload_to='document')
Setup a view to stream the ``file`` attribute:
.. code:: python
from django_downloadview import ObjectDownloadView
download = ObjectDownloadView.as_view(model=Document)
.. note::
If the file field you want to serve is not named "file", pass the right
name as "file_field" argument, i.e. adapt
``ObjectDownloadView.as_view(model=Document, file_field='file')``.
.. literalinclude:: /../demo/demoproject/object/views.py
:language: python
:lines: 1-5, 7
:class:`~django_downloadview.views.object.ObjectDownloadView` inherits from
:class:`~django.views.generic.detail.BaseDetailView`, i.e. it expects either
``slug`` or ``pk``:
.. code:: python
.. literalinclude:: /../demo/demoproject/object/urls.py
:language: python
:lines: 1-7, 8-10, 17
from django.conf.urls import url, url_patterns
url_patterns = ('',
url('^download/(?P<slug>[A-Za-z0-9_-]+)/$', download, name='download'),
)
***************************
Serving specific file field
***************************
If your model holds several file fields, or if the file field name is not
"file", you can use :attr:`ObjectDownloadView.file_field` to specify the field
to use.
Here is a model where there are two file fields:
.. literalinclude:: /../demo/demoproject/object/models.py
:language: python
:lines: 1-6, 7
Then here is the code to serve "another_file" instead of the default "file":
.. literalinclude:: /../demo/demoproject/object/views.py
:language: python
:lines: 1-5, 10-12
**********************************
Mapping file attributes to model's
@ -65,24 +71,19 @@ can be used when you serve the file.
As an example, let's consider the client-side basename lives in model and not
in storage:
.. code:: python
from django.db import models
class Document(models.Model):
file = models.FileField(upload_to='document')
basename = models.CharField(max_length=100)
.. literalinclude:: /../demo/demoproject/object/models.py
:language: python
:lines: 1-6, 8
Then you can configure the :attr:`ObjectDownloadView.basename_field` option:
.. code:: python
.. literalinclude:: /../demo/demoproject/object/views.py
:language: python
:lines: 1-5, 16-18
from django_downloadview import ObjectDownloadView
.. note::
download = ObjectDownloadView.as_view(model=Document,
basename_field='basename')
.. note:: ``basename`` could have been a property instead of a database field.
``basename`` could have been a model's property instead of a ``CharField``.
See details below for a full list of options.

View file

@ -22,45 +22,10 @@ Simple example
Setup a view to stream files given path:
.. code:: python
from django_downloadview import PathDownloadView
download = PathDownloadView.as_view()
The view accepts a ``path`` argument you can setup either in ``as_view`` or
via URLconfs:
.. code:: python
from django.conf.urls import patterns, url
urlpatterns = patterns(
'',
url(r'^(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$', download),
)
************************************
A convenient shortcut in other views
************************************
:class:`PathDownloadView` is straight to call, so you can use it to simplify
code in more complex views, provided you have an absolute path to a local file:
.. code:: python
from django_downloadview import PathDownloadView
def some_complex_view(request, *args, **kwargs):
"""Does many things, then stream a file."""
local_path = do_many_things()
return PathDownloadView.as_view(path=local_path)(request)
.. note::
`django-sendfile`_ users may like something such as
``sendfile = lambda request, path: PathDownloadView.as_view(path=path)(request)``
.. literalinclude:: /../demo/demoproject/path/views.py
:language: python
:lines: 1-14
:emphasize-lines: 14
**************************
@ -70,20 +35,16 @@ Computing path dynamically
Override the :meth:`PathDownloadView.get_path` method to adapt path
resolution to your needs:
.. code:: python
.. literalinclude:: /../demo/demoproject/path/views.py
:language: python
:lines: 1-9, 15-
import glob
import os
import random
from django.conf import settings
from django_downloadview import PathDownloadView
The view accepts a ``path`` argument you can setup either in ``as_view`` or
via URLconfs:
class RandomImageView(PathDownloadView):
"""Stream a random image in ``MEDIA_ROOT``."""
def get_path(self):
"""Return the path of a random JPG image in ``MEDIA_ROOT``."""
image_list = glob.glob(os.path.join(settings.MEDIA_ROOT, '*.jpg'))
return random.choice(image_list)
.. literalinclude:: /../demo/demoproject/path/urls.py
:language: python
:lines: 1-7, 11-13, 14
*************
@ -95,10 +56,3 @@ API reference
:undoc-members:
:show-inheritance:
:member-order: bysource
.. rubric:: Notes & references
.. target-notes::
.. _`django-sendfile`: https://pypi.python.org/pypi/django-sendfile

View file

@ -16,31 +16,22 @@ Simple example
Given a storage:
.. code:: python
from django.core.files.storage import FileSystemStorage
storage = FileSystemStorage(location='/somewhere')
.. literalinclude:: /../demo/demoproject/storage/views.py
:language: python
:lines: 1, 4-6
Setup a view to stream files in storage:
.. code:: python
from django_downloadview import StorageDownloadView
download = StorageDownloadView.as_view(storage=storage)
.. literalinclude:: /../demo/demoproject/storage/views.py
:language: python
:lines: 3-5, 10
The view accepts a ``path`` argument you can setup either in ``as_view`` or
via URLconfs:
.. code:: python
from django.conf.urls import patterns, url
urlpatterns = patterns(
'',
url(r'^(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$', download),
)
.. literalinclude:: /../demo/demoproject/storage/urls.py
:language: python
:lines: 1-7, 8-10, 14
**************************
@ -50,6 +41,13 @@ Computing path dynamically
Override the :meth:`StorageDownloadView.get_path` method to adapt path
resolution to your needs.
As an example, here is the same view as above, but the path is converted to
uppercase:
.. literalinclude:: /../demo/demoproject/storage/views.py
:language: python
:lines: 3-5, 13-20
*************
API reference

View file

@ -26,15 +26,9 @@ Let's consider you build text dynamically, as a bytes or string or unicode
object. Serve it with Django's builtin
:class:`~django.core.files.base.ContentFile` wrapper:
.. code:: python
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!", name='hello-world.txt')
.. literalinclude:: /../demo/demoproject/virtual/views.py
:language: python
:lines: 3-5, 8-13
**************
@ -44,16 +38,9 @@ Serve StringIO
:class:`~StringIO.StringIO` object lives in memory. Let's wrap it in some
download view via :class:`~django_downloadview.files.VirtualFile`:
.. code:: python
from StringIO import StringIO
from django_downloadview import VirtualDownloadView, VirtualFile
class StringIODownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIO`` object."""
file_obj = StringIO(u"Hello world!\n".encode('utf-8'))
return VirtualFile(file_obj, name='hello-world.txt')
.. literalinclude:: /../demo/demoproject/virtual/views.py
:language: python
:lines: 1-2, 5-6, 14-20
************************
@ -63,64 +50,19 @@ Stream generated content
Let's consider you have a generator function (``yield``) or an iterator object
(``__iter__()``):
.. code:: python
def generate_hello():
yield u'Hello '
yield u'world!'
.. literalinclude:: /../demo/demoproject/virtual/views.py
:language: python
:lines: 23-26
Stream generated content using :class:`VirtualDownloadView`,
:class:`~django_downloadview.files.VirtualFile` and
:class:`~django_downloadview.file.StringIteratorIO`:
.. code:: python
from django_downloadview import (VirtualDownloadView,
VirtualFile,
StringIteratorIO)
class GeneratedDownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIteratorIO`` object."""
file_obj = StringIteratorIO(generate_hello())
return VirtualFile(file_obj, name='hello-world.txt')
************************************
Handling http not modified responses
************************************
Sometimes, you know the latest date and time the content was generated at, and
you know a new request would generate exactly the same content. In such a case,
you should implement :py:meth:`~VirtualDownloadView.was_modified_since` in your
view.
.. note::
Default :py:meth:`~VirtualDownloadView.was_modified_since` implementation
trusts file wrapper's ``was_modified_since`` if any. Else (if calling
``was_modified_since()`` raises ``NotImplementedError`` or
``AttributeError``) it returns ``True``, i.e. it assumes the file was
modified.
As an example, the download views above always generate "Hello world!"... so,
if the client already downloaded it, we can safely return some HTTP "304 Not
Modified" response:
.. code:: python
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!", name='hello-world.txt')
def was_modified_since(self, file_instance, since):
return False # Never modified, always u"Hello world!".
.. literalinclude:: /../demo/demoproject/virtual/views.py
:language: python
:lines: 5-9, 29-33
*************