Hopiu/django-downloadview
1
0
Fork 0
mirror of https://github.com/jazzband/django-downloadview.git synced 2026-03-16 22:40:25 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Refs #39 - Improved narrative documentation. Work in progress.

Browse source
This commit is contained in:
Benoît Bryon 2013-10-21 15:06:19 +02:00
parent cb68d7f8e5
commit 413f7a9052
15 changed files with 805 additions and 122 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

89
docs/files.txt Normal file
View file

@ -0,0 +1,89 @@
#############
File wrappers
#############
.. py:module:: django_downloadview.files
A view return :class:`~django_downloadview.response.DownloadResponse` which
itself carries a file wrapper. Here are file wrappers distributed by Django
and django-downloadview.
*****************
Django's builtins
*****************
`Django itself provides some file wrappers`_ you can use within
``django-downloadview``:
* :py:class:`django.core.files.File` wraps a file that live on local
filesystem, initialized with a path. ``django-downloadview`` uses this
wrapper in :doc:`/views/path`.
* :py:class:`django.db.models.fields.files.FieldFile` wraps a file that is
managed in a model. ``django-downloadview`` uses this wrapper in
:doc:`/views/object`.
* :py:class:`django.core.files.base.ContentFile` wraps a bytes, string or
unicode object. You may use it with :doc:`VirtualDownloadView
</views/virtual>`.
****************************
django-downloadview builtins
****************************
``django-downloadview`` implements additional file wrappers:
* :class:`StorageFile` wraps a file that is
managed via a storage (but not necessarily via a model).
:doc:`/views/storage` uses this wrapper.
* :class:`HTTPFile` wraps a file that lives at
some (remote) location, initialized with an URL.
:doc:`/views/http` uses this wrapper.
* :class:`VirtualFile` wraps a file that lives in
memory, i.e. built as a string.
This is a convenient wrapper to use in :doc:`/views/virtual` subclasses.
*************
API reference
*************
StorageFile
===========
.. autoclass:: StorageFile
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
HTTPFile
========
.. autoclass:: HTTPFile
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
VirtualFile
===========
.. autoclass:: VirtualFile
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
.. rubric:: Notes & references
.. target-notes::
.. _`Django itself provides some file wrappers`:
https://docs.djangoproject.com/en/1.5/ref/files/file/

106
docs/healthchecks.txt Normal file
View file

@ -0,0 +1,106 @@
##################
Write healthchecks
##################
In the previous :doc:`testing </testing>` topic, you made sure the views and
middlewares work as expected... within a test environment.
One common issue when deploying in production is that the reverse-proxy's
configuration does not fit. You cannot check that within test environment.
**Healthchecks are made to diagnose issues in live (production) environments**.
************************
Introducing healthchecks
************************
Healthchecks (sometimes called "smoke tests" or "diagnosis") are assertions you
run on a live (typically production) service, as opposed to fake/mock service
used during tests (unit, integration, functional).
See `hospital`_ and `django-doctor`_ projects about writing healthchecks for
Python and Django.
********************
Typical healthchecks
********************
Here is a typical healthcheck setup for download views with reverse-proxy
optimizations.
When you run this healthcheck suite, you get a good overview if a problem
occurs: you can compare expected results and learn which part (Django,
reverse-proxy or remote storage) is guilty.
.. note::
In the examples below, we use "localhost" and ports "80" (reverse-proxy) or
"8000" (Django). Adapt them to your configuration.
Check storage
=============
Put a dummy file on the storage Django uses.
The write a healthcheck that asserts you can read the dummy file from storage.
**On success, you know remote storage is ok.**
Issues may involve permissions or communications (remote storage).
.. note::
This healthcheck may be outside Django.
Check Django VS storage
=======================
Implement a download view dedicated to healthchecks. It is typically a public
(but not referenced) view that streams a dummy file from real storage.
Let's say you register it as ``/healthcheck-utils/download/`` URL.
Write a healthcheck that asserts ``GET
http://localhost:8000/healtcheck-utils/download/`` (notice the `8000` port:
local Django server) returns the expected reverse-proxy response (X-Accel,
X-Sendfile...).
**On success, you know there is no configuration issue on the Django side.**
Check reverse proxy VS storage
==============================
Write a location in your reverse-proxy's configuration that proxy-pass to a
dummy file on storage.
Write a healthcheck that asserts this location returns the expected dummy file.
**On success, you know the reverse proxy can serve files from storage.**
Check them all together
=======================
We just checked all parts separately, so let's make sure they can work
together.
Configure the reverse-proxy so that `/healthcheck-utils/download/` is proxied
to Django. Then write a healthcheck that asserts ``GET
http://localhost:80/healthcheck-utils/download`` (notice the `80` port:
reverse-proxy server) returns the expected dummy file.
**On success, you know everything is ok.**
On failure, there is an issue in the X-Accel/X-Sendfile configuration.
.. note::
This last healthcheck should be the first one to run, i.e. if it passes,
others should pass too. The others are useful when this one fails.
.. rubric:: Notes & references
.. target-notes::
.. _`hospital`: https://pypi.python.org/pypi/hospital
.. _`django-doctor`: https://pypi.python.org/pypi/django-doctor

9
docs/index.txt
View file

@ -9,12 +9,15 @@ Contents
:maxdepth: 2
:titlesonly:
demo
overview
install
settings
views
views/index
optimizations/index
testing
api/index
healthchecks
files
responses
demo
about/index
dev

6
docs/optimizations/index.txt
View file

@ -1,6 +1,6 @@
#############
Optimizations
#############
##################
Optimize streaming
##################
Some reverse proxies allow applications to delegate actual download to the
proxy:

91
docs/overview.txt Normal file
View file

@ -0,0 +1,91 @@
##################
Overview, concepts
##################
Given:
* you manage files with Django (permissions, search, generation, ...)
* files are stored somewhere or generated somehow (local filesystem, remote
storage, memory...)
As a developer, you want to serve files quick and efficiently.
Here is an overview of ``django-downloadview``'s answer...
************************************
Generic views cover commons patterns
************************************
* :doc:`/views/object` when you have a model with a file field;
* :doc:`/views/storage` when you manage files in a storage;
* :doc:`/views/path` when you have an absolute filename on local filesystem;
* :doc:`/views/http` when you have an URL (the resource is proxied);
* :doc:`/views/virtual` when you generate a file dynamically.
*************************************************
Generic views and mixins allow easy customization
*************************************************
If your use case is a bit specific, you can easily extend the views above or
:doc:`create your own based on mixins </views/custom>`.
*****************************
Views return DownloadResponse
*****************************
Views return :py:class:`~django_downloadview.response.DownloadResponse`. It is
a special :py:class:`django.http.StreamingHttpResponse` where content is
encapsulated in a file wrapper. If the response is sent to the client, the file
content content is loaded.
.. note::
Middlewares and decorators are given the opportunity to optimize the
streaming before file content loading.
Learn more in :doc:`responses`.
***********************************
DownloadResponse carry file wrapper
***********************************
A download view instanciates a :doc:`file wrapper </files>` and use it to
initialize :py:class:`~django_downloadview.response.DownloadResponse`.
File wrappers describe files. They carry files properties, but not file
content. They implement loading and iterating over file content.
Learn more about available file wrappers in :doc:`files`.
*****************************************************************
Middlewares convert DownloadResponse into ProxiedDownloadResponse
*****************************************************************
Decorators and middlewares may capture
:py:class:`~django_downloadview.response.DownloadResponse` instances in order
to optimize the streaming. A good optimization is to delegate streaming to
reverse proxies such as Nginx via X-Accel redirections or Apache via
X-Sendfile.
Learn more in :doc:`optimizations/index`.
***************
Testing matters
***************
``django-downloadview`` also helps you :doc:`test the views you customized
<testing>`.
************
What's next?
************
Convinced? Let's :doc:`install django-downloadview <install>`.

16
docs/responses.txt Normal file
View file

@ -0,0 +1,16 @@
#########
Responses
#########
********************
``DownloadResponse``
********************
***************************
``ProxiedDownloadResponse``
***************************

6
docs/testing.txt
View file

@ -1,6 +1,6 @@
######################
Testing download views
######################
###########
Write tests
###########
This project includes shortcuts to simplify testing.

113
docs/views.txt
View file

@ -1,113 +0,0 @@
##############
Download views
##############
This section contains narrative overview about class-based views provided by
django-downloadview.
By default, all of those views would stream the file to the client.
But keep in mind that you can setup :doc:`/optimizations/index` to delegate
actual streaming to a reverse proxy.
*************
DownloadMixin
*************
The :py:class:`django_downloadview.views.DownloadMixin` class is not a view. It
is a base class which you can inherit of to create custom download views.
``DownloadMixin`` is a base of `BaseDownloadView`_, which itself is a base of
all other django_downloadview's builtin views.
****************
BaseDownloadView
****************
The :py:class:`django_downloadview.views.BaseDownloadView` class is a base
class to create download views. It inherits `DownloadMixin`_ and
:py:class:`django.views.generic.base.View`.
The only thing it does is to implement
:py:meth:`get <django_downloadview.views.BaseDownloadView.get>`: it triggers
:py:meth:`DownloadMixin's render_to_response
<django_downloadview.views.DownloadMixin.render_to_response>`.
******************
ObjectDownloadView
******************
The :py:class:`django_downloadview.views.ObjectDownloadView` class-based view
allows you to **serve files given a model with some file fields** such as
FileField or ImageField.
Use this view anywhere you could use Django's builtin ObjectDetailView.
Some options allow you to store file metadata (size, content-type, ...) in the
model, as deserialized fields.
*******************
StorageDownloadView
*******************
The :py:class:`django_downloadview.views.StorageDownloadView` class-based view
allows you to **serve files given a storage and a path**.
Use this view when you manage files in a storage (which is a good practice),
unrelated to a model.
****************
PathDownloadView
****************
The :py:class:`django_downloadview.views.PathDownloadView` class-based view
allows you to **serve files given an absolute path on local filesystem**.
Two main use cases:
* as a shortcut. This dead-simple view is straight to call, so you can use it
to simplify code in more complex views, provided you have an absolute path to
a local file.
* override. Extend :py:class:`django_downloadview.views.PathDownloadView` and
override :py:meth:`django_downloadview.views.PathDownloadView:get_path`.
****************
HTTPDownloadView
****************
The :py:class:`django_downloadview.views.HTTPDownloadView` class-based view
allows you to **serve files given an URL**. That URL is supposed to be
downloadable from the Django server.
Use it when you want to setup a proxy to remote files:
* the Django view filters input and computes target URL.
* if you setup optimizations, Django itself doesn't proxies the file,
* but, as a fallback, Django uses `requests`_ to proxy the file.
Extend :py:class:`django_downloadview.views.HTTPDownloadView` then
override :py:meth:`django_downloadview.views.HTTPDownloadView:get_url`.
*******************
VirtualDownloadView
*******************
The :py:class:`django_downloadview.views.VirtualDownloadView` class-based view
allows you to **serve files that don't live on disk**.
Use it when you want to stream a file which content is dynamically generated
or which lives in memory.
.. rubric:: References
.. target-notes::
.. _`requests`: https://pypi.python.org/pypi/requests

28
docs/views/custom.txt Normal file
View file

@ -0,0 +1,28 @@
##################
Make your own view
##################
*************
DownloadMixin
*************
The :py:class:`django_downloadview.views.DownloadMixin` class is not a view. It
is a base class which you can inherit of to create custom download views.
``DownloadMixin`` is a base of `BaseDownloadView`_, which itself is a base of
all other django_downloadview's builtin views.
****************
BaseDownloadView
****************
The :py:class:`django_downloadview.views.BaseDownloadView` class is a base
class to create download views. It inherits `DownloadMixin`_ and
:py:class:`django.views.generic.base.View`.
The only thing it does is to implement
:py:meth:`get <django_downloadview.views.BaseDownloadView.get>`: it triggers
:py:meth:`DownloadMixin's render_to_response
<django_downloadview.views.DownloadMixin.render_to_response>`.

43
docs/views/http.txt Normal file
View file

@ -0,0 +1,43 @@
################
HTTPDownloadView
################
.. py:module:: django_downloadview.views.http
:class:`HTTPDownloadView` **serves a file given an URL.**, i.e. it acts like
a proxy.
This view is particularly handy when:
* the client does not have access to the file resource, while your Django
server does.
* the client does trust your server, your server trusts a third-party, you do
not want to bother the client with the third-party.
**************
Simple example
**************
Setup a view to stream files given URL:
.. code:: python
from django_downloadview import HTTPDownloadView
class TravisStatusView(HTTPDownloadView):
def get_url(self):
"""Return URL of django-downloadview's build status."""
return u'https://travis-ci.org/benoitbryon/django-downloadview.png'
*************
API reference
*************
.. autoclass:: HTTPDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource

22
docs/views/index.txt Normal file
View file

@ -0,0 +1,22 @@
###########
Setup views
###########
Setup views depending on your needs:
* :doc:`/views/object` when you have a model with a file field;
* :doc:`/views/storage` when you manage files in a storage;
* :doc:`/views/path` when you have an absolute filename on local filesystem;
* :doc:`/views/http` when you have an URL (the resource is proxied);
* :doc:`/views/virtual` when you generate a file dynamically;
* :doc:`bases and mixins </views/custom>` to make your own.
.. toctree::
:hidden:
object
storage
path
http
virtual
custom

98
docs/views/object.txt Normal file
View file

@ -0,0 +1,98 @@
##################
ObjectDownloadView
##################
.. py:module:: django_downloadview.views.object
:class:`ObjectDownloadView` **serves files managed in models with file fields**
such as :class:`~django.db.models.FileField` or
:class:`~django.db.models.ImageField`.
Use this view like Django's builtin
:class:`~django.views.generic.detail.DetailView`.
Additional options allow you to store file metadata (size, content-type, ...)
in the model, as deserialized fields.
**************
Simple example
**************
Given a model with a :class:`~django.db.models.FileField`:
.. code:: python
from django.db import models
class Document(models.Model):
file = models.FileField(upload_to='document')
Setup a view to stream the ``file`` attribute:
.. code:: python
from django_downloadview import ObjectDownloadView
download = ObjectDownloadView.as_view(model=Document)
.. note::
If the file field you want to serve is not named "file", pass the right
name as "file_field" argument, i.e. adapt
``ObjectDownloadView.as_view(model=Document, file_field='file')``.
:class:`~django_downloadview.views.object.ObjectDownloadView` inherits from
:class:`~django.views.generic.detail.BaseDetailView`, i.e. it expects either
``slug`` or ``pk``:
.. code:: python
from django.conf.urls import url, url_patterns
url_patterns = ('',
url('^download/(?P<slug>[A-Za-z0-9_-]+)/$', download, name='download'),
)
**********************************
Mapping file attributes to model's
**********************************
Sometimes, you use Django model to store file's metadata. Some of this metadata
can be used when you serve the file.
As an example, let's consider the client-side basename lives in model and not
in storage:
.. code:: python
from django.db import models
class Document(models.Model):
file = models.FileField(upload_to='document')
basename = models.CharField(max_length=100)
Then you can configure the :attr:`ObjectDownloadView.basename_field` option:
.. code:: python
from django_downloadview import ObjectDownloadView
download = ObjectDownloadView.as_view(model=Document,
basename_field='basename')
.. note:: ``basename`` could have been a property instead of a database field.
See details below for a full list of options.
*************
API reference
*************
.. autoclass:: ObjectDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource

104
docs/views/path.txt Normal file
View file

@ -0,0 +1,104 @@
################
PathDownloadView
################
.. py:module:: django_downloadview.views.path
:class:`PathDownloadView` **serves file given a path on local filesystem**.
Use this view whenever you just have a path, outside storage or model.
.. warning::
Take care of path validation, especially if you compute paths from user
input: an attacker may be able to download files from arbitrary locations.
In most cases, you should consider managing files in storages, because they
implement default security mechanisms.
**************
Simple example
**************
Setup a view to stream files given path:
.. code:: python
from django_downloadview import PathDownloadView
download = PathDownloadView.as_view()
The view accepts a ``path`` argument you can setup either in ``as_view`` or
via URLconfs:
.. code:: python
from django.conf.urls import patterns, url
urlpatterns = patterns(
'',
url(r'^(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$', download),
)
************************************
A convenient shortcut in other views
************************************
:class:`PathDownloadView` is straight to call, so you can use it to simplify
code in more complex views, provided you have an absolute path to a local file:
.. code:: python
from django_downloadview import PathDownloadView
def some_complex_view(request, *args, **kwargs):
"""Does many things, then stream a file."""
local_path = do_many_things()
return PathDownloadView.as_view(path=local_path)(request)
.. note::
`django-sendfile`_ users may like something such as
``sendfile = lambda request, path: PathDownloadView.as_view(path=path)(request)``
**************************
Computing path dynamically
**************************
Override the :meth:`PathDownloadView.get_path` method to adapt path
resolution to your needs:
.. code:: python
import glob
import os
import random
from django.conf import settings
from django_downloadview import PathDownloadView
class RandomImageView(PathDownloadView):
"""Stream a random image in ``MEDIA_ROOT``."""
def get_path(self):
"""Return the path of a random JPG image in ``MEDIA_ROOT``."""
image_list = glob.glob(os.path.join(settings.MEDIA_ROOT, '*.jpg'))
return random.choice(image_list)
*************
API reference
*************
.. autoclass:: PathDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource
.. rubric:: Notes & references
.. target-notes::
.. _`django-sendfile`: https://pypi.python.org/pypi/django-sendfile

62
docs/views/storage.txt Normal file
View file

@ -0,0 +1,62 @@
###################
StorageDownloadView
###################
.. py:module:: django_downloadview.views.storage
:class:`StorageDownloadView` **serves files given a storage and a path**.
Use this view when you manage files in a storage (which is a good practice),
unrelated to a model.
**************
Simple example
**************
Given a storage:
.. code:: python
from django.core.files.storage import FileSystemStorage
storage = FileSystemStorage(location='/somewhere')
Setup a view to stream files in storage:
.. code:: python
from django_downloadview import StorageDownloadView
download = StorageDownloadView.as_view(storage=storage)
The view accepts a ``path`` argument you can setup either in ``as_view`` or
via URLconfs:
.. code:: python
from django.conf.urls import patterns, url
urlpatterns = patterns(
'',
url(r'^(?P<path>[a-zA-Z0-9_-]+\.[a-zA-Z0-9]{1,4})$', download),
)
**************************
Computing path dynamically
**************************
Override the :meth:`StorageDownloadView.get_path` method to adapt path
resolution to your needs.
*************
API reference
*************
.. autoclass:: StorageDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource

134
docs/views/virtual.txt Normal file
View file

@ -0,0 +1,134 @@
###################
VirtualDownloadView
###################
.. py:module:: django_downloadview.views.virtual
:class:`VirtualDownloadView` **serves files that do not live on disk**.
Use it when you want to stream a file which content is dynamically generated
or which lives in memory.
It is all about overriding :meth:`VirtualDownloadView.get_file` method so that
it returns a suitable file wrapper...
.. note::
Current implementation does not support reverse-proxy optimizations,
because there is no place reverse-proxy can load files from after Django
exited.
***************************************
Serve text (string or unicode) or bytes
***************************************
Let's consider you build text dynamically, as a bytes or string or unicode
object. Serve it with Django's builtin
:class:`~django.core.files.base.ContentFile` wrapper:
.. code:: python
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!", name='hello-world.txt')
**************
Serve StringIO
**************
:class:`~StringIO.StringIO` object lives in memory. Let's wrap it in some
download view via :class:`~django_downloadview.files.VirtualFile`:
.. code:: python
from StringIO import StringIO
from django_downloadview import VirtualDownloadView, VirtualFile
class StringIODownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIO`` object."""
file_obj = StringIO(u"Hello world!\n".encode('utf-8'))
return VirtualFile(file_obj, name='hello-world.txt')
************************
Stream generated content
************************
Let's consider you have a generator function (``yield``) or an iterator object
(``__iter__()``):
.. code:: python
def generate_hello():
yield u'Hello '
yield u'world!'
Stream generated content using :class:`VirtualDownloadView`,
:class:`~django_downloadview.files.VirtualFile` and
:class:`~django_downloadview.file.StringIteratorIO`:
.. code:: python
from django_downloadview import (VirtualDownloadView,
VirtualFile,
StringIteratorIO)
class GeneratedDownloadView(VirtualDownloadView):
def get_file(self):
"""Return wrapper on ``StringIteratorIO`` object."""
file_obj = StringIteratorIO(generate_hello())
return VirtualFile(file_obj, name='hello-world.txt')
************************************
Handling http not modified responses
************************************
Sometimes, you know the latest date and time the content was generated at, and
you know a new request would generate exactly the same content. In such a case,
you should implement :py:meth:`~VirtualDownloadView.was_modified_since` in your
view.
.. note::
Default :py:meth:`~VirtualDownloadView.was_modified_since` implementation
trusts file wrapper's ``was_modified_since`` if any. Else (if calling
``was_modified_since()`` raises ``NotImplementedError`` or
``AttributeError``) it returns ``True``, i.e. it assumes the file was
modified.
As an example, the download views above always generate "Hello world!"... so,
if the client already downloaded it, we can safely return some HTTP "304 Not
Modified" response:
.. code:: python
from django.core.files.base import ContentFile
from django_downloadview import VirtualDownloadView
class TextDownloadView(VirtualDownloadView):
def get_file(self):
"""Return :class:`django.core.files.base.ContentFile` object."""
return ContentFile(u"Hello world!", name='hello-world.txt')
def was_modified_since(self, file_instance, since):
return False # Never modified, always u"Hello world!".
*************
API reference
*************
.. autoclass:: VirtualDownloadView
:members:
:undoc-members:
:show-inheritance:
:member-order: bysource