v4.0.0 Added Hashed Id generation scheme

This commit is contained in:
AppleGrew (applegrew) 2013-03-15 11:30:40 +05:30
parent aaf24d65d6
commit e082822cdd
7 changed files with 46 additions and 12 deletions

6
README
View file

@ -33,7 +33,7 @@ External Dependencies
* Django - This is obvious.
* jQuery - This is not included in the package since it is expected that in most scenarios this would already be available.
* Memcached (python-memcached) - If you plan on running multiple python processes, then you need to turn on ``ENABLE_SELECT2_MULTI_PROCESS_SUPPORT``. In that mode it is highly recommended that you use Memcached, to minimize DB hits.
* Memcached (python-memcached) - If you plan on running multiple python processes with `GENERATE_RANDOM_SELECT2_ID` enabled, then you need to turn on `ENABLE_SELECT2_MULTI_PROCESS_SUPPORT`. In that mode it is highly recommended that you use Memcached, to minimize DB hits.
Example Application
===================
@ -49,6 +49,10 @@ Special Thanks
Changelog Summary
=================
### v4.0.0
* Main version number bumped to bring your attention to the fact that the default Id generation scheme has now changed. Now Django Select2 will use hashed paths of fields to generate their Ids. The old scheme of generating random Ids are still there. You can enable that by setting `GENERATE_RANDOM_SELECT2_ID` to `True`.
### v3.3.1
* Addressed issue[#30](https://github.com/applegrew/django-select2/issues/30).

View file

@ -33,7 +33,7 @@ External Dependencies
* Django - This is obvious.
* jQuery - This is not included in the package since it is expected that in most scenarios this would already be available.
* Memcached (python-memcached) - If you plan on running multiple python processes, then you need to turn on ``ENABLE_SELECT2_MULTI_PROCESS_SUPPORT``. In that mode it is highly recommended that you use Memcached, to minimize DB hits.
* Memcached (python-memcached) - If you plan on running multiple python processes with `GENERATE_RANDOM_SELECT2_ID` enabled, then you need to turn on `ENABLE_SELECT2_MULTI_PROCESS_SUPPORT`. In that mode it is highly recommended that you use Memcached, to minimize DB hits.
Example Application
===================
@ -49,6 +49,10 @@ Special Thanks
Changelog Summary
=================
### v4.0.0
* Main version number bumped to bring your attention to the fact that the default Id generation scheme has now changed. Now Django Select2 will use hashed paths of fields to generate their Ids. The old scheme of generating random Ids are still there. You can enable that by setting `GENERATE_RANDOM_SELECT2_ID` to `True`.
### v3.3.1
* Addressed issue[#30](https://github.com/applegrew/django-select2/issues/30).

View file

@ -74,13 +74,18 @@ The view - `Select2View`, exposed here is meant to be used with 'Heavy' fields a
"""
__version__ = "3.3.1"
import logging
logger = logging.getLogger(__name__)
__version__ = "4.0.0"
__RENDER_SELECT2_STATICS = False
__ENABLE_MULTI_PROCESS_SUPPORT = False
__MEMCACHE_HOST = None
__MEMCACHE_PORT = None
__MEMCACHE_TTL = 900
__GENERATE_RANDOM_ID = False
__SECRET_SALT = ''
try:
from django.conf import settings
@ -90,6 +95,12 @@ try:
__MEMCACHE_HOST = getattr(settings, 'SELECT2_MEMCACHE_HOST', None)
__MEMCACHE_PORT = getattr(settings, 'SELECT2_MEMCACHE_PORT', None)
__MEMCACHE_TTL = getattr(settings, 'SELECT2_MEMCACHE_TTL', 900)
__GENERATE_RANDOM_ID = getattr(settings, 'GENERATE_RANDOM_SELECT2_ID', False)
__SECRET_SALT = getattr(settings, 'SECRET_KEY', '')
if not __GENERATE_RANDOM_ID and __ENABLE_MULTI_PROCESS_SUPPORT:
logger.warn("You need not turn on ENABLE_SELECT2_MULTI_PROCESS_SUPPORT when GENERATE_RANDOM_SELECT2_ID is disabled.")
__ENABLE_MULTI_PROCESS_SUPPORT = False
from .widgets import Select2Widget, Select2MultipleWidget, HeavySelect2Widget, HeavySelect2MultipleWidget, \
AutoHeavySelect2Widget, AutoHeavySelect2MultipleWidget
@ -99,9 +110,6 @@ try:
AutoModelSelect2Field, AutoModelSelect2MultipleField
from .views import Select2View, NO_ERR_RESP
except ImportError:
import logging
logger = logging.getLogger(__name__)
if logger.isEnabledFor(logging.INFO):
logger.info("Django not found.")

View file

@ -1,4 +1,5 @@
import datetime
import hashlib
import logging
import re
import threading
@ -194,6 +195,8 @@ def convert_to_js_string_arr(lst):
from . import __ENABLE_MULTI_PROCESS_SUPPORT as ENABLE_MULTI_PROCESS_SUPPORT, \
__MEMCACHE_HOST as MEMCACHE_HOST, __MEMCACHE_PORT as MEMCACHE_PORT, __MEMCACHE_TTL as MEMCACHE_TTL
from . import __GENERATE_RANDOM_ID as GENERATE_RANDOM_ID, __SECRET_SALT as SECRET_SALT
def synchronized(f):
"Decorator to synchronize multiple calls to a functions."
f.__lock__ = threading.Lock()
@ -255,7 +258,10 @@ def register_field(key, field):
if key not in __field_store:
# Generating id
id_ = u"%d:%s" % (len(__id_store), unicode(datetime.datetime.now()))
if GENERATE_RANDOM_ID:
id_ = u"%d:%s" % (len(__id_store), unicode(datetime.datetime.now()))
else:
id_ = unicode(hashlib.sha1("%s:%s" % (key, SECRET_SALT)).hexdigest())
__field_store[key] = id_
__id_store[id_] = field

View file

@ -25,8 +25,8 @@ Installation
python manage.py syncdb
Available Setting
-----------------
Available Settings
------------------
``AUTO_RENDER_SELECT2_STATICS`` [Default ``True``]
..................................................
@ -45,13 +45,23 @@ When this settings is ``False`` then you are responsible for including the JS an
If that is set to ``1`` then only the JS and CSS libraries needed by Select2Widget (Light fields) are rendered.
That effectively leaves out ``heavy.js`` and ``extra.css``.
``GENERATE_RANDOM_SELECT2_ID`` [Default ``False``]
..................................................
As of version 4.0.0 the field's Ids are their paths which have been hashed by SHA1. This Id generation scheme should be sufficient for most applications.
However, if you have a secret government project and fear that SHA1 hashes could be cracked (which is not impossible) to reveal the path and names of your fields then you can enable this mode. This will use timestamps as Ids which have no correlation to the field's name or path.
.. tip:: The field's paths are first salted with Django generated ``SECRET_KEY`` before hashing them.
``ENABLE_SELECT2_MULTI_PROCESS_SUPPORT`` [Default ``False``]
............................................................
In production servers usually multiple server processes are run to handle the requests. This poses a problem for Django Select2's Auto fields since they generate unique Id at runtime. The clients can identify the fields in ajax query request using only these generated ids. In multi-processes scenario there is no guarantee that the process which rendered the page is the one which will respond to ajax queries.
This setting cannot be enabled as it is not required when ``GENERATE_RANDOM_SELECT2_ID`` is ``False``.
When this mode is enabled then Django Select2 maintains an id to field key mapping in DB for all processes. Whenever a process does not find an id in its internal map it looks-up in the central DB. From DB it finds the field key. Using the key the process then looks-up a field instance with that key, since all instaces with same key are assumed to be equivalent.
In production servers usually multiple server processes are run to handle the requests. This poses a problem for Django Select2's Auto fields since they generate unique Id at runtime when ``GENERATE_RANDOM_SELECT2_ID`` is enabled. The clients can identify the fields in ajax query request using only these generated ids. In multi-processes scenario there is no guarantee that the process which rendered the page is the one which will respond to ajax queries.
When this mode is enabled then Django Select2 maintains an id to field key mapping in DB for all processes. Whenever a process does not find an id in its internal map it looks-up in the central DB. From DB it finds the field key. Using the key, the process then looks-up a field instance with that key, since all instaces with same key are assumed to be equivalent.
.. tip:: Make sure to run ``python manage.py syncdb`` to create the ``KeyMap`` table.

Binary file not shown.

View file

@ -174,6 +174,8 @@ LOGGING = {
AUTO_RENDER_SELECT2_STATICS = False
#GENERATE_RANDOM_SELECT2_ID = True
##
# To test for multiple processes in developement system w/o WSGI, runserver at
# different ports. Use $('#select2_field_html_id').data('field_id') to get the id
@ -183,7 +185,7 @@ AUTO_RENDER_SELECT2_STATICS = False
# you should see a message like - "Id 7:2013-03-01 14:49:18.490212 not found in
# this process. Looking up in remote server.", in console if you have debug enabled.
##
ENABLE_SELECT2_MULTI_PROCESS_SUPPORT = True
#ENABLE_SELECT2_MULTI_PROCESS_SUPPORT = True
#SELECT2_MEMCACHE_HOST = '127.0.0.1' # Uncomment to use memcached too
#SELECT2_MEMCACHE_PORT = 11211 # Uncomment to use memcached too
#SELECT2_MEMCACHE_TTL = 9 # Default 900