Since assignments on M2M or reverse FK descriptors is composed of a `clear()`,
followed by an `add()`, `clear()` could potentially affect the value of the
assigned queryset before the `add()` step; pre-evaluating it solves the problem.
This patch fixes the issue for ForeignRelatedObjectsDescriptor,
ManyRelatedObjectsDescriptor, and ReverseGenericRelatedObjectsDescriptor.
It completes 6cb6e1 which addressed ReverseManyRelatedObjectsDescriptor.
Code borrowed from django.contrib.admin.tests. Without this, the class can
throw an exception with something like "The browser appears to have exited
before we could connect. The output was: Error: no display specified"
This commit reverts 69d4b1c and tackle the issue from a different angle.
Models remain present in the project state, but are now ignored by the
autodetector.
Added condition to prevent checking the existence of a file name of a
file like object when the name attribute is None. This is necessary
because a SpooledTemporaryFile won't exist on the file system or have a
name until it has reached its max_size. Also added tests.
db_parameters should respect an already existing db_type method and
return that as its type string. In particular, this was causing some
fields from gis to not be generated.
Thanks to @bigsassy and @blueyed for their work on the patch.
Also fixed#22260
This seems worthwhile in its own right, but also works around an Oracle
bug (in versions 10 -- 11.1) where the use of Unicode would reset the
date/time formats, causing ORA-01843 errors.
Thanks Trac users CarstenF for the report, jtiai for the initial patch,
and everyone who contributed to the discussion on the ticket.
Previously, saving a model instance to a non-related field (in
particular a FloatField) would silently convert the model to an Integer
(the pk) and save it. This is undesirable behaviour, and likely to cause
confusion so the validatio has been hardened.
Thanks to @PirosB3 for the patch and @jarshwah for the review.
GenericRelation now supports an optional related_query_name argument.
Setting related_query_name adds a relation from the related object back to
the content type for filtering, ordering and other query operations.
Thanks to Loic Bistuer for spotting a couple of important issues in
his review.
Fixed URL resolving in the case where an outer regex includes an inner
regex and both regexes use positional parameters instead of named
groups, causing the outer regex's parameters to override the inner
regex's.
Modified the regex url resolver so that it will concatenates and then
normalizes, instead of normalizing and then concatenating.
This feature allows the default `TIMEOUT` Cache argument to be set to `None`,
so that cache instances can set a non-expiring key as the default,
instead of using the default value of 5 minutes.
Previously, this was possible only by passing `None` as an argument to
the set() method of objects of type `BaseCache` (and subtypes).
The original patch for custom prefetches didn't allow usage of custom
queryset for single valued relations (along ForeignKey or OneToOneKey).
Allowing these enables calling performance oriented queryset methods like
select_related or defer/only.
Thanks @akaariai and @timgraham for the reviews. Refs #17001.
Allows custom behavior for setting initial form data in ModelAdmin.
By default, initial data is set via GET params. The new method allows
this behavior to be overridden.
Thanks egasimus for the suggestion.
Introduced a number of settings to configure max-age, path, and domain
for the language cookie: LANGUAGE_COOKIE_AGE, LANGUAGE_COOKIE_PATH and
LANGUAGE_COOKIE_DOMAIN.
Thanks sahid for the suggestion.
By removing the 'supported' keyword from the detection methods and only relying
on a cached settings.LANGUAGES, the speed of said methods has been improved;
around 4x raw performance. This allows us to stop checking Python's incomplete
list of locales, and rely on a less restrictive regular expression for
accepting certain locales.
HTTP Accept-Language is defined as being case-insensitive, based on this fact
extra performance improvements have been made; it wouldn't make sense to
check for case differences.
Added reversible property to RunPython so that migrations will not
refuse to reverse migrations including RunPython operations, so long as
reverse_code is set in the RunPython constructor. Included tests to
check the reversible property on RunPython and the similar RunSQL.
The new error message now hints that the most likely issue
is a circular import.
Thanks to trac user elena for the report and to
bpeschier for the original patch.
Due to a mixup between text and bytes, iteration over
a File instance was broken under Python 3.
Thanks to trac user pdewacht for the report and patch.
Instead of crashing with a CommandError, now a non-writable location
of mo files will only make compilemessages complain and continue.
Thanks Ramiro Morales for the review.
This option is not actually very useful in the general case
because it doesn't override sys.stdin.
It's still marginally useful for testing some features of
the createsuperuser command so it was moved there.
This commit also makes the detection of a TTY in createsuperuser
a bit more robust, after a suggestion of appolo13.
Previously, doing so resulted in invalid data or crash.
Thanks jtiai for the report and Karol Jochelson,
Jakub Nowak, Loic Bistuer, and Baptiste Mispelon for reviews.
Commit 79558c78 cleaned up the (undocumented) interface of Resolver404
exception, which breaks compatibility with code messing with .args[0]
directly. Revert the cleanup part and simply leave the fix itself.
When django.core.urlresolvers.resolve was called from a view, failed
and the exception was propagated and rendered by technical_404_response,
the URL mentioned on the page was the current URL instead of the URL
passed to resolve().
Fixed by using the path attribute from the Resolver404 exception instead
of request.path_info. Also cleaned up the exceptions to use standard
named parameters instead of stuffing a dict in args[0]
Flatten a level of sublists before checking for duplicate fields.
When given sublists such as:
```python
class FooAdmin(admin.ModelAdmin):
fields = ('one', ('one', 'two'))
```
The previous code did not correctly detect the duplicated 'one' field.
Thanks to jwa for the report.
Before 4befb30 the detection was broken because we used isinstance
against a LazyObject rather than against a Storage class. That commit
fixed it by looking directly at the object wrapped by LazyObject.
This could however be a problem to anyone who subclasses the
collectstatic management Command and directly supplies a Storage class.
Refs #21581.
When invoked as follows:
$ python manage.py dumpdata blogapp blogapp.Tag
Django would throw a TypeError. This commit fixes the problem and provides
a test.
ForeignKey or ManyToManyField attribute ``limit_choices_to`` can now
be a callable that returns either a ``Q`` object or a dict.
Thanks michael at actrix.gen.nz for the original suggestion.
Added searched_locations in finders module. Added verbosity flag level 2 on 'findstatic'
command that will output the directories on which it searched the relative paths.
Reported by ccurvey. Initial patch by Jonas Svensson and Vajrasky Kok.
When a method decorator was used in conjunction with a decorator
implemented as a descriptor, method_decorator did not correctly respect
the method binding.
Thanks for Graham Dumpleton for the report and initial patch.
Overriding the error messages now works for both unique fields, unique_together
and unique_for_date.
This patch changed the overriding logic to allow customizing NON_FIELD_ERRORS
since previously only fields' errors were customizable.
Refs #20199.
Thanks leahculver for the suggestion.
This reverts commit 2ee447fb5f.
That commit introduced a regression (#21882) and didn't really
do what it was supposed to: while it did delay the evaluation
of lazy objects passed to mark_safe(), they weren't actually
marked as such so they could end up being escaped twice.
Refs #21882.
Historically, the Django admin used to pass through the request
from an unauthorized access to the login view directly. Now we
are using a proper redirection, which is also preventing
inadvertantly changing data when POSTing login data to an admin
view when user is already authorized.
Thanks Marc Tamlyn and Tim Graham for the reviews.
This commit touchs various parts of the code base and test framework. Any
found usage of opening a cursor for the sake of initializing a connection
has been replaced with 'ensure_connection()'.
Updated SQLUpdateCompiler.execute_sql to match the behavior described in
the docstring; the 'first non-empty query' will now include all queries,
not just the main and first related update.
Added CURSOR and NO_RESULTS result_type constants to make the usages more
self documenting and allow execute_sql to explicitly close the cursor when
it is no longer needed.
Broke InspectDBTestCase.test_field_types in two:
- a test_number_field_types, which now passes on Oracle too
- a test_field_types, for all non-numeric fields, which is still expected to fail
Also made some pep8 fixes in the tests file. Refs #19884
Thanks Tim Graham for review.
The combination of BaseManager.from_queryset() and RenameMethodsBase results in
Manager.__module__ having the wrong value. This can be an issue when trying to
pickle the Manager class.
Wherever possible this filesystem path is derived automatically from the app
module's ``__path__`` and ``__file__`` attributes (this avoids any
backwards-compatibility problems).
AppConfig allows specifying an app's filesystem location explicitly, which
overrides all autodetection based on ``__path__`` and ``__file__``. This
permits Django to support any type of module as an app (namespace packages,
fake modules, modules loaded by other hypothetical non-filesystem module
loaders), as long as the app is configured with an explicit filesystem path.
Thanks Aymeric for review and discussion.
Also document the conditions under which a namespace package may or may not be
a Django app, and raise a clearer error message in those cases where it may not
be.
Thanks Aymeric for review and consultation.
AppStaticStorage only provided one thing over FileSystemStorage, which was
taking an app name (import path) and translating it into a filesystem
path. This is now something that should be done via app_config.path instead,
leaving AppStaticStorage with no reason for existence. It should be safe to
remove, as it was undocumented internal API.
There was some kind of feature in the AppDirectoriesFinder code related to a
"prefix" attribute on the storage class used by AppDirectoriesFinder. Since
this feature was undocumented, untested, and of unclear purpose, I removed it
as well.
During the admin check for list_editable _check_list_editable_item
should return an empty list if all checks pass. Additionally the
Testcase test_readonly_and_editable was changed to test what the
name implies instead of duplicating the logic of test_readonly.
This is the result of Christopher Medrela's 2013 Summer of Code project.
Thanks also to Preston Holmes, Tim Graham, Anssi Kääriäinen, Florian
Apolloner, and Alex Gaynor for review notes along the way.
Also: Fixes#8579, fixes#3055, fixes#19844.
Allowed users to specify which lookups or transforms ("nested lookus")
are available for fields. The implementation is now class based.
Squashed commit of the following:
commit fa7a7195f1952a9c8dea7f6e89ee13f81757eda7
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 18 10:53:24 2014 +0200
Added lookup registration API docs
commit eb1c8ce164325e0d8641f14202e12486c70efdb6
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Tue Jan 14 18:59:36 2014 +0200
Release notes and other minor docs changes
commit 11501c29c9352d17f22f3a0f59d3b805913dedcc
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 20:53:03 2014 +0200
Forgot to add custom_lookups tests in prev commit
commit 83173b960ea7eb2b24d573f326be59948df33536
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:59:12 2014 +0200
Renamed Extract -> Transform
commit 3b18d9f3a1bcdd93280f79654eba0efa209377bd
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 19:51:53 2014 +0200
Removed suggestion of temporary lookup registration from docs
commit 21d0c7631c161fc0c67911480be5d3f13f1afa68
Merge: 2509006 f2dc442
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 09:38:23 2014 -0800
Merge pull request #2 from mjtamlyn/lookups_3
Reworked custom lookups docs.
commit f2dc4429a1da04c858364972eea57a35a868dab4
Author: Marc Tamlyn <marc.tamlyn@gmail.com>
Date: Sun Jan 12 13:15:05 2014 +0000
Reworked custom lookups docs.
Mostly just formatting and rewording, but also replaced the example
using ``YearExtract`` to use an example which is unlikely to ever be
possible directly in the ORM.
commit 250900650628d1f11beadb22814abd666029fb81
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Jan 12 13:19:13 2014 +0200
Removed unused import
commit 4fba5dfaa022653ffa72497258ffd8f8b7476f92
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:34:41 2014 +0200
Added docs to index
commit 6d53963f375c77a1f287833b19b976d23f36c30b
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 22:10:24 2014 +0200
Dead code removal
commit f9cc0390078e21f1ea5a7bc1f15b09f8f6b0904d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 19:00:43 2014 +0200
A new try for docs
commit 33aa18a6e3c831930bda0028222a26f9c1d96e66
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:57:12 2014 +0200
Renamed get_cols to get_group_by_cols
commit c7d5f8661b7d364962bed2e6f81161c1b4f1bcc3
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 14:45:53 2014 +0200
Altered query string customization for backends vendors
The new way is trying to call first method 'as_' + connection.vendor.
If that doesn't exist, then call as_sql().
Also altered how lookup registration is done. There is now
RegisterLookupMixin class that is used by Field, Extract and
sql.Aggregate. This allows one to register lookups for extracts and
aggregates in the same way lookup registration is done for fields.
commit 90e7004ec14e15503f828cc9bde2a7dab593814d
Merge: 66649ff f7c2c0a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:21:01 2014 +0200
Merge branch 'master' into lookups_3
commit 66649ff891c7c73c7eecf6038c9a6802611b5d8a
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Jan 11 13:16:01 2014 +0200
Some rewording in docs
commit 31b8faa62714b4b6b6057a9f5cc106c4dd73caab
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 29 15:52:29 2013 +0200
Cleanup based on review comments
commit 1016159f34674c0df871ed891cde72be8340bb5d
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 18:37:04 2013 +0200
Proof-of-concept fix for #16731
Implemented only for SQLite and PostgreSQL, and only for startswith
and istartswith lookups.
commit 193cd097ca8f2cc6a911e57b8e3fb726f96ee6a6
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 28 17:57:58 2013 +0200
Fixed#11722 -- iexact=F() produced invalid SQL
commit 08ed3c3b49e100ed9019831e770c25c8f61b70f9
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:59:52 2013 +0200
Made Lookup and Extract available from django.db.models
commit b99c8d83c972786c6fcd0e84c9e5cb08c1368300
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 23:06:29 2013 +0200
Fixed review notes by Loic
commit 049eebc0703c151127f4f0265beceea7b8b39e72
Merge: ed8fab7 b80a835
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:53:10 2013 +0200
Merge branch 'master' into lookups_3
Conflicts:
django/db/models/fields/__init__.py
django/db/models/sql/compiler.py
django/db/models/sql/query.py
tests/null_queries/tests.py
commit ed8fab7fe8867ff3eb801c3697a426478387bb2f
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Dec 21 22:47:23 2013 +0200
Made Extracts aware of full lookup path
commit 27a57b7aed91b2f346abc4a77da838bffa17c727
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:10:11 2013 +0200
Removed debugger import
commit 074e0f5aca0572e368c11e6d2c73c9026e7d63d7
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 21:02:16 2013 +0200
GIS lookup support added
commit 760e28e72bae475b442b026650969b0d182dbe53
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 20:04:31 2013 +0200
Removed usage of Constraint, used Lookup instead
commit eac47766844b90e7d3269e7a8c012eee34ec0093
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:22:30 2013 +0200
Minor cleanup of Lookup API
commit 2adf50428d59a783078b0da3d5d035106640c899
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sun Dec 1 02:14:19 2013 +0200
Added documentation, polished implementation
commit 32c04357a87e3727a34f8c5e6ec0114d1fbbb303
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:10:15 2013 +0200
Avoid OrderedDict creation on lookup aggregate check
commit 7c8b3a32cc17b4dbca160921d48125f1631e0df4
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Sat Nov 30 23:04:34 2013 +0200
Implemented nested lookups
But there is no support of using lookups outside filtering yet.
commit 4d219d4cdef21d9c14e5d6b9299d583d1975fcba
Author: Anssi Kääriäinen <akaariai@gmail.com>
Date: Wed Nov 27 22:07:30 2013 +0200
Initial implementation of custom lookups
This (nearly) completes the work to isolate all the test modules from
each other. This is now more important as importing models from another
module will case PendingDeprecationWarnings if those modules are not in
INSTALLED_APPS. The only remaining obvious dependencies are:
- d.c.auth depends on d.c.admin (because of the is_admin flag to some
views), but this is not so important and d.c.admin is in
always_installed_apps
- test_client_regress depends on test_client. Eventually these should
become a single module, as the split serves no useful purpose.
When settings.DATABASES['default']['AUTOCOMMIT'] = False, the connection
wasn't in autocommit mode but Django pretended it was.
Thanks Anssi for analysing this issue.
Refs #17062.
Copied attributes into the decorated method and special case __name__
copy as this will not be present on a Class object. Added regression
test to decorator suite.
Originating WSGIRequests are now attached to the ``wsgi_request`` attribute of
the ``HttpResponse`` returned by the testing client.
Thanks rvdrijst for the suggestion.
Make use of `weakref.finalize` and `weakref.WeakMethod` on python 3.4.
Simplified the removal of receivers, the old function looked overly
complicated.
Many thanks go to Antoine Pitrou for helping me to debug and explain all
the failures I ran into while writing that patch.
Models are now attached to any application they're defined in. Since
not_installed was inside app_loading, these models were mistakenly
attached to app_loading. The test that used them passed accidentally
when run after EggLoadingTest because that class' tearDown() method
replaces apps.all_models['app_loading'] by a copy of itself, while
it should remain the same as apps.app_configs['app_loading'].models.
Surprisingly, this commit doesn't change any behavior at all. When a
model is defined with the same name as another model in the same app,
the definition of the first class is bound to the name of the class
regardless of the definition of the second class.
When django.setup() (and then configure_logging) is called inside
catch_warnings, logging setup is negatively affected (notably
warnings.showwarning definition).
When STATIC_ROOT wasn't set, collectstatic --clear would delete
every files within the current directory and its descendants.
This patch makes the following changes:
Prevent collectstatic from running if STATIC_ROOT isn't set.
Fixed an issue that prevented collectstatic from displaying the
destination directory.
Changed the warning header to notify when the command is run
in dry-run mode.
Now that the refactorings are complete, it isn't particularly useful any
more, nor very well named. Let's keep the API as simple as possible.
Fixed#21689.
Since it triggers imports, it shouldn't be done lightly.
This commit adds a public API for doing it explicitly, django.setup(),
and does it automatically when using manage.py and wsgi.py.
Returning None on errors required unpythonic error checking and was
inconsistent with get_app_config.
get_model was a private API until the previous commit, but given that it
was certainly used in third party software, the change is explained in
the release notes.
Applied the same change to get_registered_model, which is a new private
API introduced during the recent refactoring.
ContentTypes are only created for installed applications, and I could
make a case for not returning a model that isn't installed any more.
The check for stale ContentTypes in update_contenttypes doesn't use
model_class.
ModelSignal actually needs get_registered_model since the lookup happens
at import time. I took this opportunity to perform a small refactoring.
This removes the gap between the master app registry and ad-hoc app
registries created by the migration framework, specifically in terms
of behavior of the get_model[s] methods.
This commit contains a stealth feature that I'd rather not describe.
This is to provide a consistent interface (namely bytes) for the smtp
backend which after all sends bytes over the wire; encoding with as_string
yields different results since mails as unicode are not really specified.
as_string stays for backwardscompatibilty mostly and some debug outputs.
But keep in mind that the output doesn't match as_bytes!
The last component of the dotted path to the application module is
consistently referenced as the application "label". For instance it's
AppConfig.label. appname could be confused with AppConfig.name, which is
the full dotted path.
* Introduced [un]set_installed_apps to handle changes to the
INSTALLED_APPS setting.
* Refactored [un]set_available_apps to share its implementation
with [un]set_installed_apps.
* Implemented a receiver to clear some app-related caches.
* Removed test_missing_app as it is basically impossible to reproduce
this situation with public methods of the new app cache.
Currently such overrides aren't reflected in the app cache.
It would be possible to handle them. But that doesn't look like a very
good API. It makes it complicated to express "add this app" and "remove
this app", which are the most common operations on INSTALLED_APPS.
- Tested consistency the current app_configs instead of INSTALLED_APPS.
- Considered applications added with _with_app as available.
- Added docstrings.
Adjusted several tests that used it to add apps to the app cache and
then attempted to remove them by manipulating attributes directly.
Also renamed invalid_models to invalid_models_tests to avoid clashing
application labels between the outer and the inner invalid_models
applications.
It was called _populate() before I renamed it to populate(). Since it
has been superseded by populate_models() there's no reason to keep it.
Removed the can_postpone argument of load_app() as it was only used by
populate(). It's a private API and there's no replacement. Simplified
load_app() accordingly. Then new version behaves exactly like the old
one even though it's much shorter.
First stage imports app modules. It doesn't catch import errors. This
matches the previous behavior and keeps the code simple.
Second stage import models modules. It catches import errors and retries
them after walking through the entire list once. This matches the
previous behavior and seems useful.
populate_models() is intended to be equivalent to populate(). It isn't
wired yet. That is coming in the next commit.
Since applications that aren't installed no longer have an application
configuration, it is now always True in practice.
Provided an abstraction to temporarily add or remove applications as
several tests messed with app_config.installed to achieve this effect.
For now this API is _-prefixed because it looks dangerous.
Got rid of AppConfig._stub. As a side effect, app_cache.app_configs now
only contains entries for applications that are in INSTALLED_APPS, which
is a good thing and will allow dramatic simplifications (which I will
perform in the next commit). That required adjusting all methods that
iterate on app_configs without checking the "installed" flag, hence the
large changes in get_model[s].
Introduced AppCache.all_models to store models:
- while the app cache is being populated and a suitable app config
object to register models isn't available yet;
- for applications that aren't in INSTALLED_APPS since they don't have
an app config any longer.
Replaced get_model(seed_cache=False) by registered_model() which can be
kept simple and safe to call at any time, and removed the seed_cache
argument to get_model[s]. There's no replacement for that private API.
Allowed non-master app caches to go through populate() as it is now
safe to do so. They were introduced in 1.7 so backwards compatibility
isn't a concern as long as the migrations framework keeps working.
* Removed ADMIN_FOR setting and warn warning
* Group view functions by namespace instead of site
* Added a test verifying namespaces are listed
Thanks to Claude Paroz for reviewing and ideas for improvement.
Improved Andrew's hack to create temporary app caches to handle
migrations. Now the main app cache has a "master" flag set to True
(which is a non-default keyword argument, thus unlikely to be used by
mistake). Other app cache instances have "master" set to False.
The only sanctioned way to access the app cache is by importing
django.core.apps.app_cache.
If you were instanciating an app cache and relying on the Borg pattern,
you'll have to refactor your code.
Added comments in the three empty models.py files that are still needed.
Adjusted the test runner to add applications corresponding to test
labels to INSTALLED_APPS even when they don't have a models module.
Several parts of Django call get_apps() with a comment along this lines
of "this has the side effect of calling _populate()". I fail to see how
this is better than just calling populate()!
Refactored get_app() to rely on that method.
get_app() starts by calling _populate(), which goes through
INSTALLED_APPS and, for each app, imports the app module and attempts to
import the models module. At this point, no further imports are
necessary to return the models module for a given app. Therefore, the
implementation of get_app() can be simplified and the safeguards for
race conditions can be removed.
Besides, the emptyOK parameter isn't used anywhere in Django. It was
introduced in d6c95e93 but not actually used nor documented, and it has
just been carried around since then. Since it's an obscure private API,
it's acceptable to stop supporting it without a deprecation path. This
branch aims at providing first-class support for applications without a
models module eventually.
For backwards-compatibility, get_app() still raises ImproperlyConfigured
when an app isn't found, even though LookupError is technically more
correct. I haven't gone as far as to preserve the exact error messages.
I've adjusted a few tests instead.
This commit is a refactoring with no change of functionality, according
to the following invariants:
- An app_label that was in app_configs and app_models stays in
app_config and has its 'installed' attribute set to True.
- An app_label that was in app_models but not in app_configs is added to
app_configs and has its 'installed' attribute set to True.
As a consequence, all the code that iterated on app_configs is modified
to check for the 'installed' attribute. Code that iterated on app_models
is rewritten in terms of app_configs.
Many tests that stored and restored the state of the app cache were
updated.
In the long term, we should reconsider the usefulness of allowing
importing models from non-installed applications. This doesn't sound
particularly useful, can be a trap in some circumstances, and causes
significant complexity in sensitive areas of Django.
Since the original ones in django.db.models.loading were kept only for
backwards compatibility, there's no need to recreate them. However, many
internals of Django still relied on them.
They were also imported in django.db.models. They never appear in the
documentation, except a quick mention of get_models and get_app in the
1.2 release notes to document an edge case in GIS. I don't think that
makes them a public API.
This commit doesn't change the overall amount of global state but
clarifies that it's tied to the app_cache object instead of hiding it
behind half a dozen functions.
All input is now coerced to text before being normalized.
This changes nothing under Python 2 but it allows bytes
to be passed to the function without a TypeError under Python3
(bytes are assumed to be utf-8 encoded text).
Thanks to trac user vajrasky for the report.
Current language is no longer saved to session by LocaleMiddleware
on every response (the behavior introduced in #14825).
Instead language stored in session is reintroduced into new session
after logout.
Forward port of c558a43fd6 to master.
Previously when collecting static files, the directories would receive permissions
from the global umask. Now the default permission comes from FILE_UPLOAD_DIRECTORY_PERMISSIONS
and there's an option to specify the permissions by subclassing any of the
static files storage classes and setting the directory_permissions_mode parameter.
The `remove()` and `clear()` methods of the related managers created by
`ForeignKey`, `GenericForeignKey`, and `ManyToManyField` suffered from a
number of issues. Some operations ran multiple data modifying queries without
wrapping them in a transaction, and some operations didn't respect default
filtering when it was present (i.e. when the default manager on the related
model implemented a custom `get_queryset()`).
Fixing the issues introduced some backward incompatible changes:
- The implementation of `remove()` for `ForeignKey` related managers changed
from a series of `Model.save()` calls to a single `QuerySet.update()` call.
The change means that `pre_save` and `post_save` signals aren't called anymore.
- The `remove()` and `clear()` methods for `GenericForeignKey` related
managers now perform bulk delete so `Model.delete()` isn't called anymore.
- The `remove()` and `clear()` methods for `ManyToManyField` related
managers perform nested queries when filtering is involved, which may
or may not be an issue depending on the database and the data itself.
Refs. #3871, #21174.
Thanks Anssi Kääriäinen and Tim Graham for the reviews.
Updated ModelAdmin to use form.instance when passing parent model to
child inlines for add_view. There is effectively no change in the
change_view since the previously passed 'obj' is the same as form.instance.
Thanks to meshy for report, and EvilDMP and timo for review.
A test for annotations incorrectly assumed that the first instance
(in the test) of a model using AutoField for PK will always get pk=1.
The test was changed to compare against actual instance id instead.
Fixed#20187 -- Allowed repeated iteration of HttpResponse.
All this became possible when support for old-style streaming responses was
finally removed.
Removed multiple locales separated by commas variation (that wasn't
working as documented) in favor of simply allowing use of the
``--locale``/``-l`` options more than once for ``makemessages`` and
``compilemessages``.
Thanks Romain Beylerian for the report and Claude, Simon for their help.
8750296918072c97a51f6809d19828ce3f1b8d40 from stable/1.6.x.
Thanks Curtis Malony and Florian Apolloner.
Squashed commit of the following:
commit 3380495e93f5e81b80a251b03ddb0a80b17685f5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:18:07 2013 +0100
Looked up the template_fragments cache at runtime.
commit 905a74f52b24a198f802520ff06290a94dedc687
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 14:19:48 2013 +0100
Removed all uses of create_cache.
Refactored the cache tests significantly.
Made it safe to override the CACHES setting.
commit 35e289fe9285feffed3c60657af9279a6a2cfccc
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:23:57 2013 +0100
Removed create_cache function.
commit 8e274f747a1f1c0c0e6c37873e29067f7fa022e8
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Nov 23 12:04:52 2013 +0100
Updated docs to describe a simplified cache backend API.
commit ee7eb0f73e6d4699edcf5d357dce715224525cf6
Author: Curtis Maloney <curtis@tinbrain.net>
Date: Sat Oct 19 09:49:24 2013 +1100
Fixed#21012 -- Thread-local caches, like databases.
Requests made with django.test.Client.login() and logout() respect
defaults defined in django.test.Client instantiation and are processed
through middleware.
Thanks to Loic for the reviews.
This would go unnoticed by the test suite because receivers are removed
automatically when they are garbage collected.
Changed all Signal.connect calls to hold strong references to ensure we
clean up after ourselves.
The GenericRelation refactoring removed GenericRelations from
model._meta.many_to_many. This had the side effect of disallowing
editable GenericRelations in ModelForms. Editable GenericRelations
aren't officially supported, but if we don't fix this we don't offer any
upgrade path for those who used the ability to set editable=True
in GenericRelation subclass.
Thanks to Trac alias joshcartme for the report and stephencmd and Loic
for working on this issue.
Also added some tests for HttpRequest.__repr__.
Note that the added tests don't actually catch the accidental code
removal (see ticket) but they do cover a codepath that wasn't tested
before.
Thanks to Tom Christie for the report and the original patch.
Replaced the custom, untested memoize with a similar decorator from Python's
3.2 stdlib. Although some minor performance degradation (see ticket), it is
expected that in the long run lru_cache will outperform memoize once it is
implemented in C.
Thanks to EvilDMP for the report and Baptiste Mispelon for the idea of
replacing memoize with lru_cache.
``get_template_from_string`` default arguments were breaking
``assertTemplateUsed``. The solution has been to return only the names of the
templates with a ``name`` attribute distinct of ``None``. The default ``name``
kwarg of ``Template`` has been changed to ``None``, more pythonic than ``'<Unknown
Template>'``.
Add the trimmed option to the blocktrans tag to trim any newlines and
whitespace from its content.
This allows the developer to indent the blocktrans tag without adding
new lines and whitespace to the msgid in the PO file.
Thanks to mpessas for the initial patch and Dmitri Fedortchenko for the
report.
Scheme is handled correctly when making comparisons between two URLs. If
there isn't any scheme specified in the location where we are redirected to,
the original request's scheme is used. If present, the scheme in
``expected_url`` is the one used to make the comparations to.
* Safer for use in multiprocess environments
* Better random culling
* Cache files use less disk space
* Safer delete behavior
Also fixed#15806, fixed#15825.
This patch introduces the Prefetch object which allows customizing prefetch
operations.
This enables things like filtering prefetched relations, calling select_related
from a prefetched relation, or prefetching the same relation multiple times
with different querysets.
When a Prefetch instance specifies a to_attr argument, the result is stored
in a list rather than a QuerySet. This has the fortunate consequence of being
significantly faster. The preformance improvement is due to the fact that we
save the costly creation of a QuerySet instance.
Thanks @akaariai for the original patch and @bmispelon and @timgraham
for the reviews.
This commit introduced a new class JoinPromoter that can be used to
abstract away join promotion problems for complex filter conditions.
Query._add_q() and Query.combine() now use the new class.
Also, added a lot of comments about why join promotion is done the way
it is.
Thanks to Tim Graham for original report and testing the changes, and
for Loic Bistuer for review.
Language codes for Chinese are zh_Hans (Simplified) and zh_Hant (Traditional).
Added support for browsers that still send the deprecated language codes.
Thanks to Olli Wang for the report.
model_to_dict() (used when rendering forms) queries the database
to get the list of primary keys for ManyToMany fields. This is
unnecessary if the field queryset has been prefetched, all the
keys are already in memory and can be obtained with a simple
iteration.
The timezone arithmetic done in JS can be off by one hour around DST
change. We work around this issue by adding one extra hour to the test
error margin when we detect a DST change is near.
Refs #20663.
All request methods of ``django.test.client.Client`` receive a ``secure``
argument that defaults to ``False`` indicating whether or not to make the
request through https.
Thanks Aymeric Augustin for the review.
When a formset contained deletion for an existing instance, and the
instance was already deleted, django threw an exception. A common cause for
this was resubmit of the formset.
Original patch by Trac alias olau.
In addition this commit cleaned some code in _construct_form(). This
was needed as the primary key value the user submitted wasn't converted
correctly to python value in case the primary key field was also a
related field.
``ModelAdmin.view_on_site`` defines wether to show a link to the object on the
admin detail page. If ``True``, cleverness (i.e. ``Model.get_absolute_url``) is
used to get the url. If it's a callable, the callable is called with the object
as the only parameter. If ``False``, not link is displayed.
With the aim of maitaining backwards compatibility, ``True`` is the default.
We need to make sure content read from the file is decoded from UTF-8
right from the start so Python doesn't try to use another encoding
(read: ASCII/CP1252 under Windows.)
Previously, when collecting static files, the files would receive permission
from FILE_UPLOAD_PERMISSIONS. Now, there's an option to give different
permission from uploaded files permission by subclassing any of the static
files storage classes and setting the file_permissions_mode parameter.
Thanks dblack at atlassian.com for the suggestion.
Change strategy used to examine instrumented output to acommodate the
fact that on Windows, where the path separator is '\', repr() of Python
modules has changed in Python 3 to use escaped backslashes in the FS
path section (e.g.
'C:\\python3x\\Lib\\site-packages\\django\\contrib\\auth\\models.py')
without having to special-case based on platform.
Thanks dan at dlo.me for the initial patch.
- Added __pow__ and __rpow__ to ExpressionNode
- Added oracle and mysql specific power expressions
- Added used-defined power function for sqlite
The old 'django_language' variable will still be read from in order
to migrate users. The backwards-compatability shim will be removed in
Django 1.8.
Thanks to jdunck for the report and stugots for the initial patch.
This shows itself with Python 3 under Windows where UTF-8 usually isn't
the default file I/O encoding and caused one failure and three errors
in our test suite under that platform setup.
When listing available management commands, only core commands are
listed if settings have any error. This commit adds a note in this
case so errors are not totally silently skipped.
Thanks Peter Davis for the report.
select_related('foo').select_related('bar') is now equivalent to
select_related('foo', 'bar').
Also reworded docs to recommend select_related(*fields) over select_related()
When both parent and child classes are decorated with override_settings,
child class settings should take precedence.
Thanks Sephi for the report and Marc Tamlyn for the review.
The idea is that if an object implements __html__ which returns a string this is
used as HTML representation (eg: on escaping). If the object is a str or unicode
subclass and returns itself the object is a safe string type.
This is an updated patch based on jbalogh and ivank patches.
Added ``--natural-foreign`` and ``--natural-primary`` options and
deprecated the ``--natural`` option to the ``dumpdata`` management
command.
Added ``use_natural_foreign_keys`` and ``use_natural_primary_keys``
arguments and deprecated the ``use_natural_keys`` argument to
``django.core.serializers.Serializer.serialize()``.
Thanks SmileyChris for the suggestion.
Don't set a global default interpreted role function for reStructuredText.
Instead, use the `default-role` directive to change the default only within
the `parse_rst()` function.
Thanks Malcolm Tredinnick for the report.
Literals from source files with Django template language syntax don't
have a '.py' suffix anymore.
Also, the '.\' prefix is preserved to respect GNU gettext behavior on
that platform.
Refs #16903.
Squashed commit of the following:
commit 63ddb271a44df389b2c302e421fc17b7f0529755
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 29 22:51:00 2013 +0200
Clarified interactions between atomic and exceptions.
commit 2899ec299228217c876ba3aa4024e523a41c8504
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:45:32 2013 +0200
Fixed TransactionManagementError in tests.
Previous commit introduced an additional check to prevent running
queries in transactions that will be rolled back, which triggered a few
failures in the tests. In practice using transaction.atomic instead of
the low-level savepoint APIs was enough to fix the problems.
commit 4a639b059ea80aeb78f7f160a7d4b9f609b9c238
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Tue Sep 24 22:24:17 2013 +0200
Allowed nesting constraint_checks_disabled inside atomic.
Since MySQL handles transactions loosely, this isn't a problem.
commit 2a4ab1cb6e83391ff7e25d08479e230ca564bfef
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sat Sep 21 18:43:12 2013 +0200
Prevented running queries in transactions that will be rolled back.
This avoids a counter-intuitive behavior in an edge case on databases
with non-atomic transaction semantics.
It prevents using savepoint_rollback() inside an atomic block without
calling set_rollback(False) first, which is backwards-incompatible in
tests.
Refs #21134.
commit 8e3db393853c7ac64a445b66e57f3620a3fde7b0
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Sep 22 22:14:17 2013 +0200
Replaced manual savepoints by atomic blocks.
This ensures the rollback flag is handled consistently in internal APIs.
Previously, if a database request spanned a related object manager, the
first manager encountered would cause a request to the router, and this
would bind all subsequent queries to the same database returned by the
router. Unfortunately, the first router query would be performed using
a read request to the router, resulting in bad routing information being
used if the subsequent query was actually a write.
This change defers the call to the router until the final query is acutally
made.
It includes a small *BACKWARDS INCOMPATIBILITY* on an edge case - see the
release notes for details.
Thanks to Paul Collins (@paulcollinsiii) for the excellent debugging
work and patch.
This change prevents including the multiple_database test models without
duplicating the router code (we probably should do this at one point).
Refs #21148
Currently module level queries are executed against the real database
(specified in NAME) instead of the test database; since it is to late
to fix this for 1.6, we at least ensures stable builds. Refs #21443.
Our WSGIServer rewrapped the socket errors from server_bind into
WSGIServerExceptions, which is used later on to provide nicer
error messages in runserver and used by the liveserver to see if
the port is already in use. But wrapping server_bind isn't enough since
it only binds to the socket, socket.listen (which is called from
server_activate) could also raise "Address already in use".
Instead of overriding server_activate too I chose to just catch socket
errors, which seems to make more sense anyways and should be more robust
against changes in wsgiref.
Fields must render values in the current time zone.
This commit only contains tests because this ticket was just a symptom of
a regression from #18777 that was fixed separately.
We want to be able to use it for instance for discovering `tasks.py` modules
inside the INSTALLED_APPS.
This commit therefore moves the logic to `autodiscover_modules` method in
django.utils.module_loading.
This commit deprecates django.utils.tzinfo in favor of the more recent
django.utils.timezone which was introduced when Django gained support
for time zones.
* override_settings may now be imported from django.test
* removed Approximate from django.test
* updated documentation for things importable from django.test
Thanks akaariai for the suggestion.
Passed large maxlinelen to email.Header to prevent newlines from being
inserted into value returned by _convert_to_charset
Thanks mjl at laubach.at for the report.
Thanks mitsuhiko for the report.
Non-ASCII values are supported. Non-ASCII keys still aren't, because the
current parser mangles them. That's another bug.
Fixed#21058 -- Fixed debug view blowing up when no template is provided to the template rendering functions.
Assistance on this commit from @jambonrose.
The Admin widget, which can be used to filter multiple selects
can sometimes be too narrow and hide information such as
user permissions. This commit adds tooltips to the select
options so that a user can hover over and see the hidden text.
- TemporaryFile now minimally mocks the API of the Python standard
library class tempfile.NamedTemporaryFile to avoid AttributeError
exceptions.
- The symbol django.core.files.NamedTemporaryFile is actually assigned
as a different class on different operating systems.
- The bug only occurred if Django is running on Windows, hence why it
was hard to diagnose.
Merged BaseIncludeNode, ConstantIncludeNode and Include node.
This avoids raising TemplateDoesNotExist at parsing time, allows recursion
when passing a literal template name, and should make TEMPLATE_DEBUG behavior
consistant.
Thanks loic84 for help with the tests.
Fixed#3544, fixed#12064, fixed#16147
The option can be used to force pre 1.6 style SELECT on save behaviour.
This is needed in case the database returns zero updated rows even if
there is a matching row in the DB. One such case is PostgreSQL update
trigger that returns NULL.
Reviewed by Tim Graham.
Refs #16649
Modified django.utils.dateformat module, moving __init__() method and
timezone-related format methods from DateFormat class to TimeFormat
base class. Modified timezone-related format methods to return an
empty string when timezone is inappropriate for input value.
This only affests reporting not data collection and allows coverage.py
to succesfully finish the generation of e.g. the HTML report.
Cases of code whose execution data is collected during the run phase but
for which no associated source code files can be found at the reporting
phase include tests with egg files and Python module files created at
test execution-time.
Added settings.SESSION_SERIALIZER which is the import path of a serializer
to use for sessions.
Thanks apollo13, carljm, shaib, akaariai, charettes, and dstufft for reviews.
The view tried to display links to a ModelAdmin's change_view, which
resulted in NoReverseMatches if get_urls was overridden to remove the
corresponding url.
The __eq__ method now considers two instances without primary key value
equal only when they have same id(). The __hash__ method raises
TypeError for no primary key case.
Fixed#18864, fixed#18250
Thanks to Tim Graham for docs review.
In cases where the same connection (from model A to model B along the
same field) was needed multiple times in a select_related query, the
join setup code mistakenly reused an existing join.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
There was an old complaint about nullable one-to-one field cascading
even when the o2o field was saved to None value before the deletion.
Added an test to verify this doesn't happen.
Also some PEP 8 cleanup.
Cleaned up the internal implementation of m2m fields by removing
related.py _get_fk_val(). The _get_fk_val() was doing the wrong thing
if asked for the foreign key value on foreign key to parent model's
primary key when child model had different primary key field.
If LEFT JOINs are required for correct results, then trimming the join
can lead to incorrect results. Consider case:
TBL A: ID | TBL B: ID A_ID
1 1 1
2
Now A.order_by('b__a') did use a join to B, and B's a_id column. This
was seen to contain the same value as A's id, and so the join was
trimmed. But this wasn't correct as the join is LEFT JOIN, and for row
A.id = 2 the B.a_id column is NULL.
There was an old complaint about nullable one-to-one field cascading
even when the o2o field was saved to None value before the deletion.
Added an test to verify this doesn't happen.
Also some PEP 8 cleanup.
The default delete action resulted in a NoReverseMatch if it were to
list any Model with a ModelAdmin with `get_urls` overridden to remove
the change url. Catching the error and not displaying the link in that
case, as was already done for models with no registered admins.
Thanks Keryn Knight for the report.
While using USE_L10N, line numbers and IDs were printed as comma (or
locale equivalent) separated values.
Thanks Kronuz for the report and intial patch.
Fixes#20861.
The bug was already fixed by 01b9c3d519,
so only tests added.
At the same time promote_joins()'s uncoditional flag is gone, it isn't
needed for anything any more.
The `MultiValueField` class gets a new ``require_all_fields`` argument that
defaults to ``True``. If set to ``False``, individual fields can be made
optional, and a new ``incomplete`` validation error will be raised if any
required fields have empty values.
The ``incomplete`` error message can be defined on a `MultiValueField`
subclass or on each individual field. Skip duplicate errors.
The documentation promises that host validation is disabled when
DEBUG=True, that all hostnames are accepted. Domains not compliant with
RFC 1034/1035 were however being validated, this validation has now been
removed when DEBUG=True.
Additionally, when DEBUG=False a more detailed SuspiciousOperation
exception message is provided when host validation fails because the
hostname is not RFC 1034/1035 compliant.
All Promise objects were passed to force_text() deep in ORM query code.
Not only does this make it difficult or impossible for developers to
prevent or alter this behaviour, but it is also wrong for non-text
fields.
This commit changes `Field.get_prep_value()` from a no-op to one that
resolved Promise objects. All subclasses now call super() method first
to ensure that they have a real value to work with.
It has been possible to use models of wrong type in related field
lookups. For example pigs__in=[a_duck] has worked. Changes to
ForeignObject broke that.
It might be a good idea to restrict the model types usable in lookups.
This should be done intentionally, not accidentally and without any
consideration for deprecation path.
Additionally this patch solves the orthogonal problem that specialized
`QuerySet` like `ValuesQuerySet` didn't inherit from the current `QuerySet`
type. This wasn't an issue until now because we didn't officially support
custom `QuerySet` but it became necessary with the introduction of this new
feature.
Thanks aaugustin, akaariai, carljm, charettes, mjtamlyn, shaib and timgraham
for the reviews.
1. Automatically use tests/../django as the Django version.
2. If settings aren't provided through --settings or DJANGO_SETTINGS_MODULE)
then use test_sqlite.
"Fixed" by adding a test case, the original problem was already fixed
by earlier ORM changes. Thanks to david@judicata.com for the report
and test case.
In the combination of .values().aggregate() the aggregate_select_mask
didn't include the aggregates added. This resulted in bogus query.
Thanks to Trac alias debanshuk for report.
Some feed aggregators make use of the `published` element as well as
the `updated` element (within the Atom standard -- http://bit.ly/2YySb).
The standard allows for these two elements to be present in the same
entry. `Atom1Feed` had implemented the `updated` element which was
incorrectly taking the date from `pubdate`.
It doesn't work as one might expect on a certain database backend where
autocommits_when_autocommit_is_off = True. That backend happens to be
popular for running tests.
With the new test discovery system, gis tests are discovered as
other tests. They should be properly skipped now when dependencies
are missing. So let's stop special casing their inclusion.
Add support for Oracle, fix an issue with the repr of RawQuerySet,
add tests and documentations. Also added a 'supports_paramstyle_pyformat'
database feature, True by default, False for SQLite.
Thanks Donald Stufft for review of documentation.
This is useful:
- to force a rollback on the exit of an atomic block without having to
raise and catch an exception;
- to prevent a rollback after handling an exception manually.
BaseModelFormSet.forms is now a cached property instead of being
populated in the __init__ method. This behaviour also matches an
example in the documentation.
Thanks Thomasz Swiderski for the report and Simon Charette for the
review.
This attribute is only allowed on inputs of type "text", "search", "url",
"tel", "email", or "password".
Thanks to yoyoma for the report and @bmispelon for the review.
The join used by select_related was incorrectly INNER when the query
had an ORed filter for nullable join that was trimmed away. Fixed this
by forcing the join type to LOUTER even when a join was trimmed away
in ORed queries.
This reverts commit 64041f0e6e.
lookup.tests.LookupTests.test_regex_non_string fails under Postgres.
We should also try to rewrite the test using an existing model.
There was an inconsistency between how the label_tag for forms were
generated depending on which method was used: as_p, as_ul and as_table
contained code to append the label_suffix where as label_tag called on a
form field directly did NOT append the label_suffix. The code for
appending the label_suffix has been moved in to the label_tag code of
the field and the HTML generation code for as_p, as_ul and as_table now
calls this code as well.
This is a backwards incompatible change because users who have added the
label_suffix manually in their templates may now get double label_suffix
characters in their forms.
The patch for #19385 caused a regression in certain generic relations
.exclude() filters if a subquery was needed. The fix contains a
refactoring to how Query.split_exclude() and Query.trim_start()
interact.
Thanks to Trac alias nferrari for the report.
It's useful to be able to list all the (flattened) keys of a
ContextList, to help you figure out why the variable that's supposed
to be there is not.
No .values() or .items() added as the definition for those aren't clear.
The patch is Chris Wilson's patch from pull request 1065 with some
modifications by committer.
Correctly calculate the ``aggregate_start`` offset from loaded fields,
if any are deferred, instead of ``self.query.select`` which includes all
fields on the model.
Also made some PEP 8 fixes.
The SubqueryConstraint defined relabeled_clone(), but that was never
called. Instead there is now clone() and relabel_aliases() methods for
SubqueryConstraint.
A related problem was that SubqueryConstraint didn't correctly use
quote_name_unless_alias() of the outer query. This resulted in failures
when running under PostgreSQL.
SuspiciousOperations have been differentiated into subclasses, and
are now logged to a 'django.security.*' logger. SuspiciousOperations
that reach django.core.handlers.base.BaseHandler will now return a 400
instead of a 500.
Thanks to tiwoc for the report, and Carl Meyer and Donald Stufft
for review.
Allows a `GenericForeignKey` to reference proxy models. The default
for `for_concrete_model` is `True` to keep backwards compatibility.
Also added the analog `for_concrete_model` kwarg to
`generic_inlineformset_factory` to provide an API at the form level.
Don't try to be smart about building a good-looking help string
because it evaluates translations too early, simply use the same old
strategy as before. Thanks Donald Stufft for the report.
Also, actually fix the case reported by the OP by special-casing
CheckboxSelectMultiple.
Added tests.
Refs #9321.
The regex method used until now for the strip_tags utility is fast,
but subject to flaws and security issues. Consensus and good
practice lead use to use a slower but safer method.
When an exception other than IntegrityError was raised, get_or_create
could fail and leave the database connection in an unusable state.
Thanks UloPe for the report.
A regression caused by d5b93d3281 made .get() error
reporting recurse infinitely on certain rare conditions. Fixed this by
not trying to print the given lookup kwargs.
When a GenericRelation was defined on abstract model, queries on childs
of the abstract model didn't work. The problem was in the way fields and
in particular field.rel was copied from models to their children.
The regression was likely caused by #19385. Thanks to Gavin Wahl for
spotting the regression.
The failure was caused by generating the same warning from two tests.
The second time the same warning was raised it was swallowed by the
"once" simplefilter of warnings.
A decorator is easier to apply to CBVs. Backwards compatibility isn't an
issue here, except for people running on a recent clone of master.
Fixed a few minor problems in the transactions docs while I was there.
If the fixture doesn't exist, loaddata will output a warning.
The fixture named "initial_data" is exceptional though; if it
doesn't exist, the warning is not emitted. This allows syncdb and
flush management commands to attempt to load it without causing
spurious warnings.
Thanks to Derega, ptone, dirigeant and d1ffuz0r for contributions
to the ticket.
The fixture named "initial_data" is exceptional though; if it
doesn't exist, the error is not raised. This allows syncdb and
flush management commands to attempt to load it without causing
an error if it doesn't exist.
Thanks to Preston Timmons for the bulk of the work on the patch, especially
updating Django's own test suite to comply with the requirements of the new
runner. Thanks also to Jannis Leidel and Mahdi Yusuf for earlier work on the
patch and the discovery runner.
Refs #11077, #17032, and #18670.
This also updates all dependent functionality, including modelform_factory
and modelformset_factory, and the generic views `ModelFormMixin`,
`CreateView` and `UpdateView` which gain a new `fields` attribute.
Also, use Django templating for the dynamic generated JS code and use
more idiomatic coding techniques.
Thanks Matthew Tretter for the report and the patch.
I refactored RadioSelect and CheckboxSelectMultiple to
make them inherit from a base class, allowing them to share
the behavior of being able to iterate over their subwidgets.
Thanks to Matt McClanahan for the initial patch and to
Claude Paroz for the review.
Queries can contain binary data undecodable with utf-8. In this
case, using the 'replace' errors mode when decoding seems like
an acceptable representation of the query.
Thanks Marcel Ryser for the report.
This patch iproved two major parts in Django. First, the fields.related
was refactored. The main addition there was ForeignObject. Second, the
ORM now handles multicolumn joins in most cases, though there are still
cases that do not work correcly (split_exclude() for example).
In addition there were extesive changes to how GenericRelation works.
Before it was a fake m2m field, now it is a pure virtual fields and is
based on ForeignObject.
There is still much room for improvement. The related fields code is
still somewhat confusing, and how fields are represented in model._meta
should also be revisited.
This patch was written mostly by Jeremy Tillman with some final polish
by the committer.
Previously, depending on the database backend or the cursor type,
you'd need to double the percent signs in the query before passing
it to cursor.execute. Now cursor.execute consistently need percent
doubling whenever params argument is not None (placeholder substitution
will happen).
Thanks Thomas Güttler for the report and Walter Doekes for his work
on the patch.
This is provided as a new "validate_max" formset_factory option defaulting to
False, since the non-validating behavior of max_num is longstanding, and there
is certainly code relying on it. (In fact, even the Django admin relies on it
for the case where there are more existing inlines than the given max_num). It
may be that at some point we want to deprecate validate_max=False and
eventually remove the option, but this commit takes no steps in that direction.
This also fixes the DoS-prevention absolute_max enforcement so that it causes a
form validation error rather than an IndexError, and ensures that absolute_max
is always 1000 more than max_num, to prevent surprising changes in behavior
with max_num close to absolute_max.
Lastly, this commit fixes the previous inconsistency between a regular formset
and a model formset in the precedence of max_num and initial data. Previously
in a regular formset, if the provided initial data was longer than max_num, it
was truncated; in a model formset, all initial forms would be displayed
regardless of max_num. Now regular formsets are the same as model formsets; all
initial forms are displayed, even if more than max_num. (But if validate_max is
True, submitting these forms will result in a "too many forms" validation
error!) This combination of behaviors was chosen to keep the max_num validation
simple and consistent, and avoid silent data loss due to truncation of initial
data.
Thanks to Preston for discussion of the design choices.
The ticket dealt with a case where one query had .exclude() that
produced a subquery, the other query had a join to the same model that
was subqueried in the first query. This was already fixed in master, so
only test added.
This reverts commit 2cd0edaa47.
This commit was the cause of a memory leak. See ticket for more details.
Thanks Anssi Kääriäinen for identifying the source of the bug.
Before this change, the get_admin_log method would expect User model's
FK to be named `id`. When changing that FK name, admin/index.html
rendering would fail.
This includes:
* Changed the use of id for the use of pk property.
* Added a regression test that fails without the patch.
This commit refs #20088.
The original problem was that when filtering two levels up in
inheritance chain, Django optimized the join generation so that the
middle model was skipped. But then Django generated joins from top
to middle to bottom for SELECT clause, and thus there was one extra
join (top->middle->bottom + top -> bottom).
This case is fixed in master as the filtering optimization is gone.
This has the side effect that in some cases there is still extra join
if the SELECT clause doesn't contain anything from middle or bottom.
Model.save() will use UPDATE - if not updated - INSERT instead of
SELECT - if found UPDATE else INSERT. This should save a query when
updating, but will cost a little when inserting model with PK set.
Also fixed#17341 -- made sure .save() commits transactions only after
the whole model has been saved. This wasn't the case in model
inheritance situations.
The save_base implementation was refactored into multiple methods.
A typical chain for inherited save is:
save_base()
_save_parents(self)
for each parent:
_save_parents(parent)
_save_table(parent)
_save_table(self)
Thanks Anssi for haggling until I implemented this.
This change alleviates the need for atomic_if_autocommit. When
autocommit is disabled for a database, atomic will simply create and
release savepoints, and not commit anything. This honors the contract of
not doing any transaction management.
This change also makes the hack to allow using atomic within the legacy
transaction management redundant.
None of the above will work with SQLite, because of a flaw in the design
of the sqlite3 library. This is a known limitation that cannot be lifted
without unacceptable side effects eg. triggering arbitrary commits.
The sql/query.py add_q method did a lot of where/having tree hacking to
get complex queries to work correctly. The logic was refactored so that
it should be simpler to understand. The new logic should also produce
leaner WHERE conditions.
The changes cascade somewhat, as some other parts of Django (like
add_filter() and WhereNode) expect boolean trees in certain format or
they fail to work. So to fix the add_q() one must fix utils/tree.py,
some things in add_filter(), WhereNode and so on.
This commit also fixed add_filter to see negate clauses up the path.
A query like .exclude(Q(reversefk__in=a_list)) didn't work similarly to
.filter(~Q(reversefk__in=a_list)). The reason for this is that only
the immediate parent negate clauses were seen by add_filter, and thus a
tree like AND: (NOT AND: (AND: condition)) will not be handled
correctly, as there is one intermediary AND node in the tree. The
example tree is generated by .exclude(~Q(reversefk__in=a_list)).
Still, aggregation lost connectors in OR cases, and F() objects and
aggregates in same filter clause caused GROUP BY problems on some
databases.
Fixed#17600, fixed#13198, fixed#17025, fixed#17000, fixed#11293.
Before there was need to have both .relabel_aliases() and .clone() for
many structs. Now there is only relabeled_clone() for those structs
where alias is the only mutable attribute.
Replaced them with per-database options, for proper multi-db support.
Also toned down the recommendation to tie transactions to HTTP requests.
Thanks Jeremy for sharing his experience.
Since "unless managed" now means "if database-level autocommit",
committing or rolling back doesn't have any effect.
Restored transactional integrity in a few places that relied on
automatically-started transactions with a transitory API.
For users who didn't activate autocommit in their database options, this
is backwards-incompatible in "non-managed" aka "auto" transaction state.
This state now uses database-level autocommit instead of ORM-level
autocommit.
Also removed the uses_autocommit feature which lost its purpose.
Autocommit cannot be manipulated independently from an open connection.
This commit introduces a minor change in behavior: entering transaction
management forces opening a databasse connection. This shouldn't be
backwards incompatible in any practical use case.
enter_transaction_management() was nearly always followed by managed().
In three places it wasn't, but they will all be refactored eventually.
The "forced" keyword argument avoids introducing behavior changes until
then.
This is mostly backwards-compatible, except, of course, for managed
itself. There's a minor difference in _enter_transaction_management:
the top self.transaction_state now contains the new 'managed' state
rather than the previous one. Django doesn't access
self.transaction_state in _enter_transaction_management.
There were a couple of errors in ._dirty flag handling:
* It started as None, but was never reset to None.
* The _dirty flag was sometimes used to indicate if the connection
was inside transaction management, but this was not done
consistently. This also meant the flag had three separate values.
* The None value had a special meaning, causing for example inability
to commit() on new connection unless enter/leave tx management was
done.
* The _dirty was tracking "connection in transaction" state, but only
in managed transactions.
* Some tests never reset the transaction state of the used connection.
* And some additional less important changes.
This commit has some potential for regressions, but as the above list
shows, the current situation isn't perfect either.
Fixes#19584.
This implies stop storing file path command line arguments in envvars as
a security measure to start relying on with Popen's shell=False instead,
and addition of an 'utils' module.
Thanks kmichel_wgs for the report.
If Django was symlinked into site-packages the previous approach to discover
the tests subdirectory would fail. The revised version now always points to
the location of the source and not the import path.
The content of ifchanged template tag was rendered twice: first time, to
compare it with the previous value and the second time, to return the
rendered output.
Previously, the ifchanged node stored state on `self._last_seen`,
thereby giving undesired results when the node is reused by another
thread at the same time (e.g. globally caching a Template object).
Thanks to akaihola for the report and Diederik van der Boor and
Bas Peschier for the patch.
Changed the ip_address field for Comment to GenericIPAddressField. Added
instructions to the release notes on how to update the schema of existing
databases.
Change patch from https://code.djangoproject.com/ticket/5568
to work on modern Django.
Add special case for MySQL which has different syntax for DROP INDEX.
Add unit tests for the new functionality.
When iteration over a queryset raised an exception, the result cache
remained initialized with an empty list, so subsequent iterations returned
an empty list instead of raising an exception
Querying the reverse side of nullable to_field relation, where both
sides can contain null values resulted in incorrect results. The reason
was not detecting '' as NULL.
Refs #17541
There was a regression in case two models inherited the same parent,
and one contained a foreign key to other. When select_related travelled
the foreign key the other model reused the parent join made by the
first model. This was likely caused by Query.join_parent_model()
addition in commit 68985db482.
Thanks to Trac alias loic84 for report & tests.
The admin_widgets tests were issuing click() to the browser but
didn't wait for the effects of those clicks. This caused the resulting
request to be processed concurrently with the test case. When using
in-memory SQLite this caused weird failures.
Also added wait_page_loaded() to admin selenium tests for code
reuse.
Fixed#19856
The join promote=True was over-aggressive in select_related handling.
After that was removed, the only other user was query.combine(). That
use case is very easy to handle locally, so there is no more need for
the join(promote=True) flag.
Refs #19849.
This controls whether or not a database level cosntraint is created. This is useful in a few specialized circumstances, but in general should not be used!
The refactoring mainly concentrates on making sure the inner and outer
query agree about the split position. The split position is where the
multijoin happens, and thus the split position also determines the
columns used in the "WHERE col1 IN (SELECT col2 from ...)" condition.
This commit fixes a regression caused by #10790 and commit
69597e5bcc. The regression was caused
by wrong cols in the split position.
Django used to check the version of MySQL before handling the first
request, which required:
- opening a connection
- closing it, to avoid holding it idle until the first request.
This code isn't necessary any longer since Django dropped support for
some versions of MySQL, and other database backends don't implement a
similar dance. For consistency and maintenability, remove it.
Reverts 4423757c0c.
Closes#18135.
* Avoided calling BaseHttpResponse.close(). The test client take care of
that since acc5396e.
* Disconnected the request_finished signal when this method must be
called. The test client has a similar implementation since bacb097a.
Thanks Carl Meyer for the review.
Squashed commit of the following:
commit 4f290bdb60b7d8534abf4ca901bd0844612dcbda
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 21:21:30 2013 +0100
Used '0:00' instead of 'UTC' which doesn't always exist in Oracle.
Thanks Ian Kelly for the suggestion.
commit 01b6366f3ce67d57a58ca8f25e5be77911748638
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 13:38:43 2013 +0100
Made tzname a parameter of datetime_extract/trunc_sql.
This is required to work around a bug in Oracle.
commit 924a144ef8a80ba4daeeafbe9efaa826566e9d02
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Wed Feb 13 14:47:44 2013 +0100
Added support for parameters in SELECT clauses.
commit b4351d2890cd1090d3ff2d203fe148937324c935
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 22:30:22 2013 +0100
Documented backwards incompatibilities in the two previous commits.
commit 91ef84713c81bd455f559dacf790e586d08cacb9
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:42:31 2013 +0100
Used QuerySet.datetimes for the admin's date_hierarchy.
commit 0d0de288a5210fa106cd4350961eb2006535cc5c
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 09:29:38 2013 +0100
Used QuerySet.datetimes in date-based generic views.
commit 9c0859ff7c0b00734afe7fc15609d43d83215072
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:25 2013 +0100
Implemented QuerySet.datetimes on Oracle.
commit 68ab511a4ffbd2b811bf5da174d47e4dd90f28fc
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:14 2013 +0100
Implemented QuerySet.datetimes on MySQL.
commit 22d52681d347a8cdf568dc31ed032cbc61d049ef
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:42:29 2013 +0100
Implemented QuerySet.datetimes on SQLite.
commit f6800fd04c93722b45f9236976389e0b2fe436f5
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:43:03 2013 +0100
Implemented QuerySet.datetimes on PostgreSQL.
commit 0c829c23f4cf4d6804cadcc93032dd4c26b8c65e
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:41:08 2013 +0100
Added datetime-handling infrastructure in the ORM layers.
commit 104d82a7778cf3f0f5d03dfa53709c26df45daad
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Mon Feb 11 10:05:55 2013 +0100
Updated null_queries tests to avoid clashing with the __second lookup.
commit c01bbb32358201b3ac8cb4291ef87b7612a2b8e6
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 23:07:41 2013 +0100
Updated tests of .dates().
Replaced .dates() by .datetimes() for DateTimeFields.
Replaced dates with datetimes in the expected output for DateFields.
commit 50fb7a52462fecf0127b38e7f3df322aeb287c43
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 21:40:09 2013 +0100
Updated and added tests for QuerySet.datetimes.
commit a8451a5004c437190e264667b1e6fb8acc3c1eeb
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 22:34:46 2013 +0100
Documented the new time lookups and updated the date lookups.
commit 29413eab2bd1d5e004598900c0dadc0521bbf4d3
Author: Aymeric Augustin <aymeric.augustin@m4x.org>
Date: Sun Feb 10 16:15:49 2013 +0100
Documented QuerySet.datetimes and updated QuerySet.dates.
For languages with non-breaking space as thousand separator,
standard space input should also be allowed, as few people know
how to enter non-breaking space on keyboards. Refs #17217.
Thanks Alexey Boriskin for the report and initial patch.
Wrap the Parser.compile_filter method call with a try/except and call the
newly added Parser.compile_filter_error(). Overwrite this method in the
DebugParser to throw the correct error.
Since this error was otherwise catched by the compile_function try/except
block the debugger highlighted the wrong line.
The reason was that the except clause needed to remove a connection
from the django.db.connections dict, but other parts of Django do not
expect this to happen. In addition the except clause was silently
swallowing the exception messages.
Refs #19707, special thanks to Carl Meyer for pointing out that this
approach should be taken.
When a query had a complex where condition (a condition targeting more
than the base table) a subquery was used for deletion. However, the
query had default ordering from the model's meta and Oracle doesn't
work with ordered subqueries.
The regression was caused by fast-path deletion code introduced in
1cd6e04cd4 for fixing #18676.
Thanks to Dylan Klomparens for the report.
Also, according to the comments on the ticket and its duplicates, added
tests execising saving an instance of a model with a GFK to:
* An unsaved object -- This actually doesn't generate the same failure
but another ORM-level exception. The test verifies it's the case.
* An instance of a model with a __nonzero__() method thant returns False
for it. This doesn't fail because that code path isn't executed.
* An instance of a model with a CharField PK and an empty value for it.
This doesn't fail.
They are handled independently now and the latter can be influenced by
the new BaseCommand.leave_locale_alone internal option.
Thanks chrischambers for the report, Claude, lpiatek, neaf and gabooo for
their work on a patch, originally on refs. #17379.
They are simply ignored now. This allows for a more correct behavior when
they are placed before translatable constructs on the same line.
Previously, the latter were wrongly ignored because the former were
preserved when converting template code to the internal Python-syntax
form later fed to xgettext but Python has no ``/* ... */``-style
comments.
Also, special comments directed to translators are now only taken in
account when they are located at the end of a line. e.g.::
{# Translators: ignored #}{% trans "Literal A" %}{# Translators: valid, associated with "Literal B" below #}
{% trans "Literal B" %}
Behavior of ``{% comment %}...{% endcomment %}``tags remains unchanged.
Thanks juneih at redpill-linpro dot com for the report and Claude for
his work on the issue.
The main cleanup was removal of non-necessary __unicode__ method. The
tests didn't break on py3 as the string representation was never used
in the tests.
Refs #17813. Thanks to Simon Charette for spotting this issue.
Thanks a lot to everybody participating in developing this feature.
The patch was developed by multiple people, at least Trac aliases
tonnzor, jimmysong, Fandekasp and slurms.
Stylistic changes added by committer.
It creates a `locale/django.pot` file once instead of one
`locale/<locale_code>/django.pot` file for every locale involved.
Thanks Michal Čihař for the report and patch.
In a normal relational construct, if you're listening for an event
that signals a child was deleted, you dont expect that the parent
was deleted already.
This change ensures that post_delete signals are fired immediately
after objects are deleted in the graph.
The original problem was that queryset cloning was really expensive
when filtering with F() clauses. The __deepcopy__ went too deep copying
_meta attributes of the models used. To fix this the use of
__deepcopy__ in qs cloning was removed.
This commit results in some speed improvements across the djangobench
benchmark suite. Most query_* tests are 20-30% faster, save() is 50%
faster and finally complex filtering situations can see 2x to order
of magnitude improvments.
Thanks to Suor, Alex and lrekucki for valuable feedback.
The tickets are either about different signature between qs.none() and
qs or problems with subclass types (either EmptyQS overrided the custom
qs class, or EmptyQS was overridden by another class - values() did
this).
Fixed#15959, fixed#17271, fixed#17712, fixed#19426
The guarantee that no queries will be made when accessing results is
done by new EmptyWhere class which is used for query.where and having.
Thanks to Simon Charette for reviewing and valuable suggestions.
When &'ing or |'ing querysets, wrong values could be cached, and crashes
could happen.
Thanks Marc Tamlyn for figuring out the problem and writing the patch.
* Used override_settings consistently -- changes to DEBUG could leak.
* Took advantage of assertRaisesRegexp.
* Fixed indentation -- some code was indented at 2 spaces.
An index on TextField results in a warning message when running tests
on MySQL or SQLite, and the test using the TextField was PostgreSQL
only in any case.
The added promotion logic is based on promoting any joins used in only
some of the childs of an OR clause unless the join existed before the
OR clause addition.
The ORM didn't reuse joins for direct foreign key traversals when using
chained filters. For example:
qs.filter(fk__somefield=1).filter(fk__somefield=2))
produced two joins.
As a bonus, reverse onetoone filters can now reuse joins correctly
The regression was caused by the join() method refactor in commit
68847135bc
Thanks for Simon Charette for spotting some issues with the first draft
of the patch.
This is necessary because get_model() checks are case insensitive, and if the swapable check isn't, the
swappable logic gets tied up in knots with models that are partially swapped out.
Thanks to chris@cogdon.org for the report and extensive analysis, and Preston for his work on the draft patch.
This is a rather large refactoring. The "lookup traversal" code was
splitted out from the setup_joins. There is now names_to_path() method
which does the lookup traveling, the actual work of setup_joins() is
calling names_to_path() and then adding the joins found into the query.
As a side effect it was possible to remove the "process_extra"
functionality used by genric relations. This never worked for left
joins. Now the extra restriction is appended directly to the join
condition instead of the where clause.
To generate the extra condition we need to have the join field
available in the compiler. This has the side-effect that we need more
ugly code in Query.__getstate__ and __setstate__ as Field objects
aren't pickleable.
The join trimming code got a big change - now we trim all direct joins
and never trim reverse joins. This also fixes the problem in #10790
which was join trimming in null filter cases.
F() expressions reuse joins like any lookup in a .filter() call -
reuse multijoins generated in the same .filter() call else generate
new joins. Also, lookups can now reuse joins generated by F().
This change is backwards incompatible, but it is required to prevent
dict randomization from generating different queries depending on
.filter() kwarg ordering. The new way is also more consistent in how
joins are reused.
This allows for Paginator subclasses to easily override the Page class
that gets used.
Took the opportunity to also do some non-invasive PEP8 tidying of the
paginator module.
Capture warnings in Python >= 2.7 and route through
console handler, which is subject to DEBUG==True
Thanks to dstufft for the idea, and claudep for initial patch
