Compare commits

..

No commits in common. "master" and "0.1.0" have entirely different histories.

30 changed files with 293 additions and 694 deletions

View file

@ -1,38 +0,0 @@
sudo: no
language: python
env:
- TOXENV=flake8
- TOXENV=isort
- TOXENV=py27-1.7
- TOXENV=py27-1.8
- TOXENV=py27-1.9
- TOXENV=py27-main
- TOXENV=py32-1.7
- TOXENV=py32-1.8
- TOXENV=py33-1.7
- TOXENV=py33-1.8
- TOXENV=py34-1.7
- TOXENV=py34-1.8
- TOXENV=py34-1.9
- TOXENV=py34-main
matrix:
include:
- python: 3.5
env: TOXENV=py35-1.8
- python: 3.5
env: TOXENV=py35-1.9
- python: 3.5
env: TOXENV=py35-main
allow_failures:
- env: TOXENV=py27-main
- env: TOXENV=py34-main
- env: TOXENV=py35-main
install:
- pip install tox "virtualenv<14.0"
script:
- tox

View file

@ -2,11 +2,6 @@ Authors
=======
* Brandon Konkle (https://github.com/bkonkle)
* Camilo Nova (https://github.com/camilonova)
* Jeff Triplett (https://github.com/jefftriplett)
* Vladimir Iakovlev (https://github.com/nvbn)
* Nick Lang (https://github.com/fxdgear)
* Ilya Baryshev (https://github.com/coagulant)
This project is a fork of the django-jenkins project. It would not be possible without the effort of Mikhail Podgurskiy and the django-jenkins project contributors. Thank you!

View file

@ -1,46 +0,0 @@
# Code of Conduct
As contributors and maintainers of the Jazzband projects, and in the interest of
fostering an open and welcoming community, we pledge to respect all people who
contribute through reporting issues, posting feature requests, updating documentation,
submitting pull requests or patches, and other activities.
We are committed to making participation in the Jazzband a harassment-free experience
for everyone, regardless of the level of experience, gender, gender identity and
expression, sexual orientation, disability, personal appearance, body size, race,
ethnicity, age, religion, or nationality.
Examples of unacceptable behavior by participants include:
- The use of sexualized language or imagery
- Personal attacks
- Trolling or insulting/derogatory comments
- Public or private harassment
- Publishing other's private information, such as physical or electronic addresses,
without explicit permission
- Other unethical or unprofessional conduct
The Jazzband roadies have the right and responsibility to remove, edit, or reject
comments, commits, code, wiki edits, issues, and other contributions that are not
aligned to this Code of Conduct, or to ban temporarily or permanently any contributor
for other behaviors that they deem inappropriate, threatening, offensive, or harmful.
By adopting this Code of Conduct, the roadies commit themselves to fairly and
consistently applying these principles to every aspect of managing the jazzband
projects. Roadies who do not follow or enforce the Code of Conduct may be permanently
removed from the Jazzband roadies.
This code of conduct applies both within project spaces and in public spaces when an
individual is representing the project or its community.
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by
contacting the roadies at `roadies@jazzband.co`. All complaints will be reviewed and
investigated and will result in a response that is deemed necessary and appropriate to
the circumstances. Roadies are obligated to maintain confidentiality with regard to the
reporter of an incident.
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version
1.3.0, available at [https://contributor-covenant.org/version/1/3/0/][version]
[homepage]: https://contributor-covenant.org
[version]: https://contributor-covenant.org/version/1/3/0/

View file

@ -1,3 +0,0 @@
[![Jazzband](https://jazzband.co/static/img/jazzband.svg)](https://jazzband.co/)
This is a [Jazzband](https://jazzband.co/) project. By contributing you agree to abide by the [Contributor Code of Conduct](https://jazzband.co/docs/conduct) and follow the [guidelines](https://jazzband.co/docs/guidelines).

View file

@ -1,19 +0,0 @@
Developer instructions
======================
Here you can find some instructions when developing for this package.
Release a new version
---------------------
When you are ready to release a new version you need to follow these steps.
For example version 1.0.3:
1. Change the version number at ```discover_jenkins/__init__.py```
2. Commit ```git commit -m "Release 1.0.3"```
3. Tag ```git tag 1.0.3```
4. Push changes ```git push origin master```
5. Push tags ```git push --tags```
6. Upload release to Pypi ```python setup.py sdist upload```
7. Dance!

View file

@ -1 +1 @@
include *.md LICENSE discover_jenkins/tasks/pylint.rc
include *.md LICENSE

View file

@ -1,20 +1,18 @@
django-discover-jenkins
=======================
[![Build Status](https://travis-ci.org/jazzband/django-discover-jenkins.svg?branch=master)](https://travis-ci.org/jazzband/django-discover-jenkins)
[![Jazzband](https://jazzband.co/static/img/badge.svg)](https://jazzband.co/)
[![Build Status](https://drone.io/github.com/lincolnloop/django-discover-jenkins/status.png)](https://drone.io/github.com/lincolnloop/django-discover-jenkins/latest)
A streamlined fork of django-jenkins designed to work with the default test command and the discover runner.
[Read the Docs](https://django-discover-jenkins.readthedocs.io/)
[Read the Docs](https://django-discover-jenkins.readthedocs.org/)
Why?
----
The overall goal is to run tests on Jenkins the same way you do on your local machine. This project is designed to take advantage of https://github.com/jezdez/django-discover-runner/, which is the default test runner in Django 1.6. Instead of using a setting to list which apps should be tested, or accepting Django-specific test labels, it uses the official test discovery feature of the new unittest2.
The overall goal is to run tests on Jenkins the same way you do on your local machine. This project is designed to take advantage of [django-discover-runner](https://github.com/jezdez/django-discover-runner/), which is the default test runner in Django 1.6. Instead of using a setting to list which apps should be tested, or accepting Django-specific test labels, it uses the official test discovery feature of the new unittest2.
Also, the original https://github.com/kmmbvnr/django-jenkins project doesn't take advantage of improvements to testing introduced in Django 1.4. Special management commands are no longer needed, as test runners themselves can add options that are handled by the built-in `test` command.
Also, the original [django-jenkins](https://github.com/kmmbvnr/django-jenkins) project doesn't take advantage of improvements to testing introduced in Django 1.4. Special management commands are no longer needed, as test runners themselves can add options that are handled by the built-in `test` command.
What's Changed?
@ -24,3 +22,10 @@ What's Changed?
* **No management commands are provided.** Since Django 1.4, the built in 'test' command has allowed the test runner to define additional options that the management command will accept.
* **It doesn't use signals.** Instead of using the event/callback style of signals and using `inspect.getmembers` to connect everything, the test runner simply checks for key methods on each task the same way Django's request handler checks for methods on middleware.
* **Not everything works yet.** Only a subset of the django-jenkins tasks have been ported at this point. I'd love to accept your pull requests to add more of them.
Who?
----
First and foremost, the authors of [django-jenkins](https://github.com/kmmbvnr/django-jenkins) are responsible for the basis of most of this code. I ([Brandon Konkle](https://github.com/bkonkle)) just took it apart and put it back together again in a different way, then fixed or reworked some things. Thank you to the contributors of that project!
For the full list of original authors and for the contributors to this project, see the AUTHORS.md file.

View file

@ -1,3 +1,3 @@
__author__ = 'Brandon Konkle'
__version__ = '0.1.4'
__version__ = '0.1.0'

View file

@ -2,11 +2,15 @@ import os
import traceback
from datetime import datetime
from itertools import groupby
from unittest import TextTestResult
from xml.sax.saxutils import XMLGenerator
from xml.sax.xmlreader import AttributesImpl
from django.utils.unittest import TextTestResult
from discover_jenkins.utils import total_seconds
from django.utils.encoding import smart_text
try:
from django.utils.encoding import smart_text
except ImportError:
from django.utils.encoding import smart_unicode as smart_text
STDOUT_LINE = '\nStdout:\n%s'
STDERR_LINE = '\nStderr:\n%s'
@ -172,9 +176,9 @@ class XMLTestResult(TextTestResult):
document.startDocument()
document.startElement('testsuites', AttributesImpl({}))
suites = groupby(
self.testInfos, key=lambda test_info: self.test_case_name(test_info.test_method)
)
suites = groupby(self.testInfos,
key=lambda test_info: self.test_case_name(
test_info.test_method))
for suite_name, suite in suites:
document.startElement('testsuite',
AttributesImpl({'name': suite_name}))
@ -183,26 +187,26 @@ class XMLTestResult(TextTestResult):
document.startElement('testcase', AttributesImpl({
'classname': suite_name,
'name': self.test_method_name(test_info.test_method),
'time': '%3f' % (
test_info.end_time - test_info.start_time
).total_seconds()
'time': '%3f' % total_seconds(
test_info.end_time - test_info.start_time)
}))
if test_info.result == TestInfo.RESULT.ERROR:
document.startElement('error', AttributesImpl({
'message': smart_text(test_info.err[1])
}))
document.characters(self._exc_info_to_string(test_info.err, test_info.test_method))
document.characters(self._exc_info_to_string(
test_info.err, test_info.test_method))
document.endElement('error')
elif test_info.result == TestInfo.RESULT.FAILURE:
document.startElement('failure', AttributesImpl({
'message': smart_text(test_info.err[1])
}))
document.characters(
self._exc_info_to_string(test_info.err, test_info.test_method).decode('utf-8')
)
document.characters(self._exc_info_to_string(
test_info.err, test_info.test_method))
document.endElement('failure')
elif test_info.result == TestInfo.RESULT.UNEXPECTED_SUCCESS:
elif test_info.result == \
TestInfo.RESULT.UNEXPECTED_SUCCESS:
document.startElement('error', AttributesImpl({
'message': 'Unexpected success'
}))

View file

@ -1,13 +1,20 @@
import unittest
from importlib import import_module
import inspect
from optparse import make_option
import django
from django.core.exceptions import ImproperlyConfigured
from django.test.runner import DiscoverRunner
from django.utils import unittest
from django.utils.importlib import import_module
try:
# Django 1.6
from django.test.runner import DiscoverRunner
except ImportError:
# Fallback to third-party app on Django 1.5
from discover_runner.runner import DiscoverRunner
from .results import XMLTestResult
from .settings import OUTPUT_DIR, TASKS
from .settings import TASKS, OUTPUT_DIR
def get_tasks():
@ -48,23 +55,22 @@ class CIRunner(object):
A Django test runner mixin that runs tasks for Jenkins and dumps the
results to an XML file.
"""
if django.VERSION < (1, 8):
option_list = get_task_options() + (
make_option(
'--jenkins',
action='store_true',
dest='jenkins',
default=False,
help='Process the Jenkins tasks from TEST_JENKINS_TASKS.'
),
make_option(
'--output-dir',
action='store',
dest='output_dir',
default=OUTPUT_DIR,
help='Top level of project for unittest discovery.'
),
)
option_list = get_task_options() + (
make_option(
'--jenkins',
action='store_true',
dest='jenkins',
default=False,
help='Process the Jenkins tasks from TEST_JENKINS_TASKS.'
),
make_option(
'--output-dir',
action='store',
dest='output_dir',
default=OUTPUT_DIR,
help='Top level of project for unittest discovery.'
),
)
def __init__(self, jenkins=False, output_dir=None, **options):
super(CIRunner, self).__init__(**options)
@ -82,20 +88,6 @@ class CIRunner(object):
instance = task_class(output_dir=output_dir, **options)
self.tasks.append(instance)
@classmethod
def add_arguments(cls, parser):
task_classes = get_tasks()
for task_cls in task_classes:
if hasattr(task_cls, 'add_arguments'):
task_cls.add_arguments(parser)
parser.add_argument('--jenkins',
action='store_true', dest='jenkins', default=False,
help='Process the Jenkins tasks from TEST_JENKINS_TASKS.')
parser.add_argument('--output-dir',
action='store', dest='output_dir', default=OUTPUT_DIR,
help='Top level of project for unittest discovery.')
def setup_test_environment(self, **kwargs):
super(CIRunner, self).setup_test_environment(**kwargs)
if self.jenkins:
@ -138,11 +130,4 @@ class CIRunner(object):
class DiscoverCIRunner(CIRunner, DiscoverRunner):
"""The CIRunner mixin applied to the discover runner"""
if django.VERSION < (1, 8):
option_list = DiscoverRunner.option_list + CIRunner.option_list
@classmethod
def add_arguments(cls, parser):
DiscoverRunner.add_arguments(parser)
CIRunner.add_arguments(parser)
option_list = DiscoverRunner.option_list + CIRunner.option_list

View file

@ -1,8 +1,9 @@
from django.conf import settings
TASKS = getattr(settings, 'TEST_TASKS', (
'discover_jenkins.tasks.run_pylint.PyLintTask',
'discover_jenkins.tasks.with_coverage.CoverageTask',
'discover_jenkins.tasks.pylint.PyLintTask',
'discover_jenkins.tasks.coverage.CoverageTask',
))
OUTPUT_DIR = getattr(settings, 'TEST_OUTPUT_DIR', 'reports')
PYLINT_RCFILE = getattr(settings, 'TEST_PYLINT_RCFILE', 'pylint.rc')
@ -11,7 +12,8 @@ PROJECT_APPS = getattr(settings, 'TEST_PROJECT_APPS', ())
COVERAGE_WITH_MIGRATIONS = getattr(settings, 'TEST_COVERAGE_WITH_MIGRATIONS', False)
COVERAGE_REPORT_HTML_DIR = getattr(settings, 'TEST_COVERAGE_REPORT_HTML_DIR', '')
COVERAGE_MEASURE_BRANCH = getattr(settings, 'TEST_COVERAGE_MEASURE_BRANCH', True)
COVERAGE_EXCLUDE_PATHS = getattr(settings, 'TEST_COVERAGE_EXCLUDE_PATHS', [])
COVERAGE_EXCLUDES = getattr(settings, 'TEST_COVERAGE_EXCLUDES', [])
COVERAGE_EXCLUDES_FOLDERS = getattr(settings, 'TEST_COVERAGE_EXCLUDES_FOLDERS', [])
COVERAGE_RCFILE = getattr(settings, 'TEST_COVERAGE_RCFILE', 'coverage.rc')
JSHINT_CHECKED_FILES = getattr(settings, 'TEST_JSHINT_CHECKED_FILES', None)

View file

@ -5,7 +5,7 @@ ignore=migrations
cache-size=500
[MESSAGES CONTROL]
# C0111 Missing docstring
# C0111 Missing docstring
# I0011 Warning locally suppressed using disable-msg
# I0012 Warning locally suppressed using disable-msg
# W0704 Except doesn't do anything Used when an except clause does nothing but "pass" and there is no "else" clause
@ -18,7 +18,9 @@ cache-size=500
disable=C0111,I0011,I0012,W0704,W0142,W0212,W0232,W0613,W0702,R0201
[REPORTS]
msg-template={path}:{line}: [{msg_id}({symbol}), {obj}] {msg}
output-format=parseable
include-ids=yes
[BASIC]
no-docstring-rgx=__.*__|_.*
@ -29,6 +31,7 @@ const-rgx=(([A-Z_][A-Z0-9_]*)|([a-z_][a-z0-9_]*)|(__.*__)|register|urlpatterns)$
good-names=_,i,j,k,e,qs,pk,setUp,tearDown
[TYPECHECK]
# Tells whether missing members accessed in mixin class should be ignored. A
# mixin class is detected if its name ends with "mixin" (case insensitive).
ignore-mixin-members=yes
@ -48,6 +51,7 @@ generated-members=objects,DoesNotExist,id,pk,_meta,base_fields,context
# List of method names used to declare (i.e. assign) instance attributes
defining-attr-methods=__init__,__new__,setUp
[VARIABLES]
init-import=no
dummy-variables-rgx=_|dummy
@ -57,14 +61,17 @@ min-similarity-lines=6
ignore-comments=yes
ignore-docstrings=yes
[MISCELLANEOUS]
notes=FIXME,XXX,TODO
[FORMAT]
max-line-length=160
max-module-lines=500
indent-string=' '
[DESIGN]
max-args=10
max-locals=15
@ -75,3 +82,4 @@ max-parents=7
max-attributes=7
min-public-methods=0
max-public-methods=50

View file

@ -1,72 +0,0 @@
# coding: utf-8
import os
import sys
from optparse import make_option
import django
import pep8
from flake8.engine import get_style_guide
from ..utils import get_app_locations
from .run_pep8 import Pep8Task
class Flake8Task(Pep8Task):
if django.VERSION < (1, 8):
option_list = Pep8Task.option_list + (
make_option(
'--max-complexity',
dest='max_complexity',
help='McCabe complexity threshold',
),
)
def __init__(self, **options):
super(Flake8Task, self).__init__(**options)
if options.get('flake8_file_output', True):
output_dir = options['output_dir']
if not os.path.exists(output_dir):
os.makedirs(output_dir)
self.output = open(os.path.join(output_dir, 'flake8.report'), 'w')
else:
self.output = sys.stdout
if options['max_complexity']:
self.pep8_options['max_complexity'] = int(options['max_complexity'])
@classmethod
def add_arguments(cls, parser):
Pep8Task.add_arguments(parser)
parser.add_argument('--max-complexity',
dest='max_complexity',
help='McCabe complexity threshold')
def teardown_test_environment(self, **kwargs):
class JenkinsReport(pep8.BaseReport):
def error(instance, line_number, offset, text, check):
code = super(JenkinsReport, instance).error(
line_number, offset, text, check,
)
if not code:
return
sourceline = instance.line_offset + line_number
self.output.write(
'%s:%s:%s: %s\n' %
(instance.filename, sourceline, offset + 1, text),
)
flake8style = get_style_guide(
parse_argv=False,
config_file=self.pep8_rcfile,
reporter=JenkinsReport,
**self.pep8_options)
# Jenkins pep8 validator requires relative paths
project_root = os.path.abspath(os.path.dirname(__name__))
for location in map(lambda x: os.path.relpath(x, project_root), get_app_locations()):
flake8style.input_dir(location)
self.output.close()

View file

@ -1,56 +1,53 @@
# -*- coding: utf-8 -*-
import os
import sys
import codecs
import fnmatch
import os
import subprocess
import sys
from optparse import make_option
import django
from django.conf import settings as django_settings
from ..settings import JSHINT_CHECKED_FILES, JSHINT_EXCLUDE, JSHINT_RCFILE
from ..utils import CalledProcessError, get_app_locations
from ..settings import JSHINT_CHECKED_FILES, JSHINT_RCFILE, JSHINT_EXCLUDE
class JSHintTask(object):
if django.VERSION < (1, 8):
option_list = (
make_option(
"--jshint-no-staticdirs",
dest="jshint-no-staticdirs",
default=False,
action="store_true",
help="Don't check js files located in STATICFILES_DIRS settings"
),
make_option(
"--jshint-with-minjs",
dest="jshint_with-minjs",
default=False,
action="store_true",
help="Do not ignore .min.js files"
),
make_option(
"--jshint-exclude",
dest="jshint_exclude",
default=JSHINT_EXCLUDE,
help="Exclude patterns"
),
make_option(
'--jshint-stdout',
action='store_true',
dest='jshint_stdout',
default=False,
help='Print the jshint output instead of storing it in a file'
),
make_option(
'--jshint-rcfile',
dest='jshint_rcfile',
default=JSHINT_RCFILE,
help='Provide an rcfile for jshint'
),
)
option_list = (
make_option(
"--jshint-no-staticdirs",
dest="jshint-no-staticdirs",
default=False,
action="store_true",
help="Don't check js files located in STATICFILES_DIRS settings"
),
make_option(
"--jshint-with-minjs",
dest="jshint_with-minjs",
default=False,
action="store_true",
help="Do not ignore .min.js files"
),
make_option(
"--jshint-exclude",
dest="jshint_exclude",
default=JSHINT_EXCLUDE,
help="Exclude patterns"
),
make_option(
'--jshint-stdout',
action='store_true',
dest='jshint_stdout',
default=False,
help='Print the jshint output instead of storing it in a file'
),
make_option(
'--jshint-rcfile',
dest='jshint_rcfile',
default=JSHINT_RCFILE,
help='Provide an rcfile for jshint'
),
)
def __init__(self, **options):
self.to_stdout = options['jshint_stdout']
@ -71,24 +68,6 @@ class JSHintTask(object):
if isinstance(self.exclude, str):
self.exclude = self.exclude.split(',')
@classmethod
def add_arguments(cls, parser):
parser.add_argument("--jshint-no-staticdirs",
dest="jshint-no-staticdirs", default=False, action="store_true",
help="Don't check js files located in STATICFILES_DIRS settings")
parser.add_argument("--jshint-with-minjs",
dest="jshint_with-minjs", default=False, action="store_true",
help="Do not ignore .min.js files")
parser.add_argument("--jshint-exclude",
dest="jshint_exclude", default=JSHINT_EXCLUDE,
help="Exclude patterns")
parser.add_argument('--jshint-stdout',
action='store_true', dest='jshint_stdout', default=False,
help='Print the jshint output instead of storing it in a file')
parser.add_argument('--jshint-rcfile',
dest='jshint_rcfile', default=JSHINT_RCFILE,
help='Provide an rcfile for jshint')
def teardown_test_environment(self, **kwargs):
files = [path for path in self.static_files_iterator()]

View file

@ -1,112 +0,0 @@
# -*- coding: utf-8 -*-
import os
import sys
from optparse import make_option
import django
import pep8
from django.conf import settings
from ..utils import get_app_locations
class Pep8Task(object):
if django.VERSION < (1, 8):
option_list = (
make_option(
"--pep8-exclude",
dest="pep8-exclude",
default=pep8.DEFAULT_EXCLUDE + ",migrations",
help="exclude files or directories which match these "
"comma separated patterns (default: %s)" %
pep8.DEFAULT_EXCLUDE
),
make_option(
"--pep8-select", dest="pep8-select",
help="select errors and warnings (e.g. E,W6)",
),
make_option(
"--pep8-ignore", dest="pep8-ignore",
help="skip errors and warnings (e.g. E4,W)",
),
make_option(
"--pep8-max-line-length",
dest="pep8-max-line-length", type='int',
help="set maximum allowed line length (default: %d)" %
pep8.MAX_LINE_LENGTH
),
make_option(
"--pep8-rcfile", dest="pep8-rcfile",
help="PEP8 configuration file"
),
)
def __init__(self, **options):
if options.get('pep8_file_output', True):
output_dir = options['output_dir']
if not os.path.exists(output_dir):
os.makedirs(output_dir)
self.output = open(os.path.join(output_dir, 'pep8.report'), 'w')
else:
self.output = sys.stdout
self.pep8_rcfile = options['pep8-rcfile'] or self.default_config_path()
self.pep8_options = {'exclude': options['pep8-exclude'].split(',')}
if options['pep8-select']:
self.pep8_options['select'] = options['pep8-select'].split(',')
if options['pep8-ignore']:
self.pep8_options['ignore'] = options['pep8-ignore'].split(',')
if options['pep8-max-line-length']:
self.pep8_options['max_line_length'] = options['pep8-max-line-length']
@classmethod
def add_arguments(cls, parser):
parser.add_argument("--pep8-exclude",
dest="pep8-exclude",
default=pep8.DEFAULT_EXCLUDE + ",migrations",
help="exclude files or directories which match these "
"comma separated patterns (default: %s)" %
pep8.DEFAULT_EXCLUDE)
parser.add_argument("--pep8-select", dest="pep8-select",
help="select errors and warnings (e.g. E,W6)")
parser.add_argument("--pep8-ignore", dest="pep8-ignore",
help="skip errors and warnings (e.g. E4,W)")
parser.add_argument("--pep8-max-line-length",
dest="pep8-max-line-length", type=int,
help="set maximum allowed line length (default: %d)" %
pep8.MAX_LINE_LENGTH)
parser.add_argument("--pep8-rcfile", dest="pep8-rcfile",
help="PEP8 configuration file")
def teardown_test_environment(self, **kwargs):
locations = get_app_locations()
class JenkinsReport(pep8.BaseReport):
def error(instance, line_number, offset, text, check):
code = super(JenkinsReport, instance).error(
line_number, offset, text, check,
)
if not code:
return
sourceline = instance.line_offset + line_number
self.output.write(
'%s:%s:%s: %s\n' %
(instance.filename, sourceline, offset + 1, text),
)
pep8style = pep8.StyleGuide(
parse_argv=False, config_file=self.pep8_rcfile,
reporter=JenkinsReport, **self.pep8_options
)
for location in locations:
pep8style.input_dir(os.path.relpath(location))
self.output.close()
@staticmethod
def default_config_path():
rcfile = getattr(settings, 'PEP8_RCFILE', 'pep8.rc')
return rcfile if os.path.exists(rcfile) else None

View file

@ -4,11 +4,10 @@ import os
import sys
from optparse import make_option
import django
from pylint import lint
from pylint.reporters.text import ParseableTextReporter
from ..settings import PROJECT_APPS, PYLINT_RCFILE
from ..settings import PYLINT_RCFILE, PROJECT_APPS
def default_config_path():
@ -22,23 +21,21 @@ def default_config_path():
class PyLintTask(object):
if django.VERSION < (1, 8):
option_list = (
make_option(
"--pylint-rcfile",
dest="pylint_rcfile",
default=None,
help="pylint configuration file"
),
make_option(
"--pylint-errors-only",
dest="pylint_errors_only",
action="store_true",
default=False,
help="pylint output errors only mode"
),
)
option_list = (
make_option(
"--pylint-rcfile",
dest="pylint_rcfile",
default=None,
help="pylint configuration file"
),
make_option(
"--pylint-errors-only",
dest="pylint_errors_only",
action="store_true",
default=False,
help="pylint output errors only mode"
),
)
def __init__(self, **options):
self.config_path = options['pylint_rcfile'] or default_config_path()
@ -52,15 +49,6 @@ class PyLintTask(object):
else:
self.output = sys.stdout
@classmethod
def add_arguments(cls, parser):
parser.add_argument("--pylint-rcfile",
dest="pylint_rcfile", default=None,
help="pylint configuration file")
parser.add_argument("--pylint-errors-only",
dest="pylint_errors_only", action="store_true", default=False,
help="pylint output errors only mode")
def teardown_test_environment(self, **kwargs):
if PROJECT_APPS:
args = ["--rcfile=%s" % self.config_path]
@ -68,8 +56,5 @@ class PyLintTask(object):
args += ['--errors-only']
args += PROJECT_APPS
lint.Run(
args,
reporter=ParseableTextReporter(output=self.output),
exit=False
)
lint.Run(args, reporter=ParseableTextReporter(output=self.output),
exit=False)

View file

@ -1,33 +1,28 @@
# -*- coding: utf-8 -*-
import os
import subprocess
import sys
from optparse import make_option
import django
from ..utils import get_app_locations
from discover_jenkins.utils import check_output, get_app_locations
class SlocCountTask(object):
if django.VERSION < (1, 8):
option_list = (
make_option(
"--sloccount-with-migrations",
action="store_true",
default=False,
dest="sloccount_with_migrations",
help="Count migrations sloc."
),
make_option(
'--sloccount-stdout',
action='store_true',
dest='sloccount_stdout',
default=False,
help='Print the sloccount totals instead of saving them to a file'
),
)
option_list = (
make_option(
"--sloccount-with-migrations",
action="store_true",
default=False,
dest="sloccount_with_migrations",
help="Count migrations sloc."
),
make_option(
'--sloccount-stdout',
action='store_true',
dest='sloccount_stdout',
default=False,
help='Print the sloccount totals instead of saving them to a file'
),
)
def __init__(self, **options):
self.with_migrations = options['sloccount_with_migrations']
@ -41,19 +36,10 @@ class SlocCountTask(object):
self.output = open(os.path.join(output_dir,
'sloccount.report'), 'w')
@classmethod
def add_arguments(cls, parser):
parser.add_argument("--sloccount-with-migrations",
action="store_true", default=False, dest="sloccount_with_migrations",
help="Count migrations sloc.")
parser.add_argument("--sloccount-stdout",
action="store_true", dest="sloccount_stdout", default=False,
help="Print the sloccount totals instead of saving them to a file")
def teardown_test_environment(self, **kwargs):
locations = get_app_locations()
report_output = subprocess.check_output(
report_output = check_output(
['sloccount', "--duplicates", "--wide", "--details"] + locations
)
report_output = report_output.decode('utf-8')

View file

@ -3,15 +3,12 @@
import os
from optparse import make_option
import django
from django.utils.importlib import import_module
from coverage.control import coverage
from .. import settings
try:
from coverage import coverage
except ImportError:
from coverage.control import coverage
def default_config_path():
rcfile = settings.COVERAGE_RCFILE
@ -21,44 +18,42 @@ def default_config_path():
class CoverageTask(object):
if django.VERSION < (1, 8):
option_list = (
make_option(
"--coverage-rcfile",
dest="coverage_rcfile",
default="",
help="Specify configuration file."
),
make_option(
"--coverage-html-report",
dest="coverage_html_report_dir",
default=settings.COVERAGE_REPORT_HTML_DIR,
help="Directory to which HTML coverage report should be"
" written. If not specified, no report is generated."
),
make_option(
"--coverage-no-branch-measure",
action="store_false",
default=settings.COVERAGE_MEASURE_BRANCH,
dest="coverage_measure_branch",
help="Don't measure branch coverage."
),
make_option(
"--coverage-with-migrations",
action="store_true",
default=settings.COVERAGE_WITH_MIGRATIONS,
dest="coverage_with_migrations",
help="Don't measure migrations coverage."
),
make_option(
"--coverage-exclude",
action="append",
default=settings.COVERAGE_EXCLUDE_PATHS,
dest="coverage_excludes",
help="Paths to be excluded from coverage"
)
option_list = (
make_option(
"--coverage-rcfile",
dest="coverage_rcfile",
default="",
help="Specify configuration file."
),
make_option(
"--coverage-html-report",
dest="coverage_html_report_dir",
default=settings.COVERAGE_REPORT_HTML_DIR,
help="Directory to which HTML coverage report should be"
" written. If not specified, no report is generated."
),
make_option(
"--coverage-no-branch-measure",
action="store_false",
default=settings.COVERAGE_MEASURE_BRANCH,
dest="coverage_measure_branch",
help="Don't measure branch coverage."
),
make_option(
"--coverage-with-migrations",
action="store_true",
default=settings.COVERAGE_WITH_MIGRATIONS,
dest="coverage_with_migrations",
help="Don't measure migrations coverage."
),
make_option(
"--coverage-exclude",
action="append",
default=settings.COVERAGE_EXCLUDES,
dest="coverage_excludes",
help="Module name to exclude"
)
)
def __init__(self, **options):
self.output_dir = options['output_dir']
@ -66,7 +61,21 @@ class CoverageTask(object):
self.html_dir = options['coverage_html_report_dir']
self.branch = options['coverage_measure_branch']
self.exclude_locations = options['coverage_excludes'] or None
self.exclude_locations = []
modnames = options['coverage_excludes']
for modname in modnames:
try:
self.exclude_locations.append(
os.path.dirname(
import_module(modname).__file__
)
)
except ImportError:
pass
# Extra folders to exclude. Particularly useful to specify things like
# apps/company/migrations/*
self.exclude_locations.extend(settings.COVERAGE_EXCLUDES_FOLDERS)
self.coverage = coverage(
branch=self.branch,
@ -75,60 +84,29 @@ class CoverageTask(object):
config_file=options.get('coverage_rcfile') or default_config_path()
)
@classmethod
def add_arguments(cls, parser):
parser.add_argument("--coverage-rcfile",
dest="coverage_rcfile", default="",
help="Specify configuration file.")
parser.add_argument("--coverage-html-report",
dest="coverage_html_report_dir", default=settings.COVERAGE_REPORT_HTML_DIR,
help="Directory to which HTML coverage report should be"
" written. If not specified, no report is generated.")
parser.add_argument("--coverage-no-branch-measure",
action="store_false", default=settings.COVERAGE_MEASURE_BRANCH,
dest="coverage_measure_branch",
help="Don't measure branch coverage.")
parser.add_argument("--coverage-with-migrations",
action="store_true", default=settings.COVERAGE_WITH_MIGRATIONS,
dest="coverage_with_migrations",
help="Don't measure migrations coverage.")
parser.add_argument("--coverage-exclude",
action="append", default=settings.COVERAGE_EXCLUDE_PATHS,
dest="coverage_excludes",
help="Paths to be excluded from coverage")
def setup_test_environment(self, **kwargs):
self.coverage.start()
def teardown_test_environment(self, **kwargs):
self.coverage.stop()
try:
self.coverage._harvest_data()
except AttributeError:
# coverage._harvest_data was renamed to coverage.get_data in
# coverage.py 4.0.
self.coverage.get_data()
morfs = [filename for filename in self.coverage.data.measured_files()
if self.want_file(filename)]
if not os.path.exists(self.output_dir):
os.makedirs(self.output_dir)
self.coverage.xml_report(
morfs=morfs,
outfile=os.path.join(self.output_dir, 'coverage.xml')
)
self.coverage.xml_report(morfs=morfs,
outfile=os.path.join(
self.output_dir, 'coverage.xml'))
if self.html_dir:
self.coverage.html_report(
morfs=morfs,
directory=self.html_dir
)
self.coverage.html_report(morfs=morfs, directory=self.html_dir)
def want_file(self, filename):
if not self.with_migrations and '/migrations/' in filename:
return False
for location in self.exclude_locations:
if filename.startswith(location):
return False
return True

View file

@ -1,6 +1,7 @@
import os.path
import subprocess
from importlib import import_module
from django.utils.importlib import import_module
from discover_jenkins.settings import PROJECT_APPS
@ -15,6 +16,28 @@ class CalledProcessError(subprocess.CalledProcessError):
% (self.cmd, self.returncode, self.output))
def check_output(*popenargs, **kwargs):
"""
Backport from Python2.7
"""
if getattr(subprocess, 'check_output', None) is None:
if 'stdout' in kwargs or 'stderr' in kwargs:
raise ValueError('stdout or stderr argument not allowed, '
'it will be overridden.')
process = subprocess.Popen(stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
*popenargs, **kwargs)
output, err = process.communicate()
retcode = process.poll()
if retcode:
cmd = kwargs.get("args")
if cmd is None:
cmd = popenargs[0]
raise CalledProcessError(retcode, cmd, output=output + '\n' + err)
return output
return subprocess.check_output(*popenargs, **kwargs)
def find_first_existing_executable(exe_list):
"""
Accepts list of [('executable_file_path', 'options')],
@ -32,6 +55,15 @@ def find_first_existing_executable(exe_list):
return filepath
def total_seconds(delta):
"""
Backport timedelta.total_seconds() from Python 2.7
"""
if getattr(delta, 'total_seconds', None) is None:
return delta.days * 86400.0 + delta.seconds + delta.microseconds * 1e-6
return delta.total_seconds()
def get_app_locations():
"""
Returns list of paths to tested apps

View file

@ -5,11 +5,7 @@ Installation
From PyPI::
pip install django-discover-jenkins
Due to a bug in the coverage library you have to use this specific version::
pip install coverage==3.5
pip install django-discover-runner
Configuration
-------------
@ -26,7 +22,7 @@ the ``DiscoverCIRunner`` that ``discover_jenkins`` provides::
TEST_RUNNER = 'discover_jenkins.runner.DiscoverCIRunner'
Even though ``discover_jenkins`` doesn't use app names to discover tests, it
does use them to handle tasks like coverage and pylint. Add your desired apps
does use them to handle tasks like coverahe and pylint. Add your desired apps
to setting called ``TEST_PROJECT_APPS``::
TEST_PROJECT_APPS = (

View file

@ -11,8 +11,8 @@ Settings
Default value::
TEST_TASKS = (
'discover_jenkins.tasks.run_pylint.PyLintTask',
'discover_jenkins.tasks.with_coverage.CoverageTask',
'discover_jenkins.tasks.pylint.PyLintTask',
'discover_jenkins.tasks.coverage.CoverageTask',
)
* ``TEST_OUTPUT_DIR``

View file

@ -7,7 +7,7 @@ Tasks
CoverageTask
============
``discover_jenkins.tasks.with_coverage.CoverageTask``
``discover_runner.tasks.with_coverage.CoverageTask``
Reports test coverage across your apps. Uses the ``TEST_PROJECT_APPS`` setting.
@ -39,19 +39,25 @@ Settings
TEST_COVERAGE_MEASURE_BRANCH = True
* ``TEST_COVERAGE_EXCLUDE_PATHS``
* ``TEST_COVERAGE_EXCLUDES``
File paths to exclude. Can be myapp/admin.py or myapp/management/*
Module names to exclude.
Default value::
TEST_COVERAGE_EXCLUDE_PATHS = []
TEST_COVERAGE_EXCLUDES = []
* ``TEST_COVERAGE_EXCLUDES_FOLDERS``
Extra folders to exclude.
Default value::
TEST_COVERAGE_EXCLUDES_FOLDERS = []
* ``TEST_COVERAGE_RCFILE``
Specify configuration file. Please note if you set the ``TEST_COVERAGE_EXCLUDE_PATHS``
setting, coverage will ignore your coverage.rc file. So if you want to customize
coverage settings only use this file and not the other settings.
Specify configuration file.
Default value::
@ -60,7 +66,7 @@ Settings
PyLintTask
==========
``discover_jenkins.tasks.run_pylint.PyLintTask``
``discover_runner.tasks.run_pylint.PyLintTask``
Runs pylint across your apps. Uses the ``TEST_PROJECT_APPS`` setting.
@ -78,7 +84,7 @@ Settings
JSHintTask
==========
``discover_jenkins.tasks.run_jshint.JSHintTask``
``discover_runner.tasks.run_jshint.JSHintTask``
Runs jshint across your apps. Uses the ``TEST_PROJECT_APPS`` setting.
@ -112,20 +118,6 @@ Settings
SlocCountTask
=============
``discover_jenkins.tasks.run_sloccount.SlocCountTask``
``discover_runner.tasks.run_sloccount.SlocCountTask``
Run sloccount across your apps. Uses the ``TEST_PROJECT_APPS`` setting.
Pep8Task
========
``discover_jenkins.tasks.run_pep8.Pep8Task``
Run pep8 across your apps. Uses the ``TEST_PROJECT_APPS`` setting.
Flake8Task
==========
``discover_jenkins.tasks.run_flake8.Flake8Task``
Run flake8 across your apps. Uses the ``TEST_PROJECT_APPS`` setting.

View file

@ -1,13 +1,2 @@
[flake8]
ignore = E123,E128,E402,W503,E731,W601
max-line-length = 119
exclude = docs,tests/test_project/test_app/migrations/*
[isort]
combine_as_imports=true
include_trailing_comma=true
multi_line_output=5
not_skip=__init__.py
[wheel]
universal = 1

View file

@ -31,7 +31,7 @@ if sys.argv[-1] == 'publish':
setup(
name='django-discover-jenkins',
name='django-discover-runner',
version=version,
description="A minimal fork of django-jenkins designed to work with the "
"discover runner, made with simplicity in mind",
@ -46,19 +46,10 @@ setup(
"Development Status :: 3 - Alpha",
"Environment :: Web Environment",
"Framework :: Django",
"Framework :: Django :: 1.7",
"Framework :: Django :: 1.8",
"Framework :: Django :: 1.9",
"License :: OSI Approved :: BSD License",
"Operating System :: OS Independent",
"Programming Language :: JavaScript",
"Programming Language :: Python :: 2",
"Programming Language :: Python :: 2.7",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.2",
"Programming Language :: Python :: 3.3",
"Programming Language :: Python :: 3.4",
"Programming Language :: Python :: 3.5",
"Topic :: Internet :: WWW/HTTP",
"Topic :: Internet :: WWW/HTTP :: Dynamic Content",
"Topic :: Software Development :: Libraries :: Python Modules",

5
tests/requirements.pip Normal file
View file

@ -0,0 +1,5 @@
django>=1.5
pylint>=0.23
coverage>=3.4
mock>=1.0.1
django-discover-runner>=1.0

View file

@ -1,5 +1,4 @@
import os
BASE_DIR = os.path.dirname(os.path.dirname(__file__))
DEBUG = True
@ -33,7 +32,6 @@ TEST_RUNNER = 'discover_jenkins.runner.DiscoverCIRunner'
TEST_TASKS = (
'discover_jenkins.tasks.with_coverage.CoverageTask',
'discover_jenkins.tasks.run_pylint.PyLintTask',
'discover_jenkins.tasks.run_flake8.Flake8Task',
'discover_jenkins.tasks.run_jshint.JSHintTask',
'discover_jenkins.tasks.run_sloccount.SlocCountTask',
)
@ -48,9 +46,9 @@ LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'console': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'console':{
'level':'DEBUG',
'class':'logging.StreamHandler',
},
},
'loggers': {
@ -60,4 +58,4 @@ LOGGING = {
'propagate': True,
},
}
}
}

View file

@ -1,5 +1,9 @@
from django.db import migrations
from south.v2 import SchemaMigration
class Migration(SchemaMigration):
def forwards(self, orm):
a = 1 # pyflakes/pylint violation
pass
class Migration(migrations.Migration):
a = 1 # pyflakes/pylint violation
def backwards(self, orm):
pass

View file

@ -1,13 +1,7 @@
from unittest import skipIf
import django
from discover_jenkins import runner, tasks
from mock import MagicMock, patch
from django.test import TestCase
try:
from unittest.mock import MagicMock, patch
except ImportError:
from mock import MagicMock, patch
from discover_jenkins import runner, tasks
class FakeTestRunner(object):
@ -30,6 +24,7 @@ class Runner(runner.CIRunner, FakeTestRunner):
class TestCIRunner(TestCase):
def test_get_tasks(self):
"""
Make sure the correct tasks are imported based on the
@ -38,17 +33,15 @@ class TestCIRunner(TestCase):
self.assertEqual(runner.get_tasks(),
[tasks.with_coverage.CoverageTask,
tasks.run_pylint.PyLintTask,
tasks.run_flake8.Flake8Task,
tasks.run_jshint.JSHintTask,
tasks.run_sloccount.SlocCountTask])
@skipIf(django.VERSION >= (1, 8), "optparse is not used on Django 1.8+")
def test_get_task_options(self):
"""
For now, just do a simple test to make sure the right number of options
are gleaned from the tasks.
"""
self.assertEqual(len(runner.get_task_options()), 20)
self.assertEqual(len(runner.get_task_options()), 14)
def test_setup_test_environment(self):
"""
@ -79,3 +72,4 @@ class TestCIRunner(TestCase):
cirun.teardown_test_environment()
self.assertTrue(mock_task.teardown_test_environment.called)

View file

@ -1,12 +1,21 @@
import os
from datetime import timedelta
from django.test import TestCase
import discover_jenkins
import test_project
from django.test import TestCase
class TestUtils(TestCase):
def test_total_seconds(self):
"""
The total_seconds util should show that 5 minutes is 300 seconds.
"""
delta = timedelta(minutes=5)
self.assertEqual(discover_jenkins.utils.total_seconds(delta), 300)
def test_app_locations(self):
"""
The app locations should come from the test_project settings.

48
tox.ini
View file

@ -1,48 +0,0 @@
[tox]
args_are_paths = false
envlist =
flake8,
isort,
py27-{1.7,1.8,1.9,main},
py32-{1.7,1.8},
py33-{1.7,1.8},
py34-{1.7,1.8,1.9,main},
py35-{1.8,1.9,main}
[testenv]
basepython =
py27: python2.7
py32: python3.2
py33: python3.3
py34: python3.4
py35: python3.5
usedevelop = true
setenv =
PYTHONPATH=tests
commands =
{envbindir}/python -Wonce tests/manage.py test tests
deps =
py32: coverage>=3.4,<4.0
{py27,py33,py34,py35}: coverage>=4.0
flake8>=2.1.0
{py27,py32}: mock>=1.0.1
py32: astroid==1.2
py32: logilab-common==0.62
py32: pylint==1.3
{py27,py33,py34,py35}: pylint>=0.23
1.7: Django>=1.7,<1.8
1.8: Django>=1.8,<1.9
1.9: Django>=1.9,<1.10
main: https://github.com/django/django/archive/main.tar.gz
[testenv:flake8]
basepython = python2.7
commands =
flake8
[testenv:isort]
basepython = python2.7
commands =
isort --recursive --check-only --diff discover_jenkins tests
deps =
isort