Commit graph

2188 commits

Author SHA1 Message Date
calvin
4a5a4ed0bf Fix compilation and usage under Windows system.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3827 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-18 09:51:54 +00:00
calvin
b30fb3b09c Remove duplicate code in http checker.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3820 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 19:52:09 +00:00
calvin
caf8ba6297 Really allow parsing of XHTML files; I forgot some places to adjust the MIME checking.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3818 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 13:03:48 +00:00
calvin
a6deeeb8a5 Support parsing of HTML pages served with content type application/xhtml+xml
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3817 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 09:39:49 +00:00
calvin
ff41aa8d9f Lower the MIME content-type info from HTTP headers befure using it
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3816 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 09:38:09 +00:00
calvin
d26386d03f Catch errors when getting content for title.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3814 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-12 15:38:26 +00:00
calvin
8085045ae9 Added extra status logger class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3811 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:02:22 +00:00
calvin
f7a8b16096 Do not override local variable in for loop
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:01:56 +00:00
calvin
fa0c868773 Simplified Logger.writeln() method.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3808 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:01:36 +00:00
calvin
3b5e28ff58 Improved log end output: print number of duplicate error and warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 07:33:19 +00:00
calvin
a880939c40 Initialize variables in reset(), not in subsequent methods
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3796 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-08 09:27:13 +00:00
calvin
5588df8ebf Remove trailing whitespace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3794 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-08 09:26:56 +00:00
calvin
aba08b8b7d Improved graph output: print labels as node names.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3793 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:08:54 +00:00
calvin
83bd7ce4ac Improved network interface detection on POSIX systems
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3792 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:08:36 +00:00
calvin
290528b84f Added title attribute to URL data.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:56 +00:00
calvin
4ade9a79cc Moved link and title parsing into htmlutil package
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3789 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:48 +00:00
calvin
99269d12cc Add base method for Url.get_title()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3788 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:38 +00:00
calvin
58e44dbf8e Updated documentation and todo list.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3787 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:30 +00:00
calvin
e8a4744643 Use enum utility function for thread priorities.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-05 07:14:04 +00:00
calvin
5f818f620a Backport namedtuple container from Python 2.6
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-05 07:13:55 +00:00
calvin
5f4d61e018 Use keyword arguments in translation strings.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:44:40 +00:00
calvin
97772c9700 Improved email check messages.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3779 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:44:07 +00:00
calvin
2e4d0894fc Stop checking a list of emails at the first invalid one.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3778 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:43:20 +00:00
calvin
e6e51dbc6b Overwrite old results when checking a list of emails.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3777 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:42:52 +00:00
calvin
66ff422f6b Allow overwriting of an old check result.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:42:38 +00:00
calvin
5bf875bebe Use configdata for application name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-22 12:17:24 +00:00
calvin
7297519b04 Remove or replace unused variables.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-22 12:10:08 +00:00
calvin
2cb2ee9403 Use base log level for clamav warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3766 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 17:01:49 +00:00
calvin
9352dbf5e4 Move test files to separate module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3763 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 17:01:16 +00:00
calvin
90587545f7 Catch attribute errors in logger flushing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3761 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 16:57:31 +00:00
calvin
2f25962789 Match newlines in catch-all rules
Avoid printing spurious newlines when HTML parsing. The "." does
not match newlines, correct that in the catch-all lexer rules.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3760 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 16:56:58 +00:00
calvin
670a1df9b3 Updated documentation for new Python 2.5 dependency
Removed old references of Python 2.4, together with compatibility
hacks.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3757 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 10:49:00 +00:00
calvin
ce1d41a5a0 Document the CGI access variables
The access variables ALLOWED_CLIENTS and ALLOWED_SERVERS are now
documented to help setup the CGI script more easily.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3756 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 10:48:07 +00:00
calvin
dbb498a395 Add virus checking
New option --scan-virus to check the content of URLs for
viruses with ClamAV.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3753 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 08:57:37 +00:00
calvin
bacb59597e Use relative imports from Python 2.5
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-09 06:16:03 +00:00
calvin
b96e8120d6 Add W3C Validator checks
Add new options --check-html-w3 and --check-css-w3 to allow checking
of HTML and CSS pages with the online W3C validators.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-08 10:36:54 +00:00
calvin
bc9b9ee07e Move http util function in a separate module.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3747 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-08 10:36:08 +00:00
calvin
d5ec7c1ac3 Encode error output to HTML charset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-30 14:37:49 +00:00
calvin
df9f31dcb1 Only check HTML/CSS syntax of intern URLs
The HTML and CSS syntax check now only applies to URLs
which match those given on the command line.
This makes checking of personal pages easier.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 17:48:47 +00:00
calvin
ac4d09f83d Fix errors in CSS and HTML syntax check
Properly encode the warning messages as Unicode, and prevent
overwriting of the "log" module with a local variable.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 17:48:22 +00:00
calvin
92c74ece4d Send HTTP Referer header to both http and https URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3741 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 13:33:35 +00:00
calvin
5d8bdaaa1f Use generators instead of lists where possible
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-28 00:26:02 +00:00
calvin
3eac1be9ab Require and use Python 2.5
Use Python 2.5 features and get rid of old compat code. Also some
code cleanups have been made.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-27 11:39:21 +00:00
calvin
017b73db27 Fix cookie test!
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3728 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 08:34:10 +00:00
calvin
72db31e546 Only check syntax of valid URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3726 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 07:53:11 +00:00
calvin
973da91f44 Source code cleanup: use or remove unused variables
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3724 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 07:49:52 +00:00
calvin
e266a65b64 Fix css check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3723 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-24 10:46:19 +00:00
calvin
62efec3b35 Added CSS syntax check.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-24 09:44:18 +00:00
calvin
cce6affa17 Add --check-html option to check the HTML syntax.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-23 23:04:31 +00:00
calvin
9f77f97434 Add distclean target; use Python2.5 includes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-23 23:03:55 +00:00
calvin
df857aab8d Intern patterns now accept URLs with and without "www." prefixes
as default. This allows sites to check that use both variants.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3714 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 09:18:58 +00:00
calvin
5a2f89fa3d Add redirect warning for commandline URLs
If URLs given on the commandline are redirected, the automatic
intern patterns might not match anymore. A warning makes this
more prominent.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3712 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 09:18:36 +00:00
calvin
8fa01f32c2 Use LC_ALL instead of LC_MESSAGES
Windows platforms do not have LC_MESSAGES. Use LC_ALL instead.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 07:40:40 +00:00
calvin
c58dd965af Set HTML charset according to logger output encoding.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3708 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:56 +00:00
calvin
18c6e6e38a Set default_encoding on i18n init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3707 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:47 +00:00
calvin
d9f8bd3187 Properly set the locale in CGI scripts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:19 +00:00
calvin
cfc651550a Use set() instead of a list for the set of supported languages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3703 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:10 +00:00
calvin
fa48fe354d Use LC_MESSAGES locale, not default system locale in i18n
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3699 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:27:38 +00:00
calvin
963feb2288 Double Ctrl-C stops checking immediately without cleanup.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3696 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-20 23:56:25 +00:00
calvin
9c56f03ae3 Shutdown immediately when Ctrl-C is given twice
Try sys.exit() to shutdown immediately after Ctrl-C keyboard
interrupt was given twice.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3695 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:47:16 +00:00
calvin
8ae6d94b45 Improved error messages for exceptions
Prepend the exception name before the error message of exceptions.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3694 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:47:00 +00:00
calvin
67aed38df2 Bump copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3693 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:46:52 +00:00
calvin
4968f1b3cd Prevent empty exception values.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3690 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-18 07:42:24 +00:00
calvin
ba148a9d71 Proper MX DNS request fallback
Properly fall back to DNS A requests when no MX host could be found
for a mailto: URL.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3689 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:46 +00:00
calvin
9b7cf763ff Fix test for new www.example.org URL
Fix test data using www.example.org instead of imadoofus.org URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3688 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:29 +00:00
calvin
88242b4612 Compare to singletons with "is"
Make sure comparisons with singletons like None/True/False use
"is", not "==/!=".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3687 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:16 +00:00
calvin
4055721fd4 Use internal gzip2 module
Use the internal gzip replacement module gzip2 for all GzipFile handling.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3685 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:33:55 +00:00
calvin
17cd16185f Remove timestamp from gzipped files
Remove the timestamp from gzipped files since it might be a security
and/or privacy risk to include it.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3684 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:33:42 +00:00
calvin
bf277085e9 Regenerate HTML scanner with new flex version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3683 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:24:32 +00:00
calvin
1f5a2d47ea Syntax cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3682 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:24:21 +00:00
calvin
e178405748 Use example.{com,org} for example URLs
Use the guaranteed not available example.com and example.org DNS names
in example URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3681 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:23:34 +00:00
calvin
4ce0ddd166 Changes for future Python 3.x compatibility
Replace backticks with repr(), replace .has_key() with "in".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3680 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-19 10:22:57 +00:00
calvin
370bd058ea Add htmlsax.so target for local build
Add target to build htmlsax.so locally. Also add include path
for local python SVN repository for testing.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3678 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-19 10:21:52 +00:00
calvin
91a0aad5d8 Fix buggy persistent HTTP connections
Workaround for buggy servers that break protocol synchronization of
persistent HTTP connections.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3677 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:43 +00:00
calvin
67e55d3832 Revert "Update httplib2.py from upstream SVN"
This reverts commit 00937008e0c2e6d86cf8d9e9c2d54ff5d7443dcc.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:33 +00:00
calvin
f0faf1b155 Update httplib2.py from upstream SVN
Added some bugfixes from the Python upstream httplib.py.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3675 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:26 +00:00
calvin
1730097265 Prevent Unicode errors for non-ASCII emails
Prevent Unicode errors when email address contains non-ASCII characters.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3673 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-02 23:38:02 +00:00
calvin
860def8d34 Remove superfluous path slash
Really fix the test_misc unit test by removing a superluous path slash.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3672 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:46 +00:00
calvin
13df77c0b5 Added .gitignore files
Ignore files for git version tracking system.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3671 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:29 +00:00
calvin
294261d80a Replace hardcoded test paths for test_misc
Replace the hardcoded test paths with variables. Fixes failures
in the test_misc unit test.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3670 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:11 +00:00
calvin
e1b1b7d916 Regenerate HTML lexer with flex 2.5.34
The HTML lexer .c file has been regenerated with a new upstream
release of flex 2.5.34.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3669 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:00 +00:00
calvin
f01a77bab1 Don't parse '-->' as end-of-comment in script mode. This fixes parsing errors on some sites.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3668 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-23 09:43:39 +00:00
calvin
8c4d8145a7 simplify the CDATA matching rules to be more straightforward
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3667 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-23 09:11:50 +00:00
calvin
7cf9723b10 don't parse <script for=''> as URL
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3659 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-11 16:45:30 +00:00
calvin
6499cb1a63 updated copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3658 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-02 14:31:19 +00:00
calvin
c99b9b1e8f added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-28 08:25:40 +00:00
calvin
c971ebdabf Added Shockwave Flash (SWF) parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3656 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-28 02:12:48 +00:00
calvin
ad7c9bbc76 Don't print cached errors or warnings unless verbose output is requested.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-14 10:58:10 +00:00
calvin
1919c30bdf Do not throw internal errors when writing from a thread to a non-opened file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3638 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 12:25:21 +00:00
calvin
6c07be042d Add optional leading dot for cookie domain value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3637 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 11:52:10 +00:00
calvin
fddf890bd4 Allow spaces in cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 11:44:18 +00:00
calvin
30d2b4f520 HTTP content data is only considered valid for parsing if the request was not redirected and is a GET request.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3633 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 10:50:13 +00:00
calvin
41bc0b2b32 use 'self.data is None' to test if data is already read or not
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3631 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 14:09:20 +00:00
calvin
5591bbe052 fix self.downloadtime to self.dltime
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3630 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 14:01:36 +00:00
calvin
7a4c7e9f44 remove unused imports reported by pyflakes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3629 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 13:54:44 +00:00
calvin
8e6c6455ab add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3626 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:56:15 +00:00
calvin
09ce26d5fe removed debug flag, test the LOG_CHECK logger for debug settings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:50:59 +00:00
calvin
ebb428044c Simplify option parsing: check option existance before access instead of catching an exception.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:50:33 +00:00
calvin
8d2dc781e1 Ensure unused or expired connections are closed.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 16:42:41 +00:00
calvin
042f70115f updated copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3616 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 14:57:03 +00:00
calvin
f8a54faae9 make sure internpat does not remove a trailing slash, which results in checking of URLs that are not a prefix of the given URL.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3613 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 10:37:14 +00:00
calvin
9cf3314eab Use constants for warning tags, avoiding typos in string constants. And move the constants into a separate module const.py
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-29 07:50:22 +00:00
calvin
e007ea5dae fix warning typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-29 06:37:40 +00:00
calvin
fcde8bd4d6 try to detect unknown URL schemes instead of manually setting the assume_local flag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:46:50 +00:00
calvin
a50784042f make sure URL to test for IDNA encoding errors has non-ascii characters
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3608 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:40:33 +00:00
calvin
40b3be412b revert the catch UnicodeError change
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3607 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:39:12 +00:00
calvin
6a0960aa66 only store parser contents in LinkFinder handler, not in all handlers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3602 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 19:06:39 +00:00
calvin
a1d911127b remove comments from CSS files before parsing for links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 18:46:14 +00:00
calvin
cb588a3c5d replace tabs with spaces
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3598 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:55:55 +00:00
calvin
ce8b963dd9 more code cleanups and documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3596 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:46:18 +00:00
calvin
370749cafb cleanup the code and add some documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:34:50 +00:00
calvin
17906ca1e0 use Python 2.4 for local builds
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3592 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-13 12:38:01 +00:00
calvin
c93fc79702 ignore errors in idna encoding of hostnames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3591 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-13 12:36:42 +00:00
calvin
e9c973fe06 Honor urllib.proxy_bypass() when ignoring proxy settings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3583 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-23 22:56:44 +00:00
calvin
ad121779af don't double encode info text in html output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3578 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-16 17:59:57 +00:00
calvin
b4289c6ebe added password protected site
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3577 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:07:12 +00:00
calvin
fe438941a9 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3576 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:06:24 +00:00
calvin
e4c3e81b49 add Python SVN tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3575 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:05:40 +00:00
calvin
2edfaea03e Read complete body data on persistent connections, else subsequent requests could fail.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3568 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 19:33:10 +00:00
calvin
2b94c0c161 Assume missing HEAD requests for Zope server on text/plain content type
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3567 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 18:26:55 +00:00
calvin
d5685ba4dc fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3566 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 18:08:53 +00:00
calvin
69a38bfd37 Set SSL connection object to None to release it on closing.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3565 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-07 14:50:46 +00:00
calvin
cd95ac054f fix message typo for not disclosing information
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-07-22 22:54:37 +00:00
calvin
4c0620c498 use default python
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3559 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-16 22:08:41 +00:00
calvin
88d410a445 don't use repr() on unicode names, else it prints escape chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-13 00:36:23 +00:00
calvin
5aed37dada use german server for faster testing (at least for me)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3553 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-13 00:35:32 +00:00
calvin
4093da56db add result attribute
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3552 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:57:55 +00:00
calvin
590cfc95f4 add created attribute
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:53:38 +00:00
calvin
702885d7d8 Use standard UTF-8 charset name for XML output encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3549 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:41:40 +00:00
calvin
9de237b4c2 Check that charset is not None before lowering it in set_encoding().
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3547 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-03-21 19:32:19 +00:00
calvin
32fe705358 var typo fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-21 22:22:24 +00:00
calvin
a78811a422 sanitize the configuration
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3540 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-21 22:17:02 +00:00
calvin
56d98f9d71 mention that --anchors enables logging of warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3536 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-19 19:24:52 +00:00
calvin
df48d4a905 bump up copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-01-01 14:57:38 +00:00
calvin
2e5a5d20df prepare for Py_ssize_t conversion
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3531 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-08 22:07:31 +00:00
calvin
b274787c5b prepare for Py_ssize_t conversion
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3530 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-08 22:07:23 +00:00
calvin
7397929e9e updated from SVN
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:53:47 +00:00
calvin
dcc1852afb use new exception syntax for raising
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:52:09 +00:00
calvin
be4d9a2468 added patches from SVN
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:49:02 +00:00
calvin
7fb471eb2e ignore errors on log file open, display a warning instead
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3518 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-21 17:10:27 +00:00
calvin
d208818694 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3517 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-21 17:09:49 +00:00
calvin
c217b6d441 don't set result on self.get_content() redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3515 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-17 20:42:00 +00:00
calvin
db55727f69 add missing functions to ListDict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3511 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-06 22:12:05 +00:00
calvin
61f41c4949 fix the empty useragent test case
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3509 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 20:37:21 +00:00
calvin
3f099a6438 use boolean objects for rule line allowance
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3508 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 20:36:31 +00:00
calvin
698f7183bc fix vrfy error message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3507 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:57:29 +00:00
calvin
ce0f1cd62a use Py_MEMCPY macro
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3506 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:31:14 +00:00
calvin
2af8dcbead remove unused include
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3505 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:30:55 +00:00
calvin
e29704ce61 use Py_VISIT macro for cycle traversal
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3503 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:08:21 +00:00
calvin
95f9dcbbc5 fix error messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3486 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:24:41 +00:00
calvin
bef2494211 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3482 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:13:59 +00:00
calvin
19c2ce7fe0 catch thread errors separately
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3481 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:05:10 +00:00
calvin
5d26f2a5cd remove unused import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3480 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:04:41 +00:00
calvin
2c563d8bae fix variable typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3479 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:04:33 +00:00
calvin
980fe1ea31 replace numerated config entries with numerical ones
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3477 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 14:30:27 +00:00
calvin
d3e8b99ff2 print active URLs on interrupt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3476 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 13:43:54 +00:00
calvin
1646a2e9de label check threads with 'Check-' prefix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3475 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 13:42:32 +00:00
calvin
1883b79303 follow redirections when getting HTTP contents
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3473 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 09:27:38 +00:00
calvin
0c5d34e9f9 don't discard robots.txt entries with only Allow: lines
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3471 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 09:14:28 +00:00
calvin
3f89988e67 remove unused staticforward
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3469 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-18 19:59:51 +00:00
calvin
9682ce5c3a use Py_CLEAR for reference count decrement
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3468 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-18 19:19:30 +00:00
calvin
a49896b581 fixed syntax errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3463 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 09:17:36 +00:00
calvin
5ad59225a0 use dictionaries for translations with multiple arguments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 09:12:59 +00:00
calvin
ae245287b6 ignore keyboard interrupts during shutdown
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 08:41:15 +00:00
calvin
691f285f83 fix test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3454 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:27:28 +00:00
calvin
576d404ce2 close non-idle connections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3453 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:27:19 +00:00
calvin
229e352c0c improve BadStatusLine exception message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3452 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:26:51 +00:00
calvin
04f89d0668 use get_url_from helper alias
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3451 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:18:49 +00:00
calvin
72d198efcb don't send keep-alive header, it breaks some tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3450 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:18:29 +00:00
calvin
86514bb882 activate asset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3448 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:21:06 +00:00
calvin
fcd00abc75 don't close socket if response is still open
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3447 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:19:03 +00:00
calvin
7b39b6c1b6 add HTTPConnection.is_idle(), print response in HTTPConnection.__str__()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3446 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:10:14 +00:00
calvin
ba7eaeae09 ignore geo location info lines in test output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3445 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:03:54 +00:00
calvin
6348205dcc add persistent connections back to the connection cache, close all others
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3444 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:59:30 +00:00
calvin
d6676ab0a0 more response closing, and cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3443 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:51:02 +00:00
calvin
6fe2db6755 use unicode_safe alias helper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3442 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:46:55 +00:00
calvin
27a8869783 use helper alias for unicode_safe
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3441 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:34:03 +00:00
calvin
2a75dff16e add __str__ methods, and refine a self.sock close test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3440 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 18:07:13 +00:00
calvin
15dfaf35cb cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3438 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:36:44 +00:00
calvin
4b818cb4b3 Detect more cases to close the connection, and close response objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3437 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:35:53 +00:00
calvin
d22b6b5c68 add more infos to exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3436 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:33:48 +00:00
calvin
da15b15923 Split off the host wait time function, and use it with a separate lock
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3434 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 12:18:24 +00:00
calvin
a1eff0bd3e don't use a debug lock per default
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3433 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 11:30:55 +00:00
calvin
9f39498abf don't restrict socket buffer size
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3430 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 09:02:48 +00:00
calvin
c3b5e0d59d use generators and enumerate in containers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3428 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-04 17:02:34 +00:00
calvin
0a5c03536d remove unneeded logger arguments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3425 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-21 20:50:10 +00:00
calvin
f78d9bb337 s/fields/parts/ for logger arguments, and supporess the last modified info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3424 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-21 20:50:00 +00:00
calvin
01cf921816 remove old signal check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3419 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-09 19:51:12 +00:00
calvin
9d15f9a8be allow inifinite recursion in CGI script
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3417 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-07 22:18:31 +00:00
calvin
d7d9990511 use raw config parser, no need for interpolation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3411 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-02 20:12:40 +00:00
calvin
5abe8fd8ce upstream update
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3406 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-22 23:35:18 +00:00
calvin
313e6d3caa recompiled with bison 2.3
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3405 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-22 21:23:14 +00:00
calvin
a311497e66 fall back to ASCII if default locale is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3403 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-19 16:42:43 +00:00
calvin
adc4e8c0e8 quote base reference URL, with tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3402 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 08:07:46 +00:00
calvin
c6f01faab5 improved debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3401 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:43:36 +00:00
calvin
0887002783 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3400 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:40:21 +00:00
calvin
0df2478430 improve URL norm tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:39:55 +00:00
calvin
ed98b6fc27 fix intern/extern semantic
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3382 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:58:52 +00:00
calvin
40399ca6c9 minor debug msg improvement
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3381 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:58:27 +00:00
calvin
bb9fbf6d26 fix CSS in HTML output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3380 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:56:37 +00:00
calvin
46dfeb50ef catch correct exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3372 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-14 06:44:52 +00:00
calvin
6f0dbb5058 copy Queue.Queue code, for Python2.5 compatibility
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3371 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-13 22:22:29 +00:00
calvin
7016503a45 use PyObject_Del instead of PyMem_DEL, fixing possible segfault
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3370 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-10 20:56:11 +00:00
calvin
93b84683cf remove unused LRU class, add more caseless dict tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3367 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-06 17:52:36 +00:00
calvin
84741e4f63 recompile with bison 2.2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3365 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-06 06:47:36 +00:00
calvin
6b0cf48959 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3360 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:45:15 +00:00
calvin
e07d6f024b test caseless dicts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3359 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:36:33 +00:00
calvin
2df9a7eb26 add del method to caseless dict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3358 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:36:24 +00:00
calvin
35ef77ab63 remove unused functions xmlunquote*
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3357 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:23:09 +00:00
calvin
b9150f1bf6 fix blacklist file writing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3356 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:21:01 +00:00
calvin
af255b67e0 add decode() method; fix file flush in case self.fd is None; expand user name of filenames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3355 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:20:38 +00:00
calvin
4c83e19454 get rid of unused self.fd is None checks
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3354 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:00:48 +00:00
calvin
df7bf82baf check modified time after parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3353 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:00:21 +00:00
calvin
7a31cb7ede add tests for file output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3351 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:47:09 +00:00
calvin
7781fe88ce use relative imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3350 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:46:47 +00:00
calvin
7667f3402f send short keep-alive header value for test server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3349 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:46:23 +00:00
calvin
e8e6a8af9a set modified time after parsing of robots.txt entries
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3348 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:44:59 +00:00
calvin
cf75e543b3 use self.filename, not args['filename'] in __init__()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3347 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:44:30 +00:00
calvin
12946ec9f7 more test coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3344 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 13:30:54 +00:00
calvin
43c0e3447a reduce wait/sleep time for URL queue getting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3341 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-04 23:51:32 +00:00
calvin
19a7495b9e only accept ASCII robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3339 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-04 21:07:08 +00:00
calvin
d95d8c3d96 correctly handle internal errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3338 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-03 01:14:05 +00:00
calvin
21b53215e4 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3337 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:13:12 +00:00
calvin
997e686a69 no exit on internal errors in threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3336 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:13:04 +00:00
calvin
a57618a4ad use relative imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3335 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:06:19 +00:00
calvin
c3fa9d7965 don't generate empty output files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3332 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 13:58:13 +00:00
calvin
677458b8d6 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3330 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 13:54:55 +00:00
calvin
8763a42063 skip a test on nt platforms
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3327 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 16:08:37 +00:00
calvin
4f668b8f90 added file norm tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3326 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 14:22:37 +00:00
calvin
850684b1e0 datadir as url path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3325 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 14:00:45 +00:00
calvin
2888c34859 split file tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 13:54:40 +00:00
calvin
5b8b2cbda1 use diff for file compare
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3323 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 13:43:10 +00:00
calvin
2e1102a1b5 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3322 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 12:57:51 +00:00
calvin
f5f7007c34 added abstract run_checked()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3316 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 14:25:04 +00:00
calvin
b442809838 look that cached URLs get checked quickly in large queues
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3315 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:35:24 +00:00
calvin
e211d3fd6c fix internal error call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3314 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:33:23 +00:00
calvin
694f05405a translate interrupt message, and check for changed signal handler
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3313 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:33:01 +00:00
calvin
cb853c4672 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3312 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:32:32 +00:00
calvin
4c8dcb012b added a stoppable thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3305 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 22:29:22 +00:00
calvin
c91ccc56d6 add and use a stoppable thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3303 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 22:16:36 +00:00
calvin
c3afa5c9c6 improved tmieit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3299 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:54:47 +00:00
calvin
b898cc05d3 added§
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3298 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:54:37 +00:00
calvin
e7107bc270 return function result in timeit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3297 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:52:45 +00:00
calvin
1974bdcaf1 use xrange
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3296 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:52:39 +00:00
calvin
d557b0208f whops, reintroduce url_data.url is None check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3295 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 00:00:33 +00:00
calvin
f30964679c remove return in __init__
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 23:59:09 +00:00
calvin
f746b29522 documentation added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 22:04:30 +00:00
calvin
27997b0251 add a finish() method to wait for spawned threads to finish
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3290 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 19:55:35 +00:00
calvin
7a94995345 remove unused condition
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3289 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 19:51:22 +00:00
calvin
49a21901f9 raise correct Empty()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3288 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 18:57:26 +00:00
calvin
eaf1fd4ba1 use timeout for get() method, return thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3286 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:33:09 +00:00
calvin
a1905bdb22 support timeout in get() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3285 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:33:00 +00:00
calvin
5eec9cf527 update from subversion repo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3284 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:32:50 +00:00
calvin
97e20ccdd4 move status method to status package
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3283 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-22 21:35:41 +00:00
calvin
ff1e16230b wait until status thread is finished
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3282 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-22 21:35:20 +00:00
calvin
e51bd72957 use xrange
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3280 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 23:02:46 +00:00
calvin
98597c267d quote result line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3272 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:21:20 +00:00
calvin
3142663135 added tests for UnicodeError 'label too long'
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3270 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:13:28 +00:00
calvin
7e1e01bd36 do not catch UnicodeError, handle that intern
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3269 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:13:16 +00:00
calvin
2c13d7cac1 norm test urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3267 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:43:55 +00:00
calvin
608f8ba1c3 prepare filenames as URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3266 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:38:57 +00:00
calvin
37615dba02 use datadir, curdir placeholders
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3265 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:35:52 +00:00
calvin
14a29fb015 prepare filenames as URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3264 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:35:36 +00:00
calvin
0ba1520d13 fix filename for test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3263 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:14:30 +00:00
calvin
1dbc97abe7 script moving
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3262 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:10:16 +00:00
calvin
23879d78d4 adjust test result for new cache optimization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3259 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 19:14:54 +00:00
calvin
cd8886c77f adjust test results for optimized cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3258 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 18:55:03 +00:00
calvin
7f408cce19 fix the self.in_progress optimization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3257 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 18:54:41 +00:00
calvin
d43acde696 added cookie parse tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3251 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:31:49 +00:00
calvin
06956060b5 added documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3250 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:31:33 +00:00
calvin
003532e20f put in-progress URLs back to the top of the URL queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3248 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:16:19 +00:00
calvin
17555e2fa6 use configured timeout on abort, interrupt main
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3241 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 20:19:40 +00:00
calvin
74327404a4 ensure the signal module is available
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3240 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 20:19:13 +00:00
calvin
faddd9acc3 catch Timeout from abort
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3239 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:56:14 +00:00
calvin
781ccf96c1 add irc scheme to netloc using schemes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3238 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:12:29 +00:00
calvin
2ec5c054fe merge ignoredurl and errorurl into unknownurl, updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3237 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:08:40 +00:00
calvin
a4e9b8eab1 fix debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3236 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 16:24:53 +00:00
calvin
4ec74f6f5c added robots.txt tests for the internal HTTP server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3232 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:47:48 +00:00
calvin
4bf2b361cb added more crawldelay tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3230 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:34 +00:00
calvin
e574a97798 raise ValueError if wait delay is a negative value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3229 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:27 +00:00
calvin
7dce5c4df9 fix callback call for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3228 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:09 +00:00
calvin
3adaf48b3d add callback for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3227 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:35:58 +00:00
calvin
a741d7922c add get_crawldelay method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3226 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:35:48 +00:00
calvin
fb319b3785 add callback after robots.txt parse
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3225 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 23:19:18 +00:00
calvin
2aed0f3bc5 add per-host wait times
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3224 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 23:03:44 +00:00
calvin
811f5492c4 fix --pause to delay requests to the same host
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:56:13 +00:00
calvin
ad28599e57 Note if URL is missing (instead of saying it is empty)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:15:34 +00:00
calvin
00a60c6906 check if urldata.url is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3219 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:14:34 +00:00
calvin
224f5e723b don't find link name in empty urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3218 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:05:17 +00:00
calvin
72c718b7d5 added robots.txt parse test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3212 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 21:29:31 +00:00
calvin
d73aa0e5bd parse crawl-delay parameter line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3211 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 21:29:18 +00:00
calvin
75e88c062a added --cookiefile option to set initial cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3210 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 20:56:34 +00:00
calvin
c6b5759c39 print thread name on trace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3206 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 17:00:49 +00:00
calvin
0bb2970222 remove old consumer code
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3205 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 17:00:27 +00:00
calvin
b6ad3084aa added more anchor tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3201 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 16:22:56 +00:00
calvin
7c8ab1b6f7 No maximum size due to possible deadlocks.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3195 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 16:14:42 +00:00
calvin
e8bf93bd90 inherit from StandardError, not from Exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3194 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 22:22:24 +00:00
calvin
c10da15b08 add timeout setting to configuration file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3192 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 19:04:21 +00:00
calvin
61fc7ab502 since join() is not interruptable, put in a little sleep() call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3191 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:36:55 +00:00
calvin
ca8ea4a2be print in-progress URLs on status
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3189 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:35:37 +00:00
calvin
ac682df355 be precise for threads: zero threads disable threading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3188 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:35:16 +00:00
calvin
b3b45d7f06 add Timeout exception, and return in_progress URLs on status instead of unfinished
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3187 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:34:30 +00:00
calvin
b1e7ca58a2 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3186 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:33:38 +00:00
calvin
dc62353ca2 don't catch SystemExit, and no bare except
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3185 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:06:44 +00:00
calvin
ffe233c2eb don't catch SystemExit, and no bare except
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3184 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:05:52 +00:00
calvin
938d193446 store timeout in configuration
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3183 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:05:17 +00:00
calvin
c8bcdf7d79 remove unused threader class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3182 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:02:07 +00:00
calvin
70fedc052c replace bare excepts with ones catching StandardError
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3181 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:01:29 +00:00
calvin
df8af83508 use debug lock class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3179 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 19:58:55 +00:00
calvin
ab1ba455f6 make lock class just a debug wrapper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3178 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 19:58:46 +00:00
calvin
ffeaa3484e add debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3177 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 18:03:59 +00:00
calvin
df7a2076a9 cache DNS lookups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3176 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 18:03:23 +00:00
calvin
afdccbaa80 set names of worker and status threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3175 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:50:36 +00:00
calvin
523e6e8e43 use variables in result lines
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3174 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:49:58 +00:00
calvin
91ff370ed7 on redirection to different URL scheme take caching into account; adjust tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3173 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:47:50 +00:00
calvin
2a336f8dad put redirects in url queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3172 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:01:41 +00:00
calvin
2cfcb5c0bb avoid double timeouts by raising timeout errors in robots.txt retrieval
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3171 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 12:58:31 +00:00
calvin
9a431fde40 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3170 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 10:14:07 +00:00
calvin
5ba1324a73 an intern pattern matches both http and https URLs now
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3169 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 10:06:33 +00:00
calvin
853f1d7f51 always disable status thread after checking, and log end of output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3165 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 08:41:52 +00:00
calvin
ddf3703962 put status printing in a separate module, with a flag that can be disabled
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3163 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 08:29:44 +00:00
calvin
53e1bd993e fix test output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3162 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 21:25:59 +00:00
calvin
dc9f04e6dc adjust debug asserts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3159 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 21:03:21 +00:00
calvin
2947b4710d quote slashes in cgi values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3158 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:52:30 +00:00
calvin
857582c026 honor case where fallback class is not given
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3157 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:51:11 +00:00
calvin
ee0a7bb824 don't return True on debug, add shutdown method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3156 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:50:52 +00:00
calvin
67f39e516c synchronize the log method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3152 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:53:41 +00:00
calvin
0d39d6139f no active tasks in status
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3150 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:15:40 +00:00
calvin
7afe60c841 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3149 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:08:35 +00:00
calvin
98a6d1eb00 added missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3148 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:08:06 +00:00
calvin
b0c189d143 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3147 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:07:46 +00:00
calvin
f002c5f965 Replace the old threading algorithm with a new one based on Queue.Queue and consumer threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3146 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:44:52 +00:00
calvin
d05c68ef74 added path encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3145 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:41:46 +00:00
calvin
c33e6d5907 split up invalid* tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3144 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:41:10 +00:00
calvin
3e8a221110 acquire locks directly
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3143 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:40:22 +00:00
calvin
bfd52936a6 fix config parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3142 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 09:53:53 +00:00
calvin
ac9d88cd99 added config test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3141 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 09:51:50 +00:00
calvin
394abbbab5 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3137 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-12 14:51:09 +00:00
calvin
d667abac69 Don't encode the host header with idna if not necessary. Taken from Python subversion trunk.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3136 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-04 18:24:23 +00:00
calvin
c2f5d222e1 split off configuration parsing in a separate class, put config stuff into its own module directory
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3135 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 18:24:46 +00:00
calvin
d547b0ee31 fix mail tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3134 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 17:00:08 +00:00
calvin
0bd43753d3 remove old resource check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3133 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:59:58 +00:00
calvin
c35737ff3a use new long duration format
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3132 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:39:01 +00:00
calvin
58993644d8 use format_time from quodlibet source
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3131 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:35:37 +00:00
calvin
4b10271233 use unittest.TestCase for standard test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3129 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:16:57 +00:00
calvin
f9a59e616d added caseless dict classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3128 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:13:29 +00:00
calvin
bae6c90f43 documentation cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3127 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:12:53 +00:00
calvin
a764b1484b import cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3126 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:12:40 +00:00
calvin
b4d4b2ed94 cast to size_t
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3122 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-11 23:53:25 +00:00
calvin
787d9143fe don't call write_color with None argument
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3120 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 10:26:15 +00:00
calvin
31e853e0d7 remove unnecessary global statements
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3119 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:02:14 +00:00
calvin
02ecf152c6 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3118 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:01:49 +00:00
calvin
cce1d273a7 really return value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3117 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:00:44 +00:00
calvin
4845e926a9 always use default locale for translation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3116 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:00:23 +00:00
calvin
4f67591327 no bare excepts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3115 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:59:52 +00:00
calvin
2544c90708 fix detection of </script>
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3113 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:58:31 +00:00
calvin
6dacc9bf61 use has_* flags for resource availability
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3112 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:57:52 +00:00
calvin
78dfb4c293 whitespace normalization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3106 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:15:07 +00:00
calvin
b503ea15f4 updated cgi tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3105 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:14:49 +00:00
calvin
9f7dde02f8 use WConio on Windows for colored output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3104 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:14:15 +00:00
calvin
db223a1912 write label for graph
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3103 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:13:41 +00:00
calvin
38b83415dd correctly replace spaces in cgi params
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3102 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:12:43 +00:00
calvin
893300308c updated time format tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3101 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:12:30 +00:00
calvin
0132e4ca89 better time formatting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3100 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:05:45 +00:00
calvin
1cd74500ee no literal </script>, accept empty SGML comments, compress equivalence classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3099 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:04:40 +00:00
calvin
e42f1a0cec replace sf.net with sourceforge.net
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3094 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 16:35:39 +00:00
calvin
df66edb7c3 allow empty comment in HTML parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3091 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-10 10:48:25 +00:00
calvin
aeb698cbec add localhost to local interface list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3090 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-10 10:48:07 +00:00
calvin
1d724a034a Construct equivalence classes in flex scanner
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3089 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-09 22:38:51 +00:00
calvin
1bf4fac474 removed .cvsignore
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3086 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-24 20:09:03 +00:00
calvin
00310425ac updated copyright template
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3081 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-17 23:38:12 +00:00
calvin
68a4d32212 rename _synchronized in synchronize
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3079 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-16 19:47:37 +00:00
calvin
46ed49d148 ensure log output is unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3075 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-13 21:49:25 +00:00
calvin
276437c7d8 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3067 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-09 07:57:22 +00:00
calvin
287ec3b960 syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3060 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-04 09:15:11 +00:00
calvin
e56215235c cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3058 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 23:58:30 +00:00
calvin
c334b8db13 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3056 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 23:21:20 +00:00
calvin
1e0581e718 use iter* dict methods
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3054 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 22:15:41 +00:00
calvin
2788fc2a59 add modification time information functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3052 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 20:42:56 +00:00
calvin
6cda627f0f more tests added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3046 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 17:06:25 +00:00
calvin
2880274744 more test coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3045 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 23:31:39 +00:00
calvin
38a7b15aeb accept a dict as lru init val
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3044 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 23:00:03 +00:00
calvin
902949f27e some format fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3043 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:59:42 +00:00
calvin
fac0b0f793 remove unused quote_val method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3042 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:58:40 +00:00
calvin
3f6ac53fff added tests for more coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3041 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:58:11 +00:00
calvin
3a5f06cfa9 remove unused strip_quotes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3040 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:53:11 +00:00
calvin
a3e4780b38 quote attributes with unicode entity escapes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3039 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:40:57 +00:00
calvin
75be4d0bb6 fix entity resolving
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3038 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:40:02 +00:00
calvin
cbef33ec5e fix parsing of hexadecimal entities
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3037 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 20:06:59 +00:00
calvin
94e083315b allow escaped quotes in tag attributes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3036 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 20:05:59 +00:00
calvin
9fd7b24344 check return value of acquire()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3034 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 19:48:22 +00:00
calvin
b6c947442a documentation fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3033 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 19:48:02 +00:00
calvin
1df673a79d add url_unsplit method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3029 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:56:20 +00:00
calvin
5a4f1248c3 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3028 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:56:04 +00:00
calvin
0a257847e3 add a Buffer class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3026 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:55:40 +00:00
calvin
6c14cf1d8b syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3025 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:54:11 +00:00
calvin
a2ed1b323f do not lowercase the absolute url, caused the URL matching not to work for URLs with uppercase characters
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3022 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:49:31 +00:00
calvin
0681b1889f limit amount of read chunks, and cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3017 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-11 22:03:58 +00:00
calvin
0bb41a5ec3 do no print warning about different dns message ids
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3016 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 22:32:42 +00:00
calvin
bb3dac673f added coding line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3015 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:04:29 +00:00
calvin
fc5e34b216 added more decorators
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3014 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:02:50 +00:00
calvin
e94c61529b catch value error on codec lookup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3013 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:02:32 +00:00
calvin
48cdfd32ee fix tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3011 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-04 01:20:20 +00:00
calvin
e92aee054c updated copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3010 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 19:12:47 +00:00
calvin
fe24c56e84 only split host ports on known schemes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3009 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 18:37:42 +00:00
calvin
b4748cdcee back out decorator change
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3002 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 01:09:23 +00:00
calvin
bc846a20ff use own splitport method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3001 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-02 14:26:24 +00:00
calvin
6b6adab12f new decorator_with_args helper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3000 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-02 14:25:50 +00:00
calvin
388475cbe2 new-style exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2999 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 20:17:33 +00:00
calvin
7b3949eebf cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2996 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 09:09:32 +00:00
calvin
8d67ee35ef removed is_safe_js_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2995 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 09:08:51 +00:00
calvin
a24fd65943 always quote port value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2992 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-19 14:08:34 +00:00
calvin
815fb9da83 more tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2991 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-19 14:06:53 +00:00
calvin
c84a33c7ce syntax fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2988 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 09:13:23 +00:00
calvin
856ff8ef2a assert debugs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2987 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:55:42 +00:00
calvin
aec847e570 assert debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2986 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:46:48 +00:00
calvin
1cec04018f cleanup unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2985 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:40:22 +00:00
calvin
19c0a3c2ed use new cookie parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2983 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:19:11 +00:00
calvin
5eb3061ee0 added new cookie parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2982 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:18:32 +00:00
calvin
a22d055d0b make debug() function assert'able
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2981 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:17:18 +00:00
calvin
d28567753b add is_encoding() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2980 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:16:36 +00:00
calvin
cff9b1341b improved charset parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2979 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:16:25 +00:00
calvin
24e91ffaaf fix info logging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2978 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:15:49 +00:00
calvin
d7a4a08659 updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2974 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-17 19:22:44 +00:00
calvin
3b1b95af5a added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2970 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-17 19:18:08 +00:00
calvin
9425873830 cache aliases (from redirects)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2964 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:20:21 +00:00
calvin
43fb20371f new tagged infos
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2963 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:20:04 +00:00
calvin
b34f4da7fe better error message if no nameservers could be found
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2962 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:19:48 +00:00
calvin
47b4a52134 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 17:14:39 +00:00
calvin
791bd87f6d use LinkCheckTest
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2956 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 17:08:08 +00:00
calvin
ee2d4e6ead moved ftests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2954 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:40:19 +00:00
calvin
4cf6f29afe unused import removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2953 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:40:09 +00:00
calvin
26ccd4d81c fix func name typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2952 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:39:17 +00:00
calvin
329043bc9f use datadir
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2951 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:33:14 +00:00
calvin
98e3e26644 moved
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2950 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:32:48 +00:00
calvin
ddb09ab76a added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2949 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 21:55:16 +00:00
calvin
c86c0870d6 debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2946 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 22:32:31 +00:00
calvin
b733ffdb2a remove has_incoming, debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2945 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 22:32:14 +00:00
calvin
3c03728d29 add update_meta utility func
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2942 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 17:10:17 +00:00
calvin
b482d35656 fix exception name and add 0.0.0.0 to localhost list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2941 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 11:47:38 +00:00
calvin
99c0382781 add timeit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2940 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:50:44 +00:00
calvin
e2a454e31f use new trace module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2939 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:49:37 +00:00
calvin
ae508129f1 move trace into separate module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2938 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:49:22 +00:00
calvin
7d1ea45be6 rebuild with bison 2.1
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2937 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:47:30 +00:00
calvin
bcbbe6ff60 allow https js urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2935 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:46 +00:00
calvin
a4cfa10fdf better comment parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2934 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:31 +00:00
calvin
f65cfb87f9 fix tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2933 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:01 +00:00
calvin
065665a0f1 rename builtins
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2932 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:55:54 +00:00
calvin
77ed566471 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2931 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-09 07:46:22 +00:00
calvin
9eab0056ac encode url query
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2930 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-09 07:46:05 +00:00
calvin
e29efc31fa more checkes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2928 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:06:29 +00:00
calvin
c049837cdd remove parser messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2927 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:05:59 +00:00
calvin
de2e66e713 remove parser errors/warnings, and better error detection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2926 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:05:35 +00:00
calvin
dffe4f906e documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2925 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:56:28 +00:00
calvin
35f1172481 count errors in tests, and add some more cases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2924 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:56:12 +00:00
calvin
b8aa3009d8 store a list of errors, and add more error messages to the parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2923 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:53:26 +00:00
calvin
7d5be699df documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2922 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-02 13:49:03 +00:00
calvin
0dca65accf depend on python >= 2.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2921 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-02 13:48:53 +00:00
calvin
7e90eccc84 dnspython 1.3.5 merged
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2920 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-31 08:59:29 +00:00
calvin
df34e1a8e9 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2919 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:48:30 +00:00
calvin
1d891fffec fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2918 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:45 +00:00
calvin
fa3d35beff beautify equals check msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2917 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:28 +00:00
calvin
c2b4132eb9 ensure html attr values are strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2915 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:03 +00:00
calvin
e3beb86ac1 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-17 12:18:18 +00:00
calvin
c9f5d1a0b1 catch gzip errors, and use linkchecker debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2910 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-15 00:06:48 +00:00
calvin
041df3a48d documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2906 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 22:54:23 +00:00
calvin
a2e422ce0d reindent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2900 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 22:26:12 +00:00
calvin
96f4f946b2 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2898 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 21:29:17 +00:00
calvin
d83720f05d do not parse any invalid config files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2895 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 21:15:31 +00:00
calvin
8ce45149fa re-add nofollow check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:49:56 +00:00
calvin
93c498699d disable keyboard interrupt and SystemExit while aborting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2888 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:11:36 +00:00
calvin
9e560f9ccf only return on thread wait timeout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:08:14 +00:00
calvin
c585d095b4 catch SystemExit from internal errors in subthreads, and eliminate abort wrapper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2886 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:06:57 +00:00
calvin
5303f94b76 ensure proxy value syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:59:20 +00:00
calvin
50ea41b12d use preferred locale for default encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2878 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:22:12 +00:00
calvin
753b907d51 remove unneeded imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2871 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:57 +00:00
calvin
66a30acc2d assume cStringIO is there
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2870 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:22 +00:00
calvin
6ca5dea8c3 define fs var
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2869 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:07 +00:00
calvin
19a290d81a query and anchor quoting tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2868 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:55:53 +00:00
calvin
9c8cee034a quote anchor
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2866 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:50:48 +00:00
calvin
8c809d1dc3 break long line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2865 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:50:05 +00:00
calvin
25044ce08b print errors in the preferred encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2864 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:49:52 +00:00
calvin
c205cec12d remove script tag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2859 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 20:09:40 +00:00
calvin
4e1bbfea64 adjust yyget_extra definition
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2858 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:16:19 +00:00
calvin
9eaf38d563 sane sparse path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2857 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:15:55 +00:00
calvin
9076eaa7da python2.4, add sparse check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2856 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:15:00 +00:00
calvin
732366b3f5 add string include for size_t
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2855 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 15:53:32 +00:00
calvin
8953766ccd fix url part quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 14:08:14 +00:00
calvin
2da64736ba remove unneeded init_i18n() calls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2852 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 13:00:33 +00:00
calvin
80b6ba5416 init i18n on module load
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2850 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:52:36 +00:00
calvin
6dd4ac6a65 typo fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2849 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:52:28 +00:00
calvin
7cf3f83c0f better recursion check debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:39:46 +00:00
calvin
978ba63999 catch UnicodeError on syntax check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2847 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:10:32 +00:00
calvin
fccccac818 coerce to unicode in url_norm if arg was unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2845 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:09:16 +00:00
calvin
8cfcbf075a use archived musi-cal robots.txt page
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2844 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 10:08:58 +00:00
calvin
b48c6f7e96 fix default resolver init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2843 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 09:55:26 +00:00
calvin
51f541607c catch CGI syntax errors in mailto: URLs and add a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2841 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 23:06:07 +00:00
calvin
65150ad79e coerce to unicode after relative file handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2839 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:50:40 +00:00
calvin
e1192c078f do not append non-parseable meta refresh tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2838 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:21:59 +00:00
calvin
2bba7df3e7 handle case where func.__doc__ is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2837 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:10:32 +00:00
calvin
57c9f325a9 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2836 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:10:10 +00:00
calvin
5b6782c6a9 handle ignoreX and nofollowY stuff backwards compatible
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2834 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:05:16 +00:00
calvin
5c83b7c859 fix error call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2833 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:59:46 +00:00
calvin
95081a62e6 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2829 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:53:52 +00:00
calvin
ba6a2ed4e0 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2819 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:15:09 +00:00
calvin
e11bc21593 new methods stripall and limit
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2818 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:14:05 +00:00
calvin
b28be779d7 ensure tags are ASCII, regen with bison 2.0
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2817 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:13:35 +00:00
calvin
4315e77ae1 dont use XXX markers here
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2816 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:12:41 +00:00
calvin
2ba2988d2c Add tests with invalid tag chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2815 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:12:12 +00:00
calvin
712df7d743 added quote_val
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2811 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:59:07 +00:00
calvin
a74e728246 ignore loopback interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:58:41 +00:00
calvin
fc4ce83372 dont quote exclams
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2809 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:57:36 +00:00
calvin
2f48855b32 added notimplemented
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2808 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:57:25 +00:00
calvin
da8a4d2cfe dynamic buffer for interface list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2807 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:56:54 +00:00
calvin
4b697eccea added is_ascii and indent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2806 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:55:51 +00:00
calvin
866999a883 added more tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2805 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:42:46 +00:00
calvin
37e8e21df6 add notimplemented
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-11 15:49:34 +00:00
calvin
d0dab1d0ac updated to dnspython 1.3.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2792 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-03 19:27:31 +00:00
calvin
33d3a6c974 fix invalid import in DNS resolver
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-03 18:47:32 +00:00
calvin
fe4395c41f test case for HTTP -> non-HTTP redirection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2787 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 16:05:20 +00:00
calvin
f01b84b894 rework the redirection routine a little, putting warnings specifically for redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2786 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:51:12 +00:00
calvin
ed932cb165 debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:50:27 +00:00
calvin
f24bb87e54 add missing return
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 12:37:32 +00:00
calvin
afa8750dc3 catch ValueError raised by urllib2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 09:45:30 +00:00
calvin
317ef181f0 handle all redirections to different URL schemes,
not just HTTP -> not HTTP, and fix a variable typo


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 08:53:57 +00:00
calvin
0d55ea45b7 fix variable clash
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-18 08:19:11 +00:00
calvin
b94eec79ef use correct value for status output of checked links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-17 23:26:13 +00:00
calvin
9fa3a1216c replace 'print' with 'write'
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-16 07:12:44 +00:00
calvin
c553e233b7 change the custom XML format
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2770 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-16 07:12:15 +00:00
calvin
842333a36f write parent line/col as attrs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2765 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:51:07 +00:00
calvin
c2e235a154 fix end xml tag writing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2763 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:46:13 +00:00
calvin
e2e80367d1 add copyright, add more documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2762 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:41:54 +00:00
calvin
19118a0198 make copyright ASCII
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2761 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:37:45 +00:00
calvin
0b5294ef57 set output encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2760 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:37:16 +00:00
calvin
3011eb5ee6 new custom XML logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2759 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:33:15 +00:00
calvin
6239880f15 simplify test suite generation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2756 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 14:24:25 +00:00
calvin
d347840dee use official HTTP status names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2754 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 11:31:32 +00:00
calvin
baf51d1f5d cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2752 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:56:05 +00:00
calvin
359612b6fe add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2751 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:53:27 +00:00
calvin
225c045815 do not cache unusual errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:11:13 +00:00
calvin
5d82d64c31 more result sanity check warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2749 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 13:35:18 +00:00
calvin
3fb9de0847 upstream updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 13:34:45 +00:00
calvin
c140b510b1 fix warning tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 19:33:26 +00:00
calvin
f80480469f config to ignore certain warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 15:03:17 +00:00
calvin
badb88ed68 use errno constant EPIPE
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 18:36:03 +00:00
calvin
5b5eda36e4 updated from python CVS
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2741 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 18:09:15 +00:00
calvin
b559c4f82e test for hierarchical URLs, and retain non-numeric port parts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 07:18:13 +00:00
calvin
907facaf16 put start_server into the try-finally clause
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2738 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 07:17:01 +00:00
calvin
34ced03fc2 added more norming tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 21:58:58 +00:00
calvin
5049e261fe added LOG_DNS
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2734 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 15:37:15 +00:00