Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-09 15:14:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
4173 commits 1 branch 0 tags 24 MiB
Commit graph

1975 commits

Author SHA1 Message Date
calvin
a1d36611a0 Added close button in options dialog 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-11 10:33:09 +00:00
calvin
94a960208b Upstream updates for httplib2.py 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3908 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-11 01:26:40 +00:00
calvin
2f43c25699 Remove access restrictions from CGI prog, leave that to the web server configuration. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3904 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-11 00:27:45 +00:00
calvin
73206acbcb Move GUI client into package. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3903 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-11 00:26:42 +00:00
calvin
de5b7e894e Added upstream fixes to patched Python modules. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3901 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 20:00:25 +00:00
calvin
c6cb09c4aa Add missing import 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3900 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 19:41:42 +00:00
calvin
1c50cf288a Ignore DNS MX lookup failures in py2exe. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3899 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 18:50:14 +00:00
calvin
4ffe29615e Updated DNS code 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3898 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 18:05:13 +00:00
calvin
41b02bdc27 Fix typo N0ne -> None 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3896 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 17:54:44 +00:00
calvin
cc25deac12 Only accept MX dns response types when asking for MX servers. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3895 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 17:53:10 +00:00
calvin
979132c9b5 Catch all DNS exceptions when resolving MX hosts. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3894 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 15:13:55 +00:00
calvin
a26ca4c23a Replace C ftpparse module with Python implementation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3892 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-10 14:11:17 +00:00
calvin
e28ce1a710 Rename init* functions of ftpparse to avoid name clash with Python module system. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-08 23:06:37 +00:00
calvin
e9805dbd8a Updated copyright year to 2009 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-08 14:18:03 +00:00
calvin
63d42c0d92 Replace deprecated sets module with builtin set when possible. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3886 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-08 12:38:08 +00:00
calvin
8d5d4827c3 Change ftpparse import to avoid py2exe load error. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-08 12:28:39 +00:00
calvin
4b71c92f99 updated httplib from python 2.6 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3876 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-12-05 02:17:13 +00:00
calvin
7d05f0f544 Added --complete option 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3872 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-30 19:35:56 +00:00
calvin
209d5abc18 fix timeouts by testing earlier for persistent connections with HEAD 
HEAD requests never have a body; nevertheless the http lib tries to
read() from them. This times out on some servers of course. Fix is
not to let those connections be persistent.

git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3871 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-29 08:14:28 +00:00
calvin
c20e706761 Made some format changes on translated strings. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3870 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-28 20:22:48 +00:00
calvin
1abc2c504d Filter invalid mozilla bookmark urls from places.sqlite 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3869 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-28 10:54:16 +00:00
calvin
c3b6fc5aa4 Readd 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3867 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-20 21:30:10 +00:00
calvin
42c3e71329 Improved and tested opera bookmark parser 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3863 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-20 07:52:02 +00:00
calvin
9ab895751f Support parsing of Firefox 3 bookmark files 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3862 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-20 07:51:22 +00:00
calvin
bc48ce8a96 Close robotparser URL connections; simplify line parsing. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-28 16:56:24 +00:00
calvin
0cade7b308 Improved forced shutdown on Unix and Windows. Unix uses SIGKILL now, Windows os.abort(). 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3852 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-28 16:56:02 +00:00
calvin
7b2a21c133 Fix off-by-one error in cookie domain matching code. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3850 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-13 12:52:11 +00:00
calvin
97cf700e04 Fixed wrong cookie debugging format line. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3849 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-13 12:51:56 +00:00
calvin
22e6a9e67d Fix encoding errors in robots.txt, making some sites like wikipedia.org 
accesible again.

git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-13 12:51:45 +00:00
calvin
523ee87f0c Add missing return in is_absolute_path() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3846 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-09 19:17:33 +00:00
calvin
f68872f559 Improved detection of absolute Windows paths. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3844 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-07-09 19:00:02 +00:00
calvin
573453d68b Remove trailing spaces 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3835 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-30 23:18:32 +00:00
calvin
b62c28256f Updated httplib2.py with version from 2.6 Subversion repository. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3834 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-30 23:18:19 +00:00
calvin
84355f7b94 Catch original httplib errors too since it is used indirectly by urllib functions. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3833 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-30 23:17:38 +00:00
calvin
3b94dd2810 Add more includes needed by MacOSX to compile. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3832 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-26 20:39:08 +00:00
calvin
f9695ed2e5 Set svn:eol-style native 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3831 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-26 20:36:38 +00:00
calvin
64e311e2d6 Move console related print functionality to director.console module; add --version commandline option to GUI client 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3830 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-26 11:11:09 +00:00
calvin
4a5a4ed0bf Fix compilation and usage under Windows system. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3827 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-18 09:51:54 +00:00
calvin
b30fb3b09c Remove duplicate code in http checker. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3820 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-16 19:52:09 +00:00
calvin
caf8ba6297 Really allow parsing of XHTML files; I forgot some places to adjust the MIME checking. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3818 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-16 13:03:48 +00:00
calvin
a6deeeb8a5 Support parsing of HTML pages served with content type application/xhtml+xml 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3817 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-16 09:39:49 +00:00
calvin
ff41aa8d9f Lower the MIME content-type info from HTTP headers befure using it 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3816 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-16 09:38:09 +00:00
calvin
d26386d03f Catch errors when getting content for title. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3814 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-12 15:38:26 +00:00
calvin
8085045ae9 Added extra status logger class 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3811 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-11 13:02:22 +00:00
calvin
f7a8b16096 Do not override local variable in for loop 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-11 13:01:56 +00:00
calvin
fa0c868773 Simplified Logger.writeln() method. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3808 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-11 13:01:36 +00:00
calvin
3b5e28ff58 Improved log end output: print number of duplicate error and warnings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-11 07:33:19 +00:00
calvin
a880939c40 Initialize variables in reset(), not in subsequent methods 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3796 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-08 09:27:13 +00:00
calvin
5588df8ebf Remove trailing whitespace 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3794 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-08 09:26:56 +00:00
calvin
aba08b8b7d Improved graph output: print labels as node names. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3793 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:08:54 +00:00
calvin
83bd7ce4ac Improved network interface detection on POSIX systems 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3792 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:08:36 +00:00
calvin
290528b84f Added title attribute to URL data. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:56 +00:00
calvin
4ade9a79cc Moved link and title parsing into htmlutil package 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3789 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:48 +00:00
calvin
99269d12cc Add base method for Url.get_title() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3788 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:38 +00:00
calvin
58e44dbf8e Updated documentation and todo list. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3787 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:30 +00:00
calvin
e8a4744643 Use enum utility function for thread priorities. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-05 07:14:04 +00:00
calvin
5f818f620a Backport namedtuple container from Python 2.6 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-05 07:13:55 +00:00
calvin
5f4d61e018 Use keyword arguments in translation strings. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:44:40 +00:00
calvin
97772c9700 Improved email check messages. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3779 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:44:07 +00:00
calvin
2e4d0894fc Stop checking a list of emails at the first invalid one. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3778 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:43:20 +00:00
calvin
e6e51dbc6b Overwrite old results when checking a list of emails. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3777 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:42:52 +00:00
calvin
66ff422f6b Allow overwriting of an old check result. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:42:38 +00:00
calvin
5bf875bebe Use configdata for application name 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-22 12:17:24 +00:00
calvin
7297519b04 Remove or replace unused variables. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-22 12:10:08 +00:00
calvin
2cb2ee9403 Use base log level for clamav warning 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3766 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 17:01:49 +00:00
calvin
9352dbf5e4 Move test files to separate module 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3763 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 17:01:16 +00:00
calvin
90587545f7 Catch attribute errors in logger flushing 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3761 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 16:57:31 +00:00
calvin
2f25962789 Match newlines in catch-all rules 
Avoid printing spurious newlines when HTML parsing. The "." does
not match newlines, correct that in the catch-all lexer rules.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3760 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 16:56:58 +00:00
calvin
670a1df9b3 Updated documentation for new Python 2.5 dependency 
Removed old references of Python 2.4, together with compatibility
hacks.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3757 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 10:49:00 +00:00
calvin
ce1d41a5a0 Document the CGI access variables 
The access variables ALLOWED_CLIENTS and ALLOWED_SERVERS are now
documented to help setup the CGI script more easily.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3756 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 10:48:07 +00:00
calvin
dbb498a395 Add virus checking 
New option --scan-virus to check the content of URLs for
viruses with ClamAV.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3753 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 08:57:37 +00:00
calvin
bacb59597e Use relative imports from Python 2.5 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-09 06:16:03 +00:00
calvin
b96e8120d6 Add W3C Validator checks 
Add new options --check-html-w3 and --check-css-w3 to allow checking
of HTML and CSS pages with the online W3C validators.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-08 10:36:54 +00:00
calvin
bc9b9ee07e Move http util function in a separate module. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3747 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-08 10:36:08 +00:00
calvin
d5ec7c1ac3 Encode error output to HTML charset 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-30 14:37:49 +00:00
calvin
df9f31dcb1 Only check HTML/CSS syntax of intern URLs 
The HTML and CSS syntax check now only applies to URLs
which match those given on the command line.
This makes checking of personal pages easier.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-29 17:48:47 +00:00
calvin
ac4d09f83d Fix errors in CSS and HTML syntax check 
Properly encode the warning messages as Unicode, and prevent
overwriting of the "log" module with a local variable.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-29 17:48:22 +00:00
calvin
92c74ece4d Send HTTP Referer header to both http and https URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3741 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-29 13:33:35 +00:00
calvin
5d8bdaaa1f Use generators instead of lists where possible 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-28 00:26:02 +00:00
calvin
3eac1be9ab Require and use Python 2.5 
Use Python 2.5 features and get rid of old compat code. Also some
code cleanups have been made.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-27 11:39:21 +00:00
calvin
017b73db27 Fix cookie test! 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3728 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-25 08:34:10 +00:00
calvin
72db31e546 Only check syntax of valid URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3726 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-25 07:53:11 +00:00
calvin
973da91f44 Source code cleanup: use or remove unused variables 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3724 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-25 07:49:52 +00:00
calvin
e266a65b64 Fix css check 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3723 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-24 10:46:19 +00:00
calvin
62efec3b35 Added CSS syntax check. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-24 09:44:18 +00:00
calvin
cce6affa17 Add --check-html option to check the HTML syntax. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-23 23:04:31 +00:00
calvin
9f77f97434 Add distclean target; use Python2.5 includes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-23 23:03:55 +00:00
calvin
df857aab8d Intern patterns now accept URLs with and without "www." prefixes 
as default. This allows sites to check that use both variants.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3714 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 09:18:58 +00:00
calvin
5a2f89fa3d Add redirect warning for commandline URLs 
If URLs given on the commandline are redirected, the automatic
intern patterns might not match anymore. A warning makes this
more prominent.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3712 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 09:18:36 +00:00
calvin
8fa01f32c2 Use LC_ALL instead of LC_MESSAGES 
Windows platforms do not have LC_MESSAGES. Use LC_ALL instead.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 07:40:40 +00:00
calvin
c58dd965af Set HTML charset according to logger output encoding. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3708 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 01:28:56 +00:00
calvin
18c6e6e38a Set default_encoding on i18n init 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3707 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 01:28:47 +00:00
calvin
d9f8bd3187 Properly set the locale in CGI scripts 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 01:28:19 +00:00
calvin
cfc651550a Use set() instead of a list for the set of supported languages 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3703 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 01:28:10 +00:00
calvin
fa48fe354d Use LC_MESSAGES locale, not default system locale in i18n 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3699 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 01:27:38 +00:00
calvin
963feb2288 Double Ctrl-C stops checking immediately without cleanup. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3696 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-20 23:56:25 +00:00
calvin
9c56f03ae3 Shutdown immediately when Ctrl-C is given twice 
Try sys.exit() to shutdown immediately after Ctrl-C keyboard
interrupt was given twice.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3695 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-19 07:47:16 +00:00
calvin
8ae6d94b45 Improved error messages for exceptions 
Prepend the exception name before the error message of exceptions.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3694 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-19 07:47:00 +00:00
calvin
67aed38df2 Bump copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3693 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-19 07:46:52 +00:00
calvin
4968f1b3cd Prevent empty exception values. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3690 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-18 07:42:24 +00:00
calvin
ba148a9d71 Proper MX DNS request fallback 
Properly fall back to DNS A requests when no MX host could be found
for a mailto: URL.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3689 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-14 22:34:46 +00:00
calvin
9b7cf763ff Fix test for new www.example.org URL 
Fix test data using www.example.org instead of imadoofus.org URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3688 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-14 22:34:29 +00:00
calvin
88242b4612 Compare to singletons with "is" 
Make sure comparisons with singletons like None/True/False use
"is", not "==/!=".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3687 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-14 22:34:16 +00:00
calvin
4055721fd4 Use internal gzip2 module 
Use the internal gzip replacement module gzip2 for all GzipFile handling.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3685 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-14 22:33:55 +00:00
calvin
17cd16185f Remove timestamp from gzipped files 
Remove the timestamp from gzipped files since it might be a security
and/or privacy risk to include it.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3684 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-14 22:33:42 +00:00
calvin
bf277085e9 Regenerate HTML scanner with new flex version 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3683 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-25 21:24:32 +00:00
calvin
1f5a2d47ea Syntax cleanups 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3682 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-25 21:24:21 +00:00
calvin
e178405748 Use example.{com,org} for example URLs 
Use the guaranteed not available example.com and example.org DNS names
in example URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3681 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-25 21:23:34 +00:00
calvin
4ce0ddd166 Changes for future Python 3.x compatibility 
Replace backticks with repr(), replace .has_key() with "in".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3680 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-19 10:22:57 +00:00
calvin
370bd058ea Add htmlsax.so target for local build 
Add target to build htmlsax.so locally. Also add include path
for local python SVN repository for testing.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3678 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-19 10:21:52 +00:00
calvin
91a0aad5d8 Fix buggy persistent HTTP connections 
Workaround for buggy servers that break protocol synchronization of
persistent HTTP connections.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3677 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-14 13:28:43 +00:00
calvin
67e55d3832 Revert "Update httplib2.py from upstream SVN" 
This reverts commit 00937008e0c2e6d86cf8d9e9c2d54ff5d7443dcc.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-14 13:28:33 +00:00
calvin
f0faf1b155 Update httplib2.py from upstream SVN 
Added some bugfixes from the Python upstream httplib.py.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3675 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-14 13:28:26 +00:00
calvin
1730097265 Prevent Unicode errors for non-ASCII emails 
Prevent Unicode errors when email address contains non-ASCII characters.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3673 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-03-02 23:38:02 +00:00
calvin
860def8d34 Remove superfluous path slash 
Really fix the test_misc unit test by removing a superluous path slash.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3672 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-02-08 10:38:46 +00:00
calvin
13df77c0b5 Added .gitignore files 
Ignore files for git version tracking system.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3671 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-02-08 10:38:29 +00:00
calvin
294261d80a Replace hardcoded test paths for test_misc 
Replace the hardcoded test paths with variables. Fixes failures
in the test_misc unit test.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3670 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-02-08 10:38:11 +00:00
calvin
e1b1b7d916 Regenerate HTML lexer with flex 2.5.34 
The HTML lexer .c file has been regenerated with a new upstream
release of flex 2.5.34.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3669 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-02-08 10:38:00 +00:00
calvin
f01a77bab1 Don't parse '-->' as end-of-comment in script mode. This fixes parsing errors on some sites. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3668 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-01-23 09:43:39 +00:00
calvin
8c4d8145a7 simplify the CDATA matching rules to be more straightforward 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3667 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-01-23 09:11:50 +00:00
calvin
7cf9723b10 don't parse <script for=''> as URL 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3659 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-01-11 16:45:30 +00:00
calvin
6499cb1a63 updated copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3658 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-01-02 14:31:19 +00:00
calvin
c99b9b1e8f added 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-28 08:25:40 +00:00
calvin
c971ebdabf Added Shockwave Flash (SWF) parsing 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3656 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-28 02:12:48 +00:00
calvin
ad7c9bbc76 Don't print cached errors or warnings unless verbose output is requested. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-14 10:58:10 +00:00
calvin
1919c30bdf Do not throw internal errors when writing from a thread to a non-opened file 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3638 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-13 12:25:21 +00:00
calvin
6c07be042d Add optional leading dot for cookie domain value 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3637 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-13 11:52:10 +00:00
calvin
fddf890bd4 Allow spaces in cookie values 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-13 11:44:18 +00:00
calvin
30d2b4f520 HTTP content data is only considered valid for parsing if the request was not redirected and is a GET request. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3633 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-13 10:50:13 +00:00
calvin
41bc0b2b32 use 'self.data is None' to test if data is already read or not 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3631 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-03 14:09:20 +00:00
calvin
5591bbe052 fix self.downloadtime to self.dltime 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3630 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-03 14:01:36 +00:00
calvin
7a4c7e9f44 remove unused imports reported by pyflakes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3629 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-03 13:54:44 +00:00
calvin
8e6c6455ab add missing import 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3626 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-01 15:56:15 +00:00
calvin
09ce26d5fe removed debug flag, test the LOG_CHECK logger for debug settings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-01 15:50:59 +00:00
calvin
ebb428044c Simplify option parsing: check option existance before access instead of catching an exception. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-01 15:50:33 +00:00
calvin
8d2dc781e1 Ensure unused or expired connections are closed. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-30 16:42:41 +00:00
calvin
042f70115f updated copyright 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3616 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-30 14:57:03 +00:00
calvin
f8a54faae9 make sure internpat does not remove a trailing slash, which results in checking of URLs that are not a prefix of the given URL. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3613 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-30 10:37:14 +00:00
calvin
9cf3314eab Use constants for warning tags, avoiding typos in string constants. And move the constants into a separate module const.py 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-29 07:50:22 +00:00
calvin
e007ea5dae fix warning typo 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-29 06:37:40 +00:00
calvin
fcde8bd4d6 try to detect unknown URL schemes instead of manually setting the assume_local flag 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-28 18:46:50 +00:00
calvin
a50784042f make sure URL to test for IDNA encoding errors has non-ascii characters 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3608 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-28 18:40:33 +00:00
calvin
40b3be412b revert the catch UnicodeError change 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3607 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-28 18:39:12 +00:00
calvin
6a0960aa66 only store parser contents in LinkFinder handler, not in all handlers 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3602 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 19:06:39 +00:00
calvin
a1d911127b remove comments from CSS files before parsing for links 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 18:46:14 +00:00
calvin
cb588a3c5d replace tabs with spaces 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3598 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:55:55 +00:00
calvin
ce8b963dd9 more code cleanups and documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3596 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:46:18 +00:00
calvin
370749cafb cleanup the code and add some documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:34:50 +00:00
calvin
17906ca1e0 use Python 2.4 for local builds 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3592 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-13 12:38:01 +00:00
calvin
c93fc79702 ignore errors in idna encoding of hostnames 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3591 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-13 12:36:42 +00:00
calvin
e9c973fe06 Honor urllib.proxy_bypass() when ignoring proxy settings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3583 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-10-23 22:56:44 +00:00
calvin
ad121779af don't double encode info text in html output 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3578 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-10-16 17:59:57 +00:00
calvin
b4289c6ebe added password protected site 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3577 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-10-02 01:07:12 +00:00
calvin
fe438941a9 cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3576 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-10-02 01:06:24 +00:00
calvin
e4c3e81b49 add Python SVN tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3575 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-10-02 01:05:40 +00:00
calvin
2edfaea03e Read complete body data on persistent connections, else subsequent requests could fail. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3568 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-08-08 19:33:10 +00:00
calvin
2b94c0c161 Assume missing HEAD requests for Zope server on text/plain content type 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3567 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-08-08 18:26:55 +00:00
calvin
d5685ba4dc fix typo 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3566 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-08-08 18:08:53 +00:00
calvin
69a38bfd37 Set SSL connection object to None to release it on closing. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3565 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-08-07 14:50:46 +00:00
calvin
cd95ac054f fix message typo for not disclosing information 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-07-22 22:54:37 +00:00
calvin
4c0620c498 use default python 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3559 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-06-16 22:08:41 +00:00
calvin
88d410a445 don't use repr() on unicode names, else it prints escape chars 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-06-13 00:36:23 +00:00
calvin
5aed37dada use german server for faster testing (at least for me) 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3553 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-06-13 00:35:32 +00:00
calvin
4093da56db add result attribute 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3552 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-04-03 00:57:55 +00:00
calvin
590cfc95f4 add created attribute 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-04-03 00:53:38 +00:00
calvin
702885d7d8 Use standard UTF-8 charset name for XML output encoding 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3549 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-04-03 00:41:40 +00:00
calvin
9de237b4c2 Check that charset is not None before lowering it in set_encoding(). 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3547 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-03-21 19:32:19 +00:00
calvin
32fe705358 var typo fix 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-02-21 22:22:24 +00:00
calvin
a78811a422 sanitize the configuration 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3540 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-02-21 22:17:02 +00:00
calvin
56d98f9d71 mention that --anchors enables logging of warnings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3536 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-02-19 19:24:52 +00:00
calvin
df48d4a905 bump up copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-01-01 14:57:38 +00:00
calvin
2e5a5d20df prepare for Py_ssize_t conversion 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3531 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-12-08 22:07:31 +00:00
calvin
b274787c5b prepare for Py_ssize_t conversion 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3530 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-12-08 22:07:23 +00:00
calvin
7397929e9e updated from SVN 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-12-06 23:53:47 +00:00
calvin
dcc1852afb use new exception syntax for raising 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-12-06 23:52:09 +00:00
calvin
be4d9a2468 added patches from SVN 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-12-06 23:49:02 +00:00
calvin
7fb471eb2e ignore errors on log file open, display a warning instead 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3518 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-11-21 17:10:27 +00:00
calvin
d208818694 added 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3517 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-11-21 17:09:49 +00:00
calvin
c217b6d441 don't set result on self.get_content() redirections 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3515 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-11-17 20:42:00 +00:00
calvin
db55727f69 add missing functions to ListDict 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3511 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-11-06 22:12:05 +00:00
calvin
61f41c4949 fix the empty useragent test case 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3509 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 20:37:21 +00:00
calvin
3f099a6438 use boolean objects for rule line allowance 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3508 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 20:36:31 +00:00
calvin
698f7183bc fix vrfy error message 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3507 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 19:57:29 +00:00
calvin
ce0f1cd62a use Py_MEMCPY macro 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3506 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 19:31:14 +00:00
calvin
2af8dcbead remove unused include 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3505 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 19:30:55 +00:00
calvin
e29704ce61 use Py_VISIT macro for cycle traversal 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3503 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-10-19 19:08:21 +00:00
calvin
95f9dcbbc5 fix error messages 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3486 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:24:41 +00:00
calvin
bef2494211 remove unused imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3482 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:13:59 +00:00
calvin
19c2ce7fe0 catch thread errors separately 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3481 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:05:10 +00:00
calvin
5d26f2a5cd remove unused import 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3480 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:04:41 +00:00
calvin
2c563d8bae fix variable typo 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3479 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:04:33 +00:00
calvin
980fe1ea31 replace numerated config entries with numerical ones 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3477 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-21 14:30:27 +00:00
calvin
d3e8b99ff2 print active URLs on interrupt 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3476 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-21 13:43:54 +00:00
calvin
1646a2e9de label check threads with 'Check-' prefix 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3475 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-21 13:42:32 +00:00
calvin
1883b79303 follow redirections when getting HTTP contents 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3473 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-21 09:27:38 +00:00
calvin
0c5d34e9f9 don't discard robots.txt entries with only Allow: lines 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3471 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-21 09:14:28 +00:00
calvin
3f89988e67 remove unused staticforward 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3469 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-18 19:59:51 +00:00
calvin
9682ce5c3a use Py_CLEAR for reference count decrement 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3468 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-18 19:19:30 +00:00
calvin
a49896b581 fixed syntax errors 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3463 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-16 09:17:36 +00:00
calvin
5ad59225a0 use dictionaries for translations with multiple arguments 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-16 09:12:59 +00:00
calvin
ae245287b6 ignore keyboard interrupts during shutdown 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-16 08:41:15 +00:00
calvin
691f285f83 fix test 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3454 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 21:27:28 +00:00
calvin
576d404ce2 close non-idle connections 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3453 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 21:27:19 +00:00
calvin
229e352c0c improve BadStatusLine exception message 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3452 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 21:26:51 +00:00
calvin
04f89d0668 use get_url_from helper alias 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3451 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 21:18:49 +00:00
calvin
72d198efcb don't send keep-alive header, it breaks some tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3450 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 21:18:29 +00:00
calvin
86514bb882 activate asset 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3448 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 20:21:06 +00:00
calvin
fcd00abc75 don't close socket if response is still open 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3447 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 20:19:03 +00:00
calvin
7b39b6c1b6 add HTTPConnection.is_idle(), print response in HTTPConnection.__str__() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3446 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 20:10:14 +00:00
calvin
ba7eaeae09 ignore geo location info lines in test output 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3445 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 20:03:54 +00:00
calvin
6348205dcc add persistent connections back to the connection cache, close all others 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3444 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 19:59:30 +00:00
calvin
d6676ab0a0 more response closing, and cleanups 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3443 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 19:51:02 +00:00
calvin
6fe2db6755 use unicode_safe alias helper 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3442 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 19:46:55 +00:00
calvin
27a8869783 use helper alias for unicode_safe 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3441 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 19:34:03 +00:00
calvin
2a75dff16e add __str__ methods, and refine a self.sock close test 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3440 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 18:07:13 +00:00
calvin
15dfaf35cb cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3438 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 16:36:44 +00:00
calvin
4b818cb4b3 Detect more cases to close the connection, and close response objects 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3437 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 16:35:53 +00:00
calvin
d22b6b5c68 add more infos to exceptions 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3436 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 16:33:48 +00:00
calvin
da15b15923 Split off the host wait time function, and use it with a separate lock 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3434 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 12:18:24 +00:00
calvin
a1eff0bd3e don't use a debug lock per default 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3433 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 11:30:55 +00:00
calvin
9f39498abf don't restrict socket buffer size 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3430 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 09:02:48 +00:00
calvin
c3b5e0d59d use generators and enumerate in containers 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3428 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-04 17:02:34 +00:00
calvin
0a5c03536d remove unneeded logger arguments 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3425 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-08-21 20:50:10 +00:00
calvin
f78d9bb337 s/fields/parts/ for logger arguments, and supporess the last modified info 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3424 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-08-21 20:50:00 +00:00
calvin
01cf921816 remove old signal check 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3419 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-08-09 19:51:12 +00:00
calvin
9d15f9a8be allow inifinite recursion in CGI script 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3417 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-08-07 22:18:31 +00:00
calvin
d7d9990511 use raw config parser, no need for interpolation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3411 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-08-02 20:12:40 +00:00
calvin
5abe8fd8ce upstream update 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3406 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-22 23:35:18 +00:00
calvin
313e6d3caa recompiled with bison 2.3 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3405 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-22 21:23:14 +00:00
calvin
a311497e66 fall back to ASCII if default locale is None 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3403 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-19 16:42:43 +00:00
calvin
adc4e8c0e8 quote base reference URL, with tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3402 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 08:07:46 +00:00
calvin
c6f01faab5 improved debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3401 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 07:43:36 +00:00
calvin
0887002783 cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3400 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 07:40:21 +00:00
calvin
0df2478430 improve URL norm tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 07:39:55 +00:00
calvin
ed98b6fc27 fix intern/extern semantic 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3382 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-19 20:58:52 +00:00
calvin
40399ca6c9 minor debug msg improvement 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3381 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-19 20:58:27 +00:00
calvin
bb9fbf6d26 fix CSS in HTML output 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3380 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-19 20:56:37 +00:00
calvin
46dfeb50ef catch correct exception 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3372 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-14 06:44:52 +00:00
calvin
6f0dbb5058 copy Queue.Queue code, for Python2.5 compatibility 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3371 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-13 22:22:29 +00:00
calvin
7016503a45 use PyObject_Del instead of PyMem_DEL, fixing possible segfault 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3370 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-10 20:56:11 +00:00
calvin
93b84683cf remove unused LRU class, add more caseless dict tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3367 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-06 17:52:36 +00:00
calvin
84741e4f63 recompile with bison 2.2 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3365 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-06 06:47:36 +00:00
calvin
6b0cf48959 cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3360 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:45:15 +00:00
calvin
e07d6f024b test caseless dicts 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3359 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:36:33 +00:00
calvin
2df9a7eb26 add del method to caseless dict 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3358 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:36:24 +00:00
calvin
35ef77ab63 remove unused functions xmlunquote* 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3357 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:23:09 +00:00
calvin
b9150f1bf6 fix blacklist file writing 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3356 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:21:01 +00:00
calvin
af255b67e0 add decode() method; fix file flush in case self.fd is None; expand user name of filenames 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3355 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:20:38 +00:00
calvin
4c83e19454 get rid of unused self.fd is None checks 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3354 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:00:48 +00:00
calvin
df7bf82baf check modified time after parsing 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3353 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 20:00:21 +00:00
calvin
7a31cb7ede add tests for file output 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3351 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 19:47:09 +00:00
calvin
7781fe88ce use relative imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3350 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 19:46:47 +00:00
calvin
7667f3402f send short keep-alive header value for test server 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3349 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 19:46:23 +00:00
calvin
e8e6a8af9a set modified time after parsing of robots.txt entries 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3348 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 19:44:59 +00:00
calvin
cf75e543b3 use self.filename, not args['filename'] in __init__() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3347 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 19:44:30 +00:00
calvin
12946ec9f7 more test coverage 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3344 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-05 13:30:54 +00:00
calvin
43c0e3447a reduce wait/sleep time for URL queue getting 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3341 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-04 23:51:32 +00:00
calvin
19a7495b9e only accept ASCII robots.txt 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3339 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-04 21:07:08 +00:00
calvin
d95d8c3d96 correctly handle internal errors 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3338 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-03 01:14:05 +00:00
calvin
21b53215e4 cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3337 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-01 14:13:12 +00:00
calvin
997e686a69 no exit on internal errors in threads 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3336 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-01 14:13:04 +00:00
calvin
a57618a4ad use relative imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3335 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-01 14:06:19 +00:00
calvin
c3fa9d7965 don't generate empty output files 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3332 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-01 13:58:13 +00:00
calvin
677458b8d6 fix imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3330 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-01 13:54:55 +00:00
calvin
8763a42063 skip a test on nt platforms 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3327 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 16:08:37 +00:00
calvin
4f668b8f90 added file norm tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3326 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 14:22:37 +00:00
calvin
850684b1e0 datadir as url path 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3325 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 14:00:45 +00:00
calvin
2888c34859 split file tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 13:54:40 +00:00
calvin
5b8b2cbda1 use diff for file compare 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3323 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 13:43:10 +00:00
calvin
2e1102a1b5 documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3322 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-29 12:57:51 +00:00
calvin
f5f7007c34 added abstract run_checked() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3316 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 14:25:04 +00:00
calvin
b442809838 look that cached URLs get checked quickly in large queues 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3315 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 11:35:24 +00:00
calvin
e211d3fd6c fix internal error call 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3314 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 11:33:23 +00:00
calvin
694f05405a translate interrupt message, and check for changed signal handler 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3313 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 11:33:01 +00:00
calvin
cb853c4672 documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3312 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 11:32:32 +00:00
calvin
4c8dcb012b added a stoppable thread object 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3305 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 22:29:22 +00:00
calvin
c91ccc56d6 add and use a stoppable thread object 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3303 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 22:16:36 +00:00
calvin
c3afa5c9c6 improved tmieit decorator 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3299 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 16:54:47 +00:00
calvin
b898cc05d3 added§ 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3298 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 16:54:37 +00:00
calvin
e7107bc270 return function result in timeit decorator 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3297 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 16:52:45 +00:00
calvin
1974bdcaf1 use xrange 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3296 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 16:52:39 +00:00
calvin
d557b0208f whops, reintroduce url_data.url is None check 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3295 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-24 00:00:33 +00:00
calvin
f30964679c remove return in __init__ 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 23:59:09 +00:00
calvin
f746b29522 documentation added 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 22:04:30 +00:00
calvin
27997b0251 add a finish() method to wait for spawned threads to finish 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3290 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 19:55:35 +00:00
calvin
7a94995345 remove unused condition 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3289 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 19:51:22 +00:00
calvin
49a21901f9 raise correct Empty() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3288 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 18:57:26 +00:00
calvin
eaf1fd4ba1 use timeout for get() method, return thread object 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3286 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 16:33:09 +00:00
calvin
a1905bdb22 support timeout in get() method 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3285 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 16:33:00 +00:00
calvin
5eec9cf527 update from subversion repo 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3284 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-23 16:32:50 +00:00
calvin
97e20ccdd4 move status method to status package 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3283 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-22 21:35:41 +00:00
calvin
ff1e16230b wait until status thread is finished 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3282 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-22 21:35:20 +00:00
calvin
e51bd72957 use xrange 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3280 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-19 23:02:46 +00:00
calvin
98597c267d quote result line 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3272 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-19 17:21:20 +00:00
calvin
3142663135 added tests for UnicodeError 'label too long' 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3270 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-19 17:13:28 +00:00
calvin
7e1e01bd36 do not catch UnicodeError, handle that intern 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3269 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-19 17:13:16 +00:00
calvin
2c13d7cac1 norm test urls 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3267 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-18 21:43:55 +00:00
calvin
608f8ba1c3 prepare filenames as URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3266 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-18 21:38:57 +00:00
calvin
37615dba02 use datadir, curdir placeholders 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3265 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-18 21:35:52 +00:00
calvin
14a29fb015 prepare filenames as URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3264 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-18 21:35:36 +00:00