Commit graph

3053 commits

Author SHA1 Message Date
Bastian Kleineidam
3d5c114f14 Warn on permament redirections even when URL is outside of domain filter. 2010-03-07 09:36:21 +01:00
Bastian Kleineidam
09623f066c Remove trailing spaces. 2010-03-06 11:03:25 +01:00
Bastian Kleineidam
2d73b907f1 Retry HTTP when server sent empty status line; should fix most of the BadStatusLine errors that are sporadically encountered. 2010-03-06 10:23:34 +01:00
Bastian Kleineidam
280a675698 Distinguish the empty status line from other BadStatusLine errors. 2010-03-06 10:08:08 +01:00
Bastian Kleineidam
bee8023540 Fixed URL encoding 2010-02-22 01:06:19 +01:00
Bastian Kleineidam
92272b9af9 Added missing license info to some files. 2009-12-02 21:39:29 +01:00
Bastian Kleineidam
77daf80e82 Add url encoding parameter 2009-11-28 11:56:35 +01:00
Bastian Kleineidam
d404b0b1bb Use /dev/urandom to avoid waiting for /dev/random 2009-11-21 12:13:49 +01:00
Bastian Kleineidam
66da1316bd Remove version comment in Changelog 2009-09-23 00:33:33 +02:00
Bastian Kleineidam
153c241049 Use i18n.default_encoding instead of a hardcoded one of iso-8859-1. 2009-08-22 12:21:50 +02:00
Bastian Kleineidam
92e1f90117 Use locale encoding as default, even if the language is not supported 2009-08-22 10:09:41 +02:00
Bastian Kleineidam
c52d15fd4d Improve search for lccollection.qhc on Windows systems 2009-07-31 07:18:25 +02:00
Bastian Kleineidam
3fd3ee90ec Fix GUI documentation install 2009-07-29 21:53:52 +02:00
Bastian Kleineidam
2a4f76be3b Moved HTML documentation files 2009-07-26 23:31:57 +02:00
Bastian Kleineidam
5cd7b84596 Allow digits at end of domain names in safe domain check. 2009-07-26 23:16:42 +02:00
Bastian Kleineidam
8533ade21f Add ID for each logged URL. 2009-07-26 22:31:51 +02:00
Bastian Kleineidam
5e06b6b8d4 Updated FSF address in GPL blurb 2009-07-24 23:58:20 +02:00
Bastian Kleineidam
a82e2236ae Copy system config to user config 2009-07-24 23:16:12 +02:00
Bastian Kleineidam
e6f43b6822 Fixed the no_proxy handling and added changelog entry 2009-07-24 07:19:49 +02:00
Bastian Kleineidam
d666e63d19 Print the noproxy variable in system info string 2009-07-22 23:41:46 +02:00
Bastian Kleineidam
fd610ba350 Encode spaces with %20 instead of + 2009-07-22 22:52:40 +02:00
Bastian Kleineidam
34219e2a12 Improved logging message. 2009-07-22 22:29:48 +02:00
Bastian Kleineidam
3121e3385e Use direct HTML documentation for the GUI client; moved the homepage content to a separate package. 2009-07-20 18:33:09 +02:00
Bastian Kleineidam
9faa7d33d2 Fix Python2.6 deprecation warnings 2009-06-26 23:19:36 +02:00
Bastian Kleineidam
4389cc28eb Fix Python2.6 deprecation warnings 2009-06-26 22:43:18 +02:00
Bastian Kleineidam
7f67027abf ignore the fragment part (ie. the anchor) of URIs when
+  getting and caching content
2009-06-26 07:22:36 +02:00
Bastian Kleineidam
00c15eef6a Minor code cleanups 2009-06-22 20:40:00 +02:00
Bastian Kleineidam
62922d48e8 Fix previous introduce config parsing error. 2009-06-18 20:29:31 +02:00
Bastian Kleineidam
c3450c7924 Fix uninitialized file descriptor in CSV output 2009-06-18 19:59:54 +02:00
Bastian Kleineidam
561430f184 Minor code cleanups and documentation improvements in the logger output base class. 2009-06-18 19:51:18 +02:00
Bastian Kleineidam
fcdaedd46f Always write csv column names, not just when the intro part is configured. 2009-06-18 06:52:33 +02:00
Bastian Kleineidam
c7b7af877f Read Mozilla bookmark titles correctly from places.sqlite. 2009-05-20 07:50:46 +02:00
Bastian Kleineidam
bd2e76bb93 Use install_data path for finding the help file. 2009-03-27 23:12:52 +01:00
Bastian Kleineidam
8dbb5e517d Fix help start page URL 2009-03-09 23:50:08 +01:00
Bastian Kleineidam
79adf68962 Move options button into menu 2009-03-09 23:44:49 +01:00
Bastian Kleineidam
5a0a281c92 Add missing pyqt signatures 2009-03-07 19:15:48 +01:00
Bastian Kleineidam
4672f48d2b Add GUI context menu 2009-03-07 14:40:13 +01:00
Bastian Kleineidam
388c7e4fd4 Added actionViewOnline to GUI actions 2009-03-07 14:38:05 +01:00
Bastian Kleineidam
a7e53bc12f Remove t.py 2009-03-07 14:02:52 +01:00
Bastian Kleineidam
cb5444c35a Improved spacing for UI 2009-03-07 14:02:32 +01:00
Bastian Kleineidam
ede5654c85 Enable allColumnsShowFocus treewidget option. 2009-03-07 13:46:31 +01:00
Bastian Kleineidam
ea3b237c5c Use more autoconnect methods. 2009-03-07 13:46:19 +01:00
Bastian Kleineidam
4b4945309c Enable allColumnsShowFocus treewidget option. 2009-03-07 13:45:52 +01:00
Bastian Kleineidam
72b5f1838c Use the autoconnect feature for signals 2009-03-07 13:33:36 +01:00
Bastian Kleineidam
7f5b24aa07 Set custom menu policy for tree widget 2009-03-07 13:32:37 +01:00
Bastian Kleineidam
7509fdfc4e Added new online.png icon to resources 2009-03-07 13:32:01 +01:00
Bastian Kleineidam
86c2d08c8e Double clicking on URL column opens URL in webbrowser. 2009-03-07 10:57:29 +01:00
Bastian Kleineidam
475e40b599 Add ability to error out of link checking with a predefined interrupt exception. 2009-03-07 10:15:14 +01:00
Bastian Kleineidam
59ffbd43f0 Use AttrDict for transport object in loggers. 2009-03-07 09:43:55 +01:00
Bastian Kleineidam
e149644c5c Added attribute dictionary container 2009-03-07 00:45:58 +01:00
Bastian Kleineidam
7a59763508 Remove unused SetList container 2009-03-07 00:42:27 +01:00
Bastian Kleineidam
2351506752 Use plain list for info strings. 2009-03-07 00:19:19 +01:00
Bastian Kleineidam
897b68ae9b Fix copying of httpurl info 2009-03-07 00:17:17 +01:00
Bastian Kleineidam
88dbcb30cd Remove unused url_data.info tags - the tags were always None 2009-03-06 21:20:09 +01:00
Bastian Kleineidam
0b5f525f76 Print NNTP server welcome string as info 2009-03-06 20:57:35 +01:00
Bastian Kleineidam
4ee0fb0181 Add NNTP debugging. 2009-03-06 20:53:12 +01:00
Bastian Kleineidam
0bc2fbb47a Only try 3 times connecting to a busy NNTP server, not 5 times. 2009-03-06 20:52:53 +01:00
Bastian Kleineidam
29adfe92fd Minor syntax fix 2009-03-06 20:14:50 +01:00
Bastian Kleineidam
0afd5f7cc6 Properly detect a callable object in robots.txt callback 2009-03-06 20:10:26 +01:00
Bastian Kleineidam
6024f2e43e Add missing reset of self.reused_connection flag 2009-03-06 20:10:03 +01:00
Bastian Kleineidam
ba160350dd Introduced transport object API for logging. 2009-03-06 19:30:58 +01:00
Bastian Kleineidam
58925b21d3 Improved persistent connection handling by retrying closed connections. 2009-03-06 08:15:34 +01:00
Bastian Kleineidam
29599e4c74 Make sure persistent connection will not close after reading contents. 2009-03-05 19:15:44 +01:00
Bastian Kleineidam
bf9ed8c659 Make sure file descriptors are closed after decoding HTTP content. 2009-03-05 19:15:03 +01:00
Bastian Kleineidam
f23c3ec10b Updated copied modules from upstream. 2009-03-04 23:49:00 +01:00
Bastian Kleineidam
714147cb29 Improved language and encoding detection by using local.getdefaultlocale() instead of locale.getlocale(category=LC_ALL) 2009-03-04 22:43:51 +01:00
Bastian Kleineidam
b3858742a0 Style change when looking for LOCPATH 2009-03-04 22:43:10 +01:00
Bastian Kleineidam
2890e879a3 Use gettext translation in PyQT GUI client 2009-03-04 22:41:25 +01:00
Bastian Kleineidam
f2f0a0f530 Output parent URL in extra column 2009-03-02 23:42:13 +01:00
Bastian Kleineidam
3fd9e6db2b Do not log duplicate cached URLs 2009-03-02 23:41:56 +01:00
Bastian Kleineidam
59772e4837 Fixed log_filter_url documentation 2009-03-02 23:40:36 +01:00
Bastian Kleineidam
8d98337ffe Add thread name to log output when threads are enabled. 2009-03-02 23:27:05 +01:00
Bastian Kleineidam
b8944e493a Use new exception log keyword when logging errors 2009-03-02 13:18:36 +01:00
Bastian Kleineidam
9475f97edb Improved check thread name to include the word 'thread' 2009-03-02 13:18:08 +01:00
Bastian Kleineidam
24a30bae44 Add new exception printing keyword argument to log functions 2009-03-02 12:28:10 +01:00
Bastian Kleineidam
cea482f7fc Use generic keyword arguments in log functions 2009-03-02 12:27:38 +01:00
Bastian Kleineidam
f5f70a7149 Reorder text logger output fields. 2009-03-02 08:19:46 +01:00
Bastian Kleineidam
a9335fb3e8 Make file list an iterator, and add missing slash if needed to manually given file URLs. 2009-03-02 08:02:27 +01:00
Bastian Kleineidam
7862147ca3 Fix showing content size. 2009-03-01 23:04:48 +01:00
Bastian Kleineidam
a0ba9a7446 Improved Python 2.6 compatibility in HTML parser 2009-02-28 13:47:25 +01:00
Bastian Kleineidam
b106d4cc03 Improved treeview layout 2009-02-25 02:16:39 +01:00
Bastian Kleineidam
4ded3a413f Improved treeview layout 2009-02-25 02:12:55 +01:00
Bastian Kleineidam
77fa70ac43 Increase main window width 2009-02-24 23:34:59 +01:00
Bastian Kleineidam
7041202c50 Remove unused imports 2009-02-24 22:41:42 +01:00
Bastian Kleineidam
7f2644b624 Make GUI columns sortable 2009-02-24 22:36:35 +01:00
Bastian Kleineidam
8736b6ff07 Improved progress status log 2009-02-24 21:51:44 +01:00
Bastian Kleineidam
c7a77d1979 Make status print interval configurable 2009-02-24 21:46:25 +01:00
Bastian Kleineidam
52a0f70aa0 Rename LOG to more descriptive LOG_ROOT 2009-02-24 20:03:44 +01:00
Bastian Kleineidam
8e82c31fb6 Fix text output to filenames 2009-02-24 15:27:14 +01:00
Bastian Kleineidam
554f0e88b0 simplify text logger init of file output 2009-02-24 15:12:09 +01:00
Bastian Kleineidam
4b0956e574 Fix GUI status logging method name 2009-02-24 15:11:02 +01:00
Bastian Kleineidam
e7c7f74546 Set some default colors for output logging. 2009-02-24 15:10:41 +01:00
Bastian Kleineidam
6d21ca31fd Fix GUI logging of warnings 2009-02-24 15:10:20 +01:00
Bastian Kleineidam
ce531869c8 Fix console status logger 2009-02-24 14:50:10 +01:00
Bastian Kleineidam
8c221da089 Remove the custom set datatype 2009-02-24 13:15:07 +01:00
Bastian Kleineidam
8caa601a7e Python 3.0 compatibility: use exc.args[] instead of exc[] 2009-02-24 12:41:45 +01:00
Bastian Kleineidam
2c9b8d6858 Use slash as path separator in file names 2009-02-24 12:41:28 +01:00
Bastian Kleineidam
9e2fe66dc9 Python 3.0 compatibility: use exc.args[] instead of exc[] 2009-02-24 11:13:46 +01:00
Bastian Kleineidam
3f38776ce2 Replace has_key() with 'in' tests to be compatible for Python 3.0 2009-02-23 23:59:41 +01:00
Bastian Kleineidam
fb33210605 Show GUI status in progress dialog; split GUI files 2009-02-23 23:44:49 +01:00
Bastian Kleineidam
0734f7420c Added Download size column to GUI output 2009-02-22 17:00:59 +01:00
Bastian Kleineidam
323958951c Add name to unnamed file URLs. 2009-02-20 14:03:34 +01:00
Bastian Kleineidam
063a7f8310 Rename D/L size to size 2009-02-20 14:03:09 +01:00
calvin
ab5c841c9b Display log messages in progress dialog.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3977 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-19 13:36:16 +00:00
calvin
9adaadde4b Log GUI output to a treewidget
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3976 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-19 13:35:46 +00:00
calvin
2620b7ec51 Emit URL log message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3974 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-19 09:17:00 +00:00
calvin
3aa6d63b06 Activated cancel button in progress dialog
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3971 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-19 08:15:54 +00:00
calvin
0d9828002b Added flag to stop checking from another thread.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3970 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-19 08:15:32 +00:00
calvin
afd866456a Activate cancel button in progress dialog
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3969 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 23:17:57 +00:00
calvin
06c44542f2 Print URL name in one line.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3968 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 23:15:43 +00:00
calvin
75f1473bf4 Added progress bar
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3967 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 23:14:03 +00:00
calvin
3a6a8d27f0 Added help to the GUI client
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3964 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 19:05:16 +00:00
calvin
01c417c3c3 Default output encoding is now utf-8
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3963 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 16:14:50 +00:00
calvin
2e918a7b7a Added email syntax check.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3960 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 15:35:23 +00:00
calvin
7214943f38 Remove wrong function return type documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3959 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 15:34:46 +00:00
calvin
02b7a70d2d Fix custom XML output with missing info or warning tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 15:34:28 +00:00
calvin
7b489b5897 Allow missing cache keys in the in_progress queue. This occurs when syntax checks already set the result.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3957 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 15:34:10 +00:00
calvin
366c711b43 Improved domain name checking
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3956 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-18 15:33:52 +00:00
calvin
4ff416662f Support py2exe configdir setting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3953 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-10 23:57:33 +00:00
calvin
1de88aeac2 Only init logging with existing config file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3952 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-10 23:14:42 +00:00
calvin
527b617f88 Regenerate with newer flex and bison versions.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3949 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-01 11:21:13 +00:00
calvin
1ad59f729c Fix whitespace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3946 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-02-01 10:23:45 +00:00
calvin
c30e28e609 Upstream updates for dnspython
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3943 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-30 21:03:08 +00:00
calvin
4369ffcf9a Added application icon to GUI client
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3933 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-24 18:17:27 +00:00
calvin
7e5a2ea23b Remove unused file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3930 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-24 17:35:06 +00:00
calvin
e03df9e709 Removed gopher URL checking.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3929 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-24 17:34:18 +00:00
calvin
1ee5d4aa2b Rename CGI FormError class to avoid name clash with dns module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3920 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-23 08:34:41 +00:00
calvin
cf81104d5a Switch documentation to the Sphinx documentation generator.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3919 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-23 08:34:24 +00:00
calvin
a1d36611a0 Added close button in options dialog
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-11 10:33:09 +00:00
calvin
94a960208b Upstream updates for httplib2.py
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3908 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-11 01:26:40 +00:00
calvin
2f43c25699 Remove access restrictions from CGI prog, leave that to the web server configuration.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3904 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-11 00:27:45 +00:00
calvin
73206acbcb Move GUI client into package.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3903 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-11 00:26:42 +00:00
calvin
de5b7e894e Added upstream fixes to patched Python modules.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3901 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 20:00:25 +00:00
calvin
c6cb09c4aa Add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3900 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 19:41:42 +00:00
calvin
1c50cf288a Ignore DNS MX lookup failures in py2exe.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3899 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 18:50:14 +00:00
calvin
4ffe29615e Updated DNS code
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3898 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 18:05:13 +00:00
calvin
41b02bdc27 Fix typo N0ne -> None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3896 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 17:54:44 +00:00
calvin
cc25deac12 Only accept MX dns response types when asking for MX servers.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3895 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 17:53:10 +00:00
calvin
979132c9b5 Catch all DNS exceptions when resolving MX hosts.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3894 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 15:13:55 +00:00
calvin
a26ca4c23a Replace C ftpparse module with Python implementation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3892 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-10 14:11:17 +00:00
calvin
e28ce1a710 Rename init* functions of ftpparse to avoid name clash with Python module system.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-08 23:06:37 +00:00
calvin
e9805dbd8a Updated copyright year to 2009
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-08 14:18:03 +00:00
calvin
63d42c0d92 Replace deprecated sets module with builtin set when possible.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3886 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-08 12:38:08 +00:00
calvin
8d5d4827c3 Change ftpparse import to avoid py2exe load error.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2009-01-08 12:28:39 +00:00
calvin
4b71c92f99 updated httplib from python 2.6
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3876 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-12-05 02:17:13 +00:00
calvin
7d05f0f544 Added --complete option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3872 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-30 19:35:56 +00:00
calvin
209d5abc18 fix timeouts by testing earlier for persistent connections with HEAD
HEAD requests never have a body; nevertheless the http lib tries to
read() from them. This times out on some servers of course. Fix is
not to let those connections be persistent.

git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3871 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-29 08:14:28 +00:00
calvin
c20e706761 Made some format changes on translated strings.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3870 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-28 20:22:48 +00:00
calvin
1abc2c504d Filter invalid mozilla bookmark urls from places.sqlite
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3869 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-28 10:54:16 +00:00
calvin
c3b6fc5aa4 Readd
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3867 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-20 21:30:10 +00:00
calvin
42c3e71329 Improved and tested opera bookmark parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3863 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-20 07:52:02 +00:00
calvin
9ab895751f Support parsing of Firefox 3 bookmark files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3862 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-11-20 07:51:22 +00:00
calvin
bc48ce8a96 Close robotparser URL connections; simplify line parsing.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-28 16:56:24 +00:00
calvin
0cade7b308 Improved forced shutdown on Unix and Windows. Unix uses SIGKILL now, Windows os.abort().
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3852 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-28 16:56:02 +00:00
calvin
7b2a21c133 Fix off-by-one error in cookie domain matching code.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3850 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-13 12:52:11 +00:00
calvin
97cf700e04 Fixed wrong cookie debugging format line.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3849 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-13 12:51:56 +00:00
calvin
22e6a9e67d Fix encoding errors in robots.txt, making some sites like wikipedia.org
accesible again.

git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-13 12:51:45 +00:00
calvin
523ee87f0c Add missing return in is_absolute_path()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3846 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-09 19:17:33 +00:00
calvin
f68872f559 Improved detection of absolute Windows paths.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3844 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-07-09 19:00:02 +00:00
calvin
573453d68b Remove trailing spaces
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3835 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-30 23:18:32 +00:00
calvin
b62c28256f Updated httplib2.py with version from 2.6 Subversion repository.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3834 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-30 23:18:19 +00:00
calvin
84355f7b94 Catch original httplib errors too since it is used indirectly by urllib functions.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3833 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-30 23:17:38 +00:00
calvin
3b94dd2810 Add more includes needed by MacOSX to compile.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3832 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-26 20:39:08 +00:00
calvin
f9695ed2e5 Set svn:eol-style native
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3831 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-26 20:36:38 +00:00
calvin
64e311e2d6 Move console related print functionality to director.console module; add --version commandline option to GUI client
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3830 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-26 11:11:09 +00:00
calvin
4a5a4ed0bf Fix compilation and usage under Windows system.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3827 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-18 09:51:54 +00:00
calvin
b30fb3b09c Remove duplicate code in http checker.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3820 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 19:52:09 +00:00
calvin
caf8ba6297 Really allow parsing of XHTML files; I forgot some places to adjust the MIME checking.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3818 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 13:03:48 +00:00
calvin
a6deeeb8a5 Support parsing of HTML pages served with content type application/xhtml+xml
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3817 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 09:39:49 +00:00
calvin
ff41aa8d9f Lower the MIME content-type info from HTTP headers befure using it
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3816 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-16 09:38:09 +00:00
calvin
d26386d03f Catch errors when getting content for title.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3814 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-12 15:38:26 +00:00
calvin
8085045ae9 Added extra status logger class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3811 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:02:22 +00:00
calvin
f7a8b16096 Do not override local variable in for loop
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:01:56 +00:00
calvin
fa0c868773 Simplified Logger.writeln() method.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3808 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 13:01:36 +00:00
calvin
3b5e28ff58 Improved log end output: print number of duplicate error and warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-11 07:33:19 +00:00
calvin
a880939c40 Initialize variables in reset(), not in subsequent methods
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3796 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-08 09:27:13 +00:00
calvin
5588df8ebf Remove trailing whitespace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3794 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-08 09:26:56 +00:00
calvin
aba08b8b7d Improved graph output: print labels as node names.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3793 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:08:54 +00:00
calvin
83bd7ce4ac Improved network interface detection on POSIX systems
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3792 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:08:36 +00:00
calvin
290528b84f Added title attribute to URL data.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:56 +00:00
calvin
4ade9a79cc Moved link and title parsing into htmlutil package
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3789 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:48 +00:00
calvin
99269d12cc Add base method for Url.get_title()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3788 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:38 +00:00
calvin
58e44dbf8e Updated documentation and todo list.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3787 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-07 13:07:30 +00:00
calvin
e8a4744643 Use enum utility function for thread priorities.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-05 07:14:04 +00:00
calvin
5f818f620a Backport namedtuple container from Python 2.6
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-06-05 07:13:55 +00:00
calvin
5f4d61e018 Use keyword arguments in translation strings.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:44:40 +00:00
calvin
97772c9700 Improved email check messages.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3779 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:44:07 +00:00
calvin
2e4d0894fc Stop checking a list of emails at the first invalid one.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3778 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:43:20 +00:00
calvin
e6e51dbc6b Overwrite old results when checking a list of emails.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3777 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:42:52 +00:00
calvin
66ff422f6b Allow overwriting of an old check result.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-27 19:42:38 +00:00
calvin
5bf875bebe Use configdata for application name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-22 12:17:24 +00:00
calvin
7297519b04 Remove or replace unused variables.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-22 12:10:08 +00:00
calvin
2cb2ee9403 Use base log level for clamav warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3766 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 17:01:49 +00:00
calvin
9352dbf5e4 Move test files to separate module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3763 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 17:01:16 +00:00
calvin
90587545f7 Catch attribute errors in logger flushing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3761 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 16:57:31 +00:00
calvin
2f25962789 Match newlines in catch-all rules
Avoid printing spurious newlines when HTML parsing. The "." does
not match newlines, correct that in the catch-all lexer rules.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3760 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 16:56:58 +00:00
calvin
670a1df9b3 Updated documentation for new Python 2.5 dependency
Removed old references of Python 2.4, together with compatibility
hacks.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3757 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 10:49:00 +00:00
calvin
ce1d41a5a0 Document the CGI access variables
The access variables ALLOWED_CLIENTS and ALLOWED_SERVERS are now
documented to help setup the CGI script more easily.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3756 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 10:48:07 +00:00
calvin
dbb498a395 Add virus checking
New option --scan-virus to check the content of URLs for
viruses with ClamAV.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3753 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-20 08:57:37 +00:00
calvin
bacb59597e Use relative imports from Python 2.5
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-09 06:16:03 +00:00
calvin
b96e8120d6 Add W3C Validator checks
Add new options --check-html-w3 and --check-css-w3 to allow checking
of HTML and CSS pages with the online W3C validators.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-08 10:36:54 +00:00
calvin
bc9b9ee07e Move http util function in a separate module.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3747 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-05-08 10:36:08 +00:00
calvin
d5ec7c1ac3 Encode error output to HTML charset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-30 14:37:49 +00:00
calvin
df9f31dcb1 Only check HTML/CSS syntax of intern URLs
The HTML and CSS syntax check now only applies to URLs
which match those given on the command line.
This makes checking of personal pages easier.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 17:48:47 +00:00
calvin
ac4d09f83d Fix errors in CSS and HTML syntax check
Properly encode the warning messages as Unicode, and prevent
overwriting of the "log" module with a local variable.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 17:48:22 +00:00
calvin
92c74ece4d Send HTTP Referer header to both http and https URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3741 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-29 13:33:35 +00:00
calvin
5d8bdaaa1f Use generators instead of lists where possible
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-28 00:26:02 +00:00
calvin
3eac1be9ab Require and use Python 2.5
Use Python 2.5 features and get rid of old compat code. Also some
code cleanups have been made.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-27 11:39:21 +00:00
calvin
017b73db27 Fix cookie test!
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3728 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 08:34:10 +00:00
calvin
72db31e546 Only check syntax of valid URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3726 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 07:53:11 +00:00
calvin
973da91f44 Source code cleanup: use or remove unused variables
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3724 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-25 07:49:52 +00:00
calvin
e266a65b64 Fix css check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3723 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-24 10:46:19 +00:00
calvin
62efec3b35 Added CSS syntax check.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-24 09:44:18 +00:00
calvin
cce6affa17 Add --check-html option to check the HTML syntax.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-23 23:04:31 +00:00
calvin
9f77f97434 Add distclean target; use Python2.5 includes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-23 23:03:55 +00:00
calvin
df857aab8d Intern patterns now accept URLs with and without "www." prefixes
as default. This allows sites to check that use both variants.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3714 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 09:18:58 +00:00
calvin
5a2f89fa3d Add redirect warning for commandline URLs
If URLs given on the commandline are redirected, the automatic
intern patterns might not match anymore. A warning makes this
more prominent.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3712 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 09:18:36 +00:00
calvin
8fa01f32c2 Use LC_ALL instead of LC_MESSAGES
Windows platforms do not have LC_MESSAGES. Use LC_ALL instead.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 07:40:40 +00:00
calvin
c58dd965af Set HTML charset according to logger output encoding.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3708 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:56 +00:00
calvin
18c6e6e38a Set default_encoding on i18n init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3707 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:47 +00:00
calvin
d9f8bd3187 Properly set the locale in CGI scripts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:19 +00:00
calvin
cfc651550a Use set() instead of a list for the set of supported languages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3703 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:28:10 +00:00
calvin
fa48fe354d Use LC_MESSAGES locale, not default system locale in i18n
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3699 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-21 01:27:38 +00:00
calvin
963feb2288 Double Ctrl-C stops checking immediately without cleanup.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3696 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-20 23:56:25 +00:00
calvin
9c56f03ae3 Shutdown immediately when Ctrl-C is given twice
Try sys.exit() to shutdown immediately after Ctrl-C keyboard
interrupt was given twice.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3695 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:47:16 +00:00
calvin
8ae6d94b45 Improved error messages for exceptions
Prepend the exception name before the error message of exceptions.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3694 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:47:00 +00:00
calvin
67aed38df2 Bump copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3693 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-19 07:46:52 +00:00
calvin
4968f1b3cd Prevent empty exception values.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3690 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-18 07:42:24 +00:00
calvin
ba148a9d71 Proper MX DNS request fallback
Properly fall back to DNS A requests when no MX host could be found
for a mailto: URL.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3689 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:46 +00:00
calvin
9b7cf763ff Fix test for new www.example.org URL
Fix test data using www.example.org instead of imadoofus.org URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3688 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:29 +00:00
calvin
88242b4612 Compare to singletons with "is"
Make sure comparisons with singletons like None/True/False use
"is", not "==/!=".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3687 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:34:16 +00:00
calvin
4055721fd4 Use internal gzip2 module
Use the internal gzip replacement module gzip2 for all GzipFile handling.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3685 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:33:55 +00:00
calvin
17cd16185f Remove timestamp from gzipped files
Remove the timestamp from gzipped files since it might be a security
and/or privacy risk to include it.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3684 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-04-14 22:33:42 +00:00
calvin
bf277085e9 Regenerate HTML scanner with new flex version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3683 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:24:32 +00:00
calvin
1f5a2d47ea Syntax cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3682 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:24:21 +00:00
calvin
e178405748 Use example.{com,org} for example URLs
Use the guaranteed not available example.com and example.org DNS names
in example URLs.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3681 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-25 21:23:34 +00:00
calvin
4ce0ddd166 Changes for future Python 3.x compatibility
Replace backticks with repr(), replace .has_key() with "in".


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3680 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-19 10:22:57 +00:00
calvin
370bd058ea Add htmlsax.so target for local build
Add target to build htmlsax.so locally. Also add include path
for local python SVN repository for testing.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3678 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-19 10:21:52 +00:00
calvin
91a0aad5d8 Fix buggy persistent HTTP connections
Workaround for buggy servers that break protocol synchronization of
persistent HTTP connections.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3677 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:43 +00:00
calvin
67e55d3832 Revert "Update httplib2.py from upstream SVN"
This reverts commit 00937008e0c2e6d86cf8d9e9c2d54ff5d7443dcc.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:33 +00:00
calvin
f0faf1b155 Update httplib2.py from upstream SVN
Added some bugfixes from the Python upstream httplib.py.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3675 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-14 13:28:26 +00:00
calvin
1730097265 Prevent Unicode errors for non-ASCII emails
Prevent Unicode errors when email address contains non-ASCII characters.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3673 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-03-02 23:38:02 +00:00
calvin
860def8d34 Remove superfluous path slash
Really fix the test_misc unit test by removing a superluous path slash.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3672 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:46 +00:00
calvin
13df77c0b5 Added .gitignore files
Ignore files for git version tracking system.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3671 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:29 +00:00
calvin
294261d80a Replace hardcoded test paths for test_misc
Replace the hardcoded test paths with variables. Fixes failures
in the test_misc unit test.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3670 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:11 +00:00
calvin
e1b1b7d916 Regenerate HTML lexer with flex 2.5.34
The HTML lexer .c file has been regenerated with a new upstream
release of flex 2.5.34.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3669 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-02-08 10:38:00 +00:00
calvin
f01a77bab1 Don't parse '-->' as end-of-comment in script mode. This fixes parsing errors on some sites.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3668 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-23 09:43:39 +00:00
calvin
8c4d8145a7 simplify the CDATA matching rules to be more straightforward
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3667 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-23 09:11:50 +00:00
calvin
7cf9723b10 don't parse <script for=''> as URL
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3659 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-11 16:45:30 +00:00
calvin
6499cb1a63 updated copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3658 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2008-01-02 14:31:19 +00:00
calvin
c99b9b1e8f added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-28 08:25:40 +00:00
calvin
c971ebdabf Added Shockwave Flash (SWF) parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3656 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-28 02:12:48 +00:00
calvin
ad7c9bbc76 Don't print cached errors or warnings unless verbose output is requested.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-14 10:58:10 +00:00
calvin
1919c30bdf Do not throw internal errors when writing from a thread to a non-opened file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3638 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 12:25:21 +00:00
calvin
6c07be042d Add optional leading dot for cookie domain value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3637 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 11:52:10 +00:00
calvin
fddf890bd4 Allow spaces in cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 11:44:18 +00:00
calvin
30d2b4f520 HTTP content data is only considered valid for parsing if the request was not redirected and is a GET request.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3633 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-13 10:50:13 +00:00
calvin
41bc0b2b32 use 'self.data is None' to test if data is already read or not
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3631 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 14:09:20 +00:00
calvin
5591bbe052 fix self.downloadtime to self.dltime
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3630 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 14:01:36 +00:00
calvin
7a4c7e9f44 remove unused imports reported by pyflakes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3629 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-03 13:54:44 +00:00
calvin
8e6c6455ab add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3626 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:56:15 +00:00
calvin
09ce26d5fe removed debug flag, test the LOG_CHECK logger for debug settings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:50:59 +00:00
calvin
ebb428044c Simplify option parsing: check option existance before access instead of catching an exception.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-12-01 15:50:33 +00:00
calvin
8d2dc781e1 Ensure unused or expired connections are closed.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 16:42:41 +00:00
calvin
042f70115f updated copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3616 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 14:57:03 +00:00
calvin
f8a54faae9 make sure internpat does not remove a trailing slash, which results in checking of URLs that are not a prefix of the given URL.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3613 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-30 10:37:14 +00:00
calvin
9cf3314eab Use constants for warning tags, avoiding typos in string constants. And move the constants into a separate module const.py
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-29 07:50:22 +00:00
calvin
e007ea5dae fix warning typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-29 06:37:40 +00:00
calvin
fcde8bd4d6 try to detect unknown URL schemes instead of manually setting the assume_local flag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:46:50 +00:00
calvin
a50784042f make sure URL to test for IDNA encoding errors has non-ascii characters
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3608 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:40:33 +00:00
calvin
40b3be412b revert the catch UnicodeError change
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3607 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-28 18:39:12 +00:00
calvin
6a0960aa66 only store parser contents in LinkFinder handler, not in all handlers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3602 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 19:06:39 +00:00
calvin
a1d911127b remove comments from CSS files before parsing for links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 18:46:14 +00:00
calvin
cb588a3c5d replace tabs with spaces
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3598 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:55:55 +00:00
calvin
ce8b963dd9 more code cleanups and documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3596 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:46:18 +00:00
calvin
370749cafb cleanup the code and add some documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-14 17:34:50 +00:00
calvin
17906ca1e0 use Python 2.4 for local builds
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3592 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-13 12:38:01 +00:00
calvin
c93fc79702 ignore errors in idna encoding of hostnames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3591 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-11-13 12:36:42 +00:00
calvin
e9c973fe06 Honor urllib.proxy_bypass() when ignoring proxy settings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3583 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-23 22:56:44 +00:00
calvin
ad121779af don't double encode info text in html output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3578 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-16 17:59:57 +00:00
calvin
b4289c6ebe added password protected site
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3577 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:07:12 +00:00
calvin
fe438941a9 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3576 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:06:24 +00:00
calvin
e4c3e81b49 add Python SVN tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3575 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-10-02 01:05:40 +00:00
calvin
2edfaea03e Read complete body data on persistent connections, else subsequent requests could fail.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3568 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 19:33:10 +00:00
calvin
2b94c0c161 Assume missing HEAD requests for Zope server on text/plain content type
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3567 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 18:26:55 +00:00
calvin
d5685ba4dc fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3566 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-08 18:08:53 +00:00
calvin
69a38bfd37 Set SSL connection object to None to release it on closing.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3565 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-08-07 14:50:46 +00:00
calvin
cd95ac054f fix message typo for not disclosing information
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-07-22 22:54:37 +00:00
calvin
4c0620c498 use default python
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3559 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-16 22:08:41 +00:00
calvin
88d410a445 don't use repr() on unicode names, else it prints escape chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-13 00:36:23 +00:00
calvin
5aed37dada use german server for faster testing (at least for me)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3553 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-06-13 00:35:32 +00:00
calvin
4093da56db add result attribute
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3552 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:57:55 +00:00
calvin
590cfc95f4 add created attribute
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:53:38 +00:00
calvin
702885d7d8 Use standard UTF-8 charset name for XML output encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3549 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-04-03 00:41:40 +00:00
calvin
9de237b4c2 Check that charset is not None before lowering it in set_encoding().
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3547 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-03-21 19:32:19 +00:00
calvin
32fe705358 var typo fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-21 22:22:24 +00:00
calvin
a78811a422 sanitize the configuration
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3540 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-21 22:17:02 +00:00
calvin
56d98f9d71 mention that --anchors enables logging of warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3536 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-02-19 19:24:52 +00:00
calvin
df48d4a905 bump up copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2007-01-01 14:57:38 +00:00
calvin
2e5a5d20df prepare for Py_ssize_t conversion
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3531 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-08 22:07:31 +00:00
calvin
b274787c5b prepare for Py_ssize_t conversion
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3530 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-08 22:07:23 +00:00
calvin
7397929e9e updated from SVN
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:53:47 +00:00
calvin
dcc1852afb use new exception syntax for raising
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:52:09 +00:00
calvin
be4d9a2468 added patches from SVN
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-12-06 23:49:02 +00:00
calvin
7fb471eb2e ignore errors on log file open, display a warning instead
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3518 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-21 17:10:27 +00:00
calvin
d208818694 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3517 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-21 17:09:49 +00:00
calvin
c217b6d441 don't set result on self.get_content() redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3515 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-17 20:42:00 +00:00
calvin
db55727f69 add missing functions to ListDict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3511 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-11-06 22:12:05 +00:00
calvin
61f41c4949 fix the empty useragent test case
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3509 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 20:37:21 +00:00
calvin
3f099a6438 use boolean objects for rule line allowance
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3508 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 20:36:31 +00:00
calvin
698f7183bc fix vrfy error message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3507 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:57:29 +00:00
calvin
ce0f1cd62a use Py_MEMCPY macro
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3506 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:31:14 +00:00
calvin
2af8dcbead remove unused include
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3505 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:30:55 +00:00
calvin
e29704ce61 use Py_VISIT macro for cycle traversal
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3503 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-10-19 19:08:21 +00:00
calvin
95f9dcbbc5 fix error messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3486 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:24:41 +00:00
calvin
bef2494211 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3482 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:13:59 +00:00
calvin
19c2ce7fe0 catch thread errors separately
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3481 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:05:10 +00:00
calvin
5d26f2a5cd remove unused import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3480 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:04:41 +00:00
calvin
2c563d8bae fix variable typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3479 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-24 10:04:33 +00:00
calvin
980fe1ea31 replace numerated config entries with numerical ones
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3477 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 14:30:27 +00:00
calvin
d3e8b99ff2 print active URLs on interrupt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3476 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 13:43:54 +00:00
calvin
1646a2e9de label check threads with 'Check-' prefix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3475 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 13:42:32 +00:00
calvin
1883b79303 follow redirections when getting HTTP contents
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3473 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 09:27:38 +00:00
calvin
0c5d34e9f9 don't discard robots.txt entries with only Allow: lines
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3471 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-21 09:14:28 +00:00
calvin
3f89988e67 remove unused staticforward
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3469 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-18 19:59:51 +00:00
calvin
9682ce5c3a use Py_CLEAR for reference count decrement
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3468 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-18 19:19:30 +00:00
calvin
a49896b581 fixed syntax errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3463 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 09:17:36 +00:00
calvin
5ad59225a0 use dictionaries for translations with multiple arguments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 09:12:59 +00:00
calvin
ae245287b6 ignore keyboard interrupts during shutdown
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-16 08:41:15 +00:00
calvin
691f285f83 fix test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3454 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:27:28 +00:00
calvin
576d404ce2 close non-idle connections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3453 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:27:19 +00:00
calvin
229e352c0c improve BadStatusLine exception message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3452 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:26:51 +00:00
calvin
04f89d0668 use get_url_from helper alias
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3451 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:18:49 +00:00
calvin
72d198efcb don't send keep-alive header, it breaks some tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3450 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 21:18:29 +00:00
calvin
86514bb882 activate asset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3448 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:21:06 +00:00
calvin
fcd00abc75 don't close socket if response is still open
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3447 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:19:03 +00:00
calvin
7b39b6c1b6 add HTTPConnection.is_idle(), print response in HTTPConnection.__str__()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3446 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:10:14 +00:00
calvin
ba7eaeae09 ignore geo location info lines in test output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3445 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 20:03:54 +00:00
calvin
6348205dcc add persistent connections back to the connection cache, close all others
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3444 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:59:30 +00:00
calvin
d6676ab0a0 more response closing, and cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3443 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:51:02 +00:00
calvin
6fe2db6755 use unicode_safe alias helper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3442 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:46:55 +00:00
calvin
27a8869783 use helper alias for unicode_safe
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3441 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 19:34:03 +00:00
calvin
2a75dff16e add __str__ methods, and refine a self.sock close test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3440 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 18:07:13 +00:00
calvin
15dfaf35cb cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3438 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:36:44 +00:00
calvin
4b818cb4b3 Detect more cases to close the connection, and close response objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3437 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:35:53 +00:00
calvin
d22b6b5c68 add more infos to exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3436 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:33:48 +00:00
calvin
da15b15923 Split off the host wait time function, and use it with a separate lock
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3434 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 12:18:24 +00:00
calvin
a1eff0bd3e don't use a debug lock per default
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3433 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 11:30:55 +00:00
calvin
9f39498abf don't restrict socket buffer size
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3430 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 09:02:48 +00:00
calvin
c3b5e0d59d use generators and enumerate in containers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3428 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-04 17:02:34 +00:00
calvin
0a5c03536d remove unneeded logger arguments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3425 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-21 20:50:10 +00:00
calvin
f78d9bb337 s/fields/parts/ for logger arguments, and supporess the last modified info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3424 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-21 20:50:00 +00:00
calvin
01cf921816 remove old signal check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3419 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-09 19:51:12 +00:00
calvin
9d15f9a8be allow inifinite recursion in CGI script
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3417 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-07 22:18:31 +00:00
calvin
d7d9990511 use raw config parser, no need for interpolation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3411 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-02 20:12:40 +00:00
calvin
5abe8fd8ce upstream update
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3406 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-22 23:35:18 +00:00
calvin
313e6d3caa recompiled with bison 2.3
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3405 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-22 21:23:14 +00:00
calvin
a311497e66 fall back to ASCII if default locale is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3403 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-19 16:42:43 +00:00
calvin
adc4e8c0e8 quote base reference URL, with tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3402 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 08:07:46 +00:00
calvin
c6f01faab5 improved debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3401 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:43:36 +00:00
calvin
0887002783 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3400 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:40:21 +00:00
calvin
0df2478430 improve URL norm tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 07:39:55 +00:00
calvin
ed98b6fc27 fix intern/extern semantic
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3382 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:58:52 +00:00
calvin
40399ca6c9 minor debug msg improvement
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3381 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:58:27 +00:00
calvin
bb9fbf6d26 fix CSS in HTML output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3380 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-19 20:56:37 +00:00
calvin
46dfeb50ef catch correct exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3372 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-14 06:44:52 +00:00
calvin
6f0dbb5058 copy Queue.Queue code, for Python2.5 compatibility
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3371 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-13 22:22:29 +00:00
calvin
7016503a45 use PyObject_Del instead of PyMem_DEL, fixing possible segfault
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3370 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-10 20:56:11 +00:00
calvin
93b84683cf remove unused LRU class, add more caseless dict tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3367 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-06 17:52:36 +00:00
calvin
84741e4f63 recompile with bison 2.2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3365 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-06 06:47:36 +00:00
calvin
6b0cf48959 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3360 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:45:15 +00:00
calvin
e07d6f024b test caseless dicts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3359 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:36:33 +00:00
calvin
2df9a7eb26 add del method to caseless dict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3358 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:36:24 +00:00
calvin
35ef77ab63 remove unused functions xmlunquote*
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3357 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:23:09 +00:00
calvin
b9150f1bf6 fix blacklist file writing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3356 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:21:01 +00:00
calvin
af255b67e0 add decode() method; fix file flush in case self.fd is None; expand user name of filenames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3355 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:20:38 +00:00
calvin
4c83e19454 get rid of unused self.fd is None checks
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3354 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:00:48 +00:00
calvin
df7bf82baf check modified time after parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3353 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 20:00:21 +00:00
calvin
7a31cb7ede add tests for file output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3351 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:47:09 +00:00
calvin
7781fe88ce use relative imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3350 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:46:47 +00:00
calvin
7667f3402f send short keep-alive header value for test server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3349 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:46:23 +00:00
calvin
e8e6a8af9a set modified time after parsing of robots.txt entries
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3348 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:44:59 +00:00
calvin
cf75e543b3 use self.filename, not args['filename'] in __init__()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3347 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 19:44:30 +00:00
calvin
12946ec9f7 more test coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3344 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 13:30:54 +00:00
calvin
43c0e3447a reduce wait/sleep time for URL queue getting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3341 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-04 23:51:32 +00:00
calvin
19a7495b9e only accept ASCII robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3339 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-04 21:07:08 +00:00
calvin
d95d8c3d96 correctly handle internal errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3338 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-03 01:14:05 +00:00
calvin
21b53215e4 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3337 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:13:12 +00:00
calvin
997e686a69 no exit on internal errors in threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3336 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:13:04 +00:00
calvin
a57618a4ad use relative imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3335 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 14:06:19 +00:00
calvin
c3fa9d7965 don't generate empty output files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3332 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 13:58:13 +00:00
calvin
677458b8d6 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3330 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 13:54:55 +00:00
calvin
8763a42063 skip a test on nt platforms
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3327 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 16:08:37 +00:00
calvin
4f668b8f90 added file norm tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3326 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 14:22:37 +00:00
calvin
850684b1e0 datadir as url path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3325 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 14:00:45 +00:00
calvin
2888c34859 split file tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 13:54:40 +00:00
calvin
5b8b2cbda1 use diff for file compare
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3323 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 13:43:10 +00:00
calvin
2e1102a1b5 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3322 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 12:57:51 +00:00
calvin
f5f7007c34 added abstract run_checked()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3316 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 14:25:04 +00:00
calvin
b442809838 look that cached URLs get checked quickly in large queues
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3315 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:35:24 +00:00
calvin
e211d3fd6c fix internal error call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3314 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:33:23 +00:00
calvin
694f05405a translate interrupt message, and check for changed signal handler
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3313 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:33:01 +00:00
calvin
cb853c4672 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3312 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:32:32 +00:00
calvin
4c8dcb012b added a stoppable thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3305 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 22:29:22 +00:00
calvin
c91ccc56d6 add and use a stoppable thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3303 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 22:16:36 +00:00
calvin
c3afa5c9c6 improved tmieit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3299 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:54:47 +00:00
calvin
b898cc05d3 added§
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3298 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:54:37 +00:00
calvin
e7107bc270 return function result in timeit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3297 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:52:45 +00:00
calvin
1974bdcaf1 use xrange
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3296 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 16:52:39 +00:00
calvin
d557b0208f whops, reintroduce url_data.url is None check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3295 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-24 00:00:33 +00:00
calvin
f30964679c remove return in __init__
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 23:59:09 +00:00
calvin
f746b29522 documentation added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 22:04:30 +00:00
calvin
27997b0251 add a finish() method to wait for spawned threads to finish
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3290 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 19:55:35 +00:00
calvin
7a94995345 remove unused condition
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3289 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 19:51:22 +00:00
calvin
49a21901f9 raise correct Empty()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3288 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 18:57:26 +00:00
calvin
eaf1fd4ba1 use timeout for get() method, return thread object
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3286 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:33:09 +00:00
calvin
a1905bdb22 support timeout in get() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3285 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:33:00 +00:00
calvin
5eec9cf527 update from subversion repo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3284 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 16:32:50 +00:00
calvin
97e20ccdd4 move status method to status package
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3283 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-22 21:35:41 +00:00
calvin
ff1e16230b wait until status thread is finished
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3282 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-22 21:35:20 +00:00
calvin
e51bd72957 use xrange
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3280 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 23:02:46 +00:00
calvin
98597c267d quote result line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3272 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:21:20 +00:00
calvin
3142663135 added tests for UnicodeError 'label too long'
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3270 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:13:28 +00:00
calvin
7e1e01bd36 do not catch UnicodeError, handle that intern
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3269 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:13:16 +00:00
calvin
2c13d7cac1 norm test urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3267 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:43:55 +00:00
calvin
608f8ba1c3 prepare filenames as URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3266 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:38:57 +00:00
calvin
37615dba02 use datadir, curdir placeholders
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3265 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:35:52 +00:00
calvin
14a29fb015 prepare filenames as URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3264 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:35:36 +00:00
calvin
0ba1520d13 fix filename for test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3263 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:14:30 +00:00
calvin
1dbc97abe7 script moving
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3262 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:10:16 +00:00
calvin
23879d78d4 adjust test result for new cache optimization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3259 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 19:14:54 +00:00
calvin
cd8886c77f adjust test results for optimized cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3258 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 18:55:03 +00:00
calvin
7f408cce19 fix the self.in_progress optimization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3257 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 18:54:41 +00:00
calvin
d43acde696 added cookie parse tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3251 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:31:49 +00:00
calvin
06956060b5 added documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3250 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:31:33 +00:00
calvin
003532e20f put in-progress URLs back to the top of the URL queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3248 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 16:16:19 +00:00
calvin
17555e2fa6 use configured timeout on abort, interrupt main
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3241 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 20:19:40 +00:00
calvin
74327404a4 ensure the signal module is available
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3240 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 20:19:13 +00:00
calvin
faddd9acc3 catch Timeout from abort
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3239 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:56:14 +00:00
calvin
781ccf96c1 add irc scheme to netloc using schemes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3238 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:12:29 +00:00
calvin
2ec5c054fe merge ignoredurl and errorurl into unknownurl, updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3237 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:08:40 +00:00
calvin
a4e9b8eab1 fix debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3236 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 16:24:53 +00:00
calvin
4ec74f6f5c added robots.txt tests for the internal HTTP server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3232 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:47:48 +00:00
calvin
4bf2b361cb added more crawldelay tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3230 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:34 +00:00
calvin
e574a97798 raise ValueError if wait delay is a negative value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3229 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:27 +00:00
calvin
7dce5c4df9 fix callback call for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3228 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:09 +00:00
calvin
3adaf48b3d add callback for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3227 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:35:58 +00:00
calvin
a741d7922c add get_crawldelay method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3226 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:35:48 +00:00
calvin
fb319b3785 add callback after robots.txt parse
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3225 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 23:19:18 +00:00
calvin
2aed0f3bc5 add per-host wait times
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3224 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 23:03:44 +00:00
calvin
811f5492c4 fix --pause to delay requests to the same host
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:56:13 +00:00
calvin
ad28599e57 Note if URL is missing (instead of saying it is empty)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:15:34 +00:00
calvin
00a60c6906 check if urldata.url is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3219 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:14:34 +00:00
calvin
224f5e723b don't find link name in empty urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3218 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:05:17 +00:00
calvin
72c718b7d5 added robots.txt parse test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3212 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 21:29:31 +00:00
calvin
d73aa0e5bd parse crawl-delay parameter line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3211 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 21:29:18 +00:00
calvin
75e88c062a added --cookiefile option to set initial cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3210 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 20:56:34 +00:00
calvin
c6b5759c39 print thread name on trace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3206 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 17:00:49 +00:00
calvin
0bb2970222 remove old consumer code
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3205 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 17:00:27 +00:00
calvin
b6ad3084aa added more anchor tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3201 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 16:22:56 +00:00
calvin
7c8ab1b6f7 No maximum size due to possible deadlocks.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3195 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 16:14:42 +00:00
calvin
e8bf93bd90 inherit from StandardError, not from Exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3194 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 22:22:24 +00:00
calvin
c10da15b08 add timeout setting to configuration file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3192 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 19:04:21 +00:00
calvin
61fc7ab502 since join() is not interruptable, put in a little sleep() call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3191 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:36:55 +00:00
calvin
ca8ea4a2be print in-progress URLs on status
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3189 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:35:37 +00:00
calvin
ac682df355 be precise for threads: zero threads disable threading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3188 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:35:16 +00:00
calvin
b3b45d7f06 add Timeout exception, and return in_progress URLs on status instead of unfinished
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3187 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:34:30 +00:00
calvin
b1e7ca58a2 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3186 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:33:38 +00:00
calvin
dc62353ca2 don't catch SystemExit, and no bare except
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3185 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:06:44 +00:00
calvin
ffe233c2eb don't catch SystemExit, and no bare except
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3184 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:05:52 +00:00
calvin
938d193446 store timeout in configuration
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3183 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:05:17 +00:00
calvin
c8bcdf7d79 remove unused threader class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3182 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:02:07 +00:00
calvin
70fedc052c replace bare excepts with ones catching StandardError
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3181 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 18:01:29 +00:00
calvin
df8af83508 use debug lock class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3179 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 19:58:55 +00:00
calvin
ab1ba455f6 make lock class just a debug wrapper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3178 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 19:58:46 +00:00
calvin
ffeaa3484e add debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3177 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 18:03:59 +00:00
calvin
df7a2076a9 cache DNS lookups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3176 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 18:03:23 +00:00
calvin
afdccbaa80 set names of worker and status threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3175 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:50:36 +00:00
calvin
523e6e8e43 use variables in result lines
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3174 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:49:58 +00:00
calvin
91ff370ed7 on redirection to different URL scheme take caching into account; adjust tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3173 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:47:50 +00:00
calvin
2a336f8dad put redirects in url queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3172 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 17:01:41 +00:00
calvin
2cfcb5c0bb avoid double timeouts by raising timeout errors in robots.txt retrieval
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3171 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 12:58:31 +00:00
calvin
9a431fde40 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3170 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 10:14:07 +00:00
calvin
5ba1324a73 an intern pattern matches both http and https URLs now
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3169 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 10:06:33 +00:00
calvin
853f1d7f51 always disable status thread after checking, and log end of output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3165 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 08:41:52 +00:00
calvin
ddf3703962 put status printing in a separate module, with a flag that can be disabled
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3163 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 08:29:44 +00:00
calvin
53e1bd993e fix test output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3162 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 21:25:59 +00:00
calvin
dc9f04e6dc adjust debug asserts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3159 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 21:03:21 +00:00
calvin
2947b4710d quote slashes in cgi values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3158 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:52:30 +00:00
calvin
857582c026 honor case where fallback class is not given
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3157 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:51:11 +00:00
calvin
ee0a7bb824 don't return True on debug, add shutdown method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3156 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 20:50:52 +00:00
calvin
67f39e516c synchronize the log method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3152 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:53:41 +00:00
calvin
0d39d6139f no active tasks in status
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3150 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:15:40 +00:00
calvin
7afe60c841 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3149 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:08:35 +00:00
calvin
98a6d1eb00 added missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3148 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:08:06 +00:00
calvin
b0c189d143 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3147 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 18:07:46 +00:00
calvin
f002c5f965 Replace the old threading algorithm with a new one based on Queue.Queue and consumer threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3146 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:44:52 +00:00
calvin
d05c68ef74 added path encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3145 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:41:46 +00:00
calvin
c33e6d5907 split up invalid* tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3144 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:41:10 +00:00
calvin
3e8a221110 acquire locks directly
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3143 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:40:22 +00:00
calvin
bfd52936a6 fix config parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3142 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 09:53:53 +00:00
calvin
ac9d88cd99 added config test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3141 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 09:51:50 +00:00
calvin
394abbbab5 fix imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3137 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-12 14:51:09 +00:00
calvin
d667abac69 Don't encode the host header with idna if not necessary. Taken from Python subversion trunk.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3136 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-04 18:24:23 +00:00
calvin
c2f5d222e1 split off configuration parsing in a separate class, put config stuff into its own module directory
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3135 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 18:24:46 +00:00
calvin
d547b0ee31 fix mail tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3134 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 17:00:08 +00:00
calvin
0bd43753d3 remove old resource check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3133 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:59:58 +00:00
calvin
c35737ff3a use new long duration format
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3132 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:39:01 +00:00
calvin
58993644d8 use format_time from quodlibet source
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3131 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-03 16:35:37 +00:00
calvin
4b10271233 use unittest.TestCase for standard test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3129 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:16:57 +00:00
calvin
f9a59e616d added caseless dict classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3128 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:13:29 +00:00
calvin
bae6c90f43 documentation cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3127 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:12:53 +00:00
calvin
a764b1484b import cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3126 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-24 20:12:40 +00:00
calvin
b4d4b2ed94 cast to size_t
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3122 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-11 23:53:25 +00:00
calvin
787d9143fe don't call write_color with None argument
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3120 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 10:26:15 +00:00
calvin
31e853e0d7 remove unnecessary global statements
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3119 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:02:14 +00:00
calvin
02ecf152c6 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3118 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:01:49 +00:00
calvin
cce1d273a7 really return value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3117 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:00:44 +00:00
calvin
4845e926a9 always use default locale for translation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3116 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 09:00:23 +00:00
calvin
4f67591327 no bare excepts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3115 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:59:52 +00:00
calvin
2544c90708 fix detection of </script>
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3113 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:58:31 +00:00
calvin
6dacc9bf61 use has_* flags for resource availability
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3112 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:57:52 +00:00
calvin
78dfb4c293 whitespace normalization
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3106 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:15:07 +00:00
calvin
b503ea15f4 updated cgi tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3105 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:14:49 +00:00
calvin
9f7dde02f8 use WConio on Windows for colored output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3104 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:14:15 +00:00
calvin
db223a1912 write label for graph
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3103 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:13:41 +00:00
calvin
38b83415dd correctly replace spaces in cgi params
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3102 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:12:43 +00:00
calvin
893300308c updated time format tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3101 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:12:30 +00:00
calvin
0132e4ca89 better time formatting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3100 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:05:45 +00:00
calvin
1cd74500ee no literal </script>, accept empty SGML comments, compress equivalence classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3099 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:04:40 +00:00
calvin
e42f1a0cec replace sf.net with sourceforge.net
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3094 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 16:35:39 +00:00
calvin
df66edb7c3 allow empty comment in HTML parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3091 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-10 10:48:25 +00:00
calvin
aeb698cbec add localhost to local interface list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3090 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-10 10:48:07 +00:00
calvin
1d724a034a Construct equivalence classes in flex scanner
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3089 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-03-09 22:38:51 +00:00
calvin
1bf4fac474 removed .cvsignore
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3086 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-24 20:09:03 +00:00
calvin
00310425ac updated copyright template
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3081 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-17 23:38:12 +00:00
calvin
68a4d32212 rename _synchronized in synchronize
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3079 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-16 19:47:37 +00:00
calvin
46ed49d148 ensure log output is unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3075 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-13 21:49:25 +00:00
calvin
276437c7d8 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3067 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-09 07:57:22 +00:00
calvin
287ec3b960 syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3060 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-04 09:15:11 +00:00
calvin
e56215235c cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3058 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 23:58:30 +00:00
calvin
c334b8db13 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3056 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 23:21:20 +00:00
calvin
1e0581e718 use iter* dict methods
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3054 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 22:15:41 +00:00
calvin
2788fc2a59 add modification time information functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3052 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 20:42:56 +00:00
calvin
6cda627f0f more tests added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3046 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-03 17:06:25 +00:00
calvin
2880274744 more test coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3045 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 23:31:39 +00:00
calvin
38a7b15aeb accept a dict as lru init val
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3044 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 23:00:03 +00:00
calvin
902949f27e some format fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3043 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:59:42 +00:00
calvin
fac0b0f793 remove unused quote_val method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3042 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:58:40 +00:00
calvin
3f6ac53fff added tests for more coverage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3041 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 22:58:11 +00:00
calvin
3a5f06cfa9 remove unused strip_quotes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3040 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:53:11 +00:00
calvin
a3e4780b38 quote attributes with unicode entity escapes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3039 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:40:57 +00:00
calvin
75be4d0bb6 fix entity resolving
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3038 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 21:40:02 +00:00
calvin
cbef33ec5e fix parsing of hexadecimal entities
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3037 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 20:06:59 +00:00
calvin
94e083315b allow escaped quotes in tag attributes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3036 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-02 20:05:59 +00:00
calvin
9fd7b24344 check return value of acquire()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3034 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 19:48:22 +00:00
calvin
b6c947442a documentation fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3033 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 19:48:02 +00:00
calvin
1df673a79d add url_unsplit method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3029 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:56:20 +00:00
calvin
5a4f1248c3 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3028 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:56:04 +00:00
calvin
0a257847e3 add a Buffer class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3026 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:55:40 +00:00
calvin
6c14cf1d8b syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3025 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:54:11 +00:00
calvin
a2ed1b323f do not lowercase the absolute url, caused the URL matching not to work for URLs with uppercase characters
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3022 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:49:31 +00:00
calvin
0681b1889f limit amount of read chunks, and cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3017 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-11 22:03:58 +00:00
calvin
0bb41a5ec3 do no print warning about different dns message ids
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3016 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 22:32:42 +00:00
calvin
bb3dac673f added coding line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3015 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:04:29 +00:00
calvin
fc5e34b216 added more decorators
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3014 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:02:50 +00:00
calvin
e94c61529b catch value error on codec lookup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3013 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-06 15:02:32 +00:00
calvin
48cdfd32ee fix tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3011 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-04 01:20:20 +00:00
calvin
e92aee054c updated copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3010 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 19:12:47 +00:00
calvin
fe24c56e84 only split host ports on known schemes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3009 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 18:37:42 +00:00
calvin
b4748cdcee back out decorator change
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3002 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 01:09:23 +00:00
calvin
bc846a20ff use own splitport method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3001 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-02 14:26:24 +00:00
calvin
6b6adab12f new decorator_with_args helper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3000 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-02 14:25:50 +00:00
calvin
388475cbe2 new-style exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2999 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 20:17:33 +00:00
calvin
7b3949eebf cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2996 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 09:09:32 +00:00
calvin
8d67ee35ef removed is_safe_js_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2995 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 09:08:51 +00:00
calvin
a24fd65943 always quote port value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2992 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-19 14:08:34 +00:00
calvin
815fb9da83 more tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2991 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-19 14:06:53 +00:00
calvin
c84a33c7ce syntax fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2988 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 09:13:23 +00:00
calvin
856ff8ef2a assert debugs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2987 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:55:42 +00:00
calvin
aec847e570 assert debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2986 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:46:48 +00:00
calvin
1cec04018f cleanup unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2985 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:40:22 +00:00
calvin
19c0a3c2ed use new cookie parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2983 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:19:11 +00:00
calvin
5eb3061ee0 added new cookie parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2982 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:18:32 +00:00
calvin
a22d055d0b make debug() function assert'able
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2981 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:17:18 +00:00
calvin
d28567753b add is_encoding() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2980 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:16:36 +00:00
calvin
cff9b1341b improved charset parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2979 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:16:25 +00:00
calvin
24e91ffaaf fix info logging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2978 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-18 08:15:49 +00:00
calvin
d7a4a08659 updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2974 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-17 19:22:44 +00:00
calvin
3b1b95af5a added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2970 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-17 19:18:08 +00:00
calvin
9425873830 cache aliases (from redirects)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2964 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:20:21 +00:00
calvin
43fb20371f new tagged infos
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2963 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:20:04 +00:00
calvin
b34f4da7fe better error message if no nameservers could be found
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2962 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:19:48 +00:00
calvin
47b4a52134 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 17:14:39 +00:00
calvin
791bd87f6d use LinkCheckTest
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2956 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 17:08:08 +00:00
calvin
ee2d4e6ead moved ftests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2954 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:40:19 +00:00
calvin
4cf6f29afe unused import removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2953 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:40:09 +00:00
calvin
26ccd4d81c fix func name typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2952 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:39:17 +00:00
calvin
329043bc9f use datadir
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2951 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:33:14 +00:00
calvin
98e3e26644 moved
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2950 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 22:32:48 +00:00
calvin
ddb09ab76a added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2949 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-07 21:55:16 +00:00
calvin
c86c0870d6 debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2946 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 22:32:31 +00:00
calvin
b733ffdb2a remove has_incoming, debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2945 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 22:32:14 +00:00
calvin
3c03728d29 add update_meta utility func
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2942 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 17:10:17 +00:00
calvin
b482d35656 fix exception name and add 0.0.0.0 to localhost list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2941 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-06 11:47:38 +00:00
calvin
99c0382781 add timeit decorator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2940 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:50:44 +00:00
calvin
e2a454e31f use new trace module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2939 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:49:37 +00:00
calvin
ae508129f1 move trace into separate module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2938 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:49:22 +00:00
calvin
7d1ea45be6 rebuild with bison 2.1
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2937 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-14 22:47:30 +00:00
calvin
bcbbe6ff60 allow https js urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2935 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:46 +00:00
calvin
a4cfa10fdf better comment parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2934 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:31 +00:00
calvin
f65cfb87f9 fix tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2933 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:56:01 +00:00
calvin
065665a0f1 rename builtins
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2932 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-12 19:55:54 +00:00
calvin
77ed566471 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2931 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-09 07:46:22 +00:00
calvin
9eab0056ac encode url query
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2930 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-09 07:46:05 +00:00
calvin
e29efc31fa more checkes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2928 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:06:29 +00:00
calvin
c049837cdd remove parser messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2927 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:05:59 +00:00
calvin
de2e66e713 remove parser errors/warnings, and better error detection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2926 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-04 20:05:35 +00:00
calvin
dffe4f906e documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2925 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:56:28 +00:00
calvin
35f1172481 count errors in tests, and add some more cases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2924 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:56:12 +00:00
calvin
b8aa3009d8 store a list of errors, and add more error messages to the parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2923 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-03 16:53:26 +00:00
calvin
7d5be699df documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2922 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-02 13:49:03 +00:00
calvin
0dca65accf depend on python >= 2.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2921 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-11-02 13:48:53 +00:00
calvin
7e90eccc84 dnspython 1.3.5 merged
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2920 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-31 08:59:29 +00:00
calvin
df34e1a8e9 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2919 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:48:30 +00:00
calvin
1d891fffec fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2918 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:45 +00:00
calvin
fa3d35beff beautify equals check msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2917 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:28 +00:00
calvin
c2b4132eb9 ensure html attr values are strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2915 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:47:03 +00:00
calvin
e3beb86ac1 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-17 12:18:18 +00:00
calvin
c9f5d1a0b1 catch gzip errors, and use linkchecker debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2910 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-15 00:06:48 +00:00
calvin
041df3a48d documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2906 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 22:54:23 +00:00
calvin
a2e422ce0d reindent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2900 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 22:26:12 +00:00
calvin
96f4f946b2 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2898 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 21:29:17 +00:00
calvin
d83720f05d do not parse any invalid config files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2895 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 21:15:31 +00:00
calvin
8ce45149fa re-add nofollow check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:49:56 +00:00
calvin
93c498699d disable keyboard interrupt and SystemExit while aborting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2888 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:11:36 +00:00
calvin
9e560f9ccf only return on thread wait timeout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:08:14 +00:00
calvin
c585d095b4 catch SystemExit from internal errors in subthreads, and eliminate abort wrapper
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2886 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:06:57 +00:00
calvin
5303f94b76 ensure proxy value syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:59:20 +00:00
calvin
50ea41b12d use preferred locale for default encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2878 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:22:12 +00:00
calvin
753b907d51 remove unneeded imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2871 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:57 +00:00
calvin
66a30acc2d assume cStringIO is there
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2870 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:22 +00:00
calvin
6ca5dea8c3 define fs var
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2869 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:56:07 +00:00
calvin
19a290d81a query and anchor quoting tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2868 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:55:53 +00:00
calvin
9c8cee034a quote anchor
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2866 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:50:48 +00:00
calvin
8c809d1dc3 break long line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2865 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:50:05 +00:00
calvin
25044ce08b print errors in the preferred encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2864 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 12:49:52 +00:00
calvin
c205cec12d remove script tag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2859 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 20:09:40 +00:00
calvin
4e1bbfea64 adjust yyget_extra definition
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2858 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:16:19 +00:00
calvin
9eaf38d563 sane sparse path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2857 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:15:55 +00:00
calvin
9076eaa7da python2.4, add sparse check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2856 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 16:15:00 +00:00
calvin
732366b3f5 add string include for size_t
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2855 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 15:53:32 +00:00
calvin
8953766ccd fix url part quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 14:08:14 +00:00
calvin
2da64736ba remove unneeded init_i18n() calls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2852 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 13:00:33 +00:00
calvin
80b6ba5416 init i18n on module load
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2850 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:52:36 +00:00
calvin
6dd4ac6a65 typo fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2849 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:52:28 +00:00
calvin
7cf3f83c0f better recursion check debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:39:46 +00:00
calvin
978ba63999 catch UnicodeError on syntax check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2847 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:10:32 +00:00
calvin
fccccac818 coerce to unicode in url_norm if arg was unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2845 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 12:09:16 +00:00
calvin
8cfcbf075a use archived musi-cal robots.txt page
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2844 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 10:08:58 +00:00
calvin
b48c6f7e96 fix default resolver init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2843 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 09:55:26 +00:00
calvin
51f541607c catch CGI syntax errors in mailto: URLs and add a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2841 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 23:06:07 +00:00
calvin
65150ad79e coerce to unicode after relative file handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2839 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:50:40 +00:00
calvin
e1192c078f do not append non-parseable meta refresh tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2838 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:21:59 +00:00
calvin
2bba7df3e7 handle case where func.__doc__ is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2837 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:10:32 +00:00
calvin
57c9f325a9 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2836 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:10:10 +00:00
calvin
5b6782c6a9 handle ignoreX and nofollowY stuff backwards compatible
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2834 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 22:05:16 +00:00
calvin
5c83b7c859 fix error call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2833 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:59:46 +00:00
calvin
95081a62e6 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2829 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:53:52 +00:00
calvin
ba6a2ed4e0 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2819 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:15:09 +00:00
calvin
e11bc21593 new methods stripall and limit
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2818 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:14:05 +00:00
calvin
b28be779d7 ensure tags are ASCII, regen with bison 2.0
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2817 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:13:35 +00:00
calvin
4315e77ae1 dont use XXX markers here
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2816 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:12:41 +00:00
calvin
2ba2988d2c Add tests with invalid tag chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2815 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-10 21:12:12 +00:00
calvin
712df7d743 added quote_val
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2811 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:59:07 +00:00
calvin
a74e728246 ignore loopback interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:58:41 +00:00
calvin
fc4ce83372 dont quote exclams
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2809 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:57:36 +00:00
calvin
2f48855b32 added notimplemented
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2808 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:57:25 +00:00
calvin
da8a4d2cfe dynamic buffer for interface list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2807 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:56:54 +00:00
calvin
4b697eccea added is_ascii and indent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2806 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:55:51 +00:00
calvin
866999a883 added more tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2805 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-16 13:42:46 +00:00
calvin
37e8e21df6 add notimplemented
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-11 15:49:34 +00:00
calvin
d0dab1d0ac updated to dnspython 1.3.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2792 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-03 19:27:31 +00:00
calvin
33d3a6c974 fix invalid import in DNS resolver
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-08-03 18:47:32 +00:00
calvin
fe4395c41f test case for HTTP -> non-HTTP redirection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2787 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 16:05:20 +00:00
calvin
f01b84b894 rework the redirection routine a little, putting warnings specifically for redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2786 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:51:12 +00:00
calvin
ed932cb165 debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:50:27 +00:00
calvin
f24bb87e54 add missing return
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 12:37:32 +00:00
calvin
afa8750dc3 catch ValueError raised by urllib2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 09:45:30 +00:00
calvin
317ef181f0 handle all redirections to different URL schemes,
not just HTTP -> not HTTP, and fix a variable typo


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 08:53:57 +00:00
calvin
0d55ea45b7 fix variable clash
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-18 08:19:11 +00:00
calvin
b94eec79ef use correct value for status output of checked links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-17 23:26:13 +00:00
calvin
9fa3a1216c replace 'print' with 'write'
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-16 07:12:44 +00:00
calvin
c553e233b7 change the custom XML format
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2770 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-16 07:12:15 +00:00
calvin
842333a36f write parent line/col as attrs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2765 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:51:07 +00:00
calvin
c2e235a154 fix end xml tag writing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2763 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:46:13 +00:00
calvin
e2e80367d1 add copyright, add more documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2762 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:41:54 +00:00
calvin
19118a0198 make copyright ASCII
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2761 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:37:45 +00:00
calvin
0b5294ef57 set output encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2760 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:37:16 +00:00
calvin
3011eb5ee6 new custom XML logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2759 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 21:33:15 +00:00
calvin
6239880f15 simplify test suite generation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2756 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 14:24:25 +00:00
calvin
d347840dee use official HTTP status names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2754 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 11:31:32 +00:00
calvin
baf51d1f5d cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2752 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:56:05 +00:00
calvin
359612b6fe add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2751 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:53:27 +00:00
calvin
225c045815 do not cache unusual errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:11:13 +00:00
calvin
5d82d64c31 more result sanity check warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2749 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 13:35:18 +00:00
calvin
3fb9de0847 upstream updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 13:34:45 +00:00
calvin
c140b510b1 fix warning tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 19:33:26 +00:00
calvin
f80480469f config to ignore certain warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 15:03:17 +00:00
calvin
badb88ed68 use errno constant EPIPE
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 18:36:03 +00:00
calvin
5b5eda36e4 updated from python CVS
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2741 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 18:09:15 +00:00
calvin
b559c4f82e test for hierarchical URLs, and retain non-numeric port parts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 07:18:13 +00:00
calvin
907facaf16 put start_server into the try-finally clause
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2738 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-12 07:17:01 +00:00
calvin
34ced03fc2 added more norming tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 21:58:58 +00:00
calvin
5049e261fe added LOG_DNS
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2734 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 15:37:15 +00:00
calvin
bf9e55c1c8 updated documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2731 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 14:43:52 +00:00
calvin
3ce6aadfd6 updated documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2730 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 14:37:52 +00:00
calvin
e5197d429a added error check macro, and check errors on html_end_tag invocation. Also some documentation updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2729 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-11 12:02:18 +00:00
calvin
5244431a42 add ascii_safe method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 22:06:38 +00:00
calvin
56b2b422a9 ensure that thread name is ascii
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 22:06:18 +00:00
calvin
bb2b227b13 catch all errors on url check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 22:05:54 +00:00
calvin
e7ca18403a work around python bug occuring with incomplete gzip streams
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2716 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 19:56:42 +00:00
calvin
901227d436 assign extern info to local var
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 22:19:52 +00:00
calvin
aadd6fd71f fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2708 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 22:19:22 +00:00
calvin
5f2d11b613 fix logger using extern attribute
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2705 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 21:48:55 +00:00
calvin
2e207be127 allow empty relative URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 20:28:55 +00:00
calvin
55109f9111 fix log level
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2687 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-03 23:08:26 +00:00
calvin
23707a4471 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2684 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-28 13:37:13 +00:00
calvin
f09d54f75b meta tests added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2683 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-28 13:36:52 +00:00
calvin
10bbe998c7 accept more meta tag urls for checking, eg favicons
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2682 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-28 13:36:39 +00:00
calvin
506eaa21b5 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2680 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-28 10:03:04 +00:00
calvin
555e2dc2a4 fixed test output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2679 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-28 09:57:31 +00:00
calvin
49aa4ec208 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2678 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-27 15:02:10 +00:00
calvin
1df5b81f0e updated test result data
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2677 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-23 16:21:26 +00:00
calvin
0c3e78569c fix norming
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-23 16:21:05 +00:00
calvin
217d34bb54 use new extern check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2675 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-22 12:30:03 +00:00
calvin
d48eb20a28 support extern filter in ignored url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2672 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-20 14:53:23 +00:00
calvin
990d548424 don't delete from incoming queue while iterating, but after. SF #1223920
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2671 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-20 09:49:27 +00:00
calvin
51e4657cf6 add one more parent path test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 21:19:12 +00:00
calvin
ea052008a2 send the complete email address with VRFY command
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2654 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 21:17:10 +00:00
calvin
cc0f7cb113 always norm a URL before sending a request
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2653 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 21:16:47 +00:00
calvin
eba33c61e6 resync with python 2.4 httplib
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2652 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 20:58:09 +00:00
calvin
125707fceb parse combined start/end tags in tag parsers, and added a test for that
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2649 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-09 15:10:29 +00:00
calvin
1b0e300f5d use cache_url_key instead of old get_cache_key() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2648 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-09 15:08:53 +00:00
calvin
d965a387d5 catch curses setupterm errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2643 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-07 10:38:33 +00:00
calvin
c9239135b1 fix exception test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-02 21:54:41 +00:00
calvin
79d0850fed fix intern url of file: links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-22 20:28:59 +00:00
calvin
fec843814c fix intern url of file: links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2635 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-22 20:27:32 +00:00
calvin
c51a60e0b5 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2627 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 23:02:44 +00:00
calvin
afca416b67 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2625 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 20:25:01 +00:00
calvin
43c6ea202d remove unused
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2624 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:57:12 +00:00
calvin
6d8f0146d6 fix GET fallback handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:53:13 +00:00
calvin
d2107c7a6a split and fix initialization of UrlData objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:42:14 +00:00
calvin
cd32e823df fix url util call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2621 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:52 +00:00
calvin
9666421a10 add log_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2620 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:40 +00:00
calvin
1fc5eb32b6 fix extern calculation and a typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2619 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:16 +00:00
calvin
f6f04e376e fix extern calculation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2618 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:01:51 +00:00
calvin
d6e56ceba2 remove lock from check_url in case of disabled threading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 18:30:06 +00:00
calvin
3407ab362d syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2616 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 18:29:38 +00:00
calvin
55d7d5f7df use a decorator to synchronize all the checker threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2614 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 15:49:06 +00:00
calvin
3f6d2fa753 specify thread name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2613 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:16:23 +00:00
calvin
781ad5fdc0 catch exceptions on abort
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2612 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:15:54 +00:00
calvin
fd652881fb thread name and maximum wait for finish
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:15:18 +00:00
calvin
3db056f161 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:14:45 +00:00
calvin
4c270cfe74 log message formatting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 18:04:54 +00:00
calvin
e697b7a3ce more generic nt filename test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2608 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 18:04:05 +00:00
calvin
e1e7055e79 trace filtering
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2606 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 17:58:35 +00:00
calvin
7df1caf58c documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2605 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 17:57:41 +00:00
calvin
a51a0913d3 nt specific test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2604 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 17:56:52 +00:00
calvin
63b76ec642 Use HTTPMessage() in all urllib handlers, really fixing the bug noted in http://www.python.org/sf/1117588. The workaround has been removed.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2603 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 17:53:39 +00:00
calvin
cb54822dac trace ignore names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2602 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 22:46:05 +00:00
calvin
c0d9e56a4d new trace option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 22:06:47 +00:00
calvin
50c536c82a lock debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2600 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 22:06:17 +00:00
calvin
0ea778b6fd dont trace with log function, else a deadlock occurs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2599 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 21:49:09 +00:00
calvin
ed738d2436 cmdline param defaults to False
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 19:00:17 +00:00
calvin
a85ac68e16 cmdline param
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2594 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 14:12:30 +00:00
calvin
e3a049ee27 make intern pattern of parent path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2593 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 14:11:17 +00:00
calvin
6862ba1433 ability to set cmdline param in direct test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2592 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 09:40:34 +00:00
calvin
153ea4fa06 add cmdline arg to get_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2591 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 15:47:07 +00:00
calvin
1ec0c53c91 sanitize CGI env, add strip to goodaddrs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2590 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 15:46:31 +00:00
calvin
9413039083 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2589 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 15:41:44 +00:00
calvin
d821abce85 news group info string
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2588 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 14:01:53 +00:00
calvin
666ead2bb3 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2587 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 14:01:25 +00:00
calvin
51fdb7e4a3 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2586 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 13:50:41 +00:00
calvin
208a7f095f added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2585 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-09 22:07:45 +00:00
calvin
1a7a771648 new intern/extern handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2584 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-09 22:05:21 +00:00
calvin
ce2f2482eb URL consistency
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2582 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-08 20:16:52 +00:00
calvin
6ff56a53cc rename logger fields to logger parts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2579 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-08 20:06:40 +00:00
calvin
cd899b7e57 updated test results
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2574 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 22:28:49 +00:00
calvin
6919a06d8c fix backtrace printing, and only call _log when the loglevel is enabled
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2573 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 22:17:23 +00:00
calvin
829e19e7ee cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2572 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 22:15:12 +00:00
calvin
c07f42031a ignored urls only issue an info instead of a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2569 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 14:59:22 +00:00
calvin
6eb990d227 do not hardcode output locale
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2568 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 13:21:10 +00:00
calvin
b6662a3098 use startswith method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2567 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 13:06:27 +00:00
calvin
cd3f5294cb better file descriptor initialization and closing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2566 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 13:05:16 +00:00
calvin
bf8fd348aa better msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 13:22:10 +00:00
calvin
532a04e436 method change is an info, not a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2563 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 13:18:19 +00:00
calvin
96b74a1f41 collapse reduntant dot path at beginning of url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2558 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 12:58:38 +00:00
calvin
fd8c56397a updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2557 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 11:44:02 +00:00
calvin
49a2dee338 make robots.txt deny an info instead of a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2555 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 10:48:48 +00:00
calvin
a39f635b82 use some %r in warning strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 10:48:10 +00:00
calvin
b6e146750d try to read only existing default config files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2551 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-02 14:46:47 +00:00
calvin
d2a8764d18 do not store aliases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-27 12:48:48 +00:00
calvin
23e5212c7d do not remove userinfo from host part of url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-26 17:53:31 +00:00
calvin
dc2b0feca4 removed redirection warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2543 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-26 17:51:41 +00:00
calvin
82dee6ae52 reintroduce the self.url updated, this time only for redirections but without a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2542 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-26 17:51:15 +00:00
calvin
93ce3364b0 split CGI params on both ampersand and semicolon
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2541 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-26 17:28:12 +00:00
calvin
134c76de4a only use signals on posix systems
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2538 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 23:49:59 +00:00
calvin
60069a8862 improved internal error msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2537 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 23:46:37 +00:00
calvin
53b5f67745 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2536 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 23:45:54 +00:00
calvin
c95c4b11d7 added tracing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2535 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 23:02:40 +00:00
calvin
5e6317a4bb remove old warning, and move effective url test to urlbase
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2532 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 18:25:32 +00:00
calvin
006f79702e make warnings enabled as default
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 14:52:52 +00:00
calvin
71407d0399 use new logger interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 14:52:38 +00:00
calvin
040139e6d8 count errors, warnings and number of checked links in the logger classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-25 14:51:35 +00:00
calvin
f94f910209 signal handler
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2523 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-22 20:15:10 +00:00
calvin
7925d5dd56 use collections.deque for incoming queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2518 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-13 21:05:52 +00:00
calvin
813e1955d9 use pydoc.pager()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2511 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-07 13:46:24 +00:00
calvin
ba5939c7c5 allow optional empty url path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2500 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-31 13:50:11 +00:00
calvin
cf05e2cb6f correct get_cache_keys method name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2499 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-31 08:18:50 +00:00
calvin
bfe3f4cd4d do not add aliases that are in-progress
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2491 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 20:37:06 +00:00
calvin
5b64583904 add ignore_missing flag and do not cache already-cached aliases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2490 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 20:26:35 +00:00
calvin
eeb6da8330 do not cache HTTP connections with pending data
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2487 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 18:47:09 +00:00
calvin
ecd7e44935 add assertion for check queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2481 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 15:52:40 +00:00
calvin
911e2eb011 fix persistent connection detection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2477 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 13:41:22 +00:00
calvin
7429dfa209 cached connection debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2476 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 13:40:54 +00:00
calvin
ccdeaa86b4 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2475 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 12:35:03 +00:00
calvin
ebb48fa3b8 use get_true method to ensure that urls are always a unicode string and not None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2472 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:31:17 +00:00
calvin
63a71baae6 add get_true method to ListDict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2471 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:30:47 +00:00
calvin
f141255ab2 allow zero number of threads (means trheading disabled)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2470 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:30:29 +00:00
calvin
a50c3ce40a use threader init param to set number of threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2469 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:29:49 +00:00
calvin
6cbbe9265e added misc html parse test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2468 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:28:51 +00:00
calvin
cbd333955e only strip line endings from test data
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2467 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 11:28:07 +00:00
calvin
9cc3f978a6 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2465 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-29 08:52:40 +00:00
calvin
21757150f6 workaround for buggy l18n algorithm
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2464 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-28 23:52:46 +00:00
calvin
642e2d1698 do not split cgi params on semicolons
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2463 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-28 23:51:29 +00:00
calvin
1cf7426026 public method is_locked()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2462 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-28 15:56:25 +00:00
calvin
8e720030d1 check A RR records when no MX RR record could be found
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2461 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-28 14:59:17 +00:00
calvin
28d3b45819 aded
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-20 10:14:11 +00:00
calvin
bbf96868b8 document aliases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-18 14:21:12 +00:00
calvin
26e52022ea add aliases to cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2457 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-18 14:17:23 +00:00
calvin
e24c38c2aa redirect test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2456 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-18 14:16:56 +00:00
calvin
dcd407782d no double alias reset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2454 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-18 00:02:50 +00:00
calvin
0b0e672b4e cleanup of the fallbacks
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2453 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-16 17:34:02 +00:00
calvin
9b982062fc do not translate countries
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2434 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-12 00:43:52 +00:00
calvin
0baec73a77 only find redirection links with http-equiv='refresh'
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2428 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 23:47:45 +00:00
calvin
6857196b25 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2426 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 23:15:13 +00:00
calvin
964f313a89 fix return codes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2424 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 22:39:34 +00:00
calvin
2b8ca62e2e split up the big check function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2423 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 22:37:09 +00:00
calvin
9c609c0852 fix debug log keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2422 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 22:12:18 +00:00
calvin
ac450295ac put country info in local check method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2412 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-11 20:30:41 +00:00
calvin
8d206bac94 nice value +10 for low prio
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 23:43:10 +00:00
calvin
0c19d24b5d no email in the user-agent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2398 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:35:20 +00:00
calvin
882f204319 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2396 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:14:39 +00:00
calvin
131bea1a59 avoid deadlock with cache lock
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2395 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:09:41 +00:00
calvin
db299a1ead support printing tracebacks
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2394 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:03:40 +00:00
calvin
e89ef2c57a use own locking class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2393 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:03:09 +00:00
calvin
4c248fce03 avoid deadlock with cache lock
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2392 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:02:52 +00:00
calvin
4d0d76446f own locking class, avoiding multiple acquire/releases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2391 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-10 22:02:24 +00:00
calvin
7f4b062f7c fix lognames for -D option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2389 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 21:57:51 +00:00
calvin
dc960185df only set connection debug level if CHECK logging is on debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2388 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 21:55:20 +00:00
calvin
7563f0edf9 fix lock class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2387 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 21:54:56 +00:00
calvin
3373644560 make the consumer a lock object itself
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2386 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 20:28:16 +00:00
calvin
8e289cc522 dont disable threading on thread debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2385 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 20:11:44 +00:00
calvin
932c9864cb new thread log name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2384 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 20:09:21 +00:00
calvin
7f6d1b1022 more headers stuff
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2382 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 14:59:03 +00:00
calvin
b3c3c3eaaa split off header methods in separate file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2381 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 13:46:58 +00:00
calvin
ca49e04730 content encoding getter method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2380 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 13:39:20 +00:00
calvin
12b726b4df strip content type mime
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2379 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 13:33:20 +00:00
calvin
f344b75c8e documentation and syntax fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2378 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 10:26:30 +00:00
calvin
0d063a3b53 skip accept encoding, and list complete recursion list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2377 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-09 08:44:45 +00:00
calvin
4972a5b2f4 add warning on decompress errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2375 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 21:04:02 +00:00
calvin
a78ef198df disable logging on tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2373 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 20:59:49 +00:00
calvin
1e35c23986 eliminate tag attribute duplicates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2372 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 20:51:38 +00:00
calvin
3913b28b3c allow quoted CSS style url, and strip matching quotes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2371 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 20:49:00 +00:00
calvin
7d11778b5a add link parser tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2370 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 19:30:11 +00:00
calvin
0f991d47b9 add tests for matching attribute for unquoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2369 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 19:29:16 +00:00
calvin
d852672732 add matching attribute for unquoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2368 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 19:28:48 +00:00
calvin
26108790c6 supply real consumer for build url test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2366 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-08 12:26:51 +00:00
calvin
f808a03d3f no info for empty proxy
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2365 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 17:28:22 +00:00
calvin
067158d368 disable geoip for testing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2364 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 17:27:03 +00:00
calvin
5c52c35367 test noproxyfor
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2363 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 17:26:35 +00:00
calvin
7c5bef281a support setting of config args
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2362 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 17:13:23 +00:00
calvin
e1e713036b use noproxyfor config values to ignore proxy settings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2360 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 13:57:01 +00:00
calvin
52d364ae54 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2359 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 10:38:09 +00:00
calvin
9e9e42e21e added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2358 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 10:23:48 +00:00
calvin
2e996aecdb catch re.compile() errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2357 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 10:22:26 +00:00
calvin
c8920096e0 make threadsafe geoip call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2354 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 00:09:22 +00:00
calvin
aa3facf19f open geoip
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2353 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 00:08:53 +00:00
calvin
0284f58359 log country name as info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2352 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-07 00:08:48 +00:00
calvin
f3f64361cb add thread prio setter
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2348 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-04 23:18:10 +00:00
calvin
ac8490ce54 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2345 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-03-04 20:28:28 +00:00
calvin
bf89b8e9a9 str() for unicode_safe objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-18 11:29:58 +00:00
calvin
daf6f351c4 documenation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2323 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-18 11:25:05 +00:00
calvin
703da9eb65 safe decoding of non-unicode strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2322 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-18 11:22:52 +00:00
calvin
05f5123d7c make sure infos and warnings are unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2320 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-18 09:20:45 +00:00
calvin
0e12b923f6 make sure results are unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2319 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-18 09:17:46 +00:00
calvin
e71e6fb281 *** empty log message ***
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2318 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-17 22:33:00 +00:00
calvin
569fa853a8 *** empty log message ***
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2317 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-17 22:31:56 +00:00
calvin
87b6b5a6f6 *** empty log message ***
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2316 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-17 22:28:40 +00:00
calvin
d834aa0f2d documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2310 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-17 13:34:10 +00:00
calvin
b20cd18280 exception syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2308 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-17 00:02:22 +00:00
calvin
9560ad55ec sort HTTP accept languages for preference
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2307 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-16 23:57:22 +00:00
calvin
e4ca505e05 do not error out on missing locale directory
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2306 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-16 23:50:44 +00:00
calvin
44075c47bf clean up raise calls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 14:52:50 +00:00
calvin
77cea28d42 pylint cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 13:43:33 +00:00
calvin
4f3dc7a7fd syntax error
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2290 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 12:48:55 +00:00
calvin
94c769aba0 more config write stuff
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2289 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 12:33:01 +00:00
calvin
2fced4f37d documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2288 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 12:10:39 +00:00
calvin
d1fd01f669 remove expired connections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2287 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 10:41:27 +00:00
calvin
55652cabc0 fix nt path name for network paths
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2284 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-07 15:41:53 +00:00
calvin
973b6d5098 work around for 302 redirect handling error
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2283 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-07 12:10:17 +00:00
calvin
4db7e58f84 added no quote test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2281 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-07 00:07:50 +00:00
calvin
fcf18b8e1f add ! to no quote chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2280 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-07 00:07:02 +00:00
calvin
334c92708a add unicode messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2279 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-06 21:25:40 +00:00
calvin
344bfd1103 no return from add_connection
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2278 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-06 21:20:29 +00:00
calvin
bb3b972d8f always add connections, and close timed out ones
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2277 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-06 21:20:10 +00:00
calvin
3be005186a do not print XXX message, check for connected .sock attribute and always add to cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2276 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-06 21:15:28 +00:00
calvin
03a7f5f065 translate log level names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2271 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-01 02:05:34 +00:00
calvin
1d3dd0c35b documentation added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2258 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-31 20:18:17 +00:00
calvin
4f21309b9a added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2246 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-30 22:41:59 +00:00
calvin
01ceb0aafd HTTP connection pooling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2245 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-30 22:39:09 +00:00
calvin
9dea804d92 per-connection timeout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2244 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-30 11:00:37 +00:00
calvin
b698701726 only release cached FTP connections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2243 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-29 23:29:45 +00:00
calvin
e76836a607 add time import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2242 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-29 23:19:56 +00:00
calvin
372798487b start logger output before append_url is called since this can trigger logger.new_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2239 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-29 23:12:55 +00:00
calvin
879343f87b write stubs, rename norm to normpath
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2236 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 16:14:31 +00:00
calvin
afeae9fbe4 default blacklist output file is in .linkchecker user dir
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2233 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 15:34:52 +00:00
calvin
d04523b4fc move config file into a directory
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2232 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 15:08:07 +00:00
calvin
0aabca8444 do not just ignore but replace wrong encoded chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2231 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 15:06:41 +00:00
calvin
2c40c0be3c FTP connection timeout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2229 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 12:16:35 +00:00
calvin
74b8c8745e edge label quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2227 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 11:51:39 +00:00
calvin
b10c3775f4 default dot output encoding is ascii
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2226 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 11:51:12 +00:00
calvin
a749829ae5 format fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2225 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 11:33:40 +00:00
calvin
3f4f525358 fix edge writing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2224 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 01:19:17 +00:00
calvin
639aeb294a return 'busy' on busy connections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2223 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 01:03:16 +00:00
calvin
7e0c34a3d9 better FTP connection pooling, remove unnecessary LIST
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-28 01:02:15 +00:00
calvin
914f2440b7 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-27 21:14:17 +00:00
calvin
49cfa19c5d ignore all close errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2219 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-27 21:11:48 +00:00
calvin
3e79dd7e09 added and use ftpparse module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2218 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-27 19:03:33 +00:00
calvin
719143c8a1 added DOT graph output logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2216 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-27 17:11:03 +00:00
calvin
74805745e8 fix needs_quoting method by escaping chars and add [] to allowed chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2214 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-27 13:59:23 +00:00
calvin
5ae97df822 use python 2.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2209 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 20:29:55 +00:00
calvin
da8a12f408 use static dict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2208 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 20:26:43 +00:00
calvin
e226131e59 use new nntp argument usenetrc
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2207 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 19:31:13 +00:00
calvin
93e3da02b2 socket.sslerror is now a subclass of socket.error
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2206 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 19:27:11 +00:00
calvin
5fb817be93 require python 2.4
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2205 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 19:27:00 +00:00
calvin
05c9b8b5e6 use linkchecker agent on getting /robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2194 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 09:45:22 +00:00
calvin
8e2b4edd49 split status message prints
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2187 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 22:52:58 +00:00
calvin
07ec639486 read and use quotechar param
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2185 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 22:21:43 +00:00
calvin
fbb2dfbaeb change default csv separator to a comma, and add customisable quotechar param
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2184 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 22:21:18 +00:00
calvin
10e8fd9fe3 make interrupt warning a plural form
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2181 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 21:44:10 +00:00
calvin
86e90da338 fix reversed sslerror version check logic
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2175 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 12:50:38 +00:00
calvin
55cbb2f0a9 introduce MAX_NAMELEN
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2172 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 12:07:04 +00:00
calvin
f40e41a5bb cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2171 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 09:35:46 +00:00
calvin
cf30d1ae05 optimize link name search
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2170 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-20 09:32:24 +00:00
calvin
adc1d02217 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2165 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 21:11:43 +00:00
calvin
d030a5b054 documentation updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2164 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 15:56:48 +00:00
calvin
647d7167ee documentation syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2163 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 15:08:02 +00:00
calvin
255b698f44 updated to latest upstream CVS version of 20050119
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2162 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 14:52:27 +00:00
calvin
a91183afa5 documentation updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2161 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 14:38:01 +00:00
calvin
b008747f39 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2160 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 01:04:38 +00:00
calvin
46de8b6ca0 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2159 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 00:40:06 +00:00
calvin
ef8613f2c7 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2157 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 00:08:41 +00:00
calvin
8ac75ce155 only add socket.sslerror to exception list if needed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2156 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 17:04:31 +00:00
calvin
194b1f4466 move version check into library
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2155 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 17:03:17 +00:00
calvin
b51dffce7d documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2154 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 16:59:23 +00:00
calvin
cec0a01f4b only call set_encoding for meta tags, and syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2153 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 15:54:11 +00:00
calvin
83020982ba updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2152 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 15:53:34 +00:00
calvin
edfea898b4 documentation updated, and set_encoding no longer has tag attr
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2151 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 15:53:23 +00:00
calvin
4d7fa042ad use Py_CLEAR macor
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2150 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 15:52:04 +00:00
calvin
e58c070df7 supply Py_CLEAR macro
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2149 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 14:45:51 +00:00
calvin
700d564be7 documentation updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2148 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 01:00:45 +00:00
calvin
2e9525422b write column headers in first row; SF patch #1102338
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2143 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-16 23:30:16 +00:00
calvin
acbad02e8d updated mailto tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2137 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 14:34:23 +00:00
calvin
be4fb033b8 print VRFY info or warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2136 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 14:34:11 +00:00
calvin
15364fc362 fix username param
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2133 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 13:32:47 +00:00
calvin
b3e3448f96 sort MX hosts according to preference
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2132 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 13:29:34 +00:00
calvin
b06f144ced updated copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2122 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 02:22:43 +00:00
calvin
1f7563bd4e use extern optcomplete
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2118 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-10 20:37:23 +00:00
calvin
596959c121 use translator arg, not self
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2116 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-10 20:35:09 +00:00
calvin
c1a25e2e8b added tests for host matching and param splitting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2115 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-10 20:35:01 +00:00
calvin
b1d1af52a7 fix param splitting, query parsing, and host matching
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2114 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-10 20:34:26 +00:00
calvin
2fa156988e xhtml tests added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2113 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-08 16:27:03 +00:00
calvin
793862d9af *** empty log message ***
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2109 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-08 09:10:30 +00:00
calvin
4fbdbe3a51 XHTML support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2108 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-08 09:09:06 +00:00
calvin
9e843d71a7 set smtp debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2098 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-07 11:06:34 +00:00
calvin
9829268533 ignore empty filenames in filelist
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2097 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-07 08:16:20 +00:00
calvin
0778ef9d6d use internal file get method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2094 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-07 00:01:28 +00:00
calvin
1e001992ac disable recursion debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2091 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-06 22:38:43 +00:00
calvin
6d99454ebf unicode split
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2090 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-06 22:34:38 +00:00
calvin
ef0d624c0d unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2089 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-06 22:34:20 +00:00
calvin
060d6f0b78 fix url mailto splitting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2087 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-06 20:12:31 +00:00
calvin
f33586a533 comment method docs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2085 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-12-06 17:54:31 +00:00
calvin
29f3efd91a use wrap method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2083 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-30 14:10:06 +00:00
calvin
76751c04af textwrap utility method for logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2082 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-30 14:09:42 +00:00
calvin
cef58aba6e line ending for comments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2073 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-29 00:49:44 +00:00
calvin
6ffd2c60f0 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2063 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-25 16:23:13 +00:00
calvin
0c945cff8d comment and date check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2062 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-25 16:20:58 +00:00
calvin
045e517a52 check date
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2061 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-25 14:51:38 +00:00
calvin
503e482583 new fallbackklass param for translator
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2039 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-23 23:31:13 +00:00
calvin
92350d482e updated with bison 1.875d
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2038 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-23 22:51:48 +00:00
calvin
c5c3310709 use memory defines of new bison 1.875d
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2037 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-23 22:51:21 +00:00
calvin
d51418aabc rebuilt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2036 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-22 22:51:59 +00:00
calvin
dfe43e5c3f use dict.update()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2035 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-22 22:51:28 +00:00
calvin
141a3d1c94 added redirect test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2034 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-22 22:34:26 +00:00
calvin
168197a072 make http request handler customizable
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2033 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-22 22:34:03 +00:00
calvin
9151b9fe72 doc updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2024 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 20:59:32 +00:00
calvin
c35431069c added url fix method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2023 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 20:56:12 +00:00
calvin
d1a468c3b8 adjust info and warning strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2015 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 13:31:30 +00:00
calvin
3b13dd87e2 display correct url in redirect message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2014 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 13:08:26 +00:00
calvin
0ef53ab56a fix typos
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2012 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 11:52:55 +00:00
calvin
be5a59e9fd add dots to warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2010 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 11:18:06 +00:00
calvin
bc89449080 add dots to info messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2008 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-19 11:06:31 +00:00
calvin
d2a6f4ed95 recheck robots.txt allowance after redirect
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2002 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-18 00:57:35 +00:00
calvin
89616a4bba basic robots_txt check method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2001 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-18 00:55:47 +00:00
calvin
4014012c1f basic is_extern method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1999 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-18 00:49:45 +00:00
calvin
74f1935be5 recheck extern/intern filters for redirected URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1998 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-18 00:49:33 +00:00
calvin
fae80e541c use new file cache keys in test results
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1988 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-15 19:17:35 +00:00
calvin
2765c5b116 use anchor and user info in cache key
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1987 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-15 19:16:02 +00:00
calvin
cafa412343 use base cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1986 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-15 19:15:22 +00:00
calvin
4485ab19aa only norm a changed file base_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1985 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-15 16:36:56 +00:00
calvin
b0d3c575ce test leading query exclam after host without path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1982 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 11:12:34 +00:00
calvin
b150ff6ea8 handle leading query exclam after host without path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1981 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 11:12:16 +00:00
calvin
265f942ba1 add unquoted redirectino url to info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1980 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 11:02:17 +00:00
calvin
1166804eb0 use idna domain for intern patterns
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1979 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 10:52:20 +00:00
calvin
72290074ac syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1978 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 10:47:43 +00:00
calvin
ce998f63da added one idna test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1977 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 02:01:23 +00:00
calvin
6c62daba9c added one idna test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1976 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 01:58:26 +00:00
calvin
3576caf6d5 test fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1975 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 01:58:05 +00:00
calvin
2f3e7d18e0 test fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1974 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 01:55:47 +00:00
calvin
f3226afe0f do not encode empty host
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1973 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-14 01:41:29 +00:00
calvin
9c40e54e1e encode index.html data
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1971 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-13 23:40:17 +00:00
calvin
f0048d0345 add unicode domain name warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1970 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-13 23:30:14 +00:00
calvin
75ec50b156 adjust for new url_norm method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1965 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-13 11:20:33 +00:00
calvin
e429264733 support unicode domain names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1964 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-13 11:20:16 +00:00
calvin
820d07ba6f os-specific path name quote chars, and IDN support for hostnames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1963 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-12 13:34:17 +00:00
calvin
7c83266566 serialize helper function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1962 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-12 13:25:40 +00:00
calvin
8ba7bccf5a separate encode function for loggers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1961 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-11 08:56:56 +00:00
calvin
1997381c2b return unicode url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1960 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-10 16:41:41 +00:00
calvin
6ddf286c36 platform specific test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1959 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-10 16:41:13 +00:00
calvin
3d81638b8f use intern __repr__(), not repr()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-10 14:31:34 +00:00
calvin
25770a1109 encode failure messages
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1956 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-10 13:13:05 +00:00
calvin
34c2168807 use unicode url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1955 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-10 13:12:41 +00:00
calvin
e61ea956cc add url needs quoting for pipe char test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1951 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:56:17 +00:00
calvin
7ee019da0f add pipe to safe url chars
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1950 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:55:03 +00:00
calvin
088cf521a8 return unicode strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1949 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:54:20 +00:00
calvin
bb0921b1ba added no path quote test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1948 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:24:03 +00:00
calvin
e56fe74904 add | to nopathquote characters for windows file: links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1947 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:21:45 +00:00
calvin
2be584891b norm changed base url again
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1945 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 17:02:42 +00:00
calvin
85497ca439 typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1944 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 16:58:51 +00:00
calvin
d671254279 make --quiet an alias for -o none
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1943 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 16:04:05 +00:00
calvin
19ce868d8f colored output disabled for windows platforms
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1940 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 15:46:14 +00:00
calvin
60750494b8 implement windows ANSI color support check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1937 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 14:02:18 +00:00
calvin
a87b8a64d2 do not catch exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1936 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 12:39:53 +00:00
calvin
75c36e0e02 do not catch exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1935 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 12:38:02 +00:00
calvin
f70ef0198d wrap should handle line endings os-specific
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1934 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 11:10:10 +00:00
calvin
2b60a133ba retain empty text arg in wrap() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1933 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 10:28:33 +00:00
calvin
6a5fdb57c1 fix dns test on windows systems
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1932 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 10:19:01 +00:00
calvin
d238920739 added network resource
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1931 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 09:46:52 +00:00
calvin
9f95017d5b case sensitivity checking of NT filenames
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1927 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 08:11:09 +00:00
calvin
f01b2e7c4e fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1925 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 00:29:06 +00:00
calvin
c97f68f70a accept unicode in robots.txt can_fetch
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1924 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 00:00:59 +00:00
calvin
30024a4683 optimize img_re regular expression
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1919 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-06 12:43:48 +00:00
calvin
6f488fd189 remember base href in link parser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1918 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-06 12:42:33 +00:00
calvin
607425065e add cache key asserts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1917 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-05 00:24:36 +00:00
calvin
2ba4a50965 ensure unicode for redirects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1916 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-05 00:24:25 +00:00
calvin
437b56c9ad assert msgs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1915 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-05 00:24:08 +00:00
calvin
01f1c06149 no repr of url name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1913 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 22:35:29 +00:00
calvin
4cae7dc7f6 test fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1912 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 22:34:20 +00:00
calvin
9b28e3b346 unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 21:29:25 +00:00
calvin
69041b2d9a do not catch log system exceptions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1910 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 17:51:54 +00:00
calvin
00b1a426d1 convert url names to unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1909 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 13:50:54 +00:00
calvin
34081d925b unicode repr
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1908 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 13:49:59 +00:00
calvin
84498b914a unicode diffs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1907 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 13:49:23 +00:00
calvin
4894467ca9 cut off leading 'u' of repr names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1906 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-03 13:49:09 +00:00
calvin
ad78277935 accept unicode domain names according to RFC3490 (IDNA)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1903 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-02 13:16:23 +00:00
calvin
78a7af2dbf accept unicode filenames in resolver config
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1902 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-02 13:05:39 +00:00
calvin
96722d09f1 coerce urls to unicode
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1901 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-02 13:05:11 +00:00
calvin
4485cb4800 also overload NullTranslations gettext class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1900 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-28 16:03:42 +00:00
calvin
de68f20f89 unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1899 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 22:34:50 +00:00
calvin
abfd6c475d unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1898 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 21:03:02 +00:00
calvin
54b8126ccb cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1896 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:49:59 +00:00
calvin
b7f2c0eeab unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1894 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:48:50 +00:00
calvin
9ad203eef5 simplify, simplify
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1893 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:42:56 +00:00
calvin
f3a2f7e1a0 better deferred translation of field names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1892 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:31:09 +00:00
calvin
29a6fc783d unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1891 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:29:25 +00:00
calvin
b99c8a15e8 better deferred translation of field names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:27:27 +00:00
calvin
dee1c867ff encode msgs to stderr
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1889 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 19:19:29 +00:00
calvin
20bed25dd0 unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1888 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 18:43:17 +00:00
calvin
cf00b16224 honor fileoutput flag properly
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 18:39:21 +00:00
calvin
a80feec1bb install both _() and _n() into default namespace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1886 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 18:38:19 +00:00
calvin
acb3178f8f unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1885 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 18:37:56 +00:00
calvin
e6412dd549 ignore output encoding errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 18:13:20 +00:00
calvin
8cf68dfad4 added write() method to logger objects that use unicode and output_encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1882 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 13:58:37 +00:00
calvin
7cd3bad795 unicode for string constants
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1881 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 13:57:52 +00:00
calvin
97b8972bd8 fix fileoutput flag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1877 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 12:22:10 +00:00
calvin
c7603d6797 configuration and storage of output encoding for loggers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1873 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-27 09:25:05 +00:00
calvin
2e32594467 close all opened log files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1871 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-26 21:43:55 +00:00
calvin
a57b7c0924 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1867 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-26 11:07:15 +00:00
calvin
290281974b search list is a set, SIOCGIFCONF only tested under linux
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1866 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-26 11:07:05 +00:00
calvin
ba438508f4 fix ftp test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1865 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 22:17:59 +00:00
calvin
18d42179d9 fix unquote test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1864 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 21:54:02 +00:00
calvin
9f09600ff1 do not coerce to empty string in unquote()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1863 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 21:47:03 +00:00
calvin
5f0660d1f4 unicode fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1861 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 21:24:11 +00:00
calvin
64f3df05f4 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1859 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 21:19:25 +00:00
calvin
65188d735c check for SSL support when generating https connection objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1858 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-25 21:18:22 +00:00
calvin
94c605d476 decode input strings, and return unicode strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1854 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-07 15:47:09 +00:00
calvin
10209ae499 emit unicode data, store encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-06 19:05:48 +00:00
calvin
f2e2288f4b store interfaces list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1852 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-06 19:05:12 +00:00
calvin
c9162a0d54 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1851 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-10-06 19:04:25 +00:00
calvin
8a539c782a cgi param fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1850 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-25 12:18:58 +00:00
calvin
81c0fa70c1 improved documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-23 18:18:18 +00:00
calvin
5950df2473 improved the documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1845 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-22 15:53:10 +00:00
calvin
6752070eba unused import cgi removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1844 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-21 06:34:55 +00:00
calvin
b0f366dffd added missing gettext import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1843 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-21 06:31:22 +00:00
calvin
97d6f02135 better missing trailing dir slash check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1842 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 19:40:57 +00:00
calvin
5405c2844b use FTP connection cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1841 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 19:22:32 +00:00
calvin
304bdda3bf added ftp connectin cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1840 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 19:21:35 +00:00
calvin
5201de5cb6 recurse into ftp links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1839 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 18:30:44 +00:00
calvin
ee71260815 use new get_index_html utility function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1838 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 17:49:26 +00:00
calvin
6bb7a08eb5 added get_index_html utility function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1837 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 17:48:52 +00:00
calvin
0ab639242b search given filename in file list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1833 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 14:43:10 +00:00
calvin
d5972aaa8d use plural forms
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1831 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-20 12:11:32 +00:00
calvin
ff23bbf53d simplify incoming_get_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1812 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 21:40:03 +00:00
calvin
2a5d2eac61 add question mark to url_needs_quoting charset
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1810 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 21:16:42 +00:00
calvin
0dcd6a8425 assertion message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1809 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 21:14:59 +00:00
calvin
d174276fd8 also display info in repr()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1807 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 20:33:30 +00:00
calvin
f31407d2b4 remove cached urls from progress queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1806 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 20:33:14 +00:00
calvin
62b2784ebc python 2.4 compat
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1805 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 20:11:38 +00:00
calvin
d2f7955f2c cookie warning msg fixed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1804 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 20:06:06 +00:00
calvin
fda2392fc6 do not catch cookie error on check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1802 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 20:01:35 +00:00
calvin
3f9e2646fe documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1801 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 20:01:22 +00:00
calvin
5c4662499c catch cookie errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1800 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 20:00:51 +00:00
calvin
00d2b6d96e added CookieError to error list
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1799 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 19:54:46 +00:00
calvin
1a6f476ec7 allow / in cgi param values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1797 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-14 17:06:12 +00:00
calvin
b1901a3e32 init cleanup and translator method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1794 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 19:56:43 +00:00
calvin
9b83851177 also collapse redundant path segements of relative paths
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 12:40:50 +00:00
calvin
081f21f83f js allows path fragments
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1786 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 08:55:51 +00:00
calvin
4dc05df09c updated test cases for now cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 08:43:16 +00:00
calvin
ff000a8c5c documentation for cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 08:37:14 +00:00
calvin
f66c780a0c prevent collapsing of relative parent dir paths
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 07:05:58 +00:00
calvin
9ba12eb7f2 init i18n
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1779 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-10 07:02:56 +00:00
calvin
14890f7ee7 syntax cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1777 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-09 16:57:55 +00:00
calvin
76e7dc1697 import os
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1772 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-08 19:30:47 +00:00
calvin
146fa6b1bb only import ifconfig on POSIX systems
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1771 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-08 19:30:17 +00:00
calvin
7da6b5cd6c init i18n in tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1766 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-08 18:08:48 +00:00
calvin
d287959539 reinit logger after config file reading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1762 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-08 13:23:09 +00:00
calvin
4efe2eb8b4 set all text output colors to default
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1759 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-08 12:44:06 +00:00
calvin
d3cd17aec1 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1757 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-06 17:07:06 +00:00
calvin
ccd9784283 fix i18n init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1755 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-06 17:03:12 +00:00
calvin
52f1e901af pylint cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1752 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-06 15:38:30 +00:00
calvin
be4d10e450 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1751 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-06 15:19:59 +00:00
calvin
fb260882cb ensure that Python.h gets included before any system headers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-06 15:19:23 +00:00
calvin
e399999acb i18n init
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1740 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-05 20:57:50 +00:00
calvin
228221fb85 install gettext method in default namespace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1736 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-05 20:48:52 +00:00
calvin
17ce3f00b9 fix typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1732 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-04 23:23:20 +00:00
calvin
9fbf2253d0 handle case where url_data.url is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1730 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-04 08:42:44 +00:00
calvin
bf4309220e remove anchors from cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1722 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 19:55:23 +00:00
calvin
26b3f1bcf3 strip quotes from css urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1721 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 19:55:05 +00:00
calvin
3385db2018 pylint cleanups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 18:26:12 +00:00
calvin
79b0ef54f5 support url list parsing in text files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 14:54:59 +00:00
calvin
594797b5e2 content regex fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1708 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 14:43:11 +00:00
calvin
b1708fc725 get the complete content
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1707 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 14:29:07 +00:00
calvin
33f218d9a2 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1706 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 13:39:09 +00:00
calvin
a29240d868 do not error out on missing user/password of ftp link
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1705 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 13:38:46 +00:00
calvin
c79581fe31 adjust cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 13:30:24 +00:00
calvin
bacefcf242 separate cache keys for urls and recurions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1702 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 12:05:40 +00:00
calvin
1b4c190778 file output must honor verbose/quiet flag
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1693 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 08:26:10 +00:00
calvin
751d60f924 fix variable typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1692 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-03 06:39:37 +00:00
calvin
40fad49126 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1674 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-01 10:25:32 +00:00
calvin
98144c5fb0 remove ccache incoming debug
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 22:23:09 +00:00
calvin
ce9dc6fbe9 increment robotparser version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1655 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:45:34 +00:00
calvin
bc6bd34ffc fix password manager interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1654 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:42:34 +00:00
calvin
b257534a60 user/password authentication for robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1653 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:39:57 +00:00
calvin
68e1f03094 reset aliases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1650 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:31:07 +00:00
calvin
bffdfa68fd robots.txt password support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1649 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:20:51 +00:00
calvin
6c97b96aa8 check cache for redirected urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1648 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 20:43:58 +00:00
calvin
fc1f5fc5bb fix cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1646 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 20:35:09 +00:00
calvin
5944aa0c88 adjusted cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1644 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 20:29:32 +00:00
calvin
66d78a3fa0 get cache_key attribute directory
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1643 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:48:02 +00:00
calvin
a1f9f2af2b remove some asserts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1642 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:28:12 +00:00
calvin
1979a410a9 write error/valid before result
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1641 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:27:13 +00:00
calvin
3f9c9cdd84 do not append error/valid to result
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:23:52 +00:00
calvin
e2875acea3 new cache logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1638 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:15:33 +00:00
calvin
fc65d8efb9 new in_progress cache, fix cache_key setting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1637 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:15:22 +00:00
calvin
a0970a62a0 cope with empty url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 19:14:46 +00:00
calvin
60dc606510 forgot some cache adds
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1634 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-30 15:54:31 +00:00
calvin
07f25c6429 more code documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1629 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-29 15:48:17 +00:00
calvin
bbba28f781 downgrade the 'outside of domain filter' warning to an info message since after all the user is in full control over extern/intern stuff
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1626 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-29 12:45:03 +00:00
calvin
7a5bc30dda intify booleans
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 20:47:00 +00:00
calvin
62773dea21 recursionLevel -> recursion_level
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 20:41:39 +00:00
calvin
03086d7793 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 16:54:44 +00:00
calvin
fc1356d97e escape and repr()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1600 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:55:58 +00:00
calvin
9e76c8e70c key error url->uri
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1599 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:50:31 +00:00
calvin
a7a527b912 missing td
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1598 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:49:49 +00:00
calvin
137a5b96c5 rename html logger config args to colors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1596 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:39:12 +00:00
calvin
44f0e7c3b5 valign top and non-breaking fields
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:38:54 +00:00
calvin
08670b67d6 move self.errors var into base class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1594 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:24:15 +00:00
calvin
41beea6ee3 fixed cgi check run
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1593 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 12:03:53 +00:00
calvin
d18de2a6fb dont quote AT sign in paths
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1587 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 10:46:03 +00:00
calvin
c266c7e5fd added robots.txt test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1583 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 09:27:45 +00:00
calvin
7fb8e33ab3 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1582 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 09:27:29 +00:00
calvin
6567ffbfbc debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1581 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 09:27:22 +00:00
calvin
8194034e93 text magic
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1580 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 09:24:13 +00:00
calvin
61a3cab286 library status default is False
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1568 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 07:59:40 +00:00
calvin
39a6c0aeb0 status prefix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1565 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 06:40:30 +00:00
calvin
b8f0ca1c92 strict rename to externstrictall
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-28 06:37:43 +00:00
calvin
74df7dd963 updated documentation, adjust login method for missing user or password
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1562 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 21:08:49 +00:00
calvin
388d25c949 dont connect with base class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1561 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 21:08:20 +00:00
calvin
e4ee0f9cb8 rename --strict to --extern-strict-all
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1558 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 19:36:13 +00:00
calvin
c532adfd1c use new abort() method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 19:22:55 +00:00
calvin
1100eabb62 rename finish to abort, and correct the abort state
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1553 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 19:22:17 +00:00
calvin
02a1c329e8 only send http: referers, and catch bad HEAD replies on redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1552 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 18:54:31 +00:00
calvin
3ba4314786 fix cookie cache method calls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1551 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 18:25:28 +00:00
calvin
80ce276942 use new absolute_url function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 18:21:26 +00:00
calvin
047bd4f28d fix absolute_url testing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1549 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 18:18:21 +00:00
calvin
bcffed778e pathnoquote chars adjusted
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1548 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 17:57:34 +00:00
calvin
507a3b3069 do not repr() the base_ref
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1545 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 15:34:05 +00:00
calvin
8e8eba71dc print redirection info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 15:32:40 +00:00
calvin
25f8f13a06 add query cache key test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1543 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 14:57:03 +00:00
calvin
587724ffdc ignore query of requests to test http server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1542 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 14:52:41 +00:00
calvin
1539b2c639 suggest -Dall debug run in internal error message
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1540 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 13:56:21 +00:00
calvin
b13d2e2711 adjusted for new url repr()
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1539 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 13:55:57 +00:00
calvin
548d941928 do not fix path of relative urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1538 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 13:50:41 +00:00
calvin
bd857d389f adjust to new repr() url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1537 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:58:55 +00:00
calvin
7a769024d8 adjust to new repr() url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1536 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:52:08 +00:00
calvin
fc7b2b4351 self.url should be normed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1535 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:47:29 +00:00
calvin
16e2048209 adjust to new base url quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:47:03 +00:00
calvin
e1862f6321 print repr of logged url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1533 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:46:43 +00:00
calvin
09adff7cf6 javascript urls are less quoted for compatibility
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1532 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:03:49 +00:00
calvin
3d678d6d3b adjust test cases for new unquoted base_url and its new warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1531 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:44:36 +00:00
calvin
31233bb778 leave self.base_url unquoted
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1530 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:44:03 +00:00
calvin
50f7ea04df change debug color to white, use os.linesep
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:43:48 +00:00
calvin
ea2125c3f5 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:42:36 +00:00
calvin
e7b620052e documentation, and rstrip in url quote check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:40:27 +00:00
calvin
02767fed76 put flush method in base class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1523 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:58:01 +00:00
calvin
c34abe18a2 set line/column always to int
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1520 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:30:12 +00:00
calvin
4a947d1e5a show css/html validation on errors, and split into functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1519 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:29:21 +00:00
calvin
6f41bce255 use colorize in text output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1517 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 11:52:06 +00:00
calvin
d9c9e3e8f8 write colors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1514 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 21:50:41 +00:00
calvin
0d6c2c52a9 imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1512 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 21:32:31 +00:00
calvin
108015d8be remove colored logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1508 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:10:01 +00:00
calvin
66ec6ba897 default logger is text
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1506 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:09:04 +00:00
calvin
324614ab18 default logger is text again, remove colored logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1505 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:08:53 +00:00
calvin
b4039684c3 got rid of urlconnect
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1501 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:53:08 +00:00
calvin
130b89f866 use urlbase and new is_safe url functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1500 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:51:38 +00:00
calvin
c904caa5c5 move self.is_cache(), and use urlunsplit to set self.url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1499 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:50:40 +00:00
calvin
d0b54018ee use urlbase and its modified url building
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1498 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:49:37 +00:00
calvin
9d49c37d57 fix telnet tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1497 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:48:43 +00:00