Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-18 19:41:06 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
5618 commits 1 branch 0 tags 24 MiB
Commit graph

272 commits

Author SHA1 Message Date
Bastian Kleineidam
279a1eae70 Only add geoip info for non-empty hostnames. 2010-09-29 15:59:57 +02:00
Bastian Kleineidam
cc848cdb33 Fix import for moved geoip module. 2010-09-29 15:17:27 +02:00
Bastian Kleineidam
8a1ac26c85 Warn about obfuscated IP numbers. 2010-09-05 20:11:02 +02:00
Bastian Kleineidam
8a074aeea9 Work around Python 2.6+ urljoin bug. 2010-08-31 09:16:24 +02:00
Bastian Kleineidam
c3b8ff00b3 Check content and recursion in one try/except to avoid multiple errors when getting page content. 2010-08-31 06:52:08 +02:00
Bastian Kleineidam
1faedafb33 Fix data size for HTTP requests. 2010-08-04 00:06:25 +02:00
Bastian Kleineidam
0f92b76290 Remove the unnormed URL warning. 2010-07-29 20:20:59 +02:00
Bastian Kleineidam
7ad4f7c220 Compare size from meta info and content data. 2010-07-29 19:53:41 +02:00
Bastian Kleineidam
d9bfd25a68 Add warning if content size is zero 2010-07-28 08:19:55 +02:00
Bluebird75
28f4514b67 Use object with __slots__ for wire-format of UrlBase objects. 
Saves memory since UrlBase wire-format objects are used for
logging and thus often created.

Signed-off-by: Bastian Kleineidam <calvin@debian.org>
 2010-03-27 00:07:19 +01:00
Bastian Kleineidam
3370ea1562 Reflect changes in httplib2.py: use buffered read in httplib response object and use bad status line exception attribute. 2010-03-26 20:50:38 +01:00
Bastian Kleineidam
37b4e97012 Revert "Only parse anchors if both --anchors option is given and the current link has an anchor." 
This reverts commit b238527d54.
 2010-03-10 00:04:02 +01:00
Bastian Kleineidam
b238527d54 Only parse anchors if both --anchors option is given and the current link has an anchor. 2010-03-09 11:45:50 +01:00
Bastian Kleineidam
57397e938b Improved linkname parsing by adding a new peek() HTML parser function. 2010-03-09 11:31:12 +01:00
Bastian Kleineidam
51a0ef0ad4 Speed up HTML parsing by stopping early and adding callbacks. 2010-03-08 09:04:33 +01:00
Bastian Kleineidam
1e15e55689 Fix errors in Word file parsing. 2010-03-07 19:43:08 +01:00
Bastian Kleineidam
6a2fcf8ae9 Parse links in Word files. 2010-03-07 19:20:51 +01:00
Bastian Kleineidam
77daf80e82 Add url encoding parameter 2009-11-28 11:56:35 +01:00
Bastian Kleineidam
5e06b6b8d4 Updated FSF address in GPL blurb 2009-07-24 23:58:20 +02:00
Bastian Kleineidam
7f67027abf ignore the fragment part (ie. the anchor) of URIs when 
+  getting and caching content
 2009-06-26 07:22:36 +02:00
Bastian Kleineidam
59ffbd43f0 Use AttrDict for transport object in loggers. 2009-03-07 09:43:55 +01:00
Bastian Kleineidam
7a59763508 Remove unused SetList container 2009-03-07 00:42:27 +01:00
Bastian Kleineidam
2351506752 Use plain list for info strings. 2009-03-07 00:19:19 +01:00
Bastian Kleineidam
88dbcb30cd Remove unused url_data.info tags - the tags were always None 2009-03-06 21:20:09 +01:00
Bastian Kleineidam
ba160350dd Introduced transport object API for logging. 2009-03-06 19:30:58 +01:00
Bastian Kleineidam
b8944e493a Use new exception log keyword when logging errors 2009-03-02 13:18:36 +01:00
Bastian Kleineidam
7862147ca3 Fix showing content size. 2009-03-01 23:04:48 +01:00
Bastian Kleineidam
8caa601a7e Python 3.0 compatibility: use exc.args[] instead of exc[] 2009-02-24 12:41:45 +01:00
calvin
7214943f38 Remove wrong function return type documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3959 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-02-18 15:34:46 +00:00
calvin
e9805dbd8a Updated copyright year to 2009 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3887 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2009-01-08 14:18:03 +00:00
calvin
42c3e71329 Improved and tested opera bookmark parser 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3863 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-11-20 07:52:02 +00:00
calvin
d26386d03f Catch errors when getting content for title. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3814 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-12 15:38:26 +00:00
calvin
290528b84f Added title attribute to URL data. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3790 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:56 +00:00
calvin
99269d12cc Add base method for Url.get_title() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3788 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-06-07 13:07:38 +00:00
calvin
5f4d61e018 Use keyword arguments in translation strings. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:44:40 +00:00
calvin
66ff422f6b Allow overwriting of an old check result. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3776 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-27 19:42:38 +00:00
calvin
dbb498a395 Add virus checking 
New option --scan-virus to check the content of URLs for
viruses with ClamAV.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3753 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-20 08:57:37 +00:00
calvin
bacb59597e Use relative imports from Python 2.5 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-09 06:16:03 +00:00
calvin
b96e8120d6 Add W3C Validator checks 
Add new options --check-html-w3 and --check-css-w3 to allow checking
of HTML and CSS pages with the online W3C validators.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3748 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-05-08 10:36:54 +00:00
calvin
df9f31dcb1 Only check HTML/CSS syntax of intern URLs 
The HTML and CSS syntax check now only applies to URLs
which match those given on the command line.
This makes checking of personal pages easier.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-29 17:48:47 +00:00
calvin
ac4d09f83d Fix errors in CSS and HTML syntax check 
Properly encode the warning messages as Unicode, and prevent
overwriting of the "log" module with a local variable.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3742 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-29 17:48:22 +00:00
calvin
5d8bdaaa1f Use generators instead of lists where possible 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3739 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-28 00:26:02 +00:00
calvin
3eac1be9ab Require and use Python 2.5 
Use Python 2.5 features and get rid of old compat code. Also some
code cleanups have been made.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3737 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-27 11:39:21 +00:00
calvin
72db31e546 Only check syntax of valid URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3726 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-25 07:53:11 +00:00
calvin
973da91f44 Source code cleanup: use or remove unused variables 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3724 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-25 07:49:52 +00:00
calvin
62efec3b35 Added CSS syntax check. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-24 09:44:18 +00:00
calvin
cce6affa17 Add --check-html option to check the HTML syntax. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-23 23:04:31 +00:00
calvin
5a2f89fa3d Add redirect warning for commandline URLs 
If URLs given on the commandline are redirected, the automatic
intern patterns might not match anymore. A warning makes this
more prominent.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3712 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-21 09:18:36 +00:00
calvin
8ae6d94b45 Improved error messages for exceptions 
Prepend the exception name before the error message of exceptions.


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3694 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-19 07:47:00 +00:00
calvin
4968f1b3cd Prevent empty exception values. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3690 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-04-18 07:42:24 +00:00
calvin
6499cb1a63 updated copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3658 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2008-01-02 14:31:19 +00:00
calvin
c971ebdabf Added Shockwave Flash (SWF) parsing 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3656 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-28 02:12:48 +00:00
calvin
41bc0b2b32 use 'self.data is None' to test if data is already read or not 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3631 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-12-03 14:09:20 +00:00
calvin
8d2dc781e1 Ensure unused or expired connections are closed. 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-30 16:42:41 +00:00
calvin
9cf3314eab Use constants for warning tags, avoiding typos in string constants. And move the constants into a separate module const.py 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-29 07:50:22 +00:00
calvin
fcde8bd4d6 try to detect unknown URL schemes instead of manually setting the assume_local flag 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-28 18:46:50 +00:00
calvin
6a0960aa66 only store parser contents in LinkFinder handler, not in all handlers 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3602 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 19:06:39 +00:00
calvin
a1d911127b remove comments from CSS files before parsing for links 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 18:46:14 +00:00
calvin
cb588a3c5d replace tabs with spaces 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3598 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:55:55 +00:00
calvin
ce8b963dd9 more code cleanups and documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3596 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:46:18 +00:00
calvin
370749cafb cleanup the code and add some documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-11-14 17:34:50 +00:00
calvin
df48d4a905 bump up copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2007-01-01 14:57:38 +00:00
calvin
bef2494211 remove unused imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3482 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-24 10:13:59 +00:00
calvin
5ad59225a0 use dictionaries for translations with multiple arguments 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-16 09:12:59 +00:00
calvin
27a8869783 use helper alias for unicode_safe 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3441 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 19:34:03 +00:00
calvin
15dfaf35cb cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3438 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-09-15 16:36:44 +00:00
calvin
adc4e8c0e8 quote base reference URL, with tests 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3402 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 08:07:46 +00:00
calvin
c6f01faab5 improved debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3401 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-07-18 07:43:36 +00:00
calvin
d95d8c3d96 correctly handle internal errors 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3338 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-06-03 01:14:05 +00:00
calvin
e211d3fd6c fix internal error call 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3314 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-25 11:33:23 +00:00
calvin
7e1e01bd36 do not catch UnicodeError, handle that intern 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3269 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-19 17:13:16 +00:00
calvin
811f5492c4 fix --pause to delay requests to the same host 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-16 22:56:13 +00:00
calvin
ad28599e57 Note if URL is missing (instead of saying it is empty) 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-16 22:15:34 +00:00
calvin
9a431fde40 fix imports 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3170 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-14 10:14:07 +00:00
calvin
dc9f04e6dc adjust debug asserts 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3159 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-13 21:03:21 +00:00
calvin
98a6d1eb00 added missing import 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3148 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-13 18:08:06 +00:00
calvin
f002c5f965 Replace the old threading algorithm with a new one based on Queue.Queue and consumer threads 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3146 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-05-13 13:44:52 +00:00
calvin
276437c7d8 syntax cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3067 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-02-09 07:57:22 +00:00
calvin
e92aee054c updated copyright 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3010 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2006-01-03 19:12:47 +00:00
calvin
388475cbe2 new-style exceptions 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2999 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-12-20 20:17:33 +00:00
calvin
856ff8ef2a assert debugs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2987 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-12-18 08:55:42 +00:00
calvin
9425873830 cache aliases (from redirects) 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2964 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-12-08 20:20:21 +00:00
calvin
c86c0870d6 debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2946 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-12-06 22:32:31 +00:00
calvin
e2a454e31f use new trace module 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2939 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-11-14 22:49:37 +00:00
calvin
c049837cdd remove parser messages 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2927 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-11-04 20:05:59 +00:00
calvin
041df3a48d documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2906 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-10-13 22:54:23 +00:00
calvin
8ce45149fa re-add nofollow check 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2890 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-10-13 20:49:56 +00:00
calvin
6dd4ac6a65 typo fixes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2849 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-10-11 12:52:28 +00:00
calvin
7cf3f83c0f better recursion check debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2848 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-10-11 12:39:46 +00:00
calvin
978ba63999 catch UnicodeError on syntax check 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2847 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-10-11 12:10:32 +00:00
calvin
ed932cb165 debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-20 15:50:27 +00:00
calvin
359612b6fe add missing import 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2751 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-14 17:53:27 +00:00
calvin
225c045815 do not cache unusual errors 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-14 17:11:13 +00:00
calvin
5d82d64c31 more result sanity check warnings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2749 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-14 13:35:18 +00:00
calvin
f80480469f config to ignore certain warnings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-13 15:03:17 +00:00
calvin
901227d436 assign extern info to local var 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-04 22:19:52 +00:00
calvin
2e207be127 allow empty relative URLs 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-07-04 20:28:55 +00:00
calvin
0c3e78569c fix norming 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-06-23 16:21:05 +00:00
calvin
cc0f7cb113 always norm a URL before sending a request 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2653 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-06-12 21:16:47 +00:00
calvin
d2107c7a6a split and fix initialization of UrlData objects 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-19 19:42:14 +00:00
calvin
f6f04e376e fix extern calculation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2618 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-19 19:01:51 +00:00
calvin
55d7d5f7df use a decorator to synchronize all the checker threads 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2614 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-19 15:49:06 +00:00
calvin
3db056f161 documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-18 22:14:45 +00:00
calvin
c0d9e56a4d new trace option 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-17 22:06:47 +00:00
calvin
ed738d2436 cmdline param defaults to False 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-17 19:00:17 +00:00
calvin
1a7a771648 new intern/extern handling 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2584 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-09 22:05:21 +00:00
calvin
bf8fd348aa better msg 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-04 13:22:10 +00:00
calvin
a39f635b82 use some %r in warning strings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-05-04 10:48:10 +00:00
calvin
d2a8764d18 do not store aliases 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-04-27 12:48:48 +00:00
calvin
23e5212c7d do not remove userinfo from host part of url 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-04-26 17:53:31 +00:00
calvin
60069a8862 improved internal error msg 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2537 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-04-25 23:46:37 +00:00
calvin
5e6317a4bb remove old warning, and move effective url test to urlbase 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2532 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-04-25 18:25:32 +00:00
calvin
bbf96868b8 document aliases 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-03-18 14:21:12 +00:00
calvin
ac450295ac put country info in local check method 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2412 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-03-11 20:30:41 +00:00
calvin
f344b75c8e documentation and syntax fixes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2378 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-03-09 10:26:30 +00:00
calvin
0284f58359 log country name as info 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2352 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-03-07 00:08:48 +00:00
calvin
703da9eb65 safe decoding of non-unicode strings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2322 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-02-18 11:22:52 +00:00
calvin
0e12b923f6 make sure results are unicode 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2319 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-02-18 09:17:46 +00:00
calvin
44075c47bf clean up raise calls 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-02-08 14:52:50 +00:00
calvin
77cea28d42 pylint cleanups 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-02-08 13:43:33 +00:00
calvin
2fced4f37d documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2288 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-02-08 12:10:39 +00:00
calvin
01ceb0aafd HTTP connection pooling 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2245 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-30 22:39:09 +00:00
calvin
914f2440b7 syntax cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-27 21:14:17 +00:00
calvin
d030a5b054 documentation updated 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2164 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-19 15:56:48 +00:00
calvin
647d7167ee documentation syntax 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2163 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-19 15:08:02 +00:00
calvin
b008747f39 documentation 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2160 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-19 01:04:38 +00:00
calvin
b06f144ced updated copyright year 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2122 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2005-01-11 02:22:43 +00:00
calvin
1e001992ac disable recursion debugging 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2091 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-12-06 22:38:43 +00:00
calvin
ef0d624c0d unicode fixes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2089 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-12-06 22:34:20 +00:00
calvin
be5a59e9fd add dots to warnings 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2010 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-19 11:18:06 +00:00
calvin
bc89449080 add dots to info messages 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2008 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-19 11:06:31 +00:00
calvin
4014012c1f basic is_extern method 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1999 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-18 00:49:45 +00:00
calvin
2765c5b116 use anchor and user info in cache key 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1987 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-15 19:16:02 +00:00
calvin
72290074ac syntax cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1978 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-14 10:47:43 +00:00
calvin
f0048d0345 add unicode domain name warning 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1970 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-13 23:30:14 +00:00
calvin
75ec50b156 adjust for new url_norm method 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1965 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-13 11:20:33 +00:00
calvin
7c83266566 serialize helper function 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1962 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-12 13:25:40 +00:00
calvin
3d81638b8f use intern __repr__(), not repr() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-10 14:31:34 +00:00
calvin
6f488fd189 remember base href in link parser 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1918 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-06 12:42:33 +00:00
calvin
607425065e add cache key asserts 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1917 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-05 00:24:36 +00:00
calvin
01f1c06149 no repr of url name 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1913 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-03 22:35:29 +00:00
calvin
9b28e3b346 unicode fixes 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1911 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-03 21:29:25 +00:00
calvin
34081d925b unicode repr 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1908 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-11-03 13:49:59 +00:00
calvin
54b8126ccb cleanup 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1896 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-10-27 19:49:59 +00:00
calvin
dee1c867ff encode msgs to stderr 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1889 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-10-27 19:19:29 +00:00
calvin
0dcd6a8425 assertion message 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1809 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-09-16 21:14:59 +00:00
calvin
d174276fd8 also display info in repr() 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1807 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-09-16 20:33:30 +00:00
calvin
ff000a8c5c documentation for cache keys 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-09-10 08:37:14 +00:00
calvin
228221fb85 install gettext method in default namespace 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1736 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-09-05 20:48:52 +00:00
calvin
17ce3f00b9 fix typo 
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1732 e7d03fd6-7b0d-0410-9947-9c21f3af8025
 2004-09-04 23:23:20 +00:00