Commit graph

480 commits

Author SHA1 Message Date
calvin
cdca8ef7ca updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3439 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 16:36:54 +00:00
calvin
89d70e65f3 document internlinks config option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3431 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-15 09:39:33 +00:00
calvin
634ff1d15c updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3429 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-09-04 17:04:44 +00:00
calvin
220b1da8e8 release
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3423 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-17 21:46:44 +00:00
calvin
9d15f9a8be allow inifinite recursion in CGI script
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3417 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-07 22:18:31 +00:00
calvin
af278ac217 removed the --disable-psyco option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3415 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-02 20:17:22 +00:00
calvin
8064a630c6 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3412 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-08-02 20:13:46 +00:00
calvin
a137373918 bump up version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3410 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-27 17:18:23 +00:00
calvin
6318ac653a release date
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3407 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-25 21:03:43 +00:00
calvin
a311497e66 fall back to ASCII if default locale is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3403 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-19 16:42:43 +00:00
calvin
adc4e8c0e8 quote base reference URL, with tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3402 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-18 08:07:46 +00:00
calvin
7e2415392a updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3396 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-07-15 22:35:06 +00:00
calvin
53a5b4dff1 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3378 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-15 00:00:10 +00:00
calvin
9a84b42326 release name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3343 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-05 13:30:37 +00:00
calvin
19a7495b9e only accept ASCII robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3339 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-04 21:07:08 +00:00
calvin
c3fa9d7965 don't generate empty output files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3332 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 13:58:13 +00:00
calvin
300660af72 Drop privileges when running as root under Unix; add new --allow-root option to prevent this
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3329 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-06-01 09:58:40 +00:00
calvin
0665fa9add release
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3321 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-29 12:57:33 +00:00
calvin
b442809838 look that cached URLs get checked quickly in large queues
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3315 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-25 11:35:24 +00:00
calvin
14750c862a updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3293 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 22:05:02 +00:00
calvin
356918d98f bump up version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3291 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-23 19:57:50 +00:00
calvin
7e1e01bd36 do not catch UnicodeError, handle that intern
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3269 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-19 17:13:16 +00:00
calvin
98d5387ac0 release
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3268 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-18 21:46:18 +00:00
calvin
2ec5c054fe merge ignoredurl and errorurl into unknownurl, updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3237 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 19:08:40 +00:00
calvin
2914ee1f45 added crawl-delay support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3231 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-17 15:36:44 +00:00
calvin
263a38fbd2 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3223 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 22:58:04 +00:00
calvin
75e88c062a added --cookiefile option to set initial cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3210 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-16 20:56:34 +00:00
calvin
41914d4ab6 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3193 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-15 19:09:37 +00:00
calvin
df7a2076a9 cache DNS lookups
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3176 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 18:03:23 +00:00
calvin
2cfcb5c0bb avoid double timeouts by raising timeout errors in robots.txt retrieval
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3171 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 12:58:31 +00:00
calvin
5ba1324a73 an intern pattern matches both http and https URLs now
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3169 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 10:06:33 +00:00
calvin
51bd45f163 Version bump to 4.0
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3168 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-14 08:46:46 +00:00
calvin
f002c5f965 Replace the old threading algorithm with a new one based on Queue.Queue and consumer threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3146 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-05-13 13:44:52 +00:00
calvin
4dd2167508 typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3111 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-08 08:57:08 +00:00
calvin
cc388ccc65 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3107 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-04-03 19:15:33 +00:00
calvin
ab0ad2e9ef bump up version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3069 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-13 17:48:07 +00:00
calvin
bd8bfa06a3 release
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3059 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-04 09:13:34 +00:00
calvin
226ec6fbf4 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3035 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 19:52:15 +00:00
calvin
eeae239b5c updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3023 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-02-01 18:52:59 +00:00
calvin
34eb35884d release name
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3004 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2006-01-03 10:13:34 +00:00
calvin
2e1dfae083 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2998 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-20 20:11:11 +00:00
calvin
a2236f9dbb updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2965 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-12-08 20:20:43 +00:00
calvin
9cea76e009 bump up version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2913 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-17 12:31:36 +00:00
calvin
4f939b62ce release
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2901 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 22:31:53 +00:00
calvin
2d69003107 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2889 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 20:21:30 +00:00
calvin
5303f94b76 ensure proxy value syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2883 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:59:20 +00:00
calvin
d9e43b96bd updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2881 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:28:51 +00:00
calvin
27da139a1d updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2879 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-13 17:22:20 +00:00
calvin
b2b764121e updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2862 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-12 16:20:51 +00:00
calvin
de2af2d72d updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2861 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-11 20:10:50 +00:00