calvin
|
51cab64ac7
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3235 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:54:25 +00:00 |
|
calvin
|
9cd88b9546
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3234 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:54:00 +00:00 |
|
calvin
|
bb528b7c3b
|
mention crawl-delay support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3233 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:53:46 +00:00 |
|
calvin
|
4ec74f6f5c
|
added robots.txt tests for the internal HTTP server
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3232 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:47:48 +00:00 |
|
calvin
|
2914ee1f45
|
added crawl-delay support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3231 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:36:44 +00:00 |
|
calvin
|
4bf2b361cb
|
added more crawldelay tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3230 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:36:34 +00:00 |
|
calvin
|
e574a97798
|
raise ValueError if wait delay is a negative value
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3229 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:36:27 +00:00 |
|
calvin
|
7dce5c4df9
|
fix callback call for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3228 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:36:09 +00:00 |
|
calvin
|
3adaf48b3d
|
add callback for crawldelay
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3227 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:35:58 +00:00 |
|
calvin
|
a741d7922c
|
add get_crawldelay method
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3226 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-17 15:35:48 +00:00 |
|
calvin
|
fb319b3785
|
add callback after robots.txt parse
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3225 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 23:19:18 +00:00 |
|
calvin
|
2aed0f3bc5
|
add per-host wait times
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3224 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 23:03:44 +00:00 |
|
calvin
|
263a38fbd2
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3223 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:58:04 +00:00 |
|
calvin
|
811f5492c4
|
fix --pause to delay requests to the same host
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:56:13 +00:00 |
|
calvin
|
2383915730
|
ignore *.pyc and *.pyo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3221 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:22:28 +00:00 |
|
calvin
|
ad28599e57
|
Note if URL is missing (instead of saying it is empty)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:15:34 +00:00 |
|
calvin
|
00a60c6906
|
check if urldata.url is None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3219 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:14:34 +00:00 |
|
calvin
|
224f5e723b
|
don't find link name in empty urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3218 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 22:05:17 +00:00 |
|
calvin
|
944ec79cd3
|
ignore *.pyc and *.so
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3217 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:53:31 +00:00 |
|
calvin
|
c42a15652e
|
remove _ftpparse.so on clean
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3216 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:43:41 +00:00 |
|
calvin
|
fa8b9bd83b
|
fix the description of --pause
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3215 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:35:39 +00:00 |
|
calvin
|
12223ec669
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3214 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:33:36 +00:00 |
|
calvin
|
6cd7cc282d
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3213 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:29:56 +00:00 |
|
calvin
|
72c718b7d5
|
added robots.txt parse test
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3212 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:29:31 +00:00 |
|
calvin
|
d73aa0e5bd
|
parse crawl-delay parameter line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3211 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 21:29:18 +00:00 |
|
calvin
|
75e88c062a
|
added --cookiefile option to set initial cookie values
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3210 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 20:56:34 +00:00 |
|
calvin
|
200209be2c
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3209 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 20:25:31 +00:00 |
|
calvin
|
26f6d3b24a
|
also warn on graph xml logger when no --verbose was given
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3208 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 19:25:09 +00:00 |
|
calvin
|
078bb141f7
|
updated simple glade app to latest version found on net
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3207 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 17:01:16 +00:00 |
|
calvin
|
c6b5759c39
|
print thread name on trace
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3206 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 17:00:49 +00:00 |
|
calvin
|
0bb2970222
|
remove old consumer code
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3205 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 17:00:27 +00:00 |
|
calvin
|
6bc840ade9
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3204 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 17:00:08 +00:00 |
|
calvin
|
75514b64f9
|
Clarify warning about --no-anchor-caching
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3203 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:32:07 +00:00 |
|
calvin
|
5dcaeb6c04
|
synchronize docs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3202 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:27:57 +00:00 |
|
calvin
|
b6ad3084aa
|
added more anchor tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3201 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:22:56 +00:00 |
|
calvin
|
e03a741396
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3200 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:18:56 +00:00 |
|
calvin
|
7907005182
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3199 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:18:38 +00:00 |
|
calvin
|
7c22587c91
|
fix default timeout; add note about return code for 'none' logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3198 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:18:22 +00:00 |
|
calvin
|
8c0b1be28f
|
added migration doc
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3197 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:17:56 +00:00 |
|
calvin
|
e284843640
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3196 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:17:29 +00:00 |
|
calvin
|
7c8ab1b6f7
|
No maximum size due to possible deadlocks.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3195 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-16 16:14:42 +00:00 |
|
calvin
|
e8bf93bd90
|
inherit from StandardError, not from Exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3194 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 22:22:24 +00:00 |
|
calvin
|
41914d4ab6
|
updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3193 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 19:09:37 +00:00 |
|
calvin
|
c10da15b08
|
add timeout setting to configuration file
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3192 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 19:04:21 +00:00 |
|
calvin
|
61fc7ab502
|
since join() is not interruptable, put in a little sleep() call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3191 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:36:55 +00:00 |
|
calvin
|
ec2756021f
|
get default timeout from config
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3190 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:36:33 +00:00 |
|
calvin
|
ca8ea4a2be
|
print in-progress URLs on status
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3189 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:35:37 +00:00 |
|
calvin
|
ac682df355
|
be precise for threads: zero threads disable threading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3188 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:35:16 +00:00 |
|
calvin
|
b3b45d7f06
|
add Timeout exception, and return in_progress URLs on status instead of unfinished
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3187 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:34:30 +00:00 |
|
calvin
|
b1e7ca58a2
|
remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@3186 e7d03fd6-7b0d-0410-9947-9c21f3af8025
|
2006-05-15 18:33:38 +00:00 |
|