Commit graph

110 commits

Author SHA1 Message Date
calvin
5fdf81c56c fix proxy scheme
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@983 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-08-07 10:30:26 +00:00
calvin
6d724d04a4 handle http redirections to non-http url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@981 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-08-07 10:18:39 +00:00
calvin
307e57ba0d redirection fixed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@961 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-28 12:05:24 +00:00
calvin
087eb8f3b6 restructuring
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@959 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-28 11:13:11 +00:00
calvin
33070b0923 do not warn missing backslash on .html files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@958 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-28 11:08:35 +00:00
calvin
cf17757fae check cache again on redirects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@957 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-28 10:55:16 +00:00
calvin
2be119e899 quote url before request submit, previous map() call was useless
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@950 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-23 23:26:39 +00:00
calvin
1d9a192f0e getRobotsTxtUrl function
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@938 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-04 15:06:35 +00:00
calvin
308ceb45c5 add coding line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@933 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-07-04 14:24:44 +00:00
calvin
10ddb75519 fix quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@877 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-05-01 13:39:52 +00:00
calvin
d0c3d124d0 quote urls before request or html output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@875 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-05-01 13:21:23 +00:00
calvin
7097cd85ef fix gettext call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@860 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-29 09:50:36 +00:00
calvin
fc35e1f97f fix the anchor fix
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@858 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-29 09:40:33 +00:00
calvin
63d3f241ce fix redirect bug using non-absolute url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@855 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-29 01:09:05 +00:00
calvin
703abd1ca7 remove anchor from HEAD and GET requests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@853 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-28 13:08:00 +00:00
calvin
b2a87961d7 convert urlparts to list on redirect
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@845 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-27 14:32:40 +00:00
calvin
85b93170fc updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@837 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-17 13:09:33 +00:00
calvin
1382aebcc7 more imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@835 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-17 11:36:12 +00:00
calvin
558e06b574 catch httplib.error
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@834 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-04-17 11:06:22 +00:00
calvin
b570df925b fix amazon timeouts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-02-06 01:23:36 +00:00
calvin
379bb9e342 style
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@777 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-28 23:55:43 +00:00
calvin
1c80cb6287 use included robotparser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-26 14:57:18 +00:00
calvin
412b389894 fix cvs errors, recommit
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@749 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-11 12:38:18 +00:00
calvin
6d40be35e2 debug fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@732 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-08 09:27:15 +00:00
calvin
da86b44f6b added debug.py utils and use it
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@731 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-08 08:47:49 +00:00
calvin
70586ae6c1 fix i18n
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@721 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-05 22:55:48 +00:00
calvin
a21af42394 po updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@719 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-01-05 21:00:32 +00:00
calvin
c42fe507ce fix telnet url parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@690 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-12-31 00:31:22 +00:00
calvin
a81cb0835c use userinfo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@657 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-12-06 10:33:10 +00:00
calvin
5a9729c1b8 rename urlTuple to urlparts
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@656 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-12-06 01:36:19 +00:00
calvin
6e4414104c remove skip_host check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@640 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-27 14:23:01 +00:00
calvin
113460f503 update to Python >= 2.2.1
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@631 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-26 23:27:43 +00:00
calvin
33e25d4a39 fix flushing error handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-24 19:53:37 +00:00
calvin
3f59c6b6d4 use local httplib
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-22 19:29:31 +00:00
calvin
88008d5470 only skip_host if supported
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@599 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-12 01:14:51 +00:00
calvin
06b60017b0 fix stupid typos
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@593 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-07 23:53:08 +00:00
calvin
203558d357 use new HTTPConnection and HTTPResponse object interface of httplib
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@590 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-11-07 23:41:23 +00:00
calvin
07da7a92b6 support for ftp proxies
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@569 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-10-15 00:59:12 +00:00
calvin
4e6ae8e953 correct proxy info
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@566 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-26 20:06:52 +00:00
calvin
6e03bd2e8a updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-26 15:48:13 +00:00
calvin
08e3379b39 _ call fixed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@551 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-26 15:25:44 +00:00
calvin
22291c1c0c add info about proxy usage
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@549 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-26 15:08:37 +00:00
calvin
053c63a308 identity encoding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@525 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-10 23:55:04 +00:00
calvin
0579ba6988 support content-encoding: deflate
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@524 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-10 23:36:07 +00:00
calvin
2537721402 warn about unknown encodings,
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@521 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-10 20:20:44 +00:00
calvin
1d5cdd3ffd User-Agent
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@510 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-09-07 11:54:57 +00:00
calvin
311ab1bbc5 use external DNS lib
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@478 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-06-23 17:00:40 +00:00
calvin
3675c1ebef replace empty paths with a slash
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@433 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-05-14 20:44:59 +00:00
calvin
0cefc0d650 cookie support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@414 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-05-05 10:55:56 +00:00
calvin
49a16042ca cookies and port number syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@409 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2002-05-04 13:27:02 +00:00