Commit graph

44 commits

Author SHA1 Message Date
calvin
df34e1a8e9 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2919 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-25 13:48:30 +00:00
calvin
c9f5d1a0b1 catch gzip errors, and use linkchecker debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2910 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-10-15 00:06:48 +00:00
calvin
f24bb87e54 add missing return
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2784 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 12:37:32 +00:00
calvin
afa8750dc3 catch ValueError raised by urllib2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2783 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 09:45:30 +00:00
calvin
baf51d1f5d cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2752 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:56:05 +00:00
calvin
63b76ec642 Use HTTPMessage() in all urllib handlers, really fixing the bug noted in http://www.python.org/sf/1117588. The workaround has been removed.
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2603 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 17:53:39 +00:00
calvin
44075c47bf clean up raise calls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-08 14:52:50 +00:00
calvin
973b6d5098 work around for 302 redirect handling error
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2283 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-02-07 12:10:17 +00:00
calvin
05c9b8b5e6 use linkchecker agent on getting /robots.txt
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2194 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-24 09:45:22 +00:00
calvin
adc1d02217 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2165 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 21:11:43 +00:00
calvin
d030a5b054 documentation updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2164 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 15:56:48 +00:00
calvin
647d7167ee documentation syntax
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2163 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-19 15:08:02 +00:00
calvin
700d564be7 documentation updates
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2148 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-18 01:00:45 +00:00
calvin
b06f144ced updated copyright year
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2122 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-01-11 02:22:43 +00:00
calvin
c97f68f70a accept unicode in robots.txt can_fetch
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1924 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-11-09 00:00:59 +00:00
calvin
62b2784ebc python 2.4 compat
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1805 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-09-16 20:11:38 +00:00
calvin
ce9dc6fbe9 increment robotparser version
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1655 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:45:34 +00:00
calvin
bc6bd34ffc fix password manager interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1654 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:42:34 +00:00
calvin
bffdfa68fd robots.txt password support
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1649 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-31 21:20:51 +00:00
calvin
4756641e1b source code restructuring
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1423 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:20:53 +00:00
calvin
1f6670e8cd import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 13:47:19 +00:00
calvin
96dd6ef4b8 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1398 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 12:01:52 +00:00
calvin
b674575de1 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1388 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 13:36:43 +00:00
calvin
5ad8c827b4 syntax updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1374 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-20 14:49:44 +00:00
calvin
018cf945d1 new module layout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1356 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-07 18:04:40 +00:00
calvin
8dcd8f408a copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1313 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-05 09:55:14 +00:00
calvin
95611de5c3 replace backticks with repr
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1121 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-12-20 11:28:55 +00:00
calvin
f741df2bfa catch httplib errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1081 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-11-15 15:30:22 +00:00
calvin
765f30fbb1 use internal debug logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1073 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-10-31 16:31:05 +00:00
calvin
0de34a7675 ignore decompression errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1062 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-10-21 10:11:46 +00:00
calvin
c4d243dfc7 re-added robotparser2.py, updated it
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1041 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2003-09-23 21:59:17 +00:00
calvin
14c9cbc4c4 updated tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-11-29 13:49:52 +00:00
calvin
8f4260458a docstrings and copyright updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@243 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-03-15 01:19:35 +00:00
calvin
5d1c6ef00e regression test suite
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@229 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-02-18 20:29:52 +00:00
calvin
83a0846fef link patterns
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@228 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-02-09 00:59:44 +00:00
calvin
9c87fedcf4 fix robotparser2.py
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@227 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-02-01 00:52:11 +00:00
calvin
d82d4c4d97 standards version and ctrl-c
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@221 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-06 18:00:43 +00:00
calvin
47d294502f robotparser licensing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-06 12:22:32 +00:00
calvin
77b88c4a4e robotparser
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@219 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-05 11:42:11 +00:00
calvin
c9b63453d1 robotparser2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@218 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-04 23:38:06 +00:00
calvin
29921eb3ef robotparser2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@217 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-04 23:26:53 +00:00
calvin
3254f921ee robotparser2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@216 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-04 23:20:34 +00:00
calvin
f84023c401 robotparser2
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@215 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2001-01-04 23:16:46 +00:00
calvin
e2fef02cc8 release .10
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@209 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2000-12-14 22:53:48 +00:00
Renamed from robotparser2.py (Browse further)