Commit graph

273 commits

Author SHA1 Message Date
calvin
f01b84b894 rework the redirection routine a little, putting warnings specifically for redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2786 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:51:12 +00:00
calvin
ed932cb165 debugging
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2785 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 15:50:27 +00:00
calvin
317ef181f0 handle all redirections to different URL schemes,
not just HTTP -> not HTTP, and fix a variable typo


git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2780 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-20 08:53:57 +00:00
calvin
b94eec79ef use correct value for status output of checked links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2773 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-17 23:26:13 +00:00
calvin
d347840dee use official HTTP status names
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2754 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-15 11:31:32 +00:00
calvin
359612b6fe add missing import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2751 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:53:27 +00:00
calvin
225c045815 do not cache unusual errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2750 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 17:11:13 +00:00
calvin
5d82d64c31 more result sanity check warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2749 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-14 13:35:18 +00:00
calvin
c140b510b1 fix warning tags
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2746 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 19:33:26 +00:00
calvin
f80480469f config to ignore certain warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2743 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-13 15:03:17 +00:00
calvin
56b2b422a9 ensure that thread name is ascii
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2718 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 22:06:18 +00:00
calvin
bb2b227b13 catch all errors on url check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2717 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 22:05:54 +00:00
calvin
e7ca18403a work around python bug occuring with incomplete gzip streams
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2716 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-05 19:56:42 +00:00
calvin
901227d436 assign extern info to local var
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2709 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 22:19:52 +00:00
calvin
2e207be127 allow empty relative URLs
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2704 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-07-04 20:28:55 +00:00
calvin
49aa4ec208 remove unused imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2678 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-27 15:02:10 +00:00
calvin
0c3e78569c fix norming
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2676 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-23 16:21:05 +00:00
calvin
217d34bb54 use new extern check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2675 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-22 12:30:03 +00:00
calvin
d48eb20a28 support extern filter in ignored url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2672 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-20 14:53:23 +00:00
calvin
990d548424 don't delete from incoming queue while iterating, but after. SF #1223920
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2671 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-20 09:49:27 +00:00
calvin
ea052008a2 send the complete email address with VRFY command
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2654 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 21:17:10 +00:00
calvin
cc0f7cb113 always norm a URL before sending a request
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2653 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-06-12 21:16:47 +00:00
calvin
79d0850fed fix intern url of file: links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2636 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-22 20:28:59 +00:00
calvin
fec843814c fix intern url of file: links
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2635 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-22 20:27:32 +00:00
calvin
6d8f0146d6 fix GET fallback handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2623 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:53:13 +00:00
calvin
d2107c7a6a split and fix initialization of UrlData objects
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2622 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:42:14 +00:00
calvin
cd32e823df fix url util call
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2621 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:52 +00:00
calvin
9666421a10 add log_url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2620 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:40 +00:00
calvin
1fc5eb32b6 fix extern calculation and a typo
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2619 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:02:16 +00:00
calvin
f6f04e376e fix extern calculation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2618 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 19:01:51 +00:00
calvin
d6e56ceba2 remove lock from check_url in case of disabled threading
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2617 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 18:30:06 +00:00
calvin
55d7d5f7df use a decorator to synchronize all the checker threads
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2614 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-19 15:49:06 +00:00
calvin
781ad5fdc0 catch exceptions on abort
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2612 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:15:54 +00:00
calvin
fd652881fb thread name and maximum wait for finish
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2611 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:15:18 +00:00
calvin
3db056f161 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2610 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 22:14:45 +00:00
calvin
4c270cfe74 log message formatting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2609 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-18 18:04:54 +00:00
calvin
c0d9e56a4d new trace option
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2601 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 22:06:47 +00:00
calvin
ed738d2436 cmdline param defaults to False
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2595 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 19:00:17 +00:00
calvin
e3a049ee27 make intern pattern of parent path
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2593 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-17 14:11:17 +00:00
calvin
d821abce85 news group info string
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2588 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-13 14:01:53 +00:00
calvin
208a7f095f added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2585 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-09 22:07:45 +00:00
calvin
1a7a771648 new intern/extern handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2584 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-09 22:05:21 +00:00
calvin
829e19e7ee cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2572 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 22:15:12 +00:00
calvin
c07f42031a ignored urls only issue an info instead of a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2569 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-06 14:59:22 +00:00
calvin
bf8fd348aa better msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2564 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 13:22:10 +00:00
calvin
532a04e436 method change is an info, not a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2563 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 13:18:19 +00:00
calvin
49a2dee338 make robots.txt deny an info instead of a warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2555 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 10:48:48 +00:00
calvin
a39f635b82 use some %r in warning strings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2554 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-05-04 10:48:10 +00:00
calvin
d2a8764d18 do not store aliases
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2550 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-27 12:48:48 +00:00
calvin
23e5212c7d do not remove userinfo from host part of url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@2544 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2005-04-26 17:53:31 +00:00