Commit graph

725 commits

Author SHA1 Message Date
calvin
16e2048209 adjust to new base url quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1534 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:47:03 +00:00
calvin
e1862f6321 print repr of logged url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1533 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:46:43 +00:00
calvin
09adff7cf6 javascript urls are less quoted for compatibility
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1532 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 12:03:49 +00:00
calvin
3d678d6d3b adjust test cases for new unquoted base_url and its new warning
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1531 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:44:36 +00:00
calvin
31233bb778 leave self.base_url unquoted
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1530 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:44:03 +00:00
calvin
50f7ea04df change debug color to white, use os.linesep
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1529 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:43:48 +00:00
calvin
ea2125c3f5 documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1528 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:42:36 +00:00
calvin
e7b620052e documentation, and rstrip in url quote check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1527 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-27 08:40:27 +00:00
calvin
02767fed76 put flush method in base class
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1523 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:58:01 +00:00
calvin
c34abe18a2 set line/column always to int
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1520 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:30:12 +00:00
calvin
4a947d1e5a show css/html validation on errors, and split into functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1519 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 18:29:21 +00:00
calvin
6f41bce255 use colorize in text output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1517 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-26 11:52:06 +00:00
calvin
d9c9e3e8f8 write colors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1514 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 21:50:41 +00:00
calvin
0d6c2c52a9 imports
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1512 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 21:32:31 +00:00
calvin
108015d8be remove colored logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1508 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:10:01 +00:00
calvin
66ec6ba897 default logger is text
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1506 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:09:04 +00:00
calvin
324614ab18 default logger is text again, remove colored logger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1505 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 20:08:53 +00:00
calvin
b4039684c3 got rid of urlconnect
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1501 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:53:08 +00:00
calvin
130b89f866 use urlbase and new is_safe url functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1500 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:51:38 +00:00
calvin
c904caa5c5 move self.is_cache(), and use urlunsplit to set self.url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1499 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:50:40 +00:00
calvin
d0b54018ee use urlbase and its modified url building
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1498 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:49:37 +00:00
calvin
9d49c37d57 fix telnet tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1497 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:48:43 +00:00
calvin
a2bb57d81f fix cache keys and url building
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1496 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:42:26 +00:00
calvin
45568c60f5 fix cache keys
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1495 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:41:58 +00:00
calvin
6074a6f345 add cache keys to test result
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1494 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 19:02:31 +00:00
calvin
d2b68322d0 remove bad scheme from test since the logging order is confusing otherwise (the bad scheme is logged first)
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1493 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:41:28 +00:00
calvin
2e3315cb5f pretend to be the command line in file tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1491 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:35:21 +00:00
calvin
7b51ed57d2 use new is_safe url functions
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1490 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:29:26 +00:00
calvin
b3c9db06ef add new ignored url schemes, and only fall back to file:// with urls given on the command line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1489 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:25:41 +00:00
calvin
605cd63c7d log and cache invalid syntax urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1488 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:24:59 +00:00
calvin
e3af24186b remove whitespace check, it is not needed now the url is always quoted
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1487 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:24:07 +00:00
calvin
29ea22b9bc split directory slash tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1486 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:22:37 +00:00
calvin
c4ddf8f905 syntax formatting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1485 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:22:32 +00:00
calvin
95139bc0f9 added handler for invalid urls
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1484 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:21:12 +00:00
calvin
61caa9de13 added error tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1483 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:19:47 +00:00
calvin
706c3ab63e added more needs_quoting tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1482 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:19:05 +00:00
calvin
81f43fa48a rename is_valid to is_safe, and fix url_needs_quoting to cope with ending newline
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1481 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-25 18:18:40 +00:00
calvin
0de757b5eb documentation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1480 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-24 21:34:07 +00:00
calvin
0d28d0d335 add cache key to result
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1471 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-24 15:00:18 +00:00
calvin
833201711d log cache keys, ignore empty and comment lines in result files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1470 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-24 14:59:38 +00:00
calvin
d24d8a512c do not use netrc
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1469 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-24 14:58:42 +00:00
calvin
d7cc2bc274 added new field for cache key
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1467 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-24 07:47:10 +00:00
calvin
0fe112ad4e better url joining
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1466 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 22:53:57 +00:00
calvin
311be4ac04 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1465 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 22:53:27 +00:00
calvin
01976e953b add trailing directory slash
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1463 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 22:27:24 +00:00
calvin
4e654e327b recursion of directories
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1462 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 22:20:11 +00:00
calvin
84b9c1ab2e set result for directories
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1461 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 22:15:59 +00:00
calvin
a2a1429b7e fix intern url setting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1460 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 21:34:16 +00:00
calvin
5bdea05dff fix endless loops with broken urls with a non-empty anchor
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1458 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 21:24:30 +00:00
calvin
83274d0cd3 validated html output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1456 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 21:13:12 +00:00
calvin
2964bce633 no os.linesep in translation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1455 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 20:56:41 +00:00
calvin
7669ef1b6b more case tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1452 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 20:18:30 +00:00
calvin
36db68852f errors and warnings
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1451 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 20:16:53 +00:00
calvin
d1c432f146 add real url to test result
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1449 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 20:04:54 +00:00
calvin
3f300f38e1 check resources
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1448 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 20:03:07 +00:00
calvin
e44354c3e2 fix timeout exception
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1446 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-23 15:20:49 +00:00
calvin
2bce376e61 log real url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1445 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 23:38:35 +00:00
calvin
614255a441 add intern url always on cmdline
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1444 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 23:24:25 +00:00
calvin
726aac7ac0 fix cache key
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1443 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 23:05:41 +00:00
calvin
c37d3acd50 move debug message for url queue adding
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1442 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 23:05:31 +00:00
calvin
53e89e3b39 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1435 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 21:39:08 +00:00
calvin
c3100ef518 new consumer interface
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1434 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 21:36:17 +00:00
calvin
a66a8359d1 colored logger is default now
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1433 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 21:36:00 +00:00
calvin
f2e7ca6040 split off cache and url consumer routines into separate classes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1432 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-19 21:35:47 +00:00
calvin
097624bc98 pylint
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1429 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 20:11:59 +00:00
calvin
7a0888b9eb fix mail parsing, and connection closing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1428 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 20:11:21 +00:00
calvin
60666f5abb removed unused import, changed adress to address
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1427 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 20:10:48 +00:00
calvin
e25ea13fa7 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1426 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:28:42 +00:00
calvin
4756641e1b source code restructuring
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1423 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:20:53 +00:00
calvin
1cf5a14352 functional tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1422 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:20:16 +00:00
calvin
14a9b5c426 unit tests
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1421 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:20:06 +00:00
calvin
9f7e3e67a9 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1420 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-08-16 19:17:36 +00:00
calvin
c9261c96c2 install_data
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1403 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-30 08:09:18 +00:00
calvin
8a083287fb dns config
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1401 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 14:07:22 +00:00
calvin
213b9e3cec pycheck fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1400 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 13:54:32 +00:00
calvin
1f6670e8cd import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1399 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 13:47:19 +00:00
calvin
96dd6ef4b8 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1398 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 12:01:52 +00:00
calvin
3e56e96e14 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1397 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 11:43:11 +00:00
calvin
02b177017e fix getUrlDataFrom invocation
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1396 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-26 11:18:12 +00:00
calvin
b674575de1 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1388 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 13:36:43 +00:00
calvin
6942fccb50 import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1386 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 13:19:18 +00:00
calvin
82d2ec5d51 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1385 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 11:34:53 +00:00
calvin
32bc5cd292 more import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1383 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 10:54:47 +00:00
calvin
6c2c8f78b6 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1382 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-22 10:47:55 +00:00
calvin
b60070a922 bk movements
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1375 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-20 14:50:00 +00:00
calvin
5ad8c827b4 syntax updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1374 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-20 14:49:44 +00:00
calvin
c071230c1b removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1372 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-20 14:42:18 +00:00
calvin
a5204c56d5 resynced with newest upstream
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1371 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-20 14:32:04 +00:00
calvin
6d8ae43f37 moved
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1367 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-19 09:02:52 +00:00
calvin
6476c8675d more import fixes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1364 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-19 08:58:59 +00:00
calvin
916f96cc0d checker module
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1357 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-07 18:15:17 +00:00
calvin
018cf945d1 new module layout
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1356 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-07 18:04:40 +00:00
calvin
6f37e1961d added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1355 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-07 18:01:25 +00:00
calvin
5a644b35b3 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1354 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-06 22:08:05 +00:00
calvin
3bbfac47c7 removed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1353 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-06 20:34:00 +00:00
calvin
dc90faac9e renamed
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1351 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-03 06:17:08 +00:00
calvin
bde88f9715 added string utils to parser, and sync with webcleaner
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1350 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-07-02 18:25:00 +00:00
calvin
a4925830f6 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1345 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 21:16:14 +00:00
calvin
cf0ec06ef0 do not quote None
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1343 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 21:05:14 +00:00
calvin
097bb8a143 mv contentAllowsRobots to end of recursion check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1339 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 20:43:41 +00:00
calvin
49e2b1f10d rework anchor fallback
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1336 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 20:27:59 +00:00
calvin
715a80afff ignore flush errors
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1335 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 17:31:05 +00:00
calvin
7556d4e72c correctly quote request url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1331 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 09:05:46 +00:00
calvin
58fab5a44f updated from webcleaner
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1330 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 08:29:43 +00:00
calvin
04e0a9448d updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1325 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 00:30:37 +00:00
calvin
4353d97854 added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1324 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-27 00:18:04 +00:00
calvin
1f28911a23 actually fallback to GET with Zope servers
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1321 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-26 23:54:47 +00:00
calvin
ce68bb782a cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1320 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-26 23:48:28 +00:00
calvin
e9341590d4 better err msg on bad status line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1318 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-26 23:02:47 +00:00
calvin
abccff16ea fall back to GET on bad status line
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1317 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-26 23:00:21 +00:00
calvin
37b69e16e4 uri regex url added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1314 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-05-03 08:53:38 +00:00
calvin
8dcd8f408a copyright
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1313 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-05 09:55:14 +00:00
calvin
ca081c2168 also check robots allowance of HTML files
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1304 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 10:48:31 +00:00
calvin
50bc463bb1 check cache
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1303 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 10:42:15 +00:00
calvin
2ffb97a855 get new urls from top of queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1302 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 09:45:33 +00:00
calvin
e78a8ea539 full css parsing
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1300 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 09:30:10 +00:00
calvin
f4802fd467 pychecker
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1299 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:41:30 +00:00
calvin
fa46757bd7 fix import
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1298 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:34:21 +00:00
calvin
68451e65dd O3
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1297 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:31:57 +00:00
calvin
93253954a8 updated
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1296 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:30:48 +00:00
calvin
672e118d9b use sorted dict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1295 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:30:38 +00:00
calvin
8e4e92dddd minor improvements
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1294 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:30:21 +00:00
calvin
1b148b0b4e sorted dict
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1293 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:30:01 +00:00
calvin
e183ac84dc handle missing startquotes
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1292 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-04 08:29:31 +00:00
calvin
52609e4399 pychecker
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1291 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-03 18:10:38 +00:00
calvin
6b1d124d35 no sys.path fiddling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1290 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-03 17:59:31 +00:00
calvin
8584d5bc8e only check robots.txt for http
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1285 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-04-03 16:34:58 +00:00
calvin
67fabd5d8e addd contact email and url to user-agent string
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1284 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-03-11 11:01:00 +00:00
calvin
d8e738c60b check syntax and cache before putting url objects in the queue
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1277 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-03-04 12:17:38 +00:00
calvin
d79aee3a2c xml prefix for attr var
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1272 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-03-01 15:49:32 +00:00
calvin
af5be26d2c use XmlUtils instead of xmlify for quoting
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1271 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-03-01 15:38:56 +00:00
calvin
b63fb15986 hmmmm
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1267 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-21 15:06:22 +00:00
calvin
b7e54260b0 also quote parent url
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1265 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-21 14:54:10 +00:00
calvin
033a0873be better error msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1261 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-21 11:56:24 +00:00
calvin
58057bd07f better err msg
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1260 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-21 11:48:39 +00:00
calvin
85115c2039 cleanup
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1257 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-20 14:17:49 +00:00
calvin
bd628b7de7 use new url.py
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1256 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-20 14:14:31 +00:00
calvin
5187dbc4c2 quote url in output
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1255 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-20 14:13:42 +00:00
calvin
ab9092d7a0 catch errors earlier in recursion check
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1253 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-19 23:27:21 +00:00
calvin
fefba0036d catch ValueError, raise IncompleteRead on invalid chunk length
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1250 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-02-19 23:13:30 +00:00
calvin
a02d8ae2a4 fallback in redirections
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1239 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 23:47:21 +00:00
calvin
83b7ef7ff9 break cycles
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1238 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 23:46:13 +00:00
calvin
4e8c8547ec fix typos
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1237 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 21:39:15 +00:00
calvin
7121f81aff language
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1236 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 21:37:06 +00:00
calvin
967cadaa26 fallback to GET
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1231 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 21:20:28 +00:00
calvin
76452953f8 use file instead of open
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1226 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 19:04:49 +00:00
calvin
669866a7ab add NoneLogger
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1223 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 19:02:50 +00:00
calvin
fa9023d9f8 fix file parsing, ignore comments and empty lines
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1222 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 19:02:31 +00:00
calvin
8a474914f3 added NOneLogger, adjust blacklist default file and handling
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1221 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 19:02:06 +00:00
calvin
d78d96dd0e added
git-svn-id: https://linkchecker.svn.sourceforge.net/svnroot/linkchecker/trunk/linkchecker@1220 e7d03fd6-7b0d-0410-9947-9c21f3af8025
2004-01-29 19:01:24 +00:00