Commit graph

74 commits

Author SHA1 Message Date
Chris Mayo
a15a2833ca Remove spaces after names in class method definitions
And also nested functions.

This is a PEP 8 convention, E211.
2020-05-16 20:19:42 +01:00
Chris Mayo
1663e10fe7 Remove spaces after names in function definitions
This is a PEP 8 convention, E211.
2020-05-16 20:19:42 +01:00
Chris Mayo
fc11d08968 Remove spaces after names in class definitions 2020-05-16 20:19:42 +01:00
Chris Mayo
dcbddfe045 Remove Python 2 ConfigParser import 2020-05-15 19:37:04 +01:00
Chris Mayo
44e81d27dd Remove inheriting object
All Python 3 classes are new-style.
2020-05-08 10:45:31 +01:00
Chris Mayo
b0ea72e8c1 Remove # -*- coding: lines
Except for tests that include non-unicode characters:

tests/test_po.py
tests/test_strformat.py
tests/test_url.py
tests/checker/test_error.py
tests/checker/test_news.py
2020-05-08 10:45:31 +01:00
Chris Mayo
f5e7f3a382 Remove use of the future package
It was providing Python 2 compatibility.
2020-04-15 19:49:16 +01:00
Petr Dlouhý
8f4acc3168 Python3: use str and basestring from builtins 2019-04-16 20:08:29 +01:00
Petr Dlouhý
a1b300c892 Python3: fix imports 2018-01-19 09:52:43 +01:00
Antoine Beaupré
9d899d1dfa add --no-robots commandline flag
While this flag can be abused, it seems to me like a legitimate use
case that you want to check a fairly small document for mistakes,
which includes references to a website which has a robots.txt that
denies all robots. It turns out that most websites do *not* add a
permission for LinkCheck to use their site, and some sites, like the
Debian BTS for example, are very hostile with bots in general.

Between me using linkcheck and me using my web browser to check those
links one by one, there is not a big difference. In fact, using
linkcheck may be *better* for the website because it will use HEAD
requests instead of a GET, and will not fetch all page elements
(javascript, images, etc) which can often be fairly big.

Besides, hostile users will patch the software themselves: it took me
only a few minutes to disable the check, and a few more to make that
into a proper patch.

By forcing robots.txt without any other option, we are hurting our
good users and not keeping hostile users from doing harm.

The patch is still incomplete, but works. It lacks: documentation and
unit tests.

Closes: #508
2016-05-19 14:43:59 -04:00
Bastian Kleineidam
eaa8a963ec Refactor logging configuration. 2014-05-10 21:23:06 +02:00
Bastian Kleineidam
7b34be590b Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements. 2014-03-01 00:12:34 +01:00
Bastian Kleineidam
c806be5c15 Updated copyright 2014-01-08 22:33:04 +01:00
Bastian Kleineidam
103e00b4d1 Allow disabling of ssl certificate checks. 2013-12-12 22:17:57 +01:00
Bastian Kleineidam
5736987b60 Refactor output loggers. 2013-12-11 18:41:55 +01:00
Bastian Kleineidam
123578a4cd Make per-host connection limits configurable. 2013-02-27 19:37:28 +01:00
Bastian Kleineidam
42a17cbb98 Prepare py3 port and display sys.argv on internal errors. 2012-11-26 18:49:07 +01:00
Bastian Kleineidam
f3e52f1176 loginpasswordfield is not a password 2012-11-08 22:11:35 +01:00
Bastian Kleineidam
3a51ac7662 Warn about accessible passwords in config files. 2012-10-15 14:36:10 +02:00
Bastian Kleineidam
f484a6776d Use timeout value from configuration. 2012-10-10 10:53:52 +02:00
Bastian Kleineidam
3e464e509c Do not allow empty configuration string values. 2012-09-21 16:05:34 +02:00
Bastian Kleineidam
6730fb51ee Allow maximum check time specification. 2012-09-03 20:17:49 +02:00
Bastian Kleineidam
02a9f0bacb Add utility method to read string options. 2012-08-23 04:52:25 +02:00
Bastian Kleineidam
2888528a13 Code cleanup. 2012-06-20 20:11:13 +02:00
Bastian Kleineidam
4cce99a77d Test SSL certificate expiration. 2012-06-20 20:10:40 +02:00
Bastian Kleineidam
00aa631267 Add localwebroot configuration option. 2012-06-10 14:47:27 +02:00
Bastian Kleineidam
2dee223555 Allow memory dumps to be written. 2012-06-10 13:18:35 +02:00
Bastian Kleineidam
3ebe5708db Code cleanup: add function for boolean option settings. 2012-06-10 11:22:54 +02:00
Bastian Kleineidam
52dcf101e0 Remove rest of deprecated options. 2012-04-22 17:55:12 +02:00
Bastian Kleineidam
d6d82b96f1 Add cookiefile config option. 2012-03-22 22:29:56 +01:00
Bastian Kleineidam
3b818d99c0 Allow --pause to be set in configuration file. 2012-03-16 21:28:30 +01:00
Bastian Kleineidam
71f5ee42c8 Updated copyright. 2012-01-29 17:18:28 +01:00
Bastian Kleineidam
74ea444a9a Parse logger and logging part names case insensitive. 2011-10-10 20:32:58 +02:00
Bastian Kleineidam
e71d441ce6 Raise correct error messages for configuration errors. 2011-10-09 21:45:56 +02:00
Bastian Kleineidam
51bcccfdfe Added new option --user-agent to set the User-Agent header. 2011-07-25 21:09:49 +02:00
Bastian Kleineidam
68a8de58d1 Deprecate the --interactive commandline option. 2011-02-20 11:05:29 +01:00
Bastian Kleineidam
2d55c87429 Make sure ignore URL patterns take precedence over nofollow URL patterns. 2011-02-18 07:47:22 +01:00
Bastian Kleineidam
c5884b8d87 Add function documentation. 2011-02-14 21:06:34 +01:00
Bastian Kleineidam
48e4bd8bfd Updated copyright 2011-02-06 09:50:48 +01:00
Bastian Kleineidam
815f8ce29e Remove unused import. 2011-01-09 09:55:30 -06:00
Bastian Kleineidam
5058161e40 Remove deprecated options and backward compatibility parts. 2010-11-21 11:35:56 +01:00
Bastian Kleineidam
332fa4f8f9 Prepare multi-realm auth configuration. 2010-10-25 22:07:16 +02:00
Bastian Kleineidam
b6ec28c68e Make password entries in authentication configuration optional. 2010-10-25 17:33:13 +02:00
Bastian Kleineidam
25d75bd4a2 Added login URL posting. 2010-10-14 18:36:11 +02:00
Bastian Kleineidam
4acf79c641 Add configuration file option for the --cookies commandline option. 2010-10-12 00:41:14 +02:00
Bastian Kleineidam
c4c098bd83 pep8-ify the source a little more 2010-03-13 08:47:12 +01:00
Bastian Kleineidam
34a2f4a15d Disable and deprecated the --no-proxy-for option. 2010-03-07 17:45:48 +01:00
Bastian Kleineidam
5e06b6b8d4 Updated FSF address in GPL blurb 2009-07-24 23:58:20 +02:00
Bastian Kleineidam
7f67027abf ignore the fragment part (ie. the anchor) of URIs when
+  getting and caching content
2009-06-26 07:22:36 +02:00
Bastian Kleineidam
62922d48e8 Fix previous introduce config parsing error. 2009-06-18 20:29:31 +02:00