Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-05-23 21:55:49 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
7130 commits 1 branch 0 tags 24 MiB
Commit graph

61 commits

Author SHA1 Message Date
Chris Mayo
7684a22261 Make LinkChecker HTTP header bold in linkcheckerrc(5) 2024-02-26 19:28:11 +00:00
Chris Mayo
c7621d80a4 Make referenced options bold in WARNINGS in linkcheckerrc(5) 2024-02-26 19:28:11 +00:00
Chris Mayo
ffe4a83b88 Document CssSyntaxCheck checks stylesheets 2024-02-26 19:24:07 +00:00
Chris Mayo
faa570f45c Document CssSyntaxCheck and HtmlSyntaxCheck submit URLs 2024-02-26 19:24:07 +00:00
nodet
28f6743778
Add ignorewarningsforurls to ignore specific warnings (#794) 
We want to allow specifying a warning to ignore for
each URL. If no regex is specified for the warning to ignore,
we'll ignore all warnings.

The tests still pass as they are, which means that unknown
values in the configuration file are simply ignored.

* [#782] Add values to configuration file

* [#782] Parse new configuration values

* [#782] Actually ignore a warning

* [#782] Confirm side cases work as expected

* [#782] Add logging when deciding to ignore warnings

* [#782] Documentation for ignorewarningsforurls

* [#782] Update (generated) man pages

* [#782] These tests pass without network, actually

* [#782] Fix copy/paste error in symbol naming

* [#782] The regex matches the name of the warning, not the message

* [#782] Better wording

* [#782] Update (generated) man pages

* [#782] We match the type, not the message
 2024-02-13 19:43:29 +00:00
Chris Mayo
00da28f5bc Document that --verbose overrides --no-warnings 2023-09-25 19:22:19 +01:00
Chris Mayo
0faccf2ab3
Merge pull request #752 from cjmayo/deprecated_modules 
Remove support for nntp and telnet
 2023-09-04 19:22:38 +01:00
Chris Mayo
3d630b880f
Merge pull request #751 from cjmayo/wraptext 
Make TextLogger message wrapping configurable
 2023-09-04 19:22:09 +01:00
Chris Mayo
ad48c7db2c Make TextLogger message wrapping configurable 2023-08-28 19:30:11 +01:00
Chris Mayo
b3429c4759 Remove support for nntp and telnet 
Python is dropping nntplib and telnetlib.
 2023-08-28 19:24:57 +01:00
Chris Mayo
4d9749c5ba Log ignored warning messages as info 2023-08-28 19:22:24 +01:00
Chris Mayo
77da1c1b01 Revert "Move ignorewarnings to output section in linkcheckerrc(5)" 
The section header names are used when parsing linkcheckerrc.

This reverts commit f0b6ddda27.
 2023-08-28 19:22:24 +01:00
Chris Mayo
beaf9399f8 Elevate redirection to a warning tagged http-redirected 
Include the HTTP status code and reason in the message.
 2023-08-28 19:22:24 +01:00
Chris Mayo
f0b6ddda27 Move ignorewarnings to output section in linkcheckerrc(5) 
As per ignoreerrors, this is not filtering.
 2023-04-25 19:31:39 +01:00
Chris Mayo
16bee50068 Move AnchorCheck local file handling into a new class 
When checking local files with AnchorCheck, anchors in URLs
like "example/#anchor" are not supported.

Without AnchorCheck enabled, the Real URL reported for such URLs
was changed to include the anchor when local file checking was added to
AnchorCheck, but it is the directory that is checked.
The same URL was also then used as the Parent URL for the check of each
of the contents of that directory.

For FileUrl this is a revert of:
c221afda ("Enable AnchorCheck to be used with local files", 2022-10-03)
 2022-10-24 19:30:56 +01:00
Chris Mayo
a0eb6d5187 Align documentation of debug in man pages 
Linked to:
b3967f75 ("Correct documentation of --debug in linkchecker(1)", 2022-09-30)
 2022-10-18 19:24:08 +01:00
Chris Mayo
b6eea83f63
Merge pull request #676 from cjmayo/robotmap 
Document sitemaps in linkchecker(1)
 2022-10-17 19:25:57 +01:00
Chris Mayo
93f1d3f4ac Document sitemaps in linkchecker(1) 2022-10-17 19:21:03 +01:00
Chris Mayo
8bc849dfde Make --cookiefile description in linkchecker(1) a bit clearer 2022-10-17 19:21:03 +01:00
Chris Mayo
0c5db040c8 Support maxrequestspersecond less than one 2022-10-05 19:28:01 +01:00
Chris Mayo
e88cf49c8f Enable average HTTP request rate to be above 4 per second 2022-10-05 19:28:01 +01:00
Chris Mayo
354ea933ca
Merge pull request #673 from cjmayo/sitemap 
Fix sitemap output with multiple threads
 2022-10-05 19:20:40 +01:00
Chris Mayo
8b2fb86895 Remove AnchorCheck disabled note in linkcheckerrc(5) 
A partial revert of:
fe6dea12 ("Update documentation for disabled plugins", 2021-11-29)
 2022-10-03 19:33:05 +01:00
Chris Mayo
033dcf89f9
Merge pull request #671 from cjmayo/example 
Fix formatting of ignoreerrors example in linkcheckerrc(5)
 2022-10-03 19:22:36 +01:00
Chris Mayo
b3967f75c4 Correct documentation of --debug in linkchecker(1) 
dns logger was removed in:
e1f72490 ("Move dnspython module into third_party directory.", 2011-05-24)

Threading has not been disabled with --debug since:
eaa8a963 ("Refactor logging configuration.", 2014-05-10)
 2022-09-30 19:22:17 +01:00
Chris Mayo
61071fc5dc
Merge pull request #668 from cjmayo/defaults 
Clarify default values in initial linkcheckerrc and elsewhere
 2022-09-28 19:36:44 +01:00
Chris Mayo
001212b915 Fix formatting of ignoreerrors example in linkcheckerrc(5) 
Introduced in:
8c959589 ("add option to ignore specific errors for specific URLs", 2022-07-21)
 2022-09-28 19:23:04 +01:00
Lukas Pirl
8c959589c3
add option to ignore specific errors for specific URLs 2022-09-25 22:52:04 +02:00
Chris Mayo
4962a302b3 Document default frequency of sitemap logger 2022-09-22 19:24:55 +01:00
Chris Mayo
b8d0928969 Document dialect option of csv logger 2022-09-22 19:24:55 +01:00
Chris Mayo
a0b28cc0ff Rename url-rate-limited to http-rate-limited 
Make consistent with the other warnings:

- The first part of the name represents the checker class in which the
  warning is raised

- Update initial comment
 2022-09-06 19:32:24 +01:00
Chris Mayo
d6936ceb91 Add warning url-content-type-unparseable 2022-09-02 19:29:11 +01:00
Chris Mayo
fb8df6f089 Document URL list in linkchecker(1) 
First enabled in:
b1708fc7 ("get the complete content", 2004-09-03)
 2022-09-02 19:29:11 +01:00
Chris Mayo
f018cc9fd1 Fix typo addtion in linkcheckerrc(5) 2022-09-02 19:17:45 +01:00
Chris Mayo
10f3d33041 Finish documenting the use of XDG_CONFIG_HOME and XDG_DATA_HOME 
Introduced by:
a03e2e4a ("use xdg dirs for config & data", 2017-10-17)
 2022-08-23 19:21:53 +01:00
Chris Mayo
48a1dde27a Document url-rate-limited 
Introduced in:
dcdc64e8 ("Turn status code 429 into warning instead of failure",
2020-03-25)
 2022-07-11 19:27:07 +01:00
Chris Mayo
a67e8fd0f5 Test and recommend pdfminer.six for PdfParser 
pdfminer is no longer actively maintained; distributions are now
packaging pdfminer.six as pdfminer.
 2022-05-18 19:29:54 +01:00
Chris Mayo
de3361fae2 More proxy documentation updates 
a60648e34 ("Remove support for ftp_proxy", 2021-12-13)
9504a6ddd ("Document the curl_ca_bundle environment variable", 2021-12-13)
 2021-12-17 19:22:20 +00:00
Chris Mayo
315e729ec3 Fix typo "variables" in linkchecker(1) Proxy Support 2021-12-13 19:25:23 +00:00
Chris Mayo
9504a6dddf Document the curl_ca_bundle environment variable 2021-12-13 19:25:23 +00:00
Chris Mayo
a2e379a595 Remove built-in GNOME and KDE proxy support 
Only http_proxy was ever supported.

Requests uses urllib.request.getproxies().

Fedora 35 and Ubuntu 20.04 do set proxy environment variables when
settings are added through the GUI.

GNOME location of proxy settings is subject to change:
https://wiki.gnome.org/Projects/NetworkManager/Proxies
https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/27
 2021-12-13 19:25:23 +00:00
Chris Mayo
a60648e348 Remove support for ftp_proxy 
Was limited to HTTP proxy servers and prevents simplifying and fixing
HTTP proxy support.
 2021-12-13 19:25:23 +00:00
Chris Mayo
6269134017 Correct documentation of text output encoding 
50ea41b12 ("use preferred locale for default encoding", 2005-10-13)
714147cb2 ("Improved language and encoding detection by using local.getdefaultlocale() instead of locale.getlocale(category=LC_ALL)", 2009-03-04)
 2021-11-29 19:48:50 +00:00
Chris Mayo
6de93650f9 Group output documentation 2021-11-29 19:48:50 +00:00
Chris Mayo
a4b14047d6 Make quiet/-q set application logging to warning 2021-11-29 19:48:50 +00:00
Chris Mayo
eb33b1c1e5 Remove --warning-regex documentation 
Moved to RegexCheck plugin:
7b34be590 ("Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements.", 2014-03-01)
 2021-11-29 19:48:50 +00:00
Chris Mayo
a7155db64a Correct documentation of CSV separator 
851e1121e ("Use semicolon as default CSV separator.", 2010-07-31)
 2021-11-29 19:48:50 +00:00
Chris Mayo
895e2ddb43 Improve documentation of output 2021-11-29 19:48:50 +00:00
Chris Mayo
fe6dea12de Update documentation for disabled plugins 2021-11-29 19:35:34 +00:00
Chris Mayo
b9b4936729 Fix broken Edit on GitHub links in documentation 2021-08-12 19:28:50 +01:00