From 895e2ddb4389f2b7789ff01cabaf2d5ff19ae0da Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 1/6] Improve documentation of output --- doc/src/man/linkchecker.rst | 9 ++++----- doc/src/man/linkcheckerrc.rst | 8 ++++---- 2 files changed, 8 insertions(+), 9 deletions(-) diff --git a/doc/src/man/linkchecker.rst b/doc/src/man/linkchecker.rst index f1efcb01..91aacca9 100644 --- a/doc/src/man/linkchecker.rst +++ b/doc/src/man/linkchecker.rst @@ -124,7 +124,7 @@ Output options .. option:: -F TYPE[/ENCODING][/FILENAME], --file-output=TYPE[/ENCODING][/FILENAME] Output to a file linkchecker-out.TYPE, - $HOME/.linkchecker/failures for failures output, or + $HOME/.linkchecker/failures for the failures output type, or FILENAME if specified. The ENCODING specifies the output encoding, the default is that of your locale. Valid encodings are listed at @@ -147,10 +147,9 @@ Output options .. option:: -o TYPE[/ENCODING], --output=TYPE[/ENCODING] - Specify output type as text, html, sql, csv, + Specify the console output type as text, html, sql, csv, gml, dot, xml, sitemap, none or failures. - Default type is text. The various output types are documented - below. + Default type is text. The various output types are documented below. The ENCODING specifies the output encoding, the default is that of your locale. Valid encodings are listed at https://docs.python.org/library/codecs.html#standard-encodings. @@ -158,7 +157,7 @@ Output options .. option:: -q, --quiet Quiet operation, an alias for :option:`-o` *none*. This is only useful with - :option:`-F`. + :option:`-F`, else no results will be output. .. option:: -v, --verbose diff --git a/doc/src/man/linkcheckerrc.rst b/doc/src/man/linkcheckerrc.rst index 8eb49328..10e1dd25 100644 --- a/doc/src/man/linkcheckerrc.rst +++ b/doc/src/man/linkcheckerrc.rst @@ -169,14 +169,14 @@ output Command line option: :option:`--debug` **fileoutput=**\ *TYPE*\ [**,**\ *TYPE*...] Output to a file **linkchecker-out.**\ *TYPE*, or - **$HOME/.linkchecker/failures** for **failures** output. + **$HOME/.linkchecker/failures** for the **failures** output type. Valid file output types are **text**, **html**, **sql**, **csv**, **gml**, **dot**, **xml**, **none** or **failures**. Default is no file output. The various output types are documented below. Note that you can suppress all console output with **output=none**. Command line option: :option:`--file-output` **log=**\ *TYPE*\ [**/**\ *ENCODING*] - Specify output type as **text**, **html**, **sql**, **csv**, + Specify the console output type as **text**, **html**, **sql**, **csv**, **gml**, **dot**, **xml**, **none** or **failures**. Default type is **text**. The various output types are documented below. The *ENCODING* specifies the output encoding, the default is that of @@ -185,8 +185,8 @@ output Command line option: :option:`--output` **quiet=**\ [**0**\ \|\ **1**] If set, operate quiet. An alias for **log=none**. This is only - useful with **fileoutput**. - Command line option: :option:`--verbose` + useful with **fileoutput**, else no results will be output. + Command line option: :option:`--quiet` **status=**\ [**0**\ \|\ **1**] Control printing check status messages. Default is 1. Command line option: :option:`--no-status` From a7155db64a03748cb929ad1e4d57635566a2ab38 Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 2/6] Correct documentation of CSV separator 851e1121e ("Use semicolon as default CSV separator.", 2010-07-31) --- config/linkcheckerrc | 2 +- doc/src/man/linkcheckerrc.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/config/linkcheckerrc b/config/linkcheckerrc index c2470950..918d686e 100644 --- a/config/linkcheckerrc +++ b/config/linkcheckerrc @@ -79,7 +79,7 @@ # CSV logger [csv] #filename=linkchecker-out.csv -#separator=, +#separator=; #quotechar=" #parts=all diff --git a/doc/src/man/linkcheckerrc.rst b/doc/src/man/linkcheckerrc.rst index 10e1dd25..af931d17 100644 --- a/doc/src/man/linkcheckerrc.rst +++ b/doc/src/man/linkcheckerrc.rst @@ -274,7 +274,7 @@ csv **encoding=**\ *STRING* See :ref:`[text] ` section above. **separator=**\ *CHAR* - Set CSV separator. Default is a comma (**,**). + Set CSV separator. Default is a semicolon (**;**). **quotechar=**\ *CHAR* Set CSV quote character. Default is a double quote (**"**). From eb33b1c1e5fe80658c0da9232d954b7c8eefbe72 Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 3/6] Remove --warning-regex documentation Moved to RegexCheck plugin: 7b34be590 ("Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements.", 2014-03-01) --- doc/src/man/linkchecker.rst | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/doc/src/man/linkchecker.rst b/doc/src/man/linkchecker.rst index 91aacca9..181371b9 100644 --- a/doc/src/man/linkchecker.rst +++ b/doc/src/man/linkchecker.rst @@ -162,17 +162,6 @@ Output options .. option:: -v, --verbose Log all checked URLs. Default is to log only errors and warnings. - -.. option:: -W REGEX, --warning-regex=REGEX - - Define a regular expression which prints a warning if it matches any - content of the checked link. This applies only to valid pages, so we - can get their content. - Use this to check for pages that contain some form of error, for - example "This page has moved" or "Oracle Application error". - Note that multiple values can be combined in the regular expression, - for example "(This page has moved|Oracle Application error)". - See section `REGULAR EXPRESSIONS`_ for more info. Checking options ^^^^^^^^^^^^^^^^ From a4b14047d6a157e78423916d03ab3268d9865047 Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 4/6] Make quiet/-q set application logging to warning --- doc/src/man/linkchecker.rst | 5 +++-- doc/src/man/linkcheckerrc.rst | 5 +++-- linkcheck/configuration/confparse.py | 1 + linkchecker | 2 ++ 4 files changed, 9 insertions(+), 4 deletions(-) diff --git a/doc/src/man/linkchecker.rst b/doc/src/man/linkchecker.rst index 181371b9..f36a636b 100644 --- a/doc/src/man/linkchecker.rst +++ b/doc/src/man/linkchecker.rst @@ -156,8 +156,9 @@ Output options .. option:: -q, --quiet - Quiet operation, an alias for :option:`-o` *none*. This is only useful with - :option:`-F`, else no results will be output. + Quiet operation, an alias for :option:`-o` *none* that also hides + application information messages. + This is only useful with :option:`-F`, else no results will be output. .. option:: -v, --verbose diff --git a/doc/src/man/linkcheckerrc.rst b/doc/src/man/linkcheckerrc.rst index af931d17..f603688e 100644 --- a/doc/src/man/linkcheckerrc.rst +++ b/doc/src/man/linkcheckerrc.rst @@ -184,8 +184,9 @@ output https://docs.python.org/library/codecs.html#standard-encodings. Command line option: :option:`--output` **quiet=**\ [**0**\ \|\ **1**] - If set, operate quiet. An alias for **log=none**. This is only - useful with **fileoutput**, else no results will be output. + If set, operate quiet. An alias for **log=none** that also hides + application information messages. + This is only useful with **fileoutput**, else no results will be output. Command line option: :option:`--quiet` **status=**\ [**0**\ \|\ **1**] Control printing check status messages. Default is 1. diff --git a/linkcheck/configuration/confparse.py b/linkcheck/configuration/confparse.py index 1ef86de5..fb2dd3cb 100644 --- a/linkcheck/configuration/confparse.py +++ b/linkcheck/configuration/confparse.py @@ -134,6 +134,7 @@ class LCConfigParser(RawConfigParser): if self.getboolean(section, "quiet"): self.config['output'] = 'none' self.config['quiet'] = True + logconf.reset_loglevel() # if debug will be overwritten next if self.has_option(section, "debug"): val = self.get(section, "debug") parts = [f.strip().lower() for f in val.split(',')] diff --git a/linkchecker b/linkchecker index 3e0de3e6..4b5207ac 100755 --- a/linkchecker +++ b/linkchecker @@ -529,6 +529,8 @@ if options.debug: if _name not in allowed_debugs: print_usage(_("Invalid debug level %(level)r") % {"level": _name}) logconf.set_debug(options.debug) +elif options.quiet: + logconf.reset_loglevel() log.debug( LOG_CMDLINE, _("Python %(version)s on %(platform)s") From 6de93650f992b0ae054340666e80b8ab377ed3c8 Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 5/6] Group output documentation --- doc/src/man/linkchecker.rst | 44 +++++++++++++++++++++------------ doc/src/man/linkcheckerrc.rst | 46 ++++++++++++++++++++++++----------- 2 files changed, 60 insertions(+), 30 deletions(-) diff --git a/doc/src/man/linkchecker.rst b/doc/src/man/linkchecker.rst index f36a636b..641ce2e7 100644 --- a/doc/src/man/linkchecker.rst +++ b/doc/src/man/linkchecker.rst @@ -112,14 +112,8 @@ General options Output options ^^^^^^^^^^^^^^ -.. option:: -D STRING, --debug=STRING - - Print debugging output for the given logger. Available loggers are - cmdline, checking, cache, dns, plugin and - all. Specifying all is an alias for specifying all available - loggers. The option can be given multiple times to debug with more - than one logger. For accurate results, threading will be disabled - during debug runs. +URL checking results +"""""""""""""""""""" .. option:: -F TYPE[/ENCODING][/FILENAME], --file-output=TYPE[/ENCODING][/FILENAME] @@ -137,10 +131,6 @@ Output options The various output types are documented below. Note that you can suppress all console output with the option :option:`-o` *none*. -.. option:: --no-status - - Do not print check status messages. - .. option:: --no-warnings Don't log warnings. Default is to log warnings. @@ -154,16 +144,38 @@ Output options your locale. Valid encodings are listed at https://docs.python.org/library/codecs.html#standard-encodings. +.. option:: -v, --verbose + + Log all checked URLs. Default is to log only errors and warnings. + +Progress updates +"""""""""""""""" + +.. option:: --no-status + + Do not print URL check status messages. + +Application +""""""""""" + +.. option:: -D STRING, --debug=STRING + + Print debugging output for the given logger. Available loggers are + cmdline, checking, cache, dns, plugin and + all. Specifying all is an alias for specifying all available + loggers. The option can be given multiple times to debug with more + than one logger. For accurate results, threading will be disabled + during debug runs. + +Quiet +""""" + .. option:: -q, --quiet Quiet operation, an alias for :option:`-o` *none* that also hides application information messages. This is only useful with :option:`-F`, else no results will be output. -.. option:: -v, --verbose - - Log all checked URLs. Default is to log only errors and warnings. - Checking options ^^^^^^^^^^^^^^^^ diff --git a/doc/src/man/linkcheckerrc.rst b/doc/src/man/linkcheckerrc.rst index f603688e..e5f95e04 100644 --- a/doc/src/man/linkcheckerrc.rst +++ b/doc/src/man/linkcheckerrc.rst @@ -161,12 +161,9 @@ authentication output ^^^^^^ -**debug=**\ *STRING*\ [**,**\ *STRING*...] - Print debugging output for the given modules. Available debug - modules are **cmdline**, **checking**, **cache**, **dns**, - **thread**, **plugins** and **all**. Specifying **all** is an alias - for specifying all available loggers. - Command line option: :option:`--debug` +URL checking results +"""""""""""""""""""" + **fileoutput=**\ *TYPE*\ [**,**\ *TYPE*...] Output to a file **linkchecker-out.**\ *TYPE*, or **$HOME/.linkchecker/failures** for the **failures** output type. @@ -183,14 +180,6 @@ output your locale. Valid encodings are listed at https://docs.python.org/library/codecs.html#standard-encodings. Command line option: :option:`--output` -**quiet=**\ [**0**\ \|\ **1**] - If set, operate quiet. An alias for **log=none** that also hides - application information messages. - This is only useful with **fileoutput**, else no results will be output. - Command line option: :option:`--quiet` -**status=**\ [**0**\ \|\ **1**] - Control printing check status messages. Default is 1. - Command line option: :option:`--no-status` **verbose=**\ [**0**\ \|\ **1**] If set log all checked URLs once. Default is to log only errors and warnings. @@ -199,6 +188,35 @@ output If set log warnings. Default is to log warnings. Command line option: :option:`--no-warnings` +Progress updates +"""""""""""""""" + +**status=**\ [**0**\ \|\ **1**] + Control printing URL checker status messages. Default is 1. + Command line option: :option:`--no-status` + +Application +""""""""""" + +**debug=**\ *STRING*\ [**,**\ *STRING*...] + Print debugging output for the given modules. Available debug + modules are **cmdline**, **checking**, **cache**, **dns**, + **thread**, **plugins** and **all**. Specifying **all** is an alias + for specifying all available loggers. + Command line option: :option:`--debug` + +Quiet +""""" + +**quiet=**\ [**0**\ \|\ **1**] + If set, operate quiet. An alias for **log=none** that also hides + application information messages. + This is only useful with **fileoutput**, else no results will be output. + Command line option: :option:`--quiet` + +OUTPUT TYPES +------------ + text ^^^^ From 62691340175cbf9616788edef2da81ac056b6d10 Mon Sep 17 00:00:00 2001 From: Chris Mayo Date: Mon, 29 Nov 2021 19:48:50 +0000 Subject: [PATCH 6/6] Correct documentation of text output encoding 50ea41b12 ("use preferred locale for default encoding", 2005-10-13) 714147cb2 ("Improved language and encoding detection by using local.getdefaultlocale() instead of locale.getlocale(category=LC_ALL)", 2009-03-04) --- doc/src/man/linkcheckerrc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/src/man/linkcheckerrc.rst b/doc/src/man/linkcheckerrc.rst index e5f95e04..9524aa23 100644 --- a/doc/src/man/linkcheckerrc.rst +++ b/doc/src/man/linkcheckerrc.rst @@ -231,7 +231,7 @@ text **encoding=**\ *STRING* Valid encodings are listed in https://docs.python.org/library/codecs.html#standard-encodings. - Default encoding is **iso-8859-15**. + Default encoding is the system default locale encoding. *color\** Color settings for the various log parts, syntax is *color* or *type*\ **;**\ *color*. The *type* can be **bold**, **light**,