Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-04-21 22:54:45 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
6880 commits 1 branch 0 tags 24 MiB
Commit graph

3264 commits

Author SHA1 Message Date
Nathan Arthur
6dc5ade29d Fixed a bug where the resultcachesize setting was ignored. 2022-09-20 19:36:23 +01:00
Chris Mayo
af265f3d52 Write all metadata used to _release.py 
Enables running without installing.
Removes use of importlib.metadata.
 2022-09-13 19:32:06 +01:00
Chris Mayo
30e8cfad77
Merge pull request #651 from cjmayo/rate 
Rename url-rate-limited to http-rate-limited
 2022-09-12 19:25:52 +01:00
Stefan Fisk
d2b9723612 Fix srcset parsing 
Resolves #631
 2022-09-07 21:24:23 +02:00
Chris Mayo
a0b28cc0ff Rename url-rate-limited to http-rate-limited 
Make consistent with the other warnings:

- The first part of the name represents the checker class in which the
  warning is raised

- Update initial comment
 2022-09-06 19:32:24 +01:00
Chris Mayo
3c7fb5b571 Fix checking directory containing Unicode filenames 
Non-Unicode filenames are not supported.

sys.platform has not returned "linux2" since Python 3.3.
 2022-09-05 19:28:40 +01:00
Chris Mayo
c627b00755
Merge pull request #639 from cjmayo/hatch 
Replace setuptools and setup.py with hatch and pyproject.toml
 2022-09-05 19:27:48 +01:00
Chris Mayo
d5058ecd7c
Merge pull request #643 from cjmayo/altname 
Replace deprecated urllib3.contrib.pyopenssl.get_subj_alt_name()
 2022-09-05 19:26:12 +01:00
Chris Mayo
47d1015e00 Replace setuptools and setup.py with hatch and pyproject.toml 2022-09-05 19:24:01 +01:00
Chris Mayo
f0cb2e9df9 Use cryptography.x509.not_valid_after 2022-09-05 19:20:19 +01:00
Chris Mayo
76e2712311 Replace deprecated urllib3.contrib.pyopenssl.get_subj_alt_name() 
Strictly we should add a dependency on cryptography as we are now using
it directly - but for pyopenssl x509.to_cryptography() to work
cryptography would have to be already installed.
 2022-09-05 19:20:19 +01:00
Chris Mayo
c79bc07cee Add MIME type application/vnd.adobe.flash.movie 2022-09-02 19:29:11 +01:00
Chris Mayo
6d9061b00a Ignore bs4 markup and XML parser warnings 
XMLParsedAsHTMLWarning: It looks like you're parsing an XML document
using an HTML parser.

MarkupResemblesLocatorWarning: The input looks more like a filename than
markup.

MarkupResemblesLocatorWarning: The input looks more like a URL than
markup.
 2022-09-02 19:29:11 +01:00
Chris Mayo
d6936ceb91 Add warning url-content-type-unparseable 2022-09-02 19:29:11 +01:00
Kian-Meng Ang
a70ea9ea14 Fix typos 
Found via `codespell ./linkcheck/ ./tests ./doc/man/en -L bu,noone,fo,pres,shttp`
 2022-09-02 17:20:02 +08:00
Chris Mayo
b35036af2b
Merge pull request #634 from cjmayo/pyxdg 
Remove dependency on pyxdg
 2022-08-30 19:28:03 +01:00
Chris Mayo
d72649453c
Merge pull request #632 from cjmayo/docs 
Assorted documentation updates
 2022-08-30 19:27:10 +01:00
Felix Yan
7db1a867ab
Correct a typo in i18n.py 2022-08-24 19:10:41 +03:00
Chris Mayo
fbceca5dc9 Remove dependency on pyxdg 
Read the environment variables and implement the same fallbacks.
Saves a hardly used dependency and is more explicit.
 2022-08-23 19:26:15 +01:00
Chris Mayo
10f3d33041 Finish documenting the use of XDG_CONFIG_HOME and XDG_DATA_HOME 
Introduced by:
a03e2e4a ("use xdg dirs for config & data", 2017-10-17)
 2022-08-23 19:21:53 +01:00
Chris Mayo
94781120ac Correct mention of pdfminer in WordParser comment 2022-05-18 19:29:54 +01:00
Malte Gerth
cc48a09308 Add Telegram and WhatsApp link schemes 2022-02-06 23:41:33 +01:00
Malte Gerth
067dd8edbb Update IANA schemes 2022-02-06 23:40:36 +01:00
Chris Mayo
141a811ba6 Enable creating a binary with PyOxidizer 
With PyOxidizer 0.18.0 AppName in setup.py has to be changed to the
all lower case "linkchecker".

Application translations do not work.

better_exchook2.fallback_findfile() may still need converting, first
needs a test.
 2021-12-30 19:27:04 +00:00
Chris Mayo
5768b76f6c Use pkgutil to simplify loader.get_package_modules() 
Replaces use of __file__.
 2021-12-30 19:27:04 +00:00
Chris Mayo
a55bbc5237 Write RELEASE_DATE to egg-info 2021-12-30 19:27:04 +00:00
Chris Mayo
50b2063a4b Install translation catalogs in the package data 
Custom clean command no longer needed because share directory is not
created in build.
 2021-12-30 19:27:04 +00:00
Chris Mayo
1d10fffde4 Use package metadata 2021-12-30 19:27:04 +00:00
Chris Mayo
819dacb9bb Install linkcheckerrc in the package data 
data/__init__.py needed for Python < 3.10
(namespace packages supported from importlib_resources v3.2)
 2021-12-30 19:27:04 +00:00
Chris Mayo
5c0d66dd74 Raise minimum Python requirement to 3.7 2021-12-30 19:27:04 +00:00
Chris Mayo
a9ab4d847b Remove get_share_file() 
cacert.pem not used since:
e3ab9024 ("Remove platform-specific installer stuff and ensure a build .whl wheel file can be built.", 2016-01-17)
 2021-12-30 19:27:04 +00:00
Chris Mayo
2fa0016ae9 Remove Portable 
Building portable removed in:
e3ab9024 ("Remove platform-specific installer stuff and ensure a build .whl wheel file can be built.", 2016-01-17)
 2021-12-30 19:27:04 +00:00
Chris Mayo
3359c7364f Remove is_frozen() 
Not used since:
e3ab9024 ("Remove platform-specific installer stuff and ensure a build .whl wheel file can be built.", 2016-01-17)
 2021-12-30 19:27:04 +00:00
Chris Mayo
271cb59e62 Remove unused code from i18n 2021-12-30 19:27:04 +00:00
Chris Mayo
158c401dae Update copyright to 2022 2021-12-30 19:27:04 +00:00
Chris Mayo
8bc3b39b41 One more proxy documentation update 
a2e379a5 ("Remove built-in GNOME and KDE proxy support", 2021-12-13)
 2021-12-21 19:23:00 +00:00
Chris Mayo
5fef9a3b60 Generate linkchecker command using an entry point 
drop_privileges() is only used by the linkchecker command.
Move installing SIGUSR1 handler to the linkchecker command only - fixes
intermittent test failures.
 2021-12-20 19:34:58 +00:00
Chris Mayo
efb92fbee8 Create setup_config from linkchecker 2021-12-20 19:34:58 +00:00
Chris Mayo
e501c4ffac Create ArgParser from linkchecker 2021-12-20 19:34:58 +00:00
Chris Mayo
9bc1f4d04e Use relative import for configuration in failures.py 2021-12-20 19:34:58 +00:00
Chris Mayo
4444a87eb9 Update Requests bug link 2021-12-15 19:34:24 +00:00
Chris Mayo
5f3b007934
Merge pull request #591 from cjmayo/robot 
Assume robots.txt is UTF-8
 2021-12-15 19:31:00 +00:00
Chris Mayo
d70ec6f75b Assume robots.txt is UTF-8 
Match the Python standard library and Google's interpretation:
https://developers.google.com/search/docs/advanced/robots/robots_txt#file-format

Avoid Unhandled LookupError.
 2021-12-13 19:31:55 +00:00
Chris Mayo
76815bcf47 Don't guess the URL for files that end in .html 
Fixes:
linkchecker ftp.html
failing looking for ftp://ftp.html
 2021-12-13 19:31:13 +00:00
Chris Mayo
9504a6dddf Document the curl_ca_bundle environment variable 2021-12-13 19:25:23 +00:00
Chris Mayo
a2e379a595 Remove built-in GNOME and KDE proxy support 
Only http_proxy was ever supported.

Requests uses urllib.request.getproxies().

Fedora 35 and Ubuntu 20.04 do set proxy environment variables when
settings are added through the GUI.

GNOME location of proxy settings is subject to change:
https://wiki.gnome.org/Projects/NetworkManager/Proxies
https://gitlab.gnome.org/GNOME/gsettings-desktop-schemas/-/issues/27
 2021-12-13 19:25:23 +00:00
Chris Mayo
fe5a34c68f Remove linkcheck.checker.proxysupport 
Set up the requests.Session() with the complete proxy configuration
to fix a problem with using an HTTP server as an HTTPS proxy and
potential redirection issues.

Requests handles no_proxy.
 2021-12-13 19:25:23 +00:00
Chris Mayo
35ecb7e639 Add https_proxy to internal error message 2021-12-13 19:25:23 +00:00
Chris Mayo
a60648e348 Remove support for ftp_proxy 
Was limited to HTTP proxy servers and prevents simplifying and fixing
HTTP proxy support.
 2021-12-13 19:25:23 +00:00
Chris Mayo
f2e5a435e3 Remove unused ProxySupport.proxyauth 
Not used since:
7b34be590 ("Introduce check plugins, use Python requests for http/s connections, and some code cleanups and improvements.", 2014-03-01)
 2021-12-13 19:25:23 +00:00