Hopiu/linkchecker
1
0
Fork 0
mirror of https://github.com/Hopiu/linkchecker.git synced 2026-03-24 18:00:24 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions
6987 commits 1 branch 0 tags 24 MiB
Commit graph

602 commits

Author SHA1 Message Date
Chris Mayo
b6bc366af0 Run pyupgrade --py37-plus x 2 2022-11-08 19:21:29 +00:00
Chris Mayo
0bb1576887 Run pyupgrade --py37-plus --keep-percent-format 2022-11-08 19:21:29 +00:00
Chris Mayo
9631c314dd Use \d in regexp in TestDecorators.test_timeit2() 2022-10-18 19:24:08 +01:00
Chris Mayo
deac09d2c1 Clarify note in TestConfig 2022-10-18 19:24:08 +01:00
Chris Mayo
78536c578a Fix tests failing when run with pytest 
TypeError: 'NoneType' object is not callable

As per:
2cbff492 ("Fix http tests failing with pytest due to missing _()", 2022-10-03)
 2022-10-17 19:26:53 +01:00
Chris Mayo
96c3336013
Merge pull request #677 from cjmayo/maxrate 
Enable average HTTP request rate to be above 4 per second
 2022-10-17 19:24:49 +01:00
Chris Mayo
0c5db040c8 Support maxrequestspersecond less than one 2022-10-05 19:28:01 +01:00
Chris Mayo
bbb8096df5 Add @need_network to test_no_error() in test_ignoreerrors.py 
Needs network access for DNS:

warning No MX mail host for example.com found.
 2022-10-05 19:27:13 +01:00
Chris Mayo
354ea933ca
Merge pull request #673 from cjmayo/sitemap 
Fix sitemap output with multiple threads
 2022-10-05 19:20:40 +01:00
Nathan Arthur
2d1bf6ef98 Add tests for encoded anchors for file: and http: 
I started with a test of urlencoded anchors, assuming at the URL might
have a urlencoded anchor, but the actual anchor in the HTML would NOT be
urlencoded.
 2022-10-03 19:33:05 +01:00
Nathan Arthur
5398fd2406 Add an anchor test for multiple inter-connected files 2022-10-03 19:33:05 +01:00
Nathan Arthur
c221afdab5 Enable AnchorCheck to be used with local files 
[I] discovered that fileurl.py was stripping the anchors from url_data,
which breaks AnchorCheck. So I stopped it from doing that, and
tried to fix up all the places that were assuming the url would map to a
filesystem file. The tests all pass, but I'm not 100% sure I caught all
the cases, or fixed them correctly.
 2022-10-03 19:33:05 +01:00
Chris Mayo
2cbff49221 Fix http tests failing with pytest due to missing _() 
TypeError: 'NoneType' object is not callable

Ensure LinkCheckTest.setUp() is called to initialise translations.
 2022-10-03 19:33:05 +01:00
Chris Mayo
54bcefd7d7 Revert "Disable AnchorCheck plugin" 
This reverts commit 0356524369.
 2022-10-03 19:33:05 +01:00
Chris Mayo
009f22e9b6 Remove outdated comment in TestLogger 
Configuration.init_logging() removed in:
eaa8a963 ("Refactor logging configuration.", 2014-05-10)
 2022-09-30 19:22:17 +01:00
Lukas Pirl
8c959589c3
add option to ignore specific errors for specific URLs 2022-09-25 22:52:04 +02:00
Chris Mayo
0c59cd5c1e Don't use default values in configuration tests 2022-09-20 19:36:42 +01:00
Chris Mayo
fd6f4a0160
Merge pull request #657 from pacenathan/MacTimingTest 
Make timing test more tolerant
 2022-09-10 17:24:56 +01:00
Nathan Arthur
47a83cbb27 Make timing test more tolerant 
On my M1 Mac, this was taking 1.01 seconds rather than the expected 1.00
seconds. This is OK, because sleep() is not guaranteed to be precise.
 2022-09-08 09:15:29 -04:00
Stefan Fisk
d2b9723612 Fix srcset parsing 
Resolves #631
 2022-09-07 21:24:23 +02:00
Chris Mayo
3c7fb5b571 Fix checking directory containing Unicode filenames 
Non-Unicode filenames are not supported.

sys.platform has not returned "linux2" since Python 3.3.
 2022-09-05 19:28:40 +01:00
Chris Mayo
1abd9ea10e Skip tests in TestFile rather than silently returning 2022-09-05 19:28:40 +01:00
Chris Mayo
d6936ceb91 Add warning url-content-type-unparseable 2022-09-02 19:29:11 +01:00
Kian-Meng Ang
a70ea9ea14 Fix typos 
Found via `codespell ./linkcheck/ ./tests ./doc/man/en -L bu,noone,fo,pres,shttp`
 2022-09-02 17:20:02 +08:00
Chris Mayo
53bbf28584 Re-enable ldap: test 
Disabled in:
1733c6a6 ("Fix Travis CI build.", 2014-03-11)
 2022-08-23 19:25:32 +01:00
Chris Mayo
34ba737710 Remove cchardet from Docker image, tox env and recommendation 
Beautiful Soup 4.11 will use charset-normalizer.
 2022-04-11 19:26:18 +01:00
Chris Mayo
b4c2599aef Output msgfmt error on test_pos failure 2022-01-19 19:31:01 +00:00
Chris Mayo
ae577357ed Update doc translation path in test_po.py 2022-01-19 19:31:01 +00:00
Chris Mayo
8007940770 Test more command arguments 2021-12-20 19:44:40 +00:00
Chris Mayo
5fef9a3b60 Generate linkchecker command using an entry point 
drop_privileges() is only used by the linkchecker command.
Move installing SIGUSR1 handler to the linkchecker command only - fixes
intermittent test failures.
 2021-12-20 19:34:58 +00:00
Chris Mayo
b0878dd7e8 Require TestFile.test_directory_listing to succeed 2021-12-15 19:36:53 +00:00
Chris Mayo
76815bcf47 Don't guess the URL for files that end in .html 
Fixes:
linkchecker ftp.html
failing looking for ftp://ftp.html
 2021-12-13 19:31:13 +00:00
Chris Mayo
fe5a34c68f Remove linkcheck.checker.proxysupport 
Set up the requests.Session() with the complete proxy configuration
to fix a problem with using an HTTP server as an HTTPS proxy and
potential redirection issues.

Requests handles no_proxy.
 2021-12-13 19:25:23 +00:00
Koen Van den Wijngaert
900586dc01
Better handling for link rel dns-prefetch and add preconnect support (#536) 
preconnect is only DNS checked.

This is allowed even in the Resource Hints Editor's Draft
https://w3c.github.io/resource-hints/#preconnect
 2021-12-09 19:38:30 +00:00
Chris Mayo
3b19680e97 Add guidance on character set detecting including cchardet 2021-12-06 19:24:26 +00:00
Chris Mayo
0356524369 Disable AnchorCheck plugin 
Can't be relied on. Multiple reports of expected results not returned.

https://github.com/linkchecker/linkchecker/issues/542
https://github.com/linkchecker/linkchecker/issues/555
https://github.com/linkchecker/linkchecker/issues/568

Previously a fix was needed just to get the tests working:
0912e8a2c ("Don't strip the URL fragment from cache key if using AnchorCheck", 2020-07-27)

After:
eaa538c81 ("don't check one url multiple times", 2016-11-09)
 2021-11-29 19:35:34 +00:00
Chris Mayo
ef60e9dcd6 Enable certificate verification during https test 2021-11-22 19:27:18 +00:00
Chris Mayo
bb4102da5a Replace deprecated ssl.wrap_socket() in tests 2021-11-22 19:27:18 +00:00
Chris Mayo
deed6ce231 Ensure chardet is installed when testing using tox 
Beautiful Soup uses chardet, if installed, to detect character
encodings. This can lead to different test results based on whether
chardet is installed or not.

Requests < 2.26.0 requires chardet, but since 2.26.0 Requests requires
charset_normalizer.

Explicitly installing chardet maintains consistent test results.
 2021-07-27 19:48:27 +01:00
Paul Haerle
f395c74aac
Make ResultCache max_size configurable (#544) 
* Make ResultCache max_size configurable

fixes #463

* Add tests and docs.

* fix documentation...

...adapt the source, not the auto-generated man pages themselves as
requested in #544.

* fix typo.
 2021-06-21 19:45:19 +01:00
Marius Gedminas
163ff725f8 Fix tests? 2021-05-19 16:36:16 +03:00
Chris Mayo
09b4da393e Initialise Configuration.status_logger 
Fixes failure of the LinkChecker WSGI application which does
not call Configuration.set_status_logger().
 2021-01-28 19:20:24 +00:00
Chris Mayo
7c2036b68c Drop support for Beautiful Soup < 4.8.1 
The minimum version supported was already 4.8.0 because of the use
of multi_valued_attributes [1].

Test support for < 4.8.1 is the only code that needs removing [2].

[1] 3ff3d724 ("Use BeautifulSoup element attrs directly", 2020-04-03)
[2] 607328d5 ("Support Beautiful Soup line numbers", 2019-10-05)
 2021-01-28 19:20:24 +00:00
Chris Mayo
e922dd0224 Stop using biplist 
plistlib has supported binary files since Python 3.4.
 2020-10-12 19:55:46 +01:00
Chris Mayo
e75c4b3d36 Reuse linkcheck.bookmarks.safari.has_biplist in tests 2020-09-23 19:38:17 +01:00
Chris Mayo
9891fc3f70 Python 3.9 adds suport for HTTP status code 103 EARLY_HINTS 2020-09-14 19:55:05 +01:00
Chris Mayo
f268b95cf8 biplist is not compatible with Python 3.9 
File ".tox/py39/lib/python3.9/site-packages/biplist/__init__.py", line 143, in readPlist
    line: raise InvalidPlistException(e)
    locals:
      InvalidPlistException = <global> <class 'biplist.InvalidPlistException'>
      e = <not found>

InvalidPlistException: module 'plistlib' has no attribute 'Data'
 2020-09-14 19:55:05 +01:00
Chris Mayo
b1faef93c3
Merge pull request #495 from cjmayo/mswindows 
MS Windows Python 3.7 and MS Store compatibility
 2020-09-01 19:46:44 +01:00
Chris Mayo
314ec085a3
Merge pull request #462 from cjmayo/anchor 
Fix anchor checking
 2020-09-01 19:39:29 +01:00
Chris Mayo
89613d56f2 Replace the use of Python internal test.support 
Its use is discourged and it is not present in the MS Store version of
Python.
 2020-08-29 16:57:57 +01:00